{{ message }}
Combined/all fixes v2#24
Merged
Merged
Conversation
DynamoDB uses specific message patterns for validation errors that the
conformance tests verify exactly. This fixes ~34 test failures by:
- Table name validation: use "1 validation error detected: Value 'X' at
'tableName' failed to satisfy constraint: ..." format with separate
messages for too-short, too-long, empty, and null cases
- Batch operations: "The requestItems parameter is required for BatchGetItem/
BatchWriteItem" and per-table limit messages
- Transactions: "Value '[]' at 'transactItems' ... greater than or equal to 1"
- Expression syntax errors: "Invalid {Type}: Syntax error; token: ..., near: ..."
- Empty expressions: "Invalid {Type}: The expression can not be empty;"
- Query Limit: capital 'L', no value quotes
- Scan segment: include actual segment/total values in error
- Add validation_error()/validation_errors() formatter functions - Table name: length errors include value and field path - KeySchema: >2 elements includes serialized array value - TransactGetItems/TransactWriteItems: empty/too-many use correct format - BatchGetItem: per-table >100 keys uses correct field path - BatchWriteItem: >25 items includes map value representation
The query builder used chr(1114111) (Unicode max codepoint) appended to the prefix for the exclusive upper bound. This only works for TEXT columns — on BYTEA columns it causes a PostgreSQL type error, surfacing as an InternalServerError. Fix: detect binary sort key columns (sk_b) and compute the upper bound in Rust by incrementing the prefix bytes with carry propagation. Bind both prefix and upper bound as separate BYTEA parameters.
… invalid enum values
Projecting mylist[N] was returning null because insert_nested did not
handle PathElement::Index. DynamoDB returns the projected element wrapped
in a single-element list: {"mylist": [value]}.
Verified against real DynamoDB: mylist[0] → {"mylist":{"L":[value]}},
out-of-bounds index → empty item.
- Reserved keyword enforcement (configurable via enforce_reserved_keywords) - Duplicate key detection in BatchGetItem and TransactGetItems - LSI on hash-only table rejection - Duplicate index name rejection in CreateTable - Empty string key value rejection
DynamoDB allows one level of parentheses in KeyConditionExpression: - Around individual clauses: (pk = :pk) AND (sk > :sk) - Around the full expression: (pk = :pk AND sk > :sk) - Nested parens are rejected (matches DynamoDB behavior) Adds outer-paren stripping in parse_key_condition and per-clause paren handling in parse_key_clause.
- Empty KCE/UpdateExpression: 'The expression can not be empty;' - Tokenizer syntax errors: 'Syntax error; token: "x", near: "..."' - Expression type prefix (ProjectionExpression, FilterExpression, etc.) - NULL attr false: correct ValidationException message - Empty SS/NS/BS: 'were invalid:' not 'are not valid.' - PutItem: reject EAV/EAN without ConditionExpression - prefix_expression_error helper for consistent formatting
- KeyType/ScalarAttributeType: custom Deserialize with DynamoDB enum error format - Select enum: correct member order in error message - CreateTable: route validation errors from deserialization, missing TableName message - PutItem: route validation errors from deserialization, null TableName message
…sToGet Older AWS SDKs use pre-expression parameters for Query and Scan. These were previously either silently ignored or rejected. Now they are desugared into their modern expression-based equivalents: - KeyConditions → KeyCondition struct (EQ, GT, LT, GE, LE, BETWEEN, BEGINS_WITH) - QueryFilter/ScanFilter → filter Expr AST (all comparison operators including CONTAINS, NOT_CONTAINS, BEGINS_WITH, BETWEEN, IN, NULL, NOT_NULL) - AttributesToGet → ProjectionExpression (on Query and Scan, matching existing GetItem/BatchGetItem support) Mutual exclusivity enforced: mixing legacy + expression params returns ValidationException with the standard DynamoDB conflict message.
# Conflicts: # crates/storage-postgres/src/data/index.rs
# Conflicts: # crates/core/src/expression/mod.rs # crates/engine/src/transact_get_items.rs
# Conflicts: # crates/engine/src/query.rs
# Conflicts: # crates/core/src/expression/mod.rs # crates/core/src/validation/mod.rs # crates/engine/src/get_item.rs # crates/engine/src/query.rs # crates/engine/src/scan.rs # crates/engine/src/transact_get_items.rs # crates/engine/src/update_item.rs
# Conflicts: # crates/engine/src/create_table.rs # crates/engine/src/put_item.rs
# Conflicts: # crates/engine/src/get_item.rs # crates/engine/src/put_item.rs # crates/engine/src/query.rs # crates/engine/src/scan.rs # crates/engine/src/transact_get_items.rs # crates/engine/src/update_item.rs
# Conflicts: # crates/core/src/validation/mod.rs # crates/engine/src/batch_get_item.rs # crates/engine/src/batch_write_item.rs # crates/engine/src/transact_get_items.rs # crates/engine/src/transact_write_items.rs
…, empty BS message - ListTables: add COLLATE "C" for byte-order sort matching DynamoDB - size(): return 0 for missing attributes instead of erroring (filter skips) - Query: reject Select=SPECIFIC_ATTRIBUTES without ProjectionExpression - Empty BS: fix message to "Binary sets should not be empty"
- UpdateItem: evaluate ConditionExpression against empty item when key doesn't exist, not the key-only upsert placeholder - Tags: invalid ARN format returns ValidationException
- UpdateItem: empty string UpdateExpression now reaches tokenize_for which returns "The expression can not be empty;" instead of being caught by the "must be provided" validation - PutItem: validate table name before table_key_info lookup so invalid names return ValidationException not ResourceNotFoundException
…oughput, and same-value no-ops
…ion, ProjectionExpression)
…TransactWrite condition on non-existent items
… LSI without range key
…AccessDeniedException
Collaborator
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Combined conformance fixes — 454/576 → 562/576 (97.6%)
Summary
This is a large combined branch merging 20 individual fix branches plus additional incremental fixes made directly on the combined branch. It brings ExtendDB from 64.8% to 97.6% DynamoDB parity.
Included branches
fix/conditional-write-racefix/pg-identifier-overflowfix/lsi-always-synchronousfix/condition-missing-attribute-semanticsfix/update-item-list-index-setfix/number-validation-normalizationfix/error-message-fidelityfix/validation-error-codesfix/missing-validationsfix/key-condition-parenthesesfix/transact-write-4mb-limitfix/error-message-formatfix/begins-with-binary-keyfix/projection-list-indexfix/legacy-apifix/missing-validations-batchfix/error-messages-deepfix/error-messages-quickfix/validation-message-formatfix/lsi-query-sort-keyAdditional fixes on combined branch
Remaining (14 tests, 2.4%)
Test plan