fix: size() uses UTF-16 code units for strings by LeeroyHannigan · Pull Request #67 · ExtendDB/extenddb · GitHub
Skip to content

fix: size() uses UTF-16 code units for strings#67

Merged
pdf-amzn merged 1 commit into
mainfrom
fix/size-function-utf16
May 21, 2026
Merged

fix: size() uses UTF-16 code units for strings#67
pdf-amzn merged 1 commit into
mainfrom
fix/size-function-utf16

Conversation

@LeeroyHannigan

Copy link
Copy Markdown
Collaborator

What

Changes size() condition expression function to use UTF-16 code unit count for strings instead of UTF-8 byte length.

Why

Closes #60

DynamoDB uses UTF-16 code units for size() on strings. ExtendDB was using Rust's str::len() (UTF-8 bytes), causing condition expressions to evaluate incorrectly for non-ASCII strings. For example, "é𝄞" should be size 3 (1 + 2 surrogate pair) but was returning 6.

Testing done

  • Verified against real DynamoDB: size("é𝄞") = 3 succeeds, size("é𝄞") = 6 fails
  • Verified same behavior on ExtendDB after fix
  • cargo test --workspace passes
  • cargo clippy --all-targets -- -D warnings clean

Checklist

  • I have read CONTRIBUTING.md
  • All tests pass (cargo test --workspace)
  • Code is formatted (cargo fmt --check)
  • Clippy is clean (cargo clippy -- -W clippy::pedantic)
  • I have added or updated tests for new functionality
  • I have updated documentation if behavior changed
  • Breaking changes are noted below (if any)

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

@pdf-amzn pdf-amzn self-assigned this May 21, 2026
@pdf-amzn pdf-amzn merged commit b3c7b2a into main May 21, 2026
3 checks passed
@pdf-amzn pdf-amzn deleted the fix/size-function-utf16 branch May 21, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] size condition expression function calculation is wrong

3 participants