{{ message }}
Repository navigation
Tags: documentdb/documentdb
Tags
DocumentDB v0.116-0 Upstream content only: the documentdb-local, packaging and gateway cherry-picks already synced from the internal repo, plus a single commit carrying the OSS-only fixes prepared for this release: fix(documentdb-local): interlock the data directory across containers (issues #43 and #62) fix(emulator): exit on a bare positional instead of spinning at 100% CPU (issue #61) fix(ci): publish container signatures cosign v2 clients can verify fix(packaging): forward SOURCE_DATE_EPOCH into the build containers so the shipped changelog.gz does not carry each matrix cell's wall clock docs(readme): persist data with a volume and document readiness in the quickstart (issue #39) fix(packaging): create documentdb_extended_rum on the packaged install path, so a tuned cluster can build indexes (issue #74) test(packaging): document and enforce the CodeReady Builder prerequisite for RHEL-family installs (issue #75) Set the v0.116-0 release date and mark v0.115-0 as skipped The container data-directory interlock supersedes the PID-based staleness heuristic: ownership is settled by an flock(2) claim on the data directory, so a leftover postmaster.pid is no longer judged from a PID. The packaging fix makes the release bundle's cross-cell identity check deterministic: the PG-agnostic gateway .deb is emitted by both the pg17 and pg18 cells under one filename, and an unpinned changelog timestamp changed its gzip size often enough to fail the aggregate job about half the time. documentdb-tune pins documentdb.alternate_index_handler_name='extended_rum' whenever the extended-RUM control file is present, but the access method that GUC names exists only where documentdb_extended_rum has been created, and CREATE EXTENSION documentdb CASCADE does not pull it in. The packaged tools, the gateway package banners and the documented workflows now create both extensions against the cluster they are addressing. postgis36_N -> gdal*-libs -> libqhull_r.so.7 is satisfiable only from CRB (powertools on EL8), which is disabled by default, so the previously documented PGDG + EPEL set could not install on a stock Rocky/RHEL 9 host. The prerequisite block is now single-sourced from the README and verified against stock EL hosts in CI. The v0.115-0 release was skipped; its changes are included in v0.116-0.
Merged PR 2194508: Fix bson_aggregation_pipeline_tests vector test br… …oken by pgvector 0.8.5 ## Why The `numLists > data size` IVFFLAT block in `bson_aggregation_pipeline_tests` hardcoded a **pgvector build-memory pre-flight estimate** (`ERROR: memory required is 394 MB, maintenance_work_mem is 64 MB`) in its expected output. That number is a pgvector implementation detail, not documentdb behavior. PGDG shipped **pgvector 0.8.5** (commit `b383e4d1`), which changed IVFFLAT sampling to reduce build memory for small tables. The estimate dropped **394 MB -> 383 MB**, and the test started failing at `make check` in the Debian packaging path: ``` expected: memory required is 394 MB actual: memory required is 383 MB ``` Crucially, the two CI paths pin **different** pgvector versions: - regress source build -> pgvector **0.8.1** (394 MB) - deb / PGDG -> pgvector **0.8.5** (383 MB) So hardcoding any single MB number cannot satisfy both paths, and pinning a pgvector version is brittle. The fix has to be **version independent**. The block also used `numLists: 10000` against only **two** vector rows. That always exceeded `maintenance_work_mem` and asserted the memory-rejection *failure* path, so the test never actually exercised the "successful randomized-centroid search" path its own comment describes. ## How Change `numLists` **10000 -> 3** on one SQL line. `3` is the smallest value still greater than the two-row data size, so it preserves the block's stated intent (`numLists > data size`) while letting the index build cheaply. With the index building, the volatile `memory required is N MB` assertion disappears entirely and both searches return real, deterministic results. - **Only one SQL line changes.** The expected `.out` is regenerated because the now-building index turns the former error lines into real result rows and EXPLAIN plans. - `nProbes` values are **intentionally unchanged**: query1 `nProbes:10000` auto-clamps to `numLists`, and query2 `nProbes:1` is deterministic because its query vector exactly equals a stored vector (score `1.0`). Both were proven run-to-run stable and byte-identical across pgvector versions. ## Validation - Full `make check` passes on pgvector **0.8.1** and **0.8.5** (`# All 100 tests passed.`). - Cross-version expected output is **byte-identical** (0 diff), verified with 10x determinism loops per version. - documentdb OSS fork CI green across **PG15-18** on **arm64 and amd64** (sql regress + functional deb `make check`). ## Scope / sync note This edits the OSS test under `oss/pg_documentdb/...`, which syncs to the public `documentdb/documentdb` repo. No separate public PR is needed. The distributed variant (`oss/internal/pg_documentdb_distributed/...`) has no IVFFLAT `numLists` block and is unaffected. Signed-off-by: Guanzhou Song <guanzhousong@microsoft.com> (cherry picked from commit f97c83b)
Merged PR 2081022: NameSpace Validation for config db ### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description NameSpace Validation for config and admin db ---- #### AI description (iteration 1) #### PR Classification New feature: Adds namespace validation to restrict operations on reserved system databases (`config` and `admin`), with a feature flag to control index creation permissions. #### PR Summary This PR introduces namespace validation to prevent users from creating, dropping, or modifying reserved collections in system databases (`config` and `admin`), with configurable controls for index creation. The implementation includes comprehensive test coverage and is controlled by feature flags for gradual rollout. - **`/oss/pg_documentdb/src/metadata/collection.c`**: Added `ValidateSystemNamespace()` function to restrict operations on reserved collection names (e.g., `chunks`, `shards`, `version`) in the `config` database - **`/oss/pg_documentdb/src/commands/`**: Integrated validation checks in `create_collection_core.c`, `drop_collection.c`, and `create_collection_view.c` to enforce namespace restrictions - **`/gateway/managed/Mongo.Service.Postgres/`**: Added `EnableIndexCreationConfigDB` configuration flag in `PostgresTenantConfiguration.cs` and updated `PostgresMongoClient.cs` to conditionally allow index creation on system databases - **`/oss/pg_documentdb/src/configs/feature_flag_configs.c`**: Introduced `enableNewNamespaceValidation` feature flag with default value `false` for controlled rollout - **Test files**: Added comprehensive integration tests in `CreateIndexSystemDbTests.cs` and regression tests in `commands_admin_config_db_validation.sql` to validate both restricted and allowed operations <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Parag Jain <paragjain@microsoft.com>
Merged PR 2028494: [Deadlocks] Fixing deadlock issue in bulk writes ### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description [ Deadlocks] Fixing deadlock issue in bulk writes Signed-off-by: Guanzhou Song <guanzhousong@microsoft.com> Signed-off-by: Parag Jain <paragjain@microsoft.com>
Merged PR 2025281: [OSS][GW] Uniform Resource Identifiers for Session…
…, Transaction and Cursors
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
Replace raw primitive types (`Vec<u8>`, `i64`, `(i64, String)`) with dedicated newtype wrappers - `SessionId`, `TransactionNumber`, `CursorId`, and `CursorKey` - across the gateway core. This improves type safety, prevents accidental misuse of identifiers (e.g., passing a cursor ID where a transaction number is expected), and makes the codebase more self-documenting.
#### Changes
- **`SessionId(Vec<u8>)`** - newtype with `Clone`, `Eq`, `Hash`, `Display`, `Debug`, and `From` impls
- **`TransactionNumber(i64)`** - newtype with `Copy` semantics
- **`CursorId(i64)`** - newtype with `Copy` semantics
- **`CursorKey { cursor_id, username }`** - composite key replacing `(i64, String)` tuples in `CursorStore`
- Refactored `CursorStore`, `TransactionStore`, `ConnectionContext`, request parsing, and all processor/response call sites to use the new types
#### Tests
- **`SessionId`** - 10 unit tests covering construction, `From` conversions, equality, hashing, debug formatting, and `HashMap` usage
- **`CursorId` / `CursorKey` / `CursorStore`** - 17 unit tests covering equality, hashing, copy semantics, user isolation (different users cannot access each other's cursors), store add/get/remove lifecycle, invalidation by collection/database/session, and `kill_cursors` authorization
Signed-off-by: Guanzhou Song <guanzhousong@microsoft.com>
Signed-off-by: Travis Sharp <travis.sharp@microsoft.com>
Merged PR 1974367: [Ordered Scans] Fix skip scan when there is undefi…
…ned paths in between
### Does this PR have any customer impact?
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
### Are you introducing any new config? If yes, do you have tests with and without them being set?
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
[Ordered Scans] Fix skip scan when there is undefined paths in between. We were using the compare term that we built from the original term but that one is only the bson value so if the path doesn't exist we built a
{"undefined": true} bson value which would cause the skip scan to compare diferently and sometimes think it should stop scanning and not return all the values. In cases where the path doesn't exist the value is undefined but we need to construct the term from the original index term containing the term metadata to indicate eventhough the bson value is undefined, the term points not to the literal undefined but to a non existent path which is very different when comparing index terms.
Also, make sure the support function is set for the range operator.
----
#### AI description (iteration 1)
#### PR Classification
This pull request fixes a bug in ordered scan skip logic by addressing issues with handling undefined intermediate index paths.
#### PR Summary
The changes update composite index term serialization to retain proper metadata and add comprehensive tests to validate correct skip scan behavior when encountering undefined paths.
- `/oss/pg_documentdb_distributed/src/test/regress/expected/bson_composite_prefer_ordered_tests.out` and `/pgmongo/src/test/docdb_compat/expected/bson_composite_prefer_ordered_tests.out`: Added extensive test queries to check skip scan behavior with both defined and undefined paths.
- `/oss/internal/pg_documentdb_distributed/src/test/regress/sql/bson_composite_prefer_ordered_tests.sql`: Introduced new SQL test cases to cover various filter and range scenarios.
- `/oss/pg_documentdb/src/opclass/bson_gin_composite_entrypoint.c`: Modified the index term transformation logic to use the original serialized term with metadata for accurate comparisons.
- `/pgmongo/sql/pgmongo--1.109-0--1.109-1.sql` (and its SHA1 update): Added an operational SQL file to adjust the range operator selectivity for improved full scan behavior.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Signed-off-by: Guanzhou Song <guanzhousong@microsoft.com>
PreviousNext