fix: support URI query params and stop target connection churn/leaks by aitz-art · Pull Request #1013 · alibaba/MongoShake · GitHub
Skip to content

fix: support URI query params and stop target connection churn/leaks - #1013

Merged
zhongli-james merged 1 commit into
alibaba:developfrom
aitz-art:aitz-art/upgrade-support-url-param-v2
Oct 26, 2026
Merged

zhongli-james merged 1 commit into
alibaba:developfrom
aitz-art:aitz-art/upgrade-support-url-param-v2

Conversation

@aitz-art

Copy link
Copy Markdown
Contributor

EncodeMongoURI now preserves the URI query string so options such as directConnection, replicaSet, maxPoolSize and authSource reach the driver.

Connection lifecycle fixes:

  • executor: only drop the client on network/timeout errors; close the old client before reconnecting
  • NewMongoCommunityConn: disconnect on Connect/Ping failure; bound Disconnect and the IsGood ping with a 10s timeout
  • close one-off check clients (direct_writer Prepare, sanitize checks, replication sanitize, change stream serverless Watch error)
  • wrap delete+insert fallback errors with %w so network errors classify
  • sanitize: return error on unreachable source instead of nil deref
  • doc_syncer: register wg.Done first to avoid deadlock on connect failure

EncodeMongoURI now preserves the URI query string so options such as
directConnection, replicaSet, maxPoolSize and authSource reach the driver.

Connection lifecycle fixes:
- executor: only drop the client on network/timeout errors; close the old
  client before reconnecting
- NewMongoCommunityConn: disconnect on Connect/Ping failure; bound
  Disconnect and the IsGood ping with a 10s timeout
- close one-off check clients (direct_writer Prepare, sanitize checks,
  replication sanitize, change stream serverless Watch error)
- wrap delete+insert fallback errors with %w so network errors classify
- sanitize: return error on unreachable source instead of nil deref
- doc_syncer: register wg.Done first to avoid deadlock on connect failure

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aitz-art
aitz-art force-pushed the aitz-art/upgrade-support-url-param-v2 branch 2 times, most recently from 5034324 to 40ba299 Compare October 12, 2026 11:12
@zhongli-james zhongli-james self-assigned this Oct 12, 2026
@zhongli-james

Copy link
Copy Markdown
Collaborator

@zhongli-james

Copy link
Copy Markdown
Collaborator

Reviewed locally: go build ./... clean, go vet shows only pre-existing noise, merges cleanly onto current develop (b10c9aa), and go test ./common/ (including the three new EncodeMongoURI cases), ./tunnel/..., ./collector/configure/ all pass. The executor/docsyncer test failures are environmental (no live mongod on localhost:27017), unrelated to this change.

LGTM — no blocking issues. Each change fixes a real defect and the logic is sound:

  • sanitize.go: the old sourceConn, _ := ignored the error and then dereferenced sourceConn → nil panic on connect failure; now handled. Good catch.
  • doc_syncer.go: registering defer wg.Done() before the connect avoids a genuine wg.Wait() deadlock on the early-return path.
  • community_client.go: disconnecting on Connect/Ping failure plugs a client leak, and bounding IsGood()/Disconnect with a 10s timeout matters because IsGood() runs on every batch in ensureConnection.
  • operation.go: dropping the client only on network/timeout errors is the right churn fix, and the %v→%w changes in the delete+insert fallbacks are correctly coupled so mongo.IsNetworkError can see through the wrapped error. Misclassified dead connections are still caught by the next IsGood() ping.
  • Verified the chain end-to-end: EncodeMongoURI output feeds options.Client().ApplyURI(), so directConnection/replicaSet/maxPoolSize/authSource now reach the driver.

Compatibility with the existing special-char password handling — verified, fully compatible.
The conf template documents that passwords containing @ etc. are URL-encoded (per the Aliyun connection docs). That encoding path (url.UserPassword) is untouched here. Parsing order is what keeps it safe: the userinfo is split first via LastIndex("@"), and only then is the query cut from afterUserInfo — so special chars in the password (which live before the last @) are never affected by the new ? split. I ran the existing special-char cases plus combined ones against both this branch and develop:

input output (this PR)
root:1234@abcd@…/admin?replicaSet=rs0 root:1234%40abcd@…/admin?replicaSet=rs0
user:p?ss@…/admin?authSource=admin user:p%3Fss@…/admin?authSource=admin
~!@#$^&*()_-=@…?replicaSet=rs0 identical to the existing expected encoding, query preserved
user:p@ss?@…?directConnection=true user:p%40ss%3F@…?directConnection=true
no-auth localhost:27017/?directConnection=true returned verbatim

All existing TestEncodeMongoURI cases still pass unchanged.

One edge worth noting (pre-existing, not introduced by this PR, out of scope): an unencoded @ inside a query value (e.g. …/db?authSource=ad@min) still mis-splits at LastIndex("@"). I confirmed develop produces the exact same broken output, since the mis-split happens before the new ? cut. Per the URI spec such values must be percent-encoded (authSource=ad%40min), which parses correctly. No action needed here.

A few other minor, non-blocking notes:

  1. The commit/PR doesn't reference an issue — recent convention here is fix: ... (#NNNN). Worth linking one.
  2. This is a behavior change: query params that were previously dropped silently are now honored. A stray replicaSet, or directConnection=true with multiple hosts, will now surface as a driver error instead of being ignored. Suggest a one-line release note.
  3. Minor scope observation: this bundles the query-param feature with ~6 connection-leak/churn fixes. Thematically cohesive (connection lifecycle), so fine to keep, but the feature could arguably stand alone.

@zhongli-james
zhongli-james merged commit feca5a6 into alibaba:develop Oct 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants