Repository navigation
Add R2DBC instrumentation via r2dbc-proxy listener SPI - #12441
gh-worker-dd-mergequeue-cf854d[bot] merged 38 commits into
Conversation
…e hand-writing method advice Adds a general heuristic to instrumenter-module.md: before instrumenting a client library via manual method advice, check whether the library (or an official companion, e.g. r2dbc-proxy for R2DBC, ClientInterceptor for gRPC, Producer/ConsumerInterceptor for Kafka) exposes a first-class interception/listener SPI, and prefer it. Rationale, aimed at reactive/async clients: a listener SPI delivers one well-defined start/end/error/cancel callback per operation with metadata already assembled, whereas hand-written advice on a reactive method must re-implement the lifecycle by wrapping the returned Publisher/Mono/Flux — a wrapper that finishes the span only on onComplete/onError leaks every cancelled span (reactive pipelines routinely cancel: take(1), timeouts, DiscardOnCancel), so spans are created but never finished/exported, and it's easy to wrap at the wrong granularity. Notes the no-forced-dependency approach: declare the interception library compileOnly and inject the listener via helperClassNames() (or shade it), so it does not land on the user's runtime classpath. Cross-links the existing R2DBC connection-metadata guidance to this section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Concrete R2DBC/r2dbc-proxy example showing the factory-hook + AssignReturned + wrap-helper + listener shape, so the existing "check for an interception SPI" guidance is operationalizable, not just prose.
Adds tracing for R2DBC (reactive database access) by installing an official r2dbc-proxy ProxyMethodExecutionListener instead of hand-writing advice on Statement.execute()/Batch.execute(). The listener owns the reactive lifecycle (complete/error/cancel) so spans can't leak on cancellation, which a hand-rolled Publisher wrapper is prone to since reactive pipelines cancel constantly (take(1), timeouts, driver-level DiscardOnCancel). - R2dbcInstrumentation hooks ConnectionFactories.find(ConnectionFactoryOptions) via @Advice.AssignReturned-style exit advice and wraps the returned ConnectionFactory with r2dbc-proxy. - TraceProxyExecutionListener implements ProxyMethodExecutionListener, starting/finishing spans in beforeQuery/afterQuery. - R2dbcConnectionCallbackInstrumentation + R2dbcSqlCommentInjector add DBM SQL comment injection, mirroring JDBC's SQLCommenter. - r2dbc-proxy is declared implementation (not compileOnly) and its full class set is listed in helperClassNames() in dependency order: ordinary R2DBC apps don't depend on r2dbc-proxy, so its classes must be injected into the app's classloader (and added to the muzzle validation classpath via extraDependency) or the module is blocked/fails to load at runtime despite passing build-time checks. Verified via a live trace comparison against the OpenTelemetry Java agent on a Spring WebFlux + R2DBC/PostgreSQL sample app: matching INSERT/CREATE TABLE/error-SELECT spans with correct DBM tags (db.instance, db.type, db.user, peer.hostname, _dd.dbm_trace_injected).
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 58cfdba670
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
The new module fails the required integration-list check. It also uses raw SQL resource names and puts the parent span context in full-mode DBM comments.
🤖 Datadog Autotest · Commit 58cfdba · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
The r2dbc module was missing from expected.integrations, so the agent-jar structural check never verified it's bundled in the shaded agent jar.
R2dbcConnectionCallbackInstrumentation listed ConnectionCallbackHandler in its own helperClassNames() — that class is this module's instrumentedType(), so ByteBuddy already defines it on the target classloader. Listing it as a helper injects a second, redundant definition of the class currently being transformed.
r2dbc-proxy is bundled as a real dependency, so its own ConnectionFactoryProvider (DRIVER="proxy") is discoverable by ConnectionFactories.find() alongside real drivers. If an application requests that URL scheme directly, find() already returns an r2dbc-proxy-wrapped factory; wrapping it again would double the query listener callbacks and produce duplicate spans. Skip wrapping in that case — this means apps using DRIVER="proxy" directly get no DD spans, which is preferable to silently duplicating them.
extractDbType returned DRIVER unconditionally. For pooled URLs like
r2dbc:pool:postgresql://..., DRIVER is the wrapper's own name ("pool"),
losing the real database type. Prefer PROTOCOL only when DRIVER is a
known wrapper (currently "pool") — other drivers legitimately use the
driver:protocol URL shape for their own sub-mode designator (e.g.
r2dbc:h2:mem:///..., where DRIVER="h2" is already the real type and
PROTOCOL="mem" must not override it).
…atches Several interlocking fixes to the DBM/resource-name path: - Helper-injection bug: ConnectionCallbackHandler (and BatchCallbackHandler) were listed as helper classes in the other r2dbc modules. Helper-injected classes are loaded untransformed, so the createStatement/add advice that injects the DBM comment never actually ran — the _dd.dbm_trace_injected tag only appeared because it was set unconditionally. Removed both instrumented types from every helper list so their advice is applied. - _dd.dbm_trace_injected is now set only when the executed query actually carries a DD comment (SharedDBCommenter.containsTraceComment), instead of unconditionally on every query. - Statement-creation-time injection no longer embeds a per-execution traceparent (it referenced the ambient span, not the query span, which does not exist yet). It injects only static connection metadata, matching JDBC's prepareStatement path; the tag reflects that. - Resource name goes through DBQueryInfo/SQLNormalizer (literals stripped), with any injected DBM comment stripped first so it doesn't leak into it. - Batch#add(String) is now instrumented (R2dbcBatchCallbackInstrumentation), so batched statements get DBM comments too. - Connection metadata is held in a WeakHashMap keyed by ConnectionInfo, so entries are evicted when the connection is unreachable rather than relying on a close() callback that abrupt disconnects never fire. Tests updated for normalized resource names and comment-aware assertions.
The bundled r2dbc-proxy requires Reactor at runtime (e.g. DelegatingContextView implements reactor.util.context.ContextView), but the R2DBC SPI itself only requires Reactive Streams. On an R2DBC app without reactor-core, injecting those proxy helper classes would fail and leave the factory uninstrumented. Guard all three modules with a classLoaderMatcher on reactor.core.publisher.Flux so instrumentation only applies when Reactor is available.
The repo reserves Java/JUnit for unit tests and requires Groovy/Spock for instrumentation tests (AGENTS.md). Port all three r2dbc instrumentation test classes to Spock specs under src/test/groovy, preserving every test and assertion (span shape, normalized resource names, DBM tag/comment behavior, peer.service, error tags). No production changes.
The listener-SPI guidance told authors to declare the interception library compileOnly and inject its classes via helperClassNames(). That does not work: a compileOnly dependency is absent from the agent jar, so there is nothing to inject and both build-time muzzle and runtime helper definition fail. Correct it to use implementation scope (bundled into the agent), and note that bundling pulls in transitive deps (guard on their presence with classLoaderMatcher) and that the full transitive class set must be listed in topological order.
…Spock" This reverts commit 551ab66. A review comment asked to move the tests to Groovy/Spock, but the repo's "Enforce Groovy migration" CI check rejects any new .groovy file — the project has migrated to JUnit 5 for instrumentation tests (see docs/how_to_test_with_junit.md). Keep the tests as Java/JUnit, which is the current convention. The resource-name normalization and DBM assertion updates remain (they were made before the port).
This comment has been minimized.
This comment has been minimized.
…-spi-instrumentation-20260904
…per execution and suitable for caching
8b4fe75 to
bd3c373
Compare
…urce name if no query string
|
@autotest review |
…irst - Append the DBM comment (instead of prepending) for CALL statements, for PostgreSQL SQL carrying a pg_hint_plan hint (`/*+`), and when dd.dbm.always_append_sql_comment is set, keeping a closing semicolon last. Mirrors JDBC's SQLCommenter. - Check for an existing DD comment before building the new one, in both prepend and append positions. - Rename R2dbcDecorator#extractDbType to getDbType, matching getDbService. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
JDBC's DBMCompatibleConnectionInstrumentation always appends the comment for SQL Server at prepareStatement; createStatement is R2DBC's equivalent. r2dbc-mssql registers the drivers "sqlserver" and "mssql". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Move JDBC's SQLCommenter into agent-bootstrap (next to SharedDBCommenter) and have R2DBC call it instead of keeping its own copy of the placement and duplicate-comment logic. R2DBC now only decides the caller-side reasons to append (dd.dbm.always_append_sql_comment, SQL Server), in one method, like JDBC's call sites. getFirstWord becomes public @VisibleForTesting for the JDBC test and benchmarks. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
The Reactor version guard in R2dbcModule.java (line 28) checks only for Flux, which is present in Reactor 3.3, but the bundled r2dbc-proxy requires reactor.util.context.ContextView (introduced in Reactor 3.4+). On Reactor 3.3, instrumentation is incorrectly enabled and throws NoClassDefFoundError during connection or query subscription instead of leaving the application uninstrumented.
🤖 Bits Code Review · Commit 6e4675c · @DataDog review to ask questions
…cted Drop the redundant classLoaderMatcher() on reactor.core.publisher.Flux: the relocated r2dbc-proxy classes are claimed as helpers, so runtime muzzle already blocks the module when the application lacks a compatible Reactor (no Flux, or no ContextView on Reactor < 3.4). Reword the muzzle comment, and add a muzzle fail directive on Reactor 3.3.22 so CI proves it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vandonr
left a comment
There was a problem hiding this comment.
thanks for addressing comments
amarziali
left a comment
There was a problem hiding this comment.
Looks OK from the LP standpoint. However I would verify few raised points before merging
r2dbc-proxy runs on Reactor, so add the reactive-streams-1.0 and reactor-core-3.1 instrumentations as testRuntimeOnly. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Besides take(1), check spans still finish when a query is cancelled via next(), blockFirst() and Mono.from(...) (the shape behind Spring DatabaseClient's first()). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
/merge |
afff86a
into
master

What
Adds R2DBC tracing (
r2dbc-1.0, integration namer2dbc), on by default.ConnectionFactories.findand wraps the returned factory withr2dbc-proxy. A listener creates one span per query. This is the same hook point and approach as the OpenTelemetry agent.io.r2dbc.spi.Connection#createStatement(String), likeDBMCompatibleConnectionInstrumentationdoes for JDBC. OpenTelemetry has no equivalent. The comment carries static service metadata only, with notraceparent, because the query span doesn't exist yet when the statement is created.DBQueryInfo).r2dbc:pool:factories are left unwrapped, because the pool resolves its delegate through a nested, instrumentedfind. That gives one span per query and keeps theConnectionPoolobject intact.For reviewers
io.r2dbc:r2dbc-proxy:1.1.6.RELEASE`` (Apache-2.0). 77 of its classes are injected as helpers. The module only applies when Reactor 3.4+ is present (enforced by muzzle). Adds ~104 KiBinstanceof VendorConnection, sees the proxy instead. Pooled factories keep their type.Known gaps
ConnectionFactories.find, such as ones built directly or supplied as framework beans.Batch#add.Release notes
R2DBC connections are traced by wrapping them with
r2dbc-proxy. Applications that cast R2DBC connections to driver-specific types, or checkinstanceofon them, should verify behavior with the integration enabled (DD_TRACE_R2DBC_ENABLED=falseturns it off).Testing
take(1),next(),blockFirst()andMono.from(...), and anr2dbc:pool:URL (one span per query; the pool keeps its type and disposes).muzzleandlatestDepTest(Java 11+).DD_DBM_PROPAGATION_MODE=full, the server receives/*ddps=...,dddbs=...,ddh=...,dddb=...*/on every statement, and resource names stay normalized.🤖 Generated with Claude Code