ConnectionRecoveryErrorAsync is never raised when topology recovery fails, contrary to its documentation · Issue #2014 · rabbitmq/rabbitmq-dotnet-client · GitHub
You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue was written by Claude (Anthropic's Claude Code) under the direction of @lukebakken. It was found while writing a regression test for #1995: the test was asserting on this event and passed whether or not the fix was present, which turned out to be because the event never fires for this path. The code paths below were read directly and confirmed with a diagnostic build against a live broker; the suggested directions are AI-drafted starting points, not settled decisions.
IConnection.ConnectionRecoveryErrorAsync documents itself as covering topology recovery:
/// <summary>/// Raised when the connection recovery fails, e.g. because reconnection or topology/// recovery failed./// </summary>
It is raised from exactly one place, and that place only handles reconnection:
Location (on main)
Method
Behaviour on failure
AutorecoveringConnection.Recovery.cs:416
TryRecoverConnectionDelegateAsync (starts at 383)
raises ConnectionRecoveryErrorAsync
AutorecoveringConnection.Recovery.cs:359
TryPerformAutomaticRecoveryAsync (starts at 309)
logs "Exception when recovering connection. Will try again after retry interval.", aborts the inner connection, returns false. No event.
Topology recovery runs inside TryPerformAutomaticRecoveryAsync, after reconnection has already succeeded: exchanges, then queues, then bindings, then channels and consumers. Any exception from those, including one that ShouldRetryRecoveryAfter deliberately classifies as retry-worthy, lands in the catch at 359. So the documented "topology recovery failed" case raises nothing at all.
Why it matters
An application that subscribes to ConnectionRecoveryErrorAsync to detect recovery trouble sees nothing while topology recovery fails and silently retries in a loop. The connection reports IsOpen, RecoverySucceededAsync eventually fires if a later attempt gets through, and the only trace of the failed attempts is an ESLog entry. That is precisely the class of silent failure #1993 was about, one level up.
It is also the reason a regression test for #1995 could not observe a failed attempt. Verified with a diagnostic build: the retry decision fired (shouldRetry=True, so the attempt was failed and retried), while a subscriber to ConnectionRecoveryErrorAsync recorded zero events.
Note that #1995 raises the stakes slightly. Now that the retry classification is applied after a custom TopologyRecoveryExceptionHandler runs, more topology failures fail the attempt rather than being swallowed, and every one of them is invisible to this event.
Possible directions, not settled
Raise ConnectionRecoveryErrorAsync from the catch in TryPerformAutomaticRecoveryAsync as well, making the event match its documentation. Applications that already subscribe would start seeing events they never saw before, which is the point, but it is a behaviour change.
Leave the event alone and correct the documentation to say it covers reconnection only. Cheapest, but leaves topology recovery failures unobservable, and there is then no event that covers them.
Add a separate event for topology recovery failure, leaving the existing one's meaning untouched. Most explicit, but new public API and so subject to the 7.3.0 API gate in Ship API changes #1923.
Option 1 looks closest to the documented intent. Whichever is chosen, TopologyRecoveryException already carries the failing entity in its message, so the event argument would be useful without new types.
Note
This issue was written by Claude (Anthropic's Claude Code) under the direction of @lukebakken. It was found while writing a regression test for #1995: the test was asserting on this event and passed whether or not the fix was present, which turned out to be because the event never fires for this path. The code paths below were read directly and confirmed with a diagnostic build against a live broker; the suggested directions are AI-drafted starting points, not settled decisions.
IConnection.ConnectionRecoveryErrorAsyncdocuments itself as covering topology recovery:https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/main/projects/RabbitMQ.Client/IConnection.cs#L166-L173
It is raised from exactly one place, and that place only handles reconnection:
main)AutorecoveringConnection.Recovery.cs:416TryRecoverConnectionDelegateAsync(starts at 383)ConnectionRecoveryErrorAsyncAutorecoveringConnection.Recovery.cs:359TryPerformAutomaticRecoveryAsync(starts at 309)"Exception when recovering connection. Will try again after retry interval.", aborts the inner connection, returnsfalse. No event.Topology recovery runs inside
TryPerformAutomaticRecoveryAsync, after reconnection has already succeeded: exchanges, then queues, then bindings, then channels and consumers. Any exception from those, including one thatShouldRetryRecoveryAfterdeliberately classifies as retry-worthy, lands in the catch at 359. So the documented "topology recovery failed" case raises nothing at all.Why it matters
An application that subscribes to
ConnectionRecoveryErrorAsyncto detect recovery trouble sees nothing while topology recovery fails and silently retries in a loop. The connection reportsIsOpen,RecoverySucceededAsynceventually fires if a later attempt gets through, and the only trace of the failed attempts is anESLogentry. That is precisely the class of silent failure #1993 was about, one level up.It is also the reason a regression test for #1995 could not observe a failed attempt. Verified with a diagnostic build: the retry decision fired (
shouldRetry=True, so the attempt was failed and retried), while a subscriber toConnectionRecoveryErrorAsyncrecorded zero events.Note that #1995 raises the stakes slightly. Now that the retry classification is applied after a custom
TopologyRecoveryExceptionHandlerruns, more topology failures fail the attempt rather than being swallowed, and every one of them is invisible to this event.Possible directions, not settled
ConnectionRecoveryErrorAsyncfrom the catch inTryPerformAutomaticRecoveryAsyncas well, making the event match its documentation. Applications that already subscribe would start seeing events they never saw before, which is the point, but it is a behaviour change.Option 1 looks closest to the documented intent. Whichever is chosen,
TopologyRecoveryExceptionalready carries the failing entity in its message, so the event argument would be useful without new types.Related
basic.consumeexceedsContinuationTimeout#1993 - the retry classification whose failures land in the un-eventful catch