Fix socket overcrowding state race by songgaoye · Pull Request #3621 · apache/brpc · GitHub
Skip to content

Fix socket overcrowding state race - #3621

Open
songgaoye wants to merge 2 commits into
apache:masterfrom
songgaoye:fix_overcrowd_race
Open

songgaoye wants to merge 2 commits into
apache:masterfrom
songgaoye:fix_overcrowd_race

Conversation

@songgaoye

@songgaoye songgaoye commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: resolve

Problem Summary:

Socket maintains _unwritten_bytes as the source of truth for pending
output, but also caches the overcrowding state in a separate volatile bool
named _overcrowded.

WriteRequest::Setup() and CancelUnwrittenBytes() can update this cached
boolean concurrently with Socket::Write() reading it. volatile does not
provide synchronization, so these accesses form a C++ data race. The cached
state can also become stale when concurrent increment and decrement paths
overwrite each other.

What is changed and the side effects?

Changed:

  • Remove the _overcrowded cached boolean.
  • Derive the overcrowding state from an atomic load of _unwritten_bytes.
  • Use is_overcrowded() in write admission checks and socket debug output.
  • Add unit coverage for the boundary behavior around
    -socket_max_unwritten_bytes.
  • Document that a non-positive unwritten-byte limit disables overcrowding.
    Update the KeepWrite comment and exercise overcrowding transitions through
    the public socket write path.

Side effects:

  • Performance effects:

  • Breaking backward compatibility:


Check List:

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review found no blocking issues in the focused race fix or boundary coverage; tests were not run.

No open issues 🎉

What changed in this PR

Removes the socket’s cached overcrowding flag to eliminate a data race and derive write-admission state directly from the atomic pending-byte count.

Changes:

  • Derive overcrowding from _unwritten_bytes.
  • Update write checks and debug output.
  • Add threshold-boundary coverage.
File Description
test/​brpc_http_rpc_protocol_unittest.cpp Updates the existing assertion and adds boundary tests.
src/​brpc/​socket.h Removes the cached flag and declares the accessor.
src/​brpc/​socket.cpp Implements atomic-derived state and updates callers.

Review effort: Balanced

@chenBright chenBright left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chenBright
chenBright requested a review from wwbmmm October 20, 2026 02:55

@wwbmmm wwbmmm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[replied by brpc-oncall robot] The derivation of the overcrowding state from the atomic _unwritten_bytes is sound and removes the racy volatile bool; only two minor points, no blocker. Also please update the leftover comment in Socket::KeepWrite ("...may turn on _overcrowded to stop pending requests from growing infinitely"), which no longer describes how the state is produced.

Comment thread src/brpc/socket.cpp Outdated
Comment thread test/brpc_http_rpc_protocol_unittest.cpp Outdated
@songgaoye
songgaoye requested a review from wwbmmm October 21, 2026 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants