Repository navigation
feat(realtime): track presence with a per-socket connections counter - #14606
Conversation
Replace the Redis/cache holder tracking with a `connections` counter on the one-per-user presenceLogs row = how many realtime workers currently hold the user. The row is deleted when it reaches zero. - Each worker keeps its holders in memory (Realtime::$presenceHolders) and touches the DB counter only on its 0<->1 transitions (first connection in, last connection out) -- heartbeats are in-memory, no per-message Redis. - State::recordPresence increments on a worker's first hold and refreshes the value otherwise; State::releaseHolder decrements and deletes + returns the row at zero. Both run in a forUpdate transaction, so the DB row lock serializes a decrement-to-zero delete against another worker's increment -- no Redis, no distributed lock, no reaper. - onClose releases the hold inline (prompt offline event) only when the worker's last connection for the user closes. - Re-auth carries the hold on same-user, releases the previous user on a user change. Schema gains `connections` + a V25 migration for existing rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Heartbeat/value-refresh messages (not a 0->1 transition) now do a plain sparse value update that leaves connections untouched -- no transaction, no forUpdate. Only the worker's first hold for a user takes the transaction. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Guard the usage +1 on firstWorkerHold so a heartbeat on a single-connection user (connections already 1) does not re-count it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Security rulesNo new WARNING or ERROR findings from security rules. 19 existing findings tracked in
|
🟢 Tier S · Ready to mergeThe pull request adds a per-user presence connection counter and tracks each realtime socket’s hold so presence remains until the final hold is released. It updates the presence schema and migration, handles socket close and re-authentication cleanup, and adds coverage for counter transitions and related behavior. Latest changes: The newest commits remove the hostname-based startup orphan sweep, its deletion helper, and the helper’s tests.
📂 Walkthrough · 11
✅ Fixed since the last review · 1
Reviewed the commits since |
- Record a connection's in-memory hold only after recordPresence commits, so a failed write can't leave a phantom hold that later decrements a count it never incremented. recordPresence retries a lost unique-userId create race as an increment instead of erroring. - On a presence delete event (HTTP or worker), clear this worker's stale holds for that user so a later close can't decrement a count the row no longer has. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
✨ Benchmark resultsComparing
Per-scenario breakdown & investigation detailsMetrics below reflect the current branch (after). Δ P95 compares against the base.
Top API waits (after)
|
There was a problem hiding this comment.
🟡 Tier B · 1 blocking finding to address. Summary
- Reserve the worker-holder marker synchronously before the awaiting write, so two concurrent first messages on one worker can't both count as the first hold and double-increment. Roll the reservation back if the write fails. - Tag each hold with its row id and clear only holds on the deleted row, so a delete event for a superseded row can't erase a hold on a recreated one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The findings that requested changes are resolved. See the summary comment for what is still open.
releaseHolder now takes the hold's presence id and no-ops when the current row's id differs (the row was deleted and recreated since this worker's increment), so a stale close can't under-count a row it never counted -- defense in depth with clearPresenceHolders. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Concurrent first messages on one connection share the connectionId marker. Roll back only while it is still the reservation sentinel (true); if a sibling message already committed and wrote the row id, leave it, so a failed request can't erase a live hold and cause a premature release. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- V25: fold the connections attribute-add into the existing presenceLogs case (which also createCollection) instead of a second case that shadowed it and skipped collection creation on installs that never ran V24. - Re-auth: carry the hold only when the same user's connection entry was restored; otherwise release it and drop the holder-index entry, so a hold can't outlive a stash onClose would need to release it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks @tenki-reviewer — addressing the 4 summary findings:
|
|
Review complete. 🟠 2 high · 🟡 2 medium 💬 Inline comments (3)
📍 Findings outside the diff (1) — 🟠 1 high — defects on lines GitHub can't attach comments to🟠 High — Re-subscribe drops the // src/Appwrite/Messaging/Adapter/Realtime.php
202 $entry = [
203 'projectId' => $projectId,
204 'roles' => \array_values(\array_unique(\array_merge($existingRoles, $roles))),
205 'userId' => $userId ?? ($existing['userId'] ?? ''),
206 'channels' => \array_values(\array_unique(\array_merge($existingChannels, $channels))),
207 'presences' => $this->connections[$identifier]['presences'] ?? []
208 ];
🧹 Nitpicks (1) — 🟢 1 low
The change introduces durable presence connection counting: a new Reviewed commit: 211e2d7 |
Add regressions for a legacy project missing the presenceLogs collection (migration must create it with connections) and for an existing presenceLogs without the column (migration must add connections). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Adds persistent presence connection counting to realtime (a connections counter on presenceLogs with hold tracking in the realtime adapter) plus a V25 migration, fixing hold bookkeeping across re-auth, close, and delete paths.
Key findings
- 🟠 Re-subscribe drops the
presencehold key — Realtime.php:202 - 🟠 Close racing a heartbeat permanently leaks the presence hold — Presence.php:125
- 🟡 Double-count
users.presencewhen REST created the row — Presence.php:135 - 🟡 Migration swallows
connectionsfailure, records version anyway — V25.php:425
- If the socket closes during a first presence write, onClose runs before the hold is tagged and can't release it; the handler now detects the gone connection after the write, releases the row it created/incremented, and drops the reservation instead of resurrecting a dead hold. - recordPresence reports whether it actually created the row; usage +1 now fires only on creation, not on incrementing a row an HTTP upsert already counted. - V25: let the presenceLogs createCollection/connections failures propagate (createCollection is idempotent), so the version isn't recorded with a missing attribute. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
If the final holder's socket closes mid-write and the cleanup deletes the row, the handler now emits presences.*.delete (not the online upsert), purges the list cache, and decrements usage when the deleted row was previously counted -- instead of announcing the user online after the row is gone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds StateTest coverage for upsertForUserWithConnections (create = 1, second socket increments, same-socket refresh does not count, a stale/gone held row recounts or recreates) and releaseConnectionForUser (decrement, delete at zero, presenceId-guard no-op). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses three review findings in the per-socket presence counter: - Same-id recreation: an HTTP delete + recreate with the same explicit presence id kept the id, so a stale socket's close could decrement and delete the replacement row another socket still held. The hold is now keyed on the row's internal `$sequence`, which a delete-and-recreate always changes, so a stale hold no longer matches the live row (upsert refresh check + release guard). - Lost create race left `created` true: reset `created`/`counted` at the start of each transaction attempt, so a retry that only increments no longer reports an online transition and over-counts usage. - Offline event vs user lookup: triggerPresenceDeleted now tolerates a transient user lookup failure and emits the delete event with an empty user, as the old close cleanup did, so subscribers are not left with stale online state. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CI failures on the per-socket presence change: - V25 presenceLogs case: now that `connections` is in the collection config, createCollection() creates the row with it, so the following attribute-add raised Duplicate and failed the migration (and every V25Test that runs execute()). Catch the benign Duplicate; other failures still propagate. - PHPStan: drop a redundant is_string() on Document::getId() (always string), and assert the release decrement/delete via DB state instead of the nullable return, which tripped an impossible-type check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…piry A silently dead realtime client (network loss, or a peer that vanished without a FIN) never ran onClose, so its presence lingered until the 30-day expiry (#14343). Configure Swoole's heartbeat so a socket idle past the idle time is closed and runs the normal onClose/release path; clients ping every ~20s, well within the window, so live connections are untouched. Shorten the realtime presence expiry from 30 days to 1 day as the backstop for a hard crash that fires no onClose at all. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A hard crash fires no onClose, so a dead instance's presence rows linger until expiry. On a clean boot every presenceLogs row tagged with this hostname is such an orphan (new connections are only accepted after onStart), so delete them across the console and every project, purging the list cache where rows were removed. CE only -- a Cloud host is shared across tenants on sharded databases, where a per-host sweep does not apply (#14343). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Realtime presence is upserted once per connection and never refreshed (a per-ping refresh is a DB write per connection, which doesn't scale). So a short expiry deletes live long-lived connections, since the row can't tell a held-but-stale presence from an orphan. #14343's real cases are handled without the expiry -- the heartbeat closes silently-dead sockets and boot reconciliation clears same-host crash orphans -- so the expiry stays a long, harmless far backstop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ffline Addresses two review findings on the hostname sweep: - Race: the detached go() runs while the server accepts connections, so a fresh presence upserted on this hostname mid-sweep could be deleted as an orphan. Capture a `$since` at boot and only sweep rows created before it; anything a live connection writes during the sweep is created after `$since` and left alone. - Side effects: the raw bulk delete bypassed the offline path, so usage was not decremented and no delete event reached other workers. Each orphan is now deleted individually and run through triggerPresenceDeleted (usage -1 + the presences.[presenceId].delete event), resiliently per row. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…heartbeat fix(realtime): detect dead presence sockets via heartbeat + boot reconciliation
There was a problem hiding this comment.
🟡 Tier B · 1 blocking finding to address. Summary
The CE orphan sweep runs asynchronously while new connections are already
accepted. A reconnect increments the same per-user row in place, which keeps
its $createdAt, so the row still matched lessThan('$createdAt', $since) and was
deleted despite the fresh live hold.
Select candidates by $updatedAt (the field a reuse actually moves) and delete
through State::deleteOrphan, which re-reads under the same forUpdate row lock
the increment takes and bails when $updatedAt >= $since. Whichever grabs the
lock first wins: a mid-sweep reconnect advances $updatedAt and is skipped; a
sweep delete lets the reconnect create a fresh row. A live hold is never lost.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add StateTest coverage for deleteOrphan: a row not touched since boot is deleted, a row updated after $since (reconnected) is preserved, and a missing row is a no-op. The test caught that the guard compared $updatedAt to $since as raw strings. $updatedAt reads back in ISO form (2026-...T...+00:00) while $since is DB format (space-separated), so 'T' > ' ' made the comparison always true and deleteOrphan never deleted anything. Compare them as \DateTime instants. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@hansi-codes re review |
|
|
@hansi-codes
This is already fixed |

Issue
Fixes #14342 — closing one of a user's realtime connections deleted their presence while another connection was still open.
Solution
Track presence with a
connectionscounter on the one-per-userpresenceLogsrow (uniqueuserId), counting the live sockets holding the user:connections = 1) or increments it; a later message from the same socket just refreshes its status.0the row is deleted and the offline event fires — so closing one connection never removes a presence another still holds (🐛 Bug Report: Closing one realtime connection deletes the user's presence while another connection is still open #14342).presenceId, so a close can't touch a row already deleted and recreated.connections[$id]['presence']and is released on a clean close.State::upsertForUserWithConnectionsandreleaseConnectionForUserown the counter transitions — transactional, serialized against the decrement-to-zero delete — andtriggerPresenceDeletedemits the shared offline side effects.Alternatives Discovered
connectionscounted workers, with a per-worker in-memory holder index bumping the row only on a worker's0↔1transition. Dropped — connections are load-balanced, so sockets-per-worker ≈ 1 and the batching saved little while adding a second in-memory structure that could outlive a connection.