mexc: `watchOrderBook` silently loses updates when the REST snapshot lands inside a buffered push — `getCacheIndex` and `handleDelta` compare `fromVersion` where MEXC specifies `toVersion` · Issue #30637 · ccxt/ccxt · GitHub
Skip to content

mexc: watchOrderBook silently loses updates when the REST snapshot lands inside a buffered push — getCacheIndex and handleDelta compare fromVersion where MEXC specifies toVersion #30637

Open

Description

@VascoSPires

Operating System

Windows 11

Programming Language

Python

CCXT Version

4.5.64

Description

Environment

  • ccxt version: reproduced on 4.5.64; the same logic is present in master (checked
    ts/src/pro/mexc.ts directly, 2026-09-24)
  • Language: Python 3.14.3 (the defect is in the TypeScript source, so all generated
    languages are affected)
  • OS: Windows 11
  • Exchange: mexc, spot, watchOrderBook
  • Channel: spot@public.aggre.depth.v3.api.pb@100ms@<symbol>

Summary

When watchOrderBook seeds a MEXC spot book, it fetches a REST snapshot and splices it
onto the websocket pushes buffered while that request was in flight. A push whose range
straddles the snapshot — fromVersion < lastUpdateId < toVersion — is discarded
whole, and every level change it carried past the snapshot is lost.

The resulting book is well-formed, has a plausible depth, and raises nothing. Deleted
price levels survive as phantoms until something touches that price again.

MEXC's own documented procedure says to keep that push. getCacheIndex and handleDelta
both select on fromVersion where the venue specifies toVersion.

What MEXC documents

From How to Properly Maintain a Local Copy of the Order Book:

  1. From the cached incremental updates, discard all pushes where toVersion <=
    lastUpdateId

  2. Take the first remaining push. If fromVersion > lastUpdateId + 1, the data is
    considered discontinuous; return to Step 3

  3. Set the local order book to the snapshot data, set the local version to
    lastUpdateId, and apply all remaining and subsequent incremental pushes in order.
    After each update, set the local version to that push's toVersion. If any push
    does not satisfy fromVersion = previous toVersion + 1, you must reinitialize.

Steps 5 and 7 both key on toVersion.

What ccxt does

ts/src/pro/mexc.ts:

override getCacheIndex (orderbook: any, cache: any): number {
    const nonce = this.safeInteger (orderbook, 'nonce');
    const firstDelta = this.safeDict (cache, 0);
    const firstDeltaNonce = this.safeIntegerN (firstDelta, [ 'r', 'version', 'fromVersion' ]);
    if ((nonce === undefined) || (firstDeltaNonce === undefined)) {
        return -1;
    }
    if (nonce < firstDeltaNonce - 1) {
        return -1;
    }
    for (let i = 0; i < cache.length; i++) {
        const delta = cache[i];
        const deltaNonce = this.safeIntegerN (delta, [ 'r', 'version', 'fromVersion' ]);
        if (deltaNonce === undefined) {
            continue;
        }
        if (deltaNonce >= nonce) {        // <-- selects on fromVersion
            return i;
        }
    }
    return cache.length;
}

override handleDelta (orderbook: any, delta: any) {
    const existingNonce = this.safeInteger (orderbook, 'nonce');
    const deltaNonce = this.safeIntegerN (delta, [ 'r', 'version', 'fromVersion' ]);
    if ((deltaNonce !== undefined) && (existingNonce !== undefined) && (deltaNonce < existingNonce)) {
        return;                            // <-- skips on fromVersion
    }
    orderbook['nonce'] = deltaNonce;       // <-- stores fromVersion, the head of the batch
    ...
}

getCacheIndex replays from the first push whose fromVersion reaches the snapshot,
so a push that started before it is skipped even when it ended after it. handleDelta
then repeats the same comparison on the live path, and stores the batch's head as the
local version rather than its tail.

Why this loses data

The aggre.depth channel is aggregated server-side over its window, so a push carries the
net state of the levels it touched as of its toVersion. It cannot be split. Applying
one whose range overlaps the snapshot is therefore safe — it assigns those levels their
newer values — while skipping it loses everything that happened between the snapshot and
that push's end.

Worked example. Suppose the venue did this:

version change
1010 price 100.0 -> 5
1011 99.0 -> 8
1012 98.0 -> 3
1013 100.0 deleted
1014 97.0 -> 2
1015 99.0 -> 9
1016 96.0 -> 1
1017 95.0 -> 4

The socket delivers two aggregated pushes, and the REST snapshot comes back at
lastUpdateId = 1012:

push A  [1010..1016]  ->  100.0:"0"  99.0:"9"  98.0:"3"  97.0:"2"  96.0:"1"
push B  [1017..1021]  ->  95.0:"4"
snapshot at 1012      ->  {100.0: 5, 99.0: 8, 98.0: 3}
  • MEXC's rule (toVersion > 1012) keeps A and B. Result:
    {99.0: 9, 98.0: 3, 97.0: 2, 96.0: 1, 95.0: 4} — correct.
  • ccxt's rule (fromVersion >= 1012) keeps only B. Result:
    {100.0: 5, 99.0: 8, 98.0: 3, 95.0: 4} — 100.0 is a phantom bid that was deleted at
    the venue, 99.0 is stale, and 97.0 and 96.0 never arrive.

Evidence from live runs

I instrumented fetchOrderBook and loadOrderBook read-only, captured the snapshot and
the buffer that getCacheIndex was about to read, and rebuilt the book by MEXC's steps 5
and 7 for comparison. 14 spot markets, snapshotDelay = 0.

Run A — current ccxt, 33 seedings audited: 4 books diverged from MEXC's procedure.
One example, verbatim from the run:

AIN/USDT   snapshot=2121839448   straddling push [2121839442..2121839451], 3 ids past it
   bid 0.02299   correct=deleted    ccxt=5166.33     PHANTOM
   ask 0.02417   correct=1623.72    ccxt=292.26      STALE

A phantom bid of 5 166 units at a price that no longer exists at the venue.

Run B — with the fix below, 51 seedings audited, with a within-run control:

books differing from MEXC's procedure 0
seedings where the current rule would have differed 6

The worst of those six, again verbatim:

BNB/USDT   snapshot=8800367700   straddling push carried 23 ids past the snapshot
   787.79   correct=deleted   current_rule=1.576     PHANTOM
   787.71   correct=1.589     current_rule=0.013     STALE
   792.51   correct=4.7133    current_rule=absent    MISSING
   792.50   correct=deleted   current_rule=4.2398    PHANTOM
   792.49   correct=4.242     current_rule=absent    MISSING
   792.48   correct=deleted   current_rule=3.7687    PHANTOM
   ... and 16 more

22 levels wrong, all within a couple of ticks of the mid.

The rate rises with batch size, which rises with activity: across 51 seedings it was 6
(12 %), but on BNB/USDT alone, whose pushes average 19 ids and reach 612, it was 5 in 20
(25 %). fromVersion/toVersion themselves were contiguous throughout — 127 147 frames
across four separate windows, zero gaps — so this is not a lost-message problem. It is the
splice.

Suggested fix

override getCacheIndex (orderbook: any, cache: any): number {
    // MEXC step 5: discard cached pushes whose toVersion <= lastUpdateId; the first
    // that remains is where the replay begins.
    const nonce = this.safeInteger (orderbook, 'nonce');
    const firstDelta = this.safeDict (cache, 0);
    const firstDeltaNonce = this.safeIntegerN (firstDelta, [ 'r', 'version', 'fromVersion' ]);
    if ((nonce === undefined) || (firstDeltaNonce === undefined)) {
        return -1;
    }
    if (nonce < firstDeltaNonce - 1) {
        return -1;
    }
    for (let i = 0; i < cache.length; i++) {
        const delta = cache[i];
        const deltaEnd = this.safeIntegerN (delta, [ 'toVersion', 'version', 'r' ]);
        if (deltaEnd === undefined) {
            continue;
        }
        if (deltaEnd > nonce) {
            return i;
        }
    }
    return cache.length;
}

override handleDelta (orderbook: any, delta: any) {
    // MEXC step 7: the local version is the push's toVersion, and a push is stale only
    // when it ends at or before it.
    const existingNonce = this.safeInteger (orderbook, 'nonce');
    const deltaEnd = this.safeIntegerN (delta, [ 'toVersion', 'version', 'r' ]);
    if ((deltaEnd !== undefined) && (existingNonce !== undefined) && (deltaEnd <= existingNonce)) {
        return;
    }
    orderbook['nonce'] = deltaEnd;
    const asks = this.safeList (delta, 'asks', []);
    const bids = this.safeList (delta, 'bids', []);
    const asksOrderSide = orderbook['asks'];
    const bidsOrderSide = orderbook['bids'];
    this.handleBooksideDelta (asksOrderSide, asks);
    this.handleBooksideDelta (bidsOrderSide, bids);
}

Notes on the change:

  • The key fallback becomes [ 'toVersion', 'version', 'r' ]. On channels that carry a
    single id per message (r), head and tail are the same value, so behaviour there is
    unchanged.
  • handleDelta must change together with getCacheIndex, for two reasons. The same
    straddle recurs on the live path: right after a seeding the local version is the
    snapshot's lastUpdateId, which runs ahead of the socket, and the next push arriving
    can start below it and end above it. And the two must agree on what the stored nonce
    means — mixing a toVersion selection with a fromVersion bookkeeping leaves the
    comparison meaningless.
  • In steady state the two rules behave identically: consecutive pushes satisfy
    fromVersion == previous toVersion + 1, so neither rule ever skips. The difference
    appears only in the window following a seeding.
  • The futures path carries begin/end, which neither the current key list nor this
    one covers; that is pre-existing and unchanged here.

I am not set up to build the transpiled output, so this is a report rather than a PR.
The patch above is yours to take as-is — no attribution needed.

Two smaller, separate observations

Happy to file these separately if you would rather keep this issue to one thing.

1. snapshotDelay is compared with ==, so lowering it at runtime strands a symbol.
In handleOrderBook:

if (cacheLength === snapshotDelay) {
    this.spawn (this.loadOrderBook, ...);
}

If the option is changed while running to a value below a symbol's current cache length,
the condition can never become true again and that symbol never gets a book — silently,
with watchOrderBook simply never resolving. Observed while sweeping the option: a market
sat at 329 frames received and 0 applied for a whole 420 s run. >= with a "already
requested" guard would be robust to it.

2. unWatchOrderBook's UNSUBSCRIPTION never reaches the venue in Python.
ts/src/pro/mexc.ts calls watchSpotPublic without awaiting it. In JavaScript that
sends the request; the transpiled Python leaves an un-awaited coroutine, which is a no-op
and emits RuntimeWarning: coroutine 'mexc.watch_spot_public' was never awaited. The
local cleanup that follows still runs, so the call appears to work, but the venue is never
told to stop sending.

Activity

  1. carlotestor commented on Sep 25, 2026

    @carlotestor
    Collaborator

    Thanks for the detailed write-up. I reproduced the main issue offline against current master (4.5.84, compiled js/). I fed getCacheIndex + handleDeltas a snapshot nonce=1449 book and two cached pushes, [1440..1450] (deletes bid 100) and [1451..1460] (adds bid 98):

    index 1  nonce 1451  bids [[100,1],[99,2],[98,3]]      <- straddling push skipped, bid 100 is a phantom
    live straddle: nonce 1449  bids [[100,1]]              <- handleDelta drops [1440..1450] on the live path too
    

    So both the seeding splice (getCacheIndex) and handleDelta key on fromVersion. MEXC's steps 5 and 7 key on toVersion. The book also records the head of the batch (1451) as its nonce instead of the tail (1460). Your proposed change is correct as I read it:

    • Selecting on toVersion > nonce keeps the straddling push.
    • The nonce < firstDeltaNonce - 1 continuity guard stays on fromVersion, which matches step 6.
    • The r channels carry a single id, so they behave the same as before.

    The two side observations (snapshotDelay compared with ===, and the un-awaited watchSpotPublic in unWatchOrderBook under Python) are separate issues. They are worth their own issues so this one stays about the splice.

    I'll open a minimal PR with the getCacheIndex/handleDelta change in ts/src/pro/mexc.ts.

  2. carlotestor commented on Sep 25, 2026

    @carlotestor
    Collaborator

    Opened #30638 with a minimal fix.

  3. VascoSPires commented on Sep 25, 2026

    @VascoSPires
    Author

    Thank you for your effort and for taking it forward! Happy to collaborate as I benefit a lot from this tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugwsWebSocket (aka ccxt.pro)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions