You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
mexc: watchOrderBook silently loses updates when the REST snapshot lands inside a buffered push — getCacheIndex and handleDelta compare fromVersion where MEXC specifies toVersion #30637
When watchOrderBook seeds a MEXC spot book, it fetches a REST snapshot and splices it
onto the websocket pushes buffered while that request was in flight. A push whose range straddles the snapshot — fromVersion < lastUpdateId < toVersion — is discarded
whole, and every level change it carried past the snapshot is lost.
The resulting book is well-formed, has a plausible depth, and raises nothing. Deleted
price levels survive as phantoms until something touches that price again.
MEXC's own documented procedure says to keep that push. getCacheIndex and handleDelta
both select on fromVersion where the venue specifies toVersion.
From the cached incremental updates, discard all pushes where toVersion <= lastUpdateId
Take the first remaining push. If fromVersion > lastUpdateId + 1, the data is
considered discontinuous; return to Step 3
Set the local order book to the snapshot data, set the local version to lastUpdateId, and apply all remaining and subsequent incremental pushes in order.
After each update, set the local version to that push's toVersion. If any push
does not satisfy fromVersion = previous toVersion + 1, you must reinitialize.
Steps 5 and 7 both key on toVersion.
What ccxt does
ts/src/pro/mexc.ts:
overridegetCacheIndex(orderbook: any,cache: any): number {constnonce=this.safeInteger(orderbook,'nonce');constfirstDelta=this.safeDict(cache,0);constfirstDeltaNonce=this.safeIntegerN(firstDelta,['r','version','fromVersion']);if((nonce===undefined)||(firstDeltaNonce===undefined)){return-1;}if(nonce<firstDeltaNonce-1){return-1;}for(leti=0;i<cache.length;i++){constdelta=cache[i];constdeltaNonce=this.safeIntegerN(delta,['r','version','fromVersion']);if(deltaNonce===undefined){continue;}if(deltaNonce>=nonce){// <-- selects on fromVersionreturni;}}returncache.length;}overridehandleDelta(orderbook: any,delta: any){constexistingNonce=this.safeInteger(orderbook,'nonce');constdeltaNonce=this.safeIntegerN(delta,['r','version','fromVersion']);if((deltaNonce!==undefined)&&(existingNonce!==undefined)&&(deltaNonce<existingNonce)){return;// <-- skips on fromVersion}orderbook['nonce']=deltaNonce;// <-- stores fromVersion, the head of the batch
...
}
getCacheIndex replays from the first push whose fromVersion reaches the snapshot,
so a push that started before it is skipped even when it ended after it. handleDelta
then repeats the same comparison on the live path, and stores the batch's head as the
local version rather than its tail.
Why this loses data
The aggre.depth channel is aggregated server-side over its window, so a push carries the net state of the levels it touched as of its toVersion. It cannot be split. Applying
one whose range overlaps the snapshot is therefore safe — it assigns those levels their
newer values — while skipping it loses everything that happened between the snapshot and
that push's end.
Worked example. Suppose the venue did this:
version
change
1010
price 100.0 -> 5
1011
99.0 -> 8
1012
98.0 -> 3
1013
100.0 deleted
1014
97.0 -> 2
1015
99.0 -> 9
1016
96.0 -> 1
1017
95.0 -> 4
The socket delivers two aggregated pushes, and the REST snapshot comes back at lastUpdateId = 1012:
push A [1010..1016] -> 100.0:"0" 99.0:"9" 98.0:"3" 97.0:"2" 96.0:"1"
push B [1017..1021] -> 95.0:"4"
snapshot at 1012 -> {100.0: 5, 99.0: 8, 98.0: 3}
MEXC's rule (toVersion > 1012) keeps A and B. Result: {99.0: 9, 98.0: 3, 97.0: 2, 96.0: 1, 95.0: 4} — correct.
ccxt's rule (fromVersion >= 1012) keeps only B. Result: {100.0: 5, 99.0: 8, 98.0: 3, 95.0: 4} — 100.0 is a phantom bid that was deleted at
the venue, 99.0 is stale, and 97.0 and 96.0 never arrive.
Evidence from live runs
I instrumented fetchOrderBook and loadOrderBook read-only, captured the snapshot and
the buffer that getCacheIndex was about to read, and rebuilt the book by MEXC's steps 5
and 7 for comparison. 14 spot markets, snapshotDelay = 0.
Run A — current ccxt, 33 seedings audited: 4 books diverged from MEXC's procedure.
One example, verbatim from the run:
AIN/USDT snapshot=2121839448 straddling push [2121839442..2121839451], 3 ids past it
bid 0.02299 correct=deleted ccxt=5166.33 PHANTOM
ask 0.02417 correct=1623.72 ccxt=292.26 STALE
A phantom bid of 5 166 units at a price that no longer exists at the venue.
Run B — with the fix below, 51 seedings audited, with a within-run control:
books differing from MEXC's procedure
0
seedings where the current rule would have differed
6
The worst of those six, again verbatim:
BNB/USDT snapshot=8800367700 straddling push carried 23 ids past the snapshot
787.79 correct=deleted current_rule=1.576 PHANTOM
787.71 correct=1.589 current_rule=0.013 STALE
792.51 correct=4.7133 current_rule=absent MISSING
792.50 correct=deleted current_rule=4.2398 PHANTOM
792.49 correct=4.242 current_rule=absent MISSING
792.48 correct=deleted current_rule=3.7687 PHANTOM
... and 16 more
22 levels wrong, all within a couple of ticks of the mid.
The rate rises with batch size, which rises with activity: across 51 seedings it was 6
(12 %), but on BNB/USDT alone, whose pushes average 19 ids and reach 612, it was 5 in 20
(25 %). fromVersion/toVersion themselves were contiguous throughout — 127 147 frames
across four separate windows, zero gaps — so this is not a lost-message problem. It is the
splice.
Suggested fix
overridegetCacheIndex(orderbook: any,cache: any): number {// MEXC step 5: discard cached pushes whose toVersion <= lastUpdateId; the first// that remains is where the replay begins.constnonce=this.safeInteger(orderbook,'nonce');constfirstDelta=this.safeDict(cache,0);constfirstDeltaNonce=this.safeIntegerN(firstDelta,['r','version','fromVersion']);if((nonce===undefined)||(firstDeltaNonce===undefined)){return-1;}if(nonce<firstDeltaNonce-1){return-1;}for(leti=0;i<cache.length;i++){constdelta=cache[i];constdeltaEnd=this.safeIntegerN(delta,['toVersion','version','r']);if(deltaEnd===undefined){continue;}if(deltaEnd>nonce){returni;}}returncache.length;}overridehandleDelta(orderbook: any,delta: any){// MEXC step 7: the local version is the push's toVersion, and a push is stale only// when it ends at or before it.constexistingNonce=this.safeInteger(orderbook,'nonce');constdeltaEnd=this.safeIntegerN(delta,['toVersion','version','r']);if((deltaEnd!==undefined)&&(existingNonce!==undefined)&&(deltaEnd<=existingNonce)){return;}orderbook['nonce']=deltaEnd;constasks=this.safeList(delta,'asks',[]);constbids=this.safeList(delta,'bids',[]);constasksOrderSide=orderbook['asks'];constbidsOrderSide=orderbook['bids'];this.handleBooksideDelta(asksOrderSide,asks);this.handleBooksideDelta(bidsOrderSide,bids);}
Notes on the change:
The key fallback becomes [ 'toVersion', 'version', 'r' ]. On channels that carry a
single id per message (r), head and tail are the same value, so behaviour there is
unchanged.
handleDelta must change together with getCacheIndex, for two reasons. The same
straddle recurs on the live path: right after a seeding the local version is the
snapshot's lastUpdateId, which runs ahead of the socket, and the next push arriving
can start below it and end above it. And the two must agree on what the stored nonce
means — mixing a toVersion selection with a fromVersion bookkeeping leaves the
comparison meaningless.
In steady state the two rules behave identically: consecutive pushes satisfy fromVersion == previous toVersion + 1, so neither rule ever skips. The difference
appears only in the window following a seeding.
The futures path carries begin/end, which neither the current key list nor this
one covers; that is pre-existing and unchanged here.
I am not set up to build the transpiled output, so this is a report rather than a PR. The patch above is yours to take as-is — no attribution needed.
Two smaller, separate observations
Happy to file these separately if you would rather keep this issue to one thing.
1. snapshotDelay is compared with ==, so lowering it at runtime strands a symbol.
In handleOrderBook:
If the option is changed while running to a value below a symbol's current cache length,
the condition can never become true again and that symbol never gets a book — silently,
with watchOrderBook simply never resolving. Observed while sweeping the option: a market
sat at 329 frames received and 0 applied for a whole 420 s run. >= with a "already
requested" guard would be robust to it.
2. unWatchOrderBook's UNSUBSCRIPTION never reaches the venue in Python. ts/src/pro/mexc.ts calls watchSpotPublic without awaiting it. In JavaScript that
sends the request; the transpiled Python leaves an un-awaited coroutine, which is a no-op
and emits RuntimeWarning: coroutine 'mexc.watch_spot_public' was never awaited. The
local cleanup that follows still runs, so the call appears to work, but the venue is never
told to stop sending.
Thanks for the detailed write-up. I reproduced the main issue offline against current master (4.5.84, compiled js/). I fed getCacheIndex + handleDeltas a snapshot nonce=1449 book and two cached pushes, [1440..1450] (deletes bid 100) and [1451..1460] (adds bid 98):
index 1 nonce 1451 bids [[100,1],[99,2],[98,3]] <- straddling push skipped, bid 100 is a phantom
live straddle: nonce 1449 bids [[100,1]] <- handleDelta drops [1440..1450] on the live path too
So both the seeding splice (getCacheIndex) and handleDelta key on fromVersion. MEXC's steps 5 and 7 key on toVersion. The book also records the head of the batch (1451) as its nonce instead of the tail (1460). Your proposed change is correct as I read it:
Selecting on toVersion > nonce keeps the straddling push.
The nonce < firstDeltaNonce - 1 continuity guard stays on fromVersion, which matches step 6.
The r channels carry a single id, so they behave the same as before.
The two side observations (snapshotDelay compared with ===, and the un-awaited watchSpotPublic in unWatchOrderBook under Python) are separate issues. They are worth their own issues so this one stays about the splice.
I'll open a minimal PR with the getCacheIndex/handleDelta change in ts/src/pro/mexc.ts.
Activity
carlotestor commentedon Sep 25, 2026
Thanks for the detailed write-up. I reproduced the main issue offline against current
master(4.5.84, compiledjs/). I fedgetCacheIndex+handleDeltasa snapshotnonce=1449book and two cached pushes,[1440..1450](deletes bid 100) and[1451..1460](adds bid 98):So both the seeding splice (
getCacheIndex) andhandleDeltakey onfromVersion. MEXC's steps 5 and 7 key ontoVersion. The book also records the head of the batch (1451) as its nonce instead of the tail (1460). Your proposed change is correct as I read it:toVersion > noncekeeps the straddling push.nonce < firstDeltaNonce - 1continuity guard stays onfromVersion, which matches step 6.rchannels carry a single id, so they behave the same as before.The two side observations (
snapshotDelaycompared with===, and the un-awaitedwatchSpotPublicinunWatchOrderBookunder Python) are separate issues. They are worth their own issues so this one stays about the splice.I'll open a minimal PR with the
getCacheIndex/handleDeltachange ints/src/pro/mexc.ts.carlotestor commentedon Sep 25, 2026
Opened #30638 with a minimal fix.
VascoSPires commentedon Sep 25, 2026
Thank you for your effort and for taking it forward! Happy to collaborate as I benefit a lot from this tool.