Hi, I'm spicysharkco.
AI research and infrastructure engineering: reliability and performance.
Go · Rust · Python · C++ · TypeScript
I send fixes upstream for what I run into: Kubernetes, LLM inference, data tooling, robotics and OpenCV. I find a failure, fix it with a test where one fits, and measure what changed. More at Learn, Research and the open-source record.
- Failure:
syncPodFirewallChainscopied and scanned the whole iptables-save buffer once per local pod and IP family. - Fix: remove the check. The rule text it looked for is unique to one pod in one sync, so it could never match.
- Result:
BenchmarkSyncPodFirewallChains(80 pods, one family, 1 MiB of existing rules, median of 5 runs) went from 152,895,797 to 477,097 ns/op, about 320x, and from 90.5 MB to 0.2 MB allocated per op. Validation was unit tests and the benchmark; no Kubernetes integration test. - Why it matters: kube-router is a pod network, firewall and service proxy for Kubernetes, and its network-policy controller runs this sync on every node. On a node with 84 pods the removed copy cost about 175 ms of CPU and 100 MB of allocation per full sync (issue #2191).
- PR #2192, merged 2026-10-12 · post
- Failure: when both directions of a connection failed together, as on a TCP reset, the second goroutine blocked forever on an unbuffered error channel. It held a 32 KiB buffer and both connections. A plain bool flag was written by both goroutines.
- Fix: the channel has room for both errors, and the flag is gone, which removes the data race.
- Result: a new test runs
Proxy10,000 times on connections that always fail. The old code left 1029 goroutines behind and-racereported the race. With the fix,go test -race -count=20passed 20 of 20. The test catches the bug on multi-core runners; withGOMAXPROCS=1the old code can pass. - Why it matters: the leak is in the server's tunnel proxy, which the default egress-selector mode uses. On a server whose tunnel connections end in resets, for example traffic to an aggregated API service, the leaked goroutines only grow until a restart, each holding a 32 KiB buffer and two connections.
- PR #14811, merged 2026-10-15 · post
- Failure: a disparity outside the valid range made the computed index
x2negative or at leastcols, and the code used it to index a buffer unchecked, in both theCV_16SandCV_32Sbranches. - Fix: skip such pixels in both branches with one unsigned bounds check.
- Result: the new test has four cases. AddressSanitizer reports a heap-buffer-overflow in all four without the fix; all four pass with it. The 5.x port is #30344.
- Why it matters:
validateDisparityis the left-right check in stereo depth. Given a disparity map computed with other parameters, it wrote 32 bytes before its buffer. Without a sanitizer that left a bad pixel marked valid: a wrong depth value with no error. - PR #30316, merged 2026-10-16 · post
- Failure: with
autoTuneon, the approximate prefix cache sized each pod's LRU from the engine's block count. Each LRU entry is one router block of at least 64 tokens, and vLLM's default engine block is 16 tokens. So the index could track four times as many tokens as the engine's KV cache held, and up to 64 times with a block size of 1. - Fix: convert the capacity to router blocks,
CacheNumBlocks * CacheBlockSize / GetBlockSize(), rounded down and at least 1. It is unchanged when the two block sizes match or the engine does not report its block size. - Result: a new table test has six cases. On main three of them failed (31250 entries instead of 7812, 2 instead of 1, 1000 instead of 64000); with the fix all six pass under
go test -race. - Why it matters: llm-d-router picks the pod for each inference request, partly by how much of the prompt each pod already has cached. An index larger than the engine's cache keeps prefixes the engine has evicted, so the prefix-cache scorer can overestimate cache hits and send a request to a pod that no longer holds its prefix. The analysis and reproduction in issue #3139 pointed at the cause.
- PR #3476, merged 2026-10-20
- sharkline: a status line for Claude Code, in Rust. A warm render takes about 1.3 ms with no child process and no network; a Node process that does nothing takes about 30 ms on the same machine. Try:
cargo install --locked --git https://github.com/spicysharkco/sharkline. - sharkswap: a Claude Code account switcher in Rust. It swaps between saved accounts, shows each account's 5-hour and weekly usage, and can move the login when one runs out. One binary, no daemon. Try:
cargo install --locked --git https://github.com/spicysharkco/sharkswap, thensharkswap probe. - chess-sim2real-benchmark: a render generator and a 20,000-image benchmark for chess-position recognition (dataset). The README walks through rendering and scoring four frozen views locally.
- Quantization. GGUF files that give each tensor group its own quant type, chosen by measured sensitivity, inside a fixed file size (model card). At the same size as the upstream Q4_K_M (16.81 GB), mean KL divergence from BF16 fell from 0.0348 to 0.0195 on 30,690 held-out tokens. About two thirds of that gain comes from the importance matrix; the rest comes from the per-tensor mix. CPU prompt processing runs at 0.85x the upstream file's speed. Limit: the evaluation is text only, and no task benchmark was run.
- Chess benchmark. Changing which corner is light in the renderer moved per-square accuracy from 54.4% to 85.2%, and only 18 of 192 boards are exact after the fix. Limit: the public reproduction covers a bounded render and scoring path, not all 20,000 cells or the adaptation-training study.
Star counts as of 2026-10-20.
Newest first, one per project.
| Project | Change | Result | Date |
|---|---|---|---|
| llm-d-router | #3476 autoTune per-pod LRU capacity is now sized in router blocks, not engine blocks |
31250 engine blocks of 16 tokens gave a 31250-entry LRU, now 7812 entries of 64 tokens; TestMakeserver_AutotuneCapacityInRouterBlocks failed before |
2026-10-20 |
| AIBrix | #3174 PodAutoscaler scalingStrategy is immutable |
Changing the strategy in place left the old HPA scaling the target next to the new scaler; the change is now rejected | 2026-10-20 |
| Xinference | #5756 a streamed reasoning reply keeps a later </think> in the answer as content |
Chunks a, </think>, b, </think>, c gave content "bc"; now "b</think>c" |
2026-10-19 |
| llm-d-inference-sim | #802 a failed dataset download no longer leaves an empty file at --dataset-path |
After a 503 the next start skipped the download and stayed blocked; now it retries; both new test cases failed before | 2026-10-19 |
The full list of merged pull requests, by area, with a write-up where there is one: spicyshark.co/open-source.