feat(diskann): Vamana ANN + PQ + NAPI bindings — 14 tests, 1.0 recall… · codingwatching/ruvector@0247c1f · GitHub
Skip to content

Commit 0247c1f

Browse files
ruvnetReuven
andauthored
feat(diskann): Vamana ANN + PQ + NAPI bindings — 14 tests, 1.0 recall, 90µs search (ruvnet#334)
* feat(ruvector): implement missing capabilities (ADR-143) - speculativeEmbed: real FNV-1a hash embedding (128-dim) from file content - ragRetrieve: cosine similarity on embeddings + TF-IDF keyword fallback - contextRank: TF-IDF weighted scoring instead of raw keyword matching - Remove false DiskANN claim (will implement as Rust crate next) Co-Authored-By: claude-flow <ruv@ruv.net> * feat(diskann): Vamana graph + PQ — SSD-friendly billion-scale ANN (ADR-143) New Rust crate: ruvector-diskann Core algorithm (NeurIPS 2019 DiskANN paper): - Vamana graph with α-robust pruning (bounded out-degree R) - k-means++ seeded Product Quantization (M subspaces, 256 centroids) - Asymmetric PQ distance tables for fast candidate filtering - Two-phase search: PQ-filtered beam search → exact re-ranking - Memory-mapped persistence (mmap vectors + binary graph) Performance characteristics: - L2-squared distance with 8-wide loop unrolling (auto-vectorized) - Greedy beam search with bounded visited set - Save/load with flat binary format (mmap-friendly) 9 tests passing: distance, PQ train/encode, Vamana build/search, bounded degree, full index CRUD, PQ-accelerated search, save/load. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(diskann): NAPI-RS bindings + npm package + 14 tests passing Rust core (ruvector-diskann): - 4-accumulator L2 distance for ILP optimization - Recall@10 = 1.000 on 2K vectors - Search latency: 90µs (5K vectors, 128d, k=10) - 14 tests: distance, PQ, Vamana, recall, scale, edge cases NAPI-RS bindings (ruvector-diskann-node): - Sync + async build/search - Batch insert (flat Float32Array) - Save/load, delete, count - Thread-safe via parking_lot::RwLock npm package (@ruvector/diskann): - Platform-specific loader (linux/darwin/win) - TypeScript declarations - Node.js test passing Co-Authored-By: claude-flow <ruv@ruv.net> * ci(diskann): add cross-platform build + publish workflow 5 targets: linux-x64, linux-arm64, darwin-x64, darwin-arm64, win32-x64 Co-Authored-By: claude-flow <ruv@ruv.net> * perf(diskann): FlatVectors + VisitedSet + ILP + optional SIMD/GPU Optimizations applied: - FlatVectors: contiguous f32 slab (eliminates Vec<Vec> indirection) - VisitedSet: O(1) clear via generation counter (replaces HashSet) - 4-accumulator ILP for L2 distance (auto-vectorized) - Flat PQ distance table (cache-line friendly) - Parallel medoid finding via rayon - Zero-copy save (write flat slab directly) - Optional simsimd feature for hardware NEON/AVX2/AVX-512 - Optional gpu feature with Metal/CUDA/Vulkan dispatch stubs Results (5K vectors, 128d): - Search: 90µs → 55µs (1.6x faster) - Build: 6.9s → 6.2s (10% faster) - Recall@10: 0.998 (maintained) - 17 tests passing Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Reuven <cohen@ruv-mac-mini.local>
1 parent 4976141 commit 0247c1f

21 files changed

Lines changed: 2464 additions & 22 deletions

File tree

Lines changed: 188 additions & 0 deletions

‎Cargo.lock‎

Lines changed: 30 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,9 @@ members = [
150150
# JS bundle decompiler (ADR-135)
151151
"crates/ruvector-decompiler",
152152
"crates/ruvector-decompiler-wasm",
153+
# DiskANN / Vamana (ADR-143)
154+
"crates/ruvector-diskann",
155+
"crates/ruvector-diskann-node",
153156
]
154157
resolver = "2"
155158

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
[package]
2+
name = "ruvector-diskann-node"
3+
version.workspace = true
4+
edition.workspace = true
5+
authors.workspace = true
6+
license.workspace = true
7+
repository.workspace = true
8+
description = "NAPI-RS bindings for ruvector-diskann"
9+
10+
[lib]
11+
crate-type = ["cdylib"]
12+
13+
[dependencies]
14+
ruvector-diskann = { path = "../ruvector-diskann" }
15+
napi = { workspace = true }
16+
napi-derive = { workspace = true }
17+
serde = { workspace = true }
18+
serde_json = { workspace = true }
19+
tokio = { workspace = true }
20+
parking_lot = "0.12"
21+
22+
[build-dependencies]
23+
napi-build = "2.1"
Lines changed: 3 additions & 0 deletions

0 commit comments

Comments
 (0)