@ruvector/tiny-dancer

Neural router for AI agent orchestration - FastGRNN-based intelligent routing with circuit breaker, uncertainty estimation, and hot-reload.

Features

• FastGRNN Neural Routing: Efficient gated recurrent network for fast inference
• Uncertainty Estimation: Know when the router is confident vs. uncertain
• Circuit Breaker: Automatic fallback when routing fails repeatedly
• Hot-Reload: Update models without restarting the application
• SIMD Optimized: Native Rust performance with SIMD acceleration
• Multi-Platform: Works on Linux, macOS, and Windows

Installation

npm install @ruvector/tiny-dancer

The package automatically installs the correct native binary for your platform.

Quick Start

import { Router } from '@ruvector/tiny-dancer';

// Create router with configuration
const router = new Router({
  modelPath: './models/fastgrnn.safetensors',
  confidenceThreshold: 0.85,
  maxUncertainty: 0.15,
  enableCircuitBreaker: true,
  circuitBreakerThreshold: 5
});

// Route a query to the best candidate
const response = await router.route({
  queryEmbedding: new Float32Array([0.1, 0.2, 0.3, ...]),
  candidates: [
    { id: 'gpt-4', embedding: new Float32Array([...]), successRate: 0.95 },
    { id: 'claude-3', embedding: new Float32Array([...]), successRate: 0.92 },
    { id: 'gemini', embedding: new Float32Array([...]), successRate: 0.88 }
  ]
});

// Get the best routing decision
const best = response.decisions[0];
console.log(`Route to: ${best.candidateId}`);
console.log(`Confidence: ${best.confidence}`);
console.log(`Use lightweight: ${best.useLightweight}`);
console.log(`Inference time: ${response.inferenceTimeUs}μs`);

API Reference

Router

Main class for neural routing.

Constructor

new Router(config: RouterConfig)

RouterConfig:

Property	Type	Default	Description
modelPath	string	required	Path to FastGRNN model file
confidenceThreshold	number	0.85	Minimum confidence for routing
maxUncertainty	number	0.15	Maximum uncertainty allowed
enableCircuitBreaker	boolean	true	Enable fault tolerance
circuitBreakerThreshold	number	5	Failures before circuit opens
enableQuantization	boolean	true	Enable memory-efficient quantization
databasePath	string	undefined	Optional persistence path

Methods

route(request: RoutingRequest): Promise<RoutingResponse>

Route a query to the best candidate.

const response = await router.route({
  queryEmbedding: new Float32Array([...]),
  candidates: [{ id: 'model-1', embedding: new Float32Array([...]) }],
  metadata: '{"context": "user-query"}'
});

reloadModel(): Promise<void>

Hot-reload the model from disk.

await router.reloadModel();

circuitBreakerStatus(): boolean | null

Check if the circuit breaker is closed (healthy) or open (unhealthy).

const isHealthy = router.circuitBreakerStatus();

Types

Candidate

interface Candidate {
  id: string;                    // Unique identifier
  embedding: Float32Array;       // Vector embedding
  metadata?: string;             // JSON metadata
  createdAt?: number;            // Timestamp
  accessCount?: number;          // Usage count
  successRate?: number;          // Historical success (0-1)
}

RoutingDecision

interface RoutingDecision {
  candidateId: string;           // Which candidate to use
  confidence: number;            // Confidence score (0-1)
  useLightweight: boolean;       // Use fast/lightweight model
  uncertainty: number;           // Uncertainty estimate (0-1)
}

RoutingResponse

interface RoutingResponse {
  decisions: RoutingDecision[];  // Ranked decisions
  inferenceTimeUs: number;       // Inference time (μs)
  candidatesProcessed: number;   // Number processed
  featureTimeUs: number;         // Feature engineering time (μs)
}

Use Cases

LLM Model Routing

Route queries to the most appropriate language model:

const router = new Router({ modelPath: './models/llm-router.safetensors' });

const response = await router.route({
  queryEmbedding: await embedQuery("Explain quantum computing"),
  candidates: [
    { id: 'gpt-4', embedding: gpt4Embedding, successRate: 0.95 },
    { id: 'gpt-3.5-turbo', embedding: gpt35Embedding, successRate: 0.85 },
    { id: 'claude-instant', embedding: claudeInstantEmbedding, successRate: 0.88 }
  ]
});

// Use lightweight model for simple queries
if (response.decisions[0].useLightweight) {
  return callModel('gpt-3.5-turbo', query);
} else {
  return callModel(response.decisions[0].candidateId, query);
}

Agent Orchestration

Route tasks to specialized AI agents:

const agents = [
  { id: 'code-agent', embedding: codeEmbedding, successRate: 0.92 },
  { id: 'research-agent', embedding: researchEmbedding, successRate: 0.89 },
  { id: 'creative-agent', embedding: creativeEmbedding, successRate: 0.91 }
];

const best = (await router.route({ queryEmbedding, candidates: agents })).decisions[0];
await agents[best.candidateId].execute(task);

Platform Support

Platform	Architecture	Package
Linux	x64	@ruvector/tiny-dancer-linux-x64-gnu
Linux	ARM64	@ruvector/tiny-dancer-linux-arm64-gnu
macOS	x64	@ruvector/tiny-dancer-darwin-x64
macOS	ARM64	@ruvector/tiny-dancer-darwin-arm64
Windows	x64	@ruvector/tiny-dancer-win32-x64-msvc

Performance

• Inference: < 100μs per routing decision
• Throughput: 10,000+ routes/second
• Memory: ~10MB base + model size

Related Packages

• @ruvector/core - Vector database
• @ruvector/gnn - Graph Neural Networks
• @ruvector/graph-node - Hypergraph database

License

MIT