ChunkBack lets you test your apps with LLM provider endpoints without having to pay for the providers.
ChunkBack is a simple express server that emulates the response input and output of popular LLM providers, currently Gemini, Anthropic and OpenAI. Chunkback accepts a custom prompt language called CBPL that lets you customize the response to your applications.
Requires at a least Node.js 22 or greater installed.
npx chunkback@latest
Then run in your terminal:
curl -X POST http://localhost:5654/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "swag",
"messages": [
{"role": "user", "content": "SAY \"First message\"\nCHUNKSIZE 3\nCHUNKLATENCY 50\nSAY \"Second message\""}
]
}'
hello.mov
- Deterministic API response - You always will get back the same content you put in
- Saves money - When testing your application, you can stub out your LLM calls with this
- Open Source - You can see the code right there!
- No extra services dependencies - There's no DB no Redis no nothing, just server code
ChunkBack Prompt Language is a case-sensistive langauge for controlling LLM response prompts. There's an autogenerated doc here for usage.
My favorite test script is:
CHUNKSIZE 3
RANDOMLATENCY 50 200
SAY "Each chunk has a random delay between 50-200ms"
Set your OPENAI_API_BASE environment variable to http://localhost:5654 and your app should work out of the box.
Or, just replace https://api.openai.com/v1 with http://localhost:5654/v1 in your API calls.
Set your ANTHROPIC_API_BASE environment variable to http://localhost:5654/v1 and your app should work out of the box.
Or, just replace https://api.anthropic.com/v1 with http://localhost:5654 in your API calls.
Set your GOOGLE_API_BASE environment variable to http://localhost:5654 and your app should work out of the box.
Or, just replace https://generativelanguage.googleapis.com/v1beta with http://localhost:5654/v1beta in your API calls.
I am hosting a version of Chunkback on api.chunkback.com (sign up on the main site chunkback.com) so you can plug it in without having to set anything up. I'm asking for 20$ a month for unlimited usage, or you can use it for free for 1000reqs/month.
See CONTRIBUTING.md
MIT License - see LICENSE for details.
This project was built with the assistance of autogenerated coding libraries - most code was reviewed by a human (painstakingly). This README.md was handwritten, and should be kep that way.