Change server approach to handle parallel requests by sergey-zinchenko · Pull Request #1550 · abetlen/llama-cpp-python · GitHub
Skip to content

Change server approach to handle parallel requests#1550

Closed
sergey-zinchenko wants to merge 2 commits into
abetlen:mainfrom
sergey-zinchenko:model_lock_per_request
Closed

Change server approach to handle parallel requests#1550
sergey-zinchenko wants to merge 2 commits into
abetlen:mainfrom
sergey-zinchenko:model_lock_per_request

[model_lock_per_request] added limit_concurrency for uvicorn

71e28b7
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs