You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Send the task / workflow run id as a request header to OpenAI-compatible endpoints (opt-in) #8792
When Skyvern runs through an OpenAI-compatible endpoint (ENABLE_OPENAI_COMPATIBLE), every LLM call for every task and workflow run looks the same to the endpoint. A gateway or proxy in front of the model can't tell which run a call belongs to, so it can't attribute cost per run or stop a single run that's looping on a page it can't get past. Skyvern already has task_id / workflow_run_id in context at each call in api_handler_factory.py, but the only headers it adds are static ones (HTTP-Referer, X-Title, the Copilot ones).
Suggestion
An opt-in setting that sends the current run's id as a request header on LLM calls, for example:
OPENAI_COMPATIBLE_RUN_ID_HEADER=X-Run-Id # unset by default: no change
When it's set, Skyvern adds {<header>: workflow_run_id or task_id} to extra_headers on each LLM call it makes for that run. Nothing changes for anyone who doesn't set it.
Why it helps
Per-run cost attribution in whatever sits in front of the model (LiteLLM, Helicone, a company's own proxy).
Per-run spend limits. For example, with Inferrail (an open-source OpenAI-compatible gateway) a self-hosted Skyvern could set OPENAI_COMPATIBLE_RUN_ID_HEADER=X-Inferrail-Attribute-Work-Id, and the gateway's budgets.per_work_default_usd would give each workflow run its own dollar ceiling, refusing calls before they reach the provider once a run is out of budget.
Alternatives considered
A static header through extra_headers can't carry the run id, and a key per run isn't practical for a self-hosted server that starts runs on demand.
Happy to send a PR if this direction works for you. Disclosure: I maintain Inferrail.
Activity