Send the task / workflow run id as a request header to OpenAI-compatible endpoints (opt-in) · Issue #8792 · Skyvern-AI/skyvern · GitHub
Skip to content

Send the task / workflow run id as a request header to OpenAI-compatible endpoints (opt-in) #8792

Open

Description

@domondi1

Problem

When Skyvern runs through an OpenAI-compatible endpoint (ENABLE_OPENAI_COMPATIBLE), every LLM call for every task and workflow run looks the same to the endpoint. A gateway or proxy in front of the model can't tell which run a call belongs to, so it can't attribute cost per run or stop a single run that's looping on a page it can't get past. Skyvern already has task_id / workflow_run_id in context at each call in api_handler_factory.py, but the only headers it adds are static ones (HTTP-Referer, X-Title, the Copilot ones).

Suggestion

An opt-in setting that sends the current run's id as a request header on LLM calls, for example:

OPENAI_COMPATIBLE_RUN_ID_HEADER=X-Run-Id   # unset by default: no change

When it's set, Skyvern adds {<header>: workflow_run_id or task_id} to extra_headers on each LLM call it makes for that run. Nothing changes for anyone who doesn't set it.

Why it helps

  • Per-run cost attribution in whatever sits in front of the model (LiteLLM, Helicone, a company's own proxy).
  • Per-run spend limits. For example, with Inferrail (an open-source OpenAI-compatible gateway) a self-hosted Skyvern could set OPENAI_COMPATIBLE_RUN_ID_HEADER=X-Inferrail-Attribute-Work-Id, and the gateway's budgets.per_work_default_usd would give each workflow run its own dollar ceiling, refusing calls before they reach the provider once a run is out of budget.

Alternatives considered

A static header through extra_headers can't carry the run id, and a key per run isn't practical for a self-hosted server that starts runs on demand.

Happy to send a PR if this direction works for you. Disclosure: I maintain Inferrail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions