Type: Bug

Description

When using a BYOK Custom Endpoint configured to use the OpenAI Responses API, prompt_cache_key is not included in the outgoing request even when github.copilot.chat.responsesApi.promptCacheKey.enabled is enabled.

The Responses API request builder itself supports prompt_cache_key, but the conversationId required to generate it is lost while the request passes through the extension-contributed language model / vscode.lm bridge used by BYOK models.

This appears to be a request-context forwarding gap rather than an issue with the prompt cache setting itself.

Expected behavior

For a normal agent chat request, DefaultToolCallingLoop.fetch() passes the current conversation ID:

return this.options.invocation.endpoint.makeChatRequest2({
    ...opts,
    // ...
    conversationId: this.options.conversation.sessionId,
    // ...
}, token);

The Responses API request builder then uses it when prompt cache keys are enabled:

const promptCacheKeyEnabled = configService.getExperimentBasedConfig(
    ConfigKey.ResponsesApiPromptCacheKeyEnabled,
    expService
);

if (promptCacheKeyEnabled && options.conversationId) {
    body.prompt_cache_key = `${options.conversationId}:${endpoint.family}`;
}

Therefore, a BYOK Responses request made from the same chat session should also contain a stable prompt_cache_key.

Actual behavior

For an extension-contributed/BYOK model, the request goes through:

DefaultToolCallingLoop
    ↓
ExtensionContributedChatEndpoint
    ↓
vscode.lm / LanguageModelChatProvider
    ↓
CustomEndpointBYOKModelProvider
    ↓
CopilotLanguageModelWrapper
    ↓
OpenAIEndpoint
    ↓
createResponsesRequestBody

DefaultToolCallingLoop supplies conversationId, but ExtensionContributedChatEndpoint.makeChatRequest2() does not destructure or forward it:

async makeChatRequest2({
    debugName,
    messages,
    ignoreStatefulMarker,
    summarizedAtRoundId,
    requestOptions,
    finishedCb,
    location,
    source,
    telemetryProperties,
    modelCapabilities,
}: IMakeChatRequestOptions, token: CancellationToken)

The bridge already forwards several pieces of internal request context through modelOptions:

modelOptions: {
    _capturingTokenCorrelationId: ourRequestId,
    _otelTraceContext: activeTraceCtx ?? null,
    ...(telemetryTurn !== undefined
        ? { _telemetryTurn: telemetryTurn }
        : {}),
    ...(modelCapabilities?.enableThinking !== undefined
        ? { _enableThinking: modelCapabilities.enableThinking }
        : {}),
}

but it does not forward conversationId.

On the provider side, CopilotLanguageModelWrapper restores these internal values and eventually calls the actual BYOK endpoint:

const makeRequest = () => endpoint.makeChatRequest2({
    debugName: 'copilotLanguageModelWrapper',
    messages,
    finishedCb: callback,
    location: ChatLocation.Other,
    source: { extensionId },
    requestOptions: options,
    userInitiatedRequest: !!extensionId,
    telemetryProperties,
    modelCapabilities: {
        enableThinking: internalModelOptions?._enableThinking,
        reasoningEffort: typeof _options.modelConfiguration?.reasoningEffort === 'string'
            ? _options.modelConfiguration.reasoningEffort
            : undefined,
    },
}, token);

At this point conversationId is no longer available, so createResponsesRequestBody() sees:

promptCacheKeyEnabled = true
options.conversationId = undefined

and consequently does not add prompt_cache_key.

The Chat Debug Raw Request confirms this: requests with debugName: "copilotLanguageModelWrapper" do not contain prompt_cache_key.

Reproduction

1. Configure a BYOK Custom Endpoint with "apiType": "responses".
2. Enable the Responses API prompt cache key setting.
3. Start an Agent chat using that custom model.
4. Open Chat Debug and inspect the Raw Request for copilotLanguageModelWrapper.
5. Observe that the Responses request does not contain prompt_cache_key.

The same request originates from a DefaultToolCallingLoop where a valid conversation.sessionId exists.

Suggested fix

Forward the conversation ID using the existing internal modelOptions bridge.

For example, extend ExtensionLanguageModelRequestOptions:

export interface ExtensionLanguageModelRequestOptions extends OTelModelOptions {
    readonly _enableThinking?: boolean;
    readonly _conversationId?: string;
}

Then preserve it in ExtensionContributedChatEndpoint.makeChatRequest2():

async makeChatRequest2({
    // ...
    conversationId,
    // ...
}: IMakeChatRequestOptions, token: CancellationToken) {

and:

modelOptions: {
    _capturingTokenCorrelationId: ourRequestId,
    _otelTraceContext: activeTraceCtx ?? null,
    _conversationId: conversationId,
    // ...
}

Finally restore it in CopilotLanguageModelWrapper:

const makeRequest = () => endpoint.makeChatRequest2({
    // ...
    conversationId: internalModelOptions?._conversationId,
    // ...
}, token);

This would preserve the existing behavior of:

body.prompt_cache_key = `${options.conversationId}:${endpoint.family}`;

for BYOK Responses endpoints as well.

Note about CapturingToken.chatSessionId

The existing _capturingTokenCorrelationId bridge can restore a CapturingToken, which already contains a chatSessionId. It may therefore be possible to derive a cache key from:

capturingToken?.chatSessionId

However, explicitly forwarding the original conversationId seems preferable because the two values do not necessarily have identical semantics.

In particular, subagent invocations intentionally use the subagent invocation ID as the CapturingToken.chatSessionId, while conversationId is the conversation identifier used by the request layer.

Tests

extChatEndpoint.spec.ts already tests forwarding of internal request context such as:

• _telemetryTurn
• _enableThinking

A regression test could verify that a supplied conversationId is forwarded through modelOptions.

An end-to-end/unit test covering:

ExtensionContributedChatEndpoint
→ CopilotLanguageModelWrapper
→ OpenAIEndpoint Responses body

could then assert that, when ResponsesApiPromptCacheKeyEnabled is enabled, the resulting request contains the expected prompt_cache_key.

Relevant files

• extensions/copilot/src/extension/intents/node/defaultIntentRequestHandler.ts
• extensions/copilot/src/platform/endpoint/vscode-node/extChatEndpoint.ts
• extensions/copilot/src/extension/conversation/vscode-node/languageModelAccess.ts
• extensions/copilot/src/platform/endpoint/node/responsesApi.ts
• extensions/copilot/src/platform/endpoint/vscode-node/test/extChatEndpoint.spec.ts

VS Code version: Code - Insiders 1.135.0-insider (e1801a0, 2026-08-21T16:58:25Z)
OS version: Windows_NT x64 10.0.26200
Modes:

vsliv368cf:30146710
pythonvspyt551:31249597
nativeloc1:31118317
dwcopilot:31158714
dwoutputs:31242946
copilot_t_ci:31333650
g012b348:31231168
pythonrdcb7:31268811
pythonpcpt1cf:31399617
6518g693:31302842
82j33506:31327384
envsactivate1:31349248
editstats-enabled:31346256
cloudbuttont:31366566
3efgi100_wstrepl:31403338
cp_cls_t_966_ss:31526232
4je02754:31455664
8hhj4413:31478653
ge8j1254_inline_auto_hint_haiku:31490507
cp_cls_c_1081:31454833
conptydll_true:31485575
e9c30283:31453065
test_treatment2:31471001
c9b86496:31447327
control_6dc23131:31555649
ei9d7968:31462942
nes-extended-on:31455475
89g7j272:31506658
7e884298:31462391
i2gc6536:31472020
52612955:31508042
h08i8180:31475367
23c7c724:31520045
ddid_c:31478205
hmra_i5g22:31518061
bjc72774_agent_sandbox:31538428
7df3h592:31491241
cp_cls_t_1082:31535311
logging_enabled_new:31490725
jb_cp_cls_t_632:31543129
32d76977:31503652
ha629193:31508444
a1ije391_t:31533808
jbcp_cls_pctr_t:31531130
cp_intellij_t_nes:31548657
gf14b233:31526830
hide-eh-duplicates:31543293
replace-eh-with-ah:31545853
ihg5j128:31534457
f412h606:31543414
enable_editor_pane_layout:31551200
52258b4d:31547953
36h42362:31560305
c7c27ce7:31551212
6577i413_copy:31558653
1h923230:31564177
0dg73464:31554309
treatment-23-1:31555779
unuse_dynamic_mcp:31555281
0c1h4866:31562347
vrbsty_fls:31561057
5b8j3302:31564600
mangle-name-treatment:31568054
signed-out-control:31566874
codex-agent-enabled:31566739
a9h1h725:31568527
9e5id457:31570526
permission_prompt_control:1332567
ccr_pr_nudge_adoption:1319471
vsc_wsm_c:1324224