Thanks to everyone who joined our Agent Quality & Token Optimization webinar! In this session we looked at how LLMs, agents, and context windows influence both agent quality and token consumption — and why optimizing token usage isn't really about spending less, but about improving the quality of your agent interactions.

We've grouped the questions we received by theme and led with the ones that came up most often!

Watch the on-demand recording (Starts at 01:30)

📍Auto model selection

By far the biggest theme of the session. Here's what came up most.

How does Auto mode pick a model? Is it based on my prompt?

Auto model selection looks at your prompt and the surrounding context, and routes each request in real time based on task complexity, model performance, and live system health. Paid plans also get a 10% discount on token costs when using Auto. More detail in the Auto model selection docs.

With so many models available — including multiple Sonnet versions — which one does Auto actually choose?

Auto is optimized for reliability and availability. It selects models under the hood based on real-time system health and observed model performance, rather than exposing a fixed version-selection rule.

Auto almost never picks a deeper reasoning model like Opus, even when I ask for one. How do I get higher-reasoning models without having to remember to switch manually?

This is intentional in the current iteration: higher-reasoning models are significantly more expensive, so Auto leaves that as a deliberate, aware decision for you to make. Improving this is exactly what Auto Mode V2 is aimed at.

What does "V2" mean in Auto V2?

V2 refers to the next version of the Auto feature — not what's available today.

Is there a way to measure the "cognitive load" or complexity of a task before running it, so I can pick the optimal model?

There's no visible complexity score or tunable routing threshold today. Auto considers task complexity and system health internally and reveals the selected model after execution. Organizations can restrict which models are eligible, but not configure the routing thresholds themselves. As general guidance: reasoning models for planning, mid-tier models for well-specified execution, and lighter models for routine work. See Optimize AI usage.

Is there an overview of what each model is good at?

Yes — the AI model comparison covers each model's strengths, recommended tasks, and when to reach for something else.

The slide showed Opus for spec work. Is such an expensive model really necessary for all planning? Is there more granular guidance?

That slide was an example, not a rule. For less complex planning, a lower-tier model is often perfectly fine. The model comparison guide gives more nuance — and honestly, building this judgment is a learning path, especially as models keep changing. That's the whole goal of this session: give you enough understanding to make the call deliberately.

I'm building a RAG setup over ~2,000 existing automated test scripts to generate new test code. Which model is most cost-effective, and how many tokens should I budget?

For a mixed workload like this, Auto model selection is a good default — it routes tasks efficiently and gives paid plans the 10% discount. Use Copilot Spaces to ground responses in your test repositories. Keep the regular context window by default and reserve the 1M-token window for genuinely complex tasks. Billing is based on input, output, and cached tokens converted to AI credits; there's no published token budget for a corpus of that size. See Models and pricing.

Measuring and monitoring token usage

How do I know when I'm at 50% of my token window?

It depends on your surface, but every surface exposes context usage:

• Copilot CLI and the Copilot app: type /context

• VS Code: click the circle in the bottom right of the Agent window for current session info

• Other IDEs: check the docs for your specific surface

If I can't see what a given call costs, I can't debug where my tokens went or avoid repeating the same mistake. Shouldn't we be able to analyze token usage per call?

Fair point — and you can. The available methods vary by surface (VS Code, CLI, Copilot app), so check the documentation for the one you're using.

Are there repos or tools that help reduce cost while increasing output value — for example, for code agents or documentation agents?

Take a look at Chronicle, which gives you insights across your agent sessions: Gain insights across your agent sessions with Chronicle.

📍 What actually consumes your context

When Copilot reads an open file in VS Code, is the whole file sent as text — the same as pasting it into chat?

Essentially yes, unless you've scoped the context more narrowly (for example, to a few specific lines). The harness only ever sends text back and forth with the model, since that's all an LLM understands. There are some optimizations for files, but "reading a file is like pasting it into the chat" is the right mental model.

Do screenshots consume a lot of tokens? If I have a list of tests, is it efficient to paste the whole list as a screenshot?

Screenshots are supported in Copilot Chat, but their token cost isn't documented. For something like a test list, use structured text or attach the source file instead — Copilot can reference it far more precisely. See asking Copilot questions in your IDE.

A number of open source projects "compress" structured input to reduce size without losing value. Are there plans to adopt something like that, or recommendations?

Copilot CLI offers context management today, including automatic and manual /compact summarization plus file previews for tool output over 20 KiB. Note that compaction is lossy summarization, not lossless structured compression. See Context management.

📍 Instructions, skills, and customization

How much of a SKILL.md is "always on" versus loaded on demand?

Only the name and description are always loaded. The rest of the markdown is pulled in on demand — which is exactly what makes skills so token-efficient.

Would breaking instructions into sections that get invoked only when needed — referenced from a main instruction file — help with context management?

Yes, and that's precisely the best use case for skills. See About agent skills.

You mentioned "the model is already highly proficient in React, so no skill is needed." How would I know that — especially across different models in Auto mode?

You often won't know from a single prompt. You learn it by treating instructions and skills as living artifacts. If your agents fail repeatedly because they're missing knowledge, that's your signal to add something. Don't add skills preemptively. It's also worth pruning periodically — our own CLI team deletes their entire custom instructions file every three months and rebuilds it from scratch based on where agents actually missed.

Will GitHub offer skills or model configurations at the org level, as preset modes?

Organization-level custom agents already work as preset modes across repositories, covering prompts, tools, MCP servers, and an IDE-specific model setting. See Create custom agents.

Can wiki-style documentation help the agent? Are there studies?

Yes — structured docs help. Check out how to leverage an openwiki-style repo with Copilot CLI.

Is there a repository showing this agentic workflow as a working example?

Yes: Agent Factory status.

📍 Prompting practices

Is prompt brevity really the issue? Isn't the bigger problem large context being pulled in — rather than writing in "caveman speak," especially when humans need to read those prompts later in skills?

It's less about brevity and more about precision: the right amount of information, no more. You don't need caveman speak. But because the prompt effectively determines what work gets done, it's worth avoiding long prose.

If I can narrow the problem down that precisely, what do I need AI for? I usually reach for it when I can't track an error down. Any way to stop the model from reading dozens of unrelated files even when I've told it where to look?

The point of that example was: share whatever knowledge you actually have at prompt time. We see a lot of developers being vague even when they know exactly where the issue lives or what change they want. If you genuinely have no lead, being vague is the right trade-off between token cost and your own time — that's a legitimate call to make.

Can I write prompts specifically tuned to a model to reduce input, output, and cached tokens?

You could, but that's a deep power-user move requiring real understanding of how each model behaves — and models change constantly, so we don't think it's a good investment. There are much bigger levers to pull. For cached tokens specifically, you have very little direct control beyond not switching models mid-session; the harness should be optimizing that, not you.

📍 Workflow, gates, and governance

Could something like /security-review be enforced pre-commit through the harness? Right now the agent often runs a full QA pass after every task, which burns tokens when I'd rather review manually first.

Great instinct, but slash commands aren't enforceable gates — /security-review is an on-demand skill. The supported pattern is to separate iterative agent behavior from deterministic commit and merge gates:

• Configure Copilot to stop after implementation and wait for your approval before QA

• Enforce the real checks deterministically with Git hooks, GitHub Actions, rulesets, required status checks, and code scanning

• Keep lightweight checks in the normal loop; reserve full QA for critical changes or pre-commit

See Comparing CLI features.

Are token optimization techniques like headroom management on the roadmap?

No specific commitment on that technique, but our harness teams continuously evaluate token-optimization strategies to bake directly into the harness. Read more in Evaluating performance and efficiency of the GitHub Copilot agentic harness.

📍 Slash commands and surfaces

Where can I learn more about /orchestrate, /rubber-duck, and similar commands?

The CLI command reference has a high-level overview plus deeper detail on all interactive slash commands.

Does /rubber-duck automatically pick a model that wasn't used for development?

Yes — it uses a model that wasn't used in the current session. So if you planned with one model, rubber-duck will reach for a different one to get you a genuinely independent perspective.

What about the Eclipse plugin — when will analysis options be available there?

Token options are available in Eclipse today. See the Eclipse BYOK, skills, and chat updates changelog.

📚 Additional resources

Model selection and pricing

• Auto model selection - ow Copilot routes each request to a model in real time, what it optimizes for, and how the 10% token discount for paid plans works.

• AI model comparison — Side-by-side breakdown of every available model: strengths, best-fit task types, and when to switch to something else.

• Models and pricing - ow input, output, and cached tokens convert into AI credits, plus the multiplier for each model. Start here to understand what you're actually paying for.

• Optimize AI usage — Practical guidance on matching model tier to task, plus the org-level controls for restricting which models are eligible.

Context and token management

• Copilot CLI context management - Now the CLI manages your context window: /context to check usage, automatic and manual /compact summarization, and file previews for large tool output.

• Evaluating performance and efficiency of the Copilot agentic harness — Engineering deep dive on how we benchmark models across tasks and bake token optimizations directly into the harness.

• Gain insights across your agent sessions with Chronicle — Session-level analytics across your agent runs, so you can see where tokens actually went and spot repeat patterns.

Customization: skills, agents, and instructions

• About agent skills — What skills are, how to author one, and why they're token-efficient: only the name and description stay loaded, with the body pulled in on demand.

• Create custom agents — Setting up reusable preset modes at the org level across repositories, covering prompts, tools, MCP servers, and an IDE-specific model setting.

• Agent Factory example workflow — A live, working agentic workflow you can inspect and borrow from, rather than building from scratch.

CLI and IDE references

• Copilot CLI command reference (slash commands) — The full list of interactive slash commands with what each one does, including /orchestrate, /rubber-duck, /context, and /compact.

• Comparing CLI features — What the CLI can and can't enforce, and where to reach for deterministic gates like Git hooks, Actions, and rulesets instead.

• Asking Copilot questions in your IDE - ow context attachment works in chat, including files, code selections, and screenshots, and how to scope it tightly.

• Copilot in Eclipse: BYOK, skills, and chat updates — The Eclipse feature update covering BYOK, skills, chat improvements, and the token options asked about during the session.