Repository navigation
Reduce fixed injected context per session: adaptive, non-duplicative tool/prompt loading #5252
Description
Describe the feature or problem you'd like to solve
Every interactive CLI session injects a large fixed context before the first user message (roughly 40k tokens in a comparable Copilot App session — mostly tool definitions and static system-prompt blocks, not conversation). Users have no way to see or trim what is injected, and several classes of injection are duplicated or rarely needed:
- Tool usage prose is duplicated between the system prompt (
<tools>guide) and the tool definitions themselves. - Long usage essays embedded in tool descriptions (e.g. a ~600-token DuckDB/SQLite guidance essay in
session_store_sql) are loaded every session even when the tool is never called. - Skill listings inject full descriptions — dozens of skills × 50–100 tokens each — though only one or two are used per session.
- Deferred/lazy tool reminders can list multiple overlapping toolsets from different MCP servers at once (two full browser-automation suites in one session), even when neither is used.
- Rarely used first-party tool groups (workflow orchestration, PR review-reply, session-broadcast tools) are eager-loaded in contexts that don't need them.
Since the CLI ships as a binary, users cannot trim any of this themselves; it is a product-side lever.
Proposed solution
Adaptive, non-duplicative context injection:
- Single source of truth for tool guidance: keep usage prose only in the tool definitions; drop the duplicated system-prompt
<tools>block (or vice versa). - Load long per-tool usage guidance lazily: keep a one-line summary in the description, return the full essay in the tool result on first invocation.
- Cap skill-list descriptions at ~15–20 words in the injected
<available_skills>block; the full text is already available when a skill is invoked. - When multiple MCP servers expose overlapping capability (e.g. two browser-automation suites), collapse the deferred-tool listing to one pointer discoverable via the tool-search mechanism; ideally detect overlap automatically.
- Defer eager loading of rarely used tool groups (dynamic workflows, PR-lifecycle/reply tools, agent-broadcast) behind tool-search discovery, and eager-load them only in contexts that actually use them (parent agent gets
task; post-review contexts get review-reply tools; etc.). - Bonus: a
--debug-context(or similar) breakdown so users and maintainers can see exactly what is injected and how much each block costs.
Estimated effect: ~15–20% smaller fixed overhead per session (~6–8k tokens), with no capability loss since everything stays discoverable.
Example prompts or workflows
- Starting a quick question session (
gh-style lookup): none of the workflow/MCP tools are used; the session should not pay for them. - First call to a lazy tool: tool-search surfaces it, and the full usage guidance arrives with the first result instead of in every session preamble.
- Session with Playwright and Ghostlight MCP registered: only one browser toolset is advertised up front; the other is opt-in.
- Running
/security-review: its output-format contract is injected only when it runs, not in every session. - Long 200-message chat: freed fixed overhead extends the usable window before compaction.
Additional context
Measured from a Copilot App chat session with 48 eager tool definitions, ~19 skills, 15 configured repos, and 5 MCP servers registered; comparable structure in the CLI. The App-side equivalent request is filed in github/app (cross-linked there). Items 1–4 apply to both products.
Activity