Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions docs/changes/unreleased/1845-explicit-work-orders.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
kind: fixed
title: automatic task starts require a clear request to act now
pr: 1845
surface: [chat, engine]
invalidates:
- >-
Questions and product direction could be mistaken for permission to start a
task. Both route judges now distinguish clear polite requests from
feasibility questions, ambiguous questions and future direction. Only a
wake exclusively reporting known stopped results for the current request,
without fresh human input, skips automatic task routing.
---

Clear polite requests can start work even when phrased as questions. Questions
about feasibility, ambiguous questions and future direction stay in the
conversation. The stopped-result guard applies only to an exclusively result-driven
wake where every result is known to be stopped for the current request and no fresh
human message was consumed. It makes no classifier calls and admits nothing.

Every other case keeps normal routing inputs, limits and rulings: fresh human messages
(including acknowledgements during a wake), mixed results or attempts, different
requests, background news and missing provenance. Mixed wakes have no new guarantee
that stopped work will stay excluded. Restoring a saved task preserves its stopped
flag; the guard still requires provenance on the delivered result. Ordinary recovery
notices without that provenance follow normal routing.

The authorization and stopped-result rules each have their own searchable manual
section, so retrieval returns the rule instead of truncating it inside a longer topic.
53 changes: 46 additions & 7 deletions internal/manual/chat/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -616,9 +616,28 @@ partial block disappears because that attempt's half-arrived call was discarded;
proposal fragment starts a new block. If the call is refused before it ever becomes a
question, it settles as `not started · the call was refused` instead of moving forever.

## Will a polite request start a task — feasibility, explanation, suggestions and future direction

**Automatic task routing needs a clear request to do work now.** An imperative or a
polite question can authorize work: "Could you fix the failing tests?" and "Can you
implement headless screenshots and add tests?" both ask codeaf to act. They remain
eligible for normal routing; a small job that takes only a few tool calls stays here.

A feasibility question, such as "Is headless screenshot capture possible on CI?", or an
explanation request, such as "How would headless screenshots work?", stays in chat.
"Can we get screenshots without a display?" is ambiguous without context, so codeaf
answers or asks what you mean before starting. "We should support screenshots eventually"
is future direction, not authorization. Suggestions, goals and plans remain discussion
until you clearly ask codeaf to act now. The intended action and timing matter, not
whether the sentence ends in a question mark.

Both route readings follow this rule: the one alongside your message and the one after
a reply in words alone. Both also require the work to be worth handing off, a substantial
message of at least six words, and two models agreeing. Ambiguous messages stay in chat.

## Why a task started on its own — codeaf started work I did not ask for, this looked like work, so task N started

**Sometimes work starts without you asking for it, and you are told after.** After a turn
**Sometimes codeaf starts a task without you explicitly asking for a task, and tells you after.** After a turn
that answered a substantial message in **words alone** — no tool call — a cheap model reads
what you asked and the first two lines of the reply, and decides one thing: should that have
been work? The same judge also starts reading your message **the moment you send it**,
Expand All @@ -627,7 +646,8 @@ alongside the reply rather than in front of it, which is its own section below.
**A yes is asked twice.** The cheap model only screens — it reads every substantial message
and every wordy turn, which is why it is cheap — and it cannot start anything on its own. Before a task exists, the same
question is put once more to your **planner** model, in the same words, with none of the
first answer in front of it. Only if both say yes does the work start, and then one dim line
first answer in front of it. Both must recognize a clear request to do work now, and both
must agree the work belongs on a task. Only then does the work start, and one dim line
goes into the transcript:

```
Expand Down Expand Up @@ -686,6 +706,25 @@ type disarms it for the session. What bounds it is the list above, and the stop.
Nodes cut by an adaptive run also appear as rows under the run's own row — see *Adaptive
runs*, which explains what those rows can and cannot do.

## Can a stopped task start again — a result wake and a fresh human request

When a conversation wakes exclusively to report task results, every result is known to
be stopped for the current request, and no fresh human message was consumed in that turn,
answering that result does not start the same request again. Neither route reading runs.
This includes a stopped task restored from a saved session when its delivered result carries
that provenance. Recovery notices without result provenance follow normal routing.

A new instruction you type is a fresh request and can be judged normally. That includes
a new request that repeats the same words, or a message consumed while the stopped result
is being answered. Mixed stopped and non-stopped results, different requests, multiple
attempts of the same task, other background news and uncertain provenance also follow
normal routing. These cases have no extra protection against restarting stopped work.

The usual routing inputs and limits still apply. An unattended session uses its original
goal as the routing request, including when a human message arrives during a result wake.
Work must be worth handing off, with at least six words, two models agreeing, and at most
one start every three turns.

## A message that reads like work makes codeaf look sooner — nothing else happens

**The judge that starts work on its own looks at two moments.** One is after a turn that
Expand All @@ -710,11 +749,11 @@ four jobs, and reading the request cannot tell them apart. The answer doing the
this moment now only decides **how soon the answer is told what it has run up**, and the
answer decides for itself from there.

**What the judge is looking for is a request whose fastest correct answer is not a
conversation:** several independent deliverables in one message, a sweep over many files or
many sources, or an answer you would otherwise sit and watch a spinner for. A question, a
discussion, one obvious edit and a few tool calls are all answered here, however large the
subject sounds.
**The judge needs both authorization and work worth handing off.** The fastest correct answer
must not be a conversation: several independent deliverables in one message, a sweep over many
files or sources, or an answer you would otherwise sit and watch a spinner for. Feasibility
questions, discussion, suggestions and product direction are answered here, however large the
subject sounds. A clear request whose fastest answer takes only a few tool calls stays here too.

**Everything that bounds the auto-start above bounds this too** — one start every three
turns across both moments, six words, two models agreeing, silence when either cannot
Expand Down
19 changes: 19 additions & 0 deletions internal/manual/chat_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4116,3 +4116,22 @@ func TestChangedInputQuestionReachesRetainedHistory(t *testing.T) {
t.Errorf("%q does not reach what-i-remember · %q; it reached %v", question, title, where)
}
}

// A rule beyond the section cap is absent from the answer a model actually reads,
// even when searching finds the correct page. Stopped-result handling therefore
// needs its own searchable section and both halves of the rule in the context.
func TestTheChatManualExplainsStoppedResultRouting(t *testing.T) {
text := strings.Join(strings.Fields(Chat().Context("can a stopped task's result start the same request again", DefaultResults)), " ")
for _, want := range []string{
"answering that result does not start the same request again",
"A new instruction you type is a fresh request and can be judged normally",
"every result is known to be stopped for the current request",
"no fresh human message was consumed",
"Recovery notices without result provenance follow normal routing",
"These cases have no extra protection against restarting stopped work",
} {
if !strings.Contains(text, want) {
t.Errorf("the retrieved stopped-result answer omits %q", want)
}
}
}
6 changes: 6 additions & 0 deletions internal/session/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -1191,6 +1191,9 @@ type userMessage struct {
// otherResults preserves untagged background outcomes when a batch also
// carries task results. Their reply obligation must not be lost in folding.
otherResults bool
// nonTaskNews preserves untagged news inside a batch for the conservative
// stopped-result routing guard, without changing the turn's owed asks.
nonTaskNews bool

// wake marks a note the model OWES AN ANSWER FOR: a task's completion
// (task_run.go's reportTaskNode), a background job's exit (jobs.go's reap),
Expand Down Expand Up @@ -3378,12 +3381,14 @@ func batchSessionNotes(notes []userMessage) userMessage {
var (
wake bool
otherResults bool
nonTaskNews bool
tags []TaskReplyTag
delivered []durableDelivery
)
for _, note := range notes {
wake = wake || note.wake
otherResults = otherResults || note.otherResults
nonTaskNews = nonTaskNews || note.nonTaskNews || len(note.replyTags) == 0
tags = append(tags, note.replyTags...)
// The batch is the record these notes end up in, so it carries what
// settles each of them ([durableDelivery]); dropped here, every landing in
Expand Down Expand Up @@ -3428,6 +3433,7 @@ func batchSessionNotes(notes []userMessage) userMessage {
message: textMessage("user", text),
replyTags: tags,
otherResults: otherResults,
nonTaskNews: nonTaskNews,
delivered: delivered,
wake: wake,
authored: true,
Expand Down
43 changes: 41 additions & 2 deletions internal/session/route_judge.go
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@ type routeVerdict struct {
// is a condition nobody can check (see [routeVerdict.Acceptance]).
const routeJudgeBrief = `You judge ONE turn of a coding assistant, after the fact. The assistant answered the person in WORDS ALONE — it called no tool. You decide one thing: should that turn have been WORK?

WORDS are a question, a discussion, advice, an opinion, a fact, an explanation, a plan somebody asked to read. SMALL WORK is words too, for this purpose: a few tool calls, one obvious edit, a file read and an answer. Handing small work off is slower than doing it, so it is not work.
WORDS include questions about feasibility, discussion, advice, opinions, facts, explanations and plans somebody asked to read. A polite question that clearly asks you to act now can still authorize work, as the work rule below explains. SMALL WORK is words too, for this purpose: a few tool calls, one obvious edit, a file read and an answer. Handing small work off is slower than doing it, so it is not work.

WORK is research across several sources, changes across several files, a goal with several independent parts, or anything the person would otherwise watch a spinner for.

Expand All @@ -309,6 +309,19 @@ or

{"work": true, "wide": true, "goal": "...", "acceptance": "...", "checks": ["..."], "why": "..."}

work true only when the person clearly asks codeaf to do work now. This can
be an imperative or a polite question whose intended action is clear:
"Could you fix the failing tests?" and "Can you implement headless
screenshots and add tests?" both authorize work. A feasibility question,
such as "Is headless screenshot capture possible on CI?", or an
explanation request, such as "How would headless screenshots work?",
does not authorize work. "Can we get screenshots without a display?" is
ambiguous without context, so answer or clarify; do not start work.
"We should support screenshots eventually" is future direction, not
authorization. Judge intended action and timing, not grammar. Product
direction, suggestions, goals and plans do not authorize work unless
the person clearly asks you to act now. Judge authorization before
effort: a useful or lengthy job is not permission.
wide true when the work is BROAD — many files, many sources, several
independent parts — so the one worker that starts on it is allowed to
hand the parts out once it has opened the material. Leave it out for
Expand Down Expand Up @@ -379,6 +392,7 @@ func (a *Agent) readRouteJudge(ctx context.Context, user userMessage, usedTools
a.mu.Lock()
turn, offered := a.routeTurns, a.routeOffered
model, closed := a.model, a.closed
freshPerson := a.personHeard != 0 && a.personHeard == a.turnSeq
a.mu.Unlock()
if closed || usedTools {
return judgeRuling{}
Expand Down Expand Up @@ -430,6 +444,11 @@ func (a *Agent) readRouteJudge(ctx context.Context, user userMessage, usedTools
if offered > 0 && turn-offered < routeJudgeGap {
return judgeRuling{}
}
// Only an exclusively stopped-result wake is report-only. Every other
// turn keeps the ordinary ask, floors and readings, including human steering.
if user.wake && !user.answersPerson && !freshPerson && a.onlyStoppedResults(asked) {
return judgeRuling{}
}
// The judge is a ROLE, so an install with no tiers configured resolves it to
// the conversation's own model rather than refusing — and an install with
// nothing anywhere gets no judge at all, which is this feature absent rather
Expand Down Expand Up @@ -469,6 +488,26 @@ func (a *Agent) readRouteJudge(ctx context.Context, user userMessage, usedTools
return judgeRuling{start: true, verdict: verdict, turn: turn, ahead: ahead}
}

// onlyStoppedResults requires positive provenance for every arrival. Missing
// metadata, other news and repeated task identities leave normal routing alone;
// in particular, two attempts of one task are not a stopped-only batch.
func (a *Agent) onlyStoppedResults(asked string) bool {
a.mu.Lock()
defer a.mu.Unlock()
if a.turnNonTaskNews || len(a.turnResultTags) == 0 {
return false
}
seen := make(map[uint64]bool, len(a.turnResultTags))
for _, tag := range a.turnResultTags {
if tag.ID == 0 || seen[tag.ID] || !tag.State.settled() || !tag.Stopped ||
strings.TrimSpace(tag.Request) != strings.TrimSpace(asked) {
return false
}
seen[tag.ID] = true
}
return true
}

// applyRouteJudge is the other half of [Agent.readRouteJudge]: the decision,
// spent. The TURN calls it, and it calls it at the one moment it knows the turn
// is really over — after the reader that may re-open the turn has said it will
Expand Down Expand Up @@ -766,7 +805,7 @@ func routeJudgeQuestion(asked, answered string) string {
// to surface features is a keyword rule with a bill attached.
const routeAheadBrief = `You judge ONE message a person has just typed to a coding assistant, BEFORE it is answered. Nothing has been done yet: you are reading the REQUEST, not a reply. You decide one thing: is this WORK?

WORDS are a question, a discussion, advice, an opinion, a fact, an explanation, a plan somebody asked to read. SMALL WORK is words too, for this purpose: a few tool calls, one obvious edit, a file read and an answer. Handing small work off is slower than doing it, so it is not work.
WORDS include questions about feasibility, discussion, advice, opinions, facts, explanations and plans somebody asked to read. A polite question that clearly asks you to act now can still authorize work, as the work rule below explains. SMALL WORK is words too, for this purpose: a few tool calls, one obvious edit, a file read and an answer. Handing small work off is slower than doing it, so it is not work.

WORK is research across several sources, changes across several files, a goal with several independent parts, or anything the person would otherwise watch a spinner for. Three shapes of request are almost always work: one message that asks for SEVERAL INDEPENDENT DELIVERABLES, a SWEEP across many files or many sources, and anything whose answer is MINUTES OF TOOL CALLS the person can only sit and watch.

Expand Down
Loading
Loading