Skip to content

4 - FEAT: Last-used Defaults - #3647

Draft
Richard Lundeen (richlundeen) wants to merge 31 commits into
microsoft:mainfrom
richlundeen:richlundeen-last-used-defaults
Draft

Richard Lundeen (richlundeen) wants to merge 31 commits into
microsoft:mainfrom
richlundeen:richlundeen-last-used-defaults

Conversation

@richlundeen

@richlundeen Richard Lundeen (richlundeen) commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Reuses last-used objective targets and optional chat scorers for new activity, with supported Registry defaults taking precedence. Removes the objective-target Registry default without adding chat controls.

Changes:

  • Reuse the last-used objective target for new chats, scans, and suite launches; an explicit target choice takes precedence.
  • Reuse the optional chat scorer, including Off, unless a Registry chat-scorer default is set. Keep explicit choices first.
  • Remove the objective-target default selector and Objective default badge from Registry. Keep supported adversarial and converter defaults.
  • Remember used choices in the current account and browser profile across restarts, not when viewing history or editing an unsent draft.
  • Keep saved attack/run settings, required scenario success scoring, and the existing chat layout and controls unchanged.

Existing objective-target defaults are not migrated; select a target once to record last use. Adversarial scans keep the existing fallback order: Registry default, then the named adversarial_chat target, then last use.

Layer 4/10 | Previous: #3651 | Final: #3648 | Blocked by: #3651

Diff is cumulative until preceding layers merge.

Before (original GUI)

Original Registry exposes separate objective and adversarial target defaults.

Original Registry exposes separate objective and adversarial target defaults.

After

Registry retains the adversarial default and removes the objective-target default.

Registry retains the adversarial default and removes the objective-target default.

Video

last-used-defaults-production.webm

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Move history into feature routes, add stored ID lookup, and retain the current chat draft.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve discard blocking before resetting the composer and use shared encoded continuation and scenario routes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep an explicit not-found page and Home recovery link at unknown URLs. Preserve the last valid chat and dataset views without changing legacy aliases or Registry index behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Prefer the existing retained draft when entering Chat from another view. Keep an already open chat in place so live work is not replaced by an older snapshot. Cover the workflow with the real History table, Chat tabs, composer and reset controls.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use shared readiness and completion watchdogs instead of a short scheduling deadline while preserving the concurrent settings edit assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@behnam-o Behnam (behnam-o) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[GHCP]: Verdict: APPROVE.

PR 4/10 - Last-used Defaults. Please address and resolve all 3 numbered inline comments in this review. Approval does not waive any comment.

  1. [GHCP]: MEDIUM - Temperature-bound chats remember an unresolvable target.
  2. [GHCP]: MEDIUM - A slow scorer catalog silently turns remembered scoring off.
  3. [GHCP]: MEDIUM - Leaving chat during an accepted send loses confirmed target use.

Scope: the feature increment 9fc4c51b4c5a34c832092a7c9d1882376e6264d0 -> 287e94a10c1b0f84e82056ef0cccd37e842de767. Static review only; tests, builds, and browser validation were not run.

initialMessages: [...messages],
navigationRevision: navigationRevisionRef.current,
converterGeneration: runtime.generation,
target: targetReference(activeTarget),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[GHCP]: MEDIUM - 1. Temperature-bound chats remember an unresolvable target.

ChatWindow.tsx:1244 remembers the source registry name together with the private, temperature-adjusted effective hash. Default resolution requires the registered name/hash pair, so subsequent new chats and scans cannot restore the selection. Normalize remembered choices to binding.source_name and binding.source_hash, or explicitly resolve the binding. Confidence: 10/10.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4d399a1. targetReference now uses binding.source_name and binding.source_hash for a temperature-bound target, so the last-used preference resolves back to the registered source. The effective hash and temperature remain unchanged for attack execution, and draft dirty tracking still distinguishes different effective bindings. The regression sends again through a reopened bound target, rather than only testing the first lazy-binding send.

Comment thread frontend/src/App.tsx
const registry = useTargetRegistry()
const scorerRegistry = useScorerRegistry()
const { preferences, updatePreferences, error: preferenceError } = useUserPreferences()
const defaultChatScorer = preferences.chatScorer !== undefined ? preferences.chatScorer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[GHCP]: MEDIUM - 2. A slow scorer catalog silently turns remembered scoring off.

App.tsx:209 treats an initially empty, still-loading scorer catalog as an absent default. With a remembered scorer and no explicit override, a user can send after targets load but before scorers do; the request omits chat_scorer, and successful completion remembers Off over the previous choice. Distinguish unresolved loading from a completed missing lookup before allowing affected submissions. Confidence: 10/10.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4d399a1. A new chat waits while its implicit remembered scorer is still loading, while Registry defaults and explicit selections keep precedence. The send receipt captures scorer-choice provenance: a missing response scorer records Off only when Off was actually selected, not when an implicit reference could not resolve. The regression uses a confirmed non-preparation send with a non-inherited response and a nonblank objective, and explicit Off remains covered through a delayed catalog update.

): ChatSendOutcome => {
const effectiveConvId = operation.conversationId
const targetResponseStatus = response.messages.target_response_status
if (!operation.rememberedUse && operation.progress?.request_turn_number != null

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[GHCP]: MEDIUM - 3. Leaving chat during an accepted send loses confirmed target use.

ChatWindow.tsx:929 records use only in applySendResponse. Navigating Home or Registry aborts/clears the component's trackers while the accepted backend send continues, so a successfully used target is never remembered. Make confirmed-use recording account-scoped rather than dependent on the chat/evidence-reader lifetime, while still excluding preparation failures. Confidence: 9/10.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4d399a1. The account-scoped preference owner starts observing the send receipt before submission acceptance is awaited. Those shared receipt reads survive ChatWindow unmount, while stale view evidence and UI updates are still discarded. Recording requires a request turn and a non-preparation stage, happens once, and uses the initiating account's updater even after an account switch. Added navigation and pending-account-switch regressions; unknown acceptance and preparation failure still cannot manufacture use.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep result details within the viewport after resizing, preserve editor assertions across modal restoration, and align unit and mock callers with feature-owned new attacks and exact ID forwarding.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve canonical routes, navigation context, hierarchy details, and last-used preferences.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate target identity inspection and converter reuse while retaining last-used objective targets and scorer preferences.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Record canonical target and scorer provenance through account-owned send receipts that survive navigation. Hydrate untouched suite target choices after registry loading while preserving explicit selections.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@richlundeen

Copy link
Copy Markdown
Contributor Author

Reply to #3647 (review)

Fixed all three findings in 4d399a1. Bound targets now remember their registered source identity; unresolved remembered scorers are no longer inferred as Off; and confirmed-use recording is owned by the initiating account rather than the chat view. Added regressions for subsequent temperature-bound sends, delayed scorer catalogs, explicit Off, navigation and account switching. Preparation failures, saved history and unsent drafts still do not record use.

Use the backend sequence and target response contract for confirmed sends without changing production preference guards.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed. No blocking concerns remain. This review records source coverage; it does not authorize merge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants