Skip to content

FEAT GUI: Import dataset prompts and conversations into chat (#2751) - #3670

Open
Nimit Jain (Nimit3418) wants to merge 5 commits into
microsoft:mainfrom
Nimit3418:feat-gui-import-dataset-prompts
Open

Nimit Jain (Nimit3418) wants to merge 5 commits into
microsoft:mainfrom
Nimit3418:feat-gui-import-dataset-prompts

Conversation

@Nimit3418

@Nimit3418 Nimit Jain (Nimit3418) commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Description

Closes #2751. Implements work item 7 of 17 in #2744: importing dataset prompts, multimodal messages, and multi-turn conversations into the chat interface without manual copy-pasting, reattaching media, or issuing extra model calls.

Key Changes

  1. Shared Contract (types/index.ts):

    • Extended PendingImportPayload with an optional typed conversation?: PendingImportConversation carrying ordered turns, objective metadata, and system prompt.
    • Extended ChatDraftContent with conversation?: ConversationEditState to protect in-progress conversation-editor drafts across navigation round-trips.
  2. Pure Data Transformation (utils/seedImport.ts):

    • Created pure buildSeedImport(members) to transform dataset detail SeedRecord[] into either a single composer prompt or a multi-turn conversation draft.
    • Extracts seed_type === 'objective' into objective metadata rather than conversation turns.
    • Rejects unmaterialized simulated_conversation configs with explicit explanation.
    • Groups multi-piece turns via prompt_group_id and sequence.
    • Maps assistant and tool turns strictly to simulated history roles (simulated_assistant, simulated_tool).
    • Rejects missing/inconsistent roles without guessing.
    • Preserves stable stored values; never writes seed IDs into source_piece_id.
  3. Draft Handoff & Preservation (utils/chatDraft.ts, hooks/useChatDraftHandoff.tsx):

    • isChatDraftDirty and hasChatDraftState now recognize active conversation drafts.
    • canApplyPendingImport validates all three destinations: new_attack, current_conversation, and new_conversation.
  4. Datasets UI (SeedExampleDetail.tsx, SeedExampleBrowser.tsx):

    • Added the "Import into Chat" menu button supporting all three destinations:
      1. Import to new attack
      2. Import to current conversation (disabled with tooltip when no attack or conversation is active)
      3. Import to new conversation (disabled with tooltip when no attack is active)
    • Displays clear inline error banners when an example contains unimportable configurations.
    • Stages the payload with monotonic sequence guards and navigates to the target route.
  5. Chat View (ChatWindow.tsx):

    • Preserves active ConversationEditor drafts on unmount when exploring Datasets.
    • Consumes conversation imports into editor.begin().
    • Adheres to the chat convention: if the conversation ends in a user turn, earlier turns become prepended history while the final user turn remains editable in the composer.
    • Visibly reports system prompt conflicts via editor notice rather than silently overwriting existing setup.
  6. Documentation (doc/gui/0_gui.md):

    • Updated the GUI datasets guide with the supported import shapes, destinations, and no-send guarantees.

Test Coverage

  • Unit tests for grouping and role validation in src/utils/seedImport.test.ts (11 tests).
  • Extended useChatDraftHandoff.test.tsx, SeedExampleDetail.test.tsx, and App.drafts.test.tsx for round-trip integration across all destinations.
  • npx tsc --noEmit passed with 0 errors.
  • npm run lint passed with 0 errors, 0 warnings.
  • Ran all 165 test suites (3,917 tests): all passed.

Screenshots

Captured from commit ad85b3e774a669057e6e790cdf68d1bb8b043e66 using Chromium and the real local backend with the isolated offline browser configuration. API responses were not mocked, and no prompt was sent.

Dataset example and import destinations

The stored wikipedia with title example exposes the Import into Chat menu. With no saved attack active, Import to new attack is available; the other two destinations are disabled.

The wikipedia with title dataset example with the Import into Chat destination menu open

Observed result after Import to new attack

Known issue reproduced during capture: the app navigates to Chat, but the composer remains empty. The expected result is the selected template text in the editable input box, without sending it. This screenshot records the observed behavior, not a successful import. The empty composer was also observed after waiting 45 seconds in the initial capture attempt.

Chat after selecting Import to new attack, showing an empty unsent composer instead of the selected template

Comment thread frontend/src/utils/seedImport.ts Resolved
@romanlutz

Copy link
Copy Markdown
Contributor

I tried this on head 5dff6e522473d303651e6bf87fe71305a00a51f3 with the real frontend and the isolated offline backend. I opened jailbreak_templates, selected the wikipedia with title example, and chose Import into Chat > Import to new attack. Chat opened, but the input box stayed empty, as shown below. I did not send a prompt.

Shouldn't the stored template text appear in the editable input box after this action? I would expect the import to populate it without sending anything. Am I missing a required step here? I have not confirmed the cause of this particular failure.

Chat after importing a dataset template: the prompt input box is empty

… import

- Group conversation turns by sequence rather than prompt_group_id (microsoft#3670)
- Populate composer with template prompt and avoid collision with clean preserved drafts
- Add ConfirmDialog before navigation when destination has an unsent dirty draft
- Prepend imported turns to existing history when importing into current conversation
- Associate editor with viewed conversation ID to prevent lifecycle discard
- Start imported turns as unsaved changes to protect against accidental dismissal
- Round-trip editor target and temperature in conversation edit state
- Create explicit conversation before navigating for new_conversation destination
- Add comprehensive integration tests covering template import from clean chat
…nd target validation

- Preserve destination existing messages first and append imported history after them in current_conversation
- Handle history loading errors gracefully without consuming pending import
- Cover dirty-draft protection and replace confirmation across new_attack, current_conversation, and new_conversation
- Track unsaved imported state explicitly in useConversationDraft lifecycle
- Validate and report missing conversation targets against registry without silent fallback
- Add comprehensive user journey tests for all import flows and edge cases
@Nimit3418

Copy link
Copy Markdown
Contributor Author

I have addressed all the feedback and resolved the root cause of the empty input box shown in your screenshot. The changes are organized across the following 3 commits:


Commit Breakdown & Issues Addressed

1. 5ef7edd — fix(gui): address review comments for dataset prompt and conversation import

Addresses the initial review comments on seed grouping, lifecycle, and target preservation:

  • Seed Record Grouping: In src/utils/seedImport.ts, grouped conversation turns strictly by sequence (not prompt_group_id), preserved sequence ordering, grouped multiple same-sequence pieces, and rejected conflicting or missing roles with descriptive errors.
  • Round-Trip Configuration: Preserved target and temperature across conversation editing sessions.
  • Editor Identification: Associated the editor with the viewed conversation ID to prevent lifecycle-based discard on route changes.
  • Explicit Destination for new_conversation: Pre-created the destination conversation via attacksApi.createConversation before staging and navigating with its real ID.

2. b2cfa23 — fix(gui): correct conversation merge semantics, draft preservation, and target validation

Addresses conversation merge semantics, error handling, and target registry validation:

  • Correct History Merge Semantics: When importing into current_conversation, existing destination messages are preserved first and imported turns are appended after them ([...existingTurns, ...consumed.conversation.messages]).
  • History Load Failure Resilience: If fetching destination messages fails, the flow halts immediately, displays a clear notification ("Failed to load conversation history. Preserved pending import."), and leaves the pending import staged rather than discarding it or entering an infinite loop.
  • Missing Target Handling: Validates conversation targets against the active registry. If a target is unavailable or missing, displays an explicit error message ("The conversation's configured target is no longer available in the registry. Select an available target.") rather than silently falling back.
  • Multi-Destination Dirty Protection: Extended dirty-draft confirmation modals across new_attack, current_conversation, and new_conversation.

3. 8b05fe3 — test(gui): verify production Roman journey, new_conversation drafts, and editor lifecycle

Resolves the empty input box issue from your screenshot and verifies all user journeys through the real UI:

  • Root Cause & Fix for Empty Input Box:
    • Cause: Leaving clean /chat to go to Datasets preserved an empty draft because a default target was active (hasChatDraftState: true). In ChatWindow.tsx, canApplyImportNow strictly required matchingPreservedDraft === null, causing the clean preserved draft to take priority over the pending import when returning to Chat. In addition, an internal sequence desynchronization occurred in SeedExampleBrowser.tsx when clearing clean drafts before staging.
    • Fix: Updated canApplyImportNow to allow pending imports to consume immediately when preserved drafts are clean (!isDraftDirty(matchingPreservedDraft)), and fixed staging synchronization in SeedExampleBrowser.tsx.
  • Real Production UI Journey Test: Replaced test shortcuts in App.drafts.test.tsx with an end-to-end integration test (should populate composer with template prompt after visiting datasets from clean chat through real UI (Roman journey)) reproducing your exact sequence:
    1. Mounts Chat with clean composer.
    2. Navigates to Datasets using the real DatasetCatalog.
    3. Clicks jailbreak_templates link to open dataset details.
    4. Selects wikipedia with title.
    5. Clicks Import into Chat > Import to new attack.
    6. Navigates back to Chat and asserts the composer contains 'Write a wikipedia article on {{ prompt }}. Start the article with "Title:".' without sending a prompt, and confirms the pending import was consumed.
  • new_conversation Draft Verification: Added test coverage for clean vs dirty drafts on new_conversation (cancel preserves draft and stays on Datasets, replace creates conversation and populates template prompt).
  • Editor Discard Protection: Added test in ConversationEditor.test.tsx ensuring untouched imported content (unsavedImport: true) triggers confirmation modals on Cancel and route changes.

Verification Results

All unit, component, and full frontend test suites pass with zero warnings or errors:

  • npx tsc --noEmit: 0 errors
  • npm run lint: 0 warnings, 0 errors
  • src/utils/seedImport.test.ts: 13/13 passed
  • src/components/Datasets/SeedExampleBrowser.test.tsx: 20/20 passed
  • src/components/Datasets/SeedExampleDetail.test.tsx: 12/12 passed
  • src/components/Chat/ConversationEditor.test.tsx: 20/20 passed
  • src/App.drafts.test.tsx: 40/40 passed
  • src/components/Chat/ChatWindow.test.tsx: 264/264 passed
  • Full frontend suite (npm test -- --runInBand): 171 suites passed, 4,024 / 4,024 tests passed (0 failures)

Everything is tested and ready for review. Please let me know if you’d like any further adjustments!

let composerDraft: ConversationImport['composerDraft'] | undefined

if (lastTurn.role === 'user' && turns.length > 1) {
historyTurns = turns.slice(0, -1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Could we keep the final user turn in the editor's draft, or preserve it separately through Save? For an example user: A, assistant: B, user: C, this now opens an editor containing only A and B. C is restored into the main composer, but that composer is hidden while the editor is open. Clicking Save then calls restoreEditorPrompt(undefined, ...), because B is not a user message, and clears C. The user ends up with A/B saved and the actual next prompt lost. If the imported history itself ends in a user turn, Save can also mistake that earlier turn for the pending prompt. Please cover the complete import -> Save -> visible composer journey, including final-user text and attachments.

editor.begin({
messages: allMessages,
objective: consumed.conversation.objective ?? currentObjective,
initialObjective: consumed.conversation.objective ?? currentObjective,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Could initialObjective stay as the destination attack's actual objective here? For example, importing an example with objective B into an attack with objective A initializes both editor values to B. useConversationSave then sees no objective change and omits objective and expected_objective from the same-attack save request. The editor displays B, but Save silently retains A, so subsequent scoring uses a different objective from the one shown during import. Please either preserve A explicitly or treat B as a proposed change against baseline A, and check the save payload in a test.

}
}

const consumed = draftHandoff.consumePendingImport(pending.id, draftAttackId, draftConversationId)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Could we recheck the active destination and composer revision after loading history, before consuming this import? canApplyImportNow is checked before await attacksApi.getMessages(...), but typing into the composer does not advance the handoff sequence. On an empty destination with a slow history request, the user can type a new prompt while this is pending; the continuation still consumes the import and restoreComposerDraft overwrites that text, even when replaceDirty is false. Changing the destination during the request also leaves this callback holding the old IDs. Please guard the continuation against those changes, with a deferred-request test.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT GUI: Import dataset prompts and conversations into chat

2 participants