Skip to content

FIX: Persist is_general_technique on seeds to preserve technique reconstruction (#3739) - #3785

Open
Yogyabhilash wants to merge 2 commits into
microsoft:mainfrom
Yogyabhilash:auto-fix-3739
Open

Yogyabhilash wants to merge 2 commits into
microsoft:mainfrom
Yogyabhilash:auto-fix-3739

Conversation

@Yogyabhilash

Copy link
Copy Markdown
Contributor

Description

When SeedPrompt or SeedSimulatedConversation are saved into memory via SeedEntry, the is_general_technique attribute was not persisted. As a result, when seeds were retrieved back from memory, is_general_technique defaulted to False. This altered the seed's hash/content identifier and caused ValidationError when reconstructing an AttackTechniqueSeedGroup from stored seeds (which requires is_general_technique=True on all constituent seeds).

Problem & Root Cause

  • SeedPrompt has is_general_technique: bool, which is included in SeedIdentifier.content_hash().
  • However, SeedEntry in pyrit/memory/memory_models.py omitted is_general_technique from the SeedPromptEntries table schema, __init__, get_seed_record(), and get_seed().
  • Reconstructing AttackTechniqueSeedGroup via get_seed_groups_async failed validation because reconstructed seeds had is_general_technique=False.

Solution

  • Added Alembic migration e7a1b3c5d9f0_add_seed_is_general_technique.py (revising c8d3e5f7a901) adding is_general_technique boolean column to SeedPromptEntries with server_default=0.
  • Updated SeedEntry mapped column, constructor, and reconstruction methods (get_seed and get_seed_record).
  • Updated SeedRecord in pyrit/models/seeds/seed_record.py and frontend/src/types/index.ts.
  • Included is_general_technique in MemoryInterface._seed_group_member_key and exposed is_general_technique filtering across seed and seed group query and deletion methods.

Closes #3739

Tests and Documentation

  • Added tests/unit/memory/test_seed_is_general_technique_migration.py:
    • Verifies SQLite migration upgrade and downgrade preserving rows.
    • Verifies MSSQL schema generation with default constraints.
  • Added tests/unit/memory/test_seed_general_technique.py:
    • End-to-end roundtrip test storing and retrieving AttackTechniqueSeedGroup from SQLite memory.
    • End-to-end roundtrip test for SeedSimulatedConversation with is_general_technique=True.
    • Filtering and deletion by is_general_technique on individual seeds and seed groups.
  • Updated tests/unit/memory/test_memory_models.py:
    • Unit test verifying SeedEntry roundtrip preserves is_general_technique.
  • Updated tests/unit/memory/test_operations_and_findings_migration.py to run migration before schema check.
  • Validated with:
    • pytest tests/unit/memory/test_seed_general_technique.py tests/unit/memory/test_seed_is_general_technique_migration.py tests/unit/memory/test_operations_and_findings_migration.py tests/unit/memory/test_memory_models.py (96 passed)
    • pytest tests/unit/memory/test_migration.py -k "test_seed_conditions_and_follow_up_template_migrations_merge or test_run_schema_migrations_applies_head_revision" (3 passed)
    • ruff check (0 errors)
    • ruff format --check (clean)
    • ty check pyrit/memory pyrit/models/seeds tests/unit/memory (clean)

…nstruction (microsoft#3739)

When SeedEntry persisted SeedPrompt or SeedSimulatedConversation, it omitted is_general_technique. Consequently, retrieving seeds from memory restored is_general_technique=False by default. This altered seed hashes and caused ValidationError when reconstructing AttackTechniqueSeedGroup from stored seeds.

This change:
- Adds is_general_technique boolean column to SeedPromptEntries via Alembic migration e7a1b3c5d9f0.
- Persists and restores is_general_technique on SeedEntry and SeedRecord.
- Adds is_general_technique filtering and group key inclusion in MemoryInterface.
- Adds comprehensive unit and migration tests.

Closes microsoft#3739
@Yogyabhilash
Yogyabhilash marked this pull request as ready for review November 1, 2026 15:11

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Saving a general technique seed loses the flag required to rebuild its technique

1 participant