Skip to content

FEAT: Add Garak LeakReplay training-data replay scenario - #3689

Open
barkha1research-ai wants to merge 1 commit into
microsoft:mainfrom
barkha1research-ai:feat-leak-replay-scenario
Open

barkha1research-ai wants to merge 1 commit into
microsoft:mainfrom
barkha1research-ai:feat-leak-replay-scenario

Conversation

@barkha1research-ai

Copy link
Copy Markdown
Contributor

Description

This adds a garak.leak_replay scenario that ports garak's leakreplay probes (Chang et al. 2023, "Speak, Memory"). It has two techniques. cloze sends a passage with one proper name replaced by [MASK] inside garak's few-shot prompt and asks for the name. completion sends only the text before the mask. Each request carries its masked name as an OutputMatches condition. Cloze passes when the response contains the name, ignoring case, after <name> tags are removed. Completion passes when the stripped response starts with the name. These are the same checks as garak's TriggerListDetector and leakreplay.StartsWith.

As mentioned in #3219, the passages are not bundled. Four remote loaders (garak_leak_replay_literature, _potter, _nyt, _guardian) download the TSVs from the garak commit already pinned in third_party/garak-provenance.json. They parse each line as one passage. garak's default csv quoting merges 185 lines of book_cloze.tsv into 5 rows (NVIDIA/garak#278), so garak loads 1,239 passages instead of 1,419. The default run samples 20 requests, with at least one per technique and corpus; max_total="all" runs every request, like garak's *Full probes. I picked both techniques as the default because garak's active probes run both, but I'm happy to make it Cloze only, as ProPILE does with Twin, if you prefer.

Fixes #3219.

Tests and Documentation

I added unit tests for the loaders and the scenario, a LeakReplay section in doc/scanner/garak.py/.ipynb, the dataset names in 1_loading_datasets, and the chang2023speak reference. The notebook output comes from a run against a local Ollama model (llama3.1:8b). I also compared the generated prompts and pass/fail decisions with garak's own probe and detector code at the pinned commit, and they matched.

pytest tests/unit passes apart from one test_attack_analytics case that also fails on main, and pre-commit passes on the changed files.

Signed-off-by: Barkha <340367068+barkha1research-ai@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT: Add Garak LeakReplay training-data replay scenario

1 participant