Skip to content

Translation language order changes seeded behavior without changing converter identity #3717

Closed

Description

@romanlutz

Describe the bug

Two RandomTranslationConverter instances with the same seed and same language set but different input order have equal identity hashes while producing different translations. Replay/config identity therefore does not describe behavior.

Steps/Code to Reproduce

Create converters with the same seed and languages ordered [Spanish, French] vs [French, Spanish], then convert the same fixed words.

Expected Results

Equivalent identities should imply equivalent deterministic behavior, or differing order must be included in the identifier.

Actual Results

equal_hash=true, yet converter one emits Spanish for words 1-3 and 5, French for word 4; converter two emits French for words 1-3 and 5, Spanish for word 4.

Verified source reference: RandomTranslationConverter identity/selection.

Proposed fix/regression: canonicalize the pool for both hashing and random selection, or include order in identity; test same-seed equivalence and hash agreement.

Screenshots

Not applicable; exact generated prompts are described above.

Versions

Tested commit d3446e0, PyRIT 1.2.0.dev0, offline deterministic scope. Evidence: files/evidence/round3/repro-output.txt; local path is not an attachment.

Activity

  1. Shizoqua commented on Nov 1, 2026

    @Shizoqua
    Contributor

    I'd like to take this. I'll canonicalize the language pool so the identifier and the seeded selection use the same order, and add a regression test for same-seed equivalence and hash agreement.

  2. romanlutz commented on Nov 2, 2026

    @romanlutz
    ContributorAuthor

    Ownership update: Lanre Shittu (@Shizoqua) claimed this first and is assigned here. #3784 is the selected implementation. I compared and closed the later duplicate #3804.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

help wantedExtra attention is needed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions