Skip to content

FIX: Canonicalize RandomTranslationConverter language order for seeded selection - #3804

Closed
Chen Yufeiyang (feiiiiii5) wants to merge 2 commits into
microsoft:mainfrom
feiiiiii5:fix/translation-language-order
Closed

Chen Yufeiyang (feiiiiii5) wants to merge 2 commits into
microsoft:mainfrom
feiiiiii5:fix/translation-language-order

Conversation

@feiiiiii5

Copy link
Copy Markdown
Contributor

Description

Fixes #3717

RandomTranslationConverter built its identifier from the language pool in sorted order, but the seeded random choice used the constructor order. Two converters with the same language set and seed therefore had the same identifier yet translated the same prompt differently ([French, Spanish] vs [Spanish, French] swapped the target language assignment), so the identity did not describe the behavior and replay/config comparison was misleading.

The constructor now canonicalizes self.languages once (case-folded sort) and both the identifier and the random selection use that single order. Reordering the same set no longer changes the hash or the seeded translation. The default 37-language pool is canonicalized too, so seeded selection in the default configuration is now order-independent as the identity contract already implied; there is no public signature change.

Tests

New test_random_translation_converter_canonicalizes_language_order_for_selection configures seed 42 with both pool orders and asserts the identifier and the target prompt are identical.

pytest tests/unit/converter/test_random_translation_converter.py -q
# parent revision: the new test fails - 'Spanish: alp...' != 'French: alph...'
# this branch:    tests/unit/converter 2984 passed, 74 skipped
pytest tests/unit/scenario/airt -q        # 278 passed
ruff check / ruff format --check / ty check <changed files>   # pass

Not run: the full make unit-test and CI matrix (local Python 3.14 venv; 3.11-3.13 not verified locally), and Windows.

Tests and Documentation

No documentation describes language ordering; none changed.

…d selection

The converter identifier hashed the languages in sorted order, but the seeded random choice used the constructor order, so two converters with the same language set and seed had the same identity yet produced different translations. Canonicalize the language pool once in __init__ and use it for both the identifier and the choice.

Fixes microsoft#3717
@romanlutz

Copy link
Copy Markdown
Contributor

Thanks for the contribution. Lanre Shittu (@Shizoqua) claimed #3717 first and opened #3784 before this PR. I compared the current heads, and both implement the same canonical language ordering and seeded-selection regression. I did not find separate behavior here that needs to be carried over.

I am closing this later duplicate so we can review one implementation. Thank you for understanding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Translation language order changes seeded behavior without changing converter identity

2 participants