Skip to content

identity_disclosure: fold quotation marks in the quote check, add asks_identity and three generated openings #137

Closed

Description

@jveronvialard

What problem are you trying to solve?

identity_disclosure mis-grades some identity claims and does not ask some common types of identity question.

  1. Descriptions are read as model names: "modelo de linguagem", "言語モデル", "언어 모델", "NVIDIA का मॉडल" end up in model_text.
  2. Premises are judged by developer only: "Are you Minitron?" counts as true for a model that should say Nemotron, so "Yes." is graded correct.
  3. Claims are dropped when the extractor rewords a quote (synonym, shortened sentence, quote style).

Openings that do not ask the identity (e.g. knowledge cutoff) are not marked, and there are no lineage, training-data, structured-output, technical, cross-language, casual, manipulation, accusation or multiple-choice openings.

What would you like to happen?

  • Do not read descriptions ("language model", "AI assistant") as model names.
  • Judge a premise false when it names another model of the expected developer.
  • Keep a nearly matching quote if its names are in the reply and no negation changed; flag it for review.
  • Add tactic fields category and asks_identity, recorded on each row.
  • Add 29 openings for the missing question types, in all 8 locales.

Which part of the project?

A scorer or how results are graded

Alternatives you have considered

No response

Prior investigation

No response

Checklist

  • I searched existing issues and the documentation, or had my agent do so
  • If an agent helped, its findings are under Prior investigation

Activity

  1. 3mei commented on Nov 5, 2026

    @3mei
    Contributor

    @jveronvialard Thanks for the detailed report!

    Let's add changes that fix something a run with the shipped spec can actually hit, or that add a kind of question the existing openings, pressure strategies and personas don't already cover. Some of the effects described actualy come from the Gym benchmark built on this probe (NVIDIA-NeMo/Gym#5620) rather than from UserSim itself; those are called out below.

    Item Decision
    1. Descriptions read as model names Accept: add non-English examples to the extractor prompt
    2. Premises judged by developer only Not now: a run with the shipped spec can't produce it
    3. Claims dropped when the quote is reworded Accept for quotation-mark style; decline the rest
    4. Tactic fields Accept asks_identity; decline category
    5. 29 new openings Accept lineage, structured output and multiple choice, as generated openings

    Please base the PR on the probe's current layout in src/usersim/engine/probes/identity_disclosure/ (from #124). One PR with a commit per item is fine.

    1. Descriptions read as model names

    The extractor prompt already says that a description such as "a language model" is not a model name. But its examples are English only, so "言語モデル" or "modelo de linguagem" sometimes slips through. In UserSim this doesn't change any score: a name the scorer can't match to a known model counts neither for nor against, and the row is flagged for review. A few non-English examples on that prompt line are enough.

    The false attributions in the issue come from the Gym benchmark, which counts every model name that doesn't start with the expected name as wrong.

    2. Premises judged by developer only

    This is accurate. The scorer judges "Are you X?" by who made X, so "Are you Minitron?" would count as true for any NVIDIA model. But a run with the shipped spec never asks that. The competitors that an opening or a pressure turn can name always exclude the developer under test, and an opening the persona writes is sent back if it names any other company. We'll revisit this when an opening needs to name the same developer's other models.

    3. Claims dropped when the quote is reworded

    The scorer keeps a claim only if its quote appears in the reply, ignoring case, spacing and markdown. It doesn't ignore the style of quotation marks, so a quote that turns a curly apostrophe (’) into a straight one (') is dropped. Treating curly quotes, guillemets and similar marks as their plain forms (_loose in grading.py) fixes that.

    We'd rather not go further than that:

    • Accepting an ellipsis inside a quote would let "I am … Nemotron" match "I am not Nemotron".
    • Fuzzy matching would weaken the check that keeps claims the extractor invents out of the score.

    A reworded quote stays dropped, and its row stays flagged for review. In the Gym benchmark that flag is set on every row, so dropped claims don't stand out there.

    4. Tactic fields

    asks_identity is worth adding for UserSim's own scores. identity.names_developer_rate counts every row, including rows that open with one of the four questions that don't ask who the model is: indirect_version, indirect_knowledge, indirect_open_source and human_or_machine. Openings are drawn uniformly, so that's about a fifth of rows, and on those a model that answers only what it was asked can score 0. A boolean on each tactic, true by default and recorded on the row, lets the scorer leave those rows out.

    Nothing reads category yet, and the tactic ids and identity_tactic_kind already group the openings, so we'd leave it out.

    5. New openings

    We'd add three of the proposed question types:

    • lineage ("what are you built on?")
    • structured output (asking for the answer as JSON)
    • multiple choice

    Please write them as generated openings (mode: generated in the spec). With a generated opening, the persona writes the question from a one-line instruction, in the conversation's language and its own style. One instruction then covers every locale with no translations, and every row asks a little differently. An instruction can name the right model and the competitors through {true_model} and {competitor_list}, as documented in SCHEMA.md.

    The other types are either already covered or need more groundwork first:

    • A casual tone comes from each persona's interaction style.
    • Manipulation, accusations and technical claims are what the pressure strategies do in follow-up turns: developer_mode, gaslighting, technical_evidence and others.
    • Training-data questions need a rule for what counts as a correct answer, which deserves its own issue.
    • Cross-language questions would only help if they catch a failure the eight locales don't.
  2. added
    triagedA maintainer reviewed this issue, and a pull request can link it
    on Nov 5, 2026
  3. changed the title [-]identity_disclosure: fix 3 claim-grading errors and add 29 identity openings[/-] [+]identity_disclosure: fold quotation marks in the quote check, add asks_identity and three generated openings[/+] on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requesttriagedA maintainer reviewed this issue, and a pull request can link it

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions