Repository navigation
Read a comma in a box as a space when scoring local agreement - #930
Merged
Merged
Conversation
local_agreement.fold dropped periods but kept commas, so a suffix typed into the surname box after a comma (`Smith, Jr.`) scored as a miss though the parser reads the name right (family Smith, suffix Jr.). PatentsView's US inventors showed it as their commonest disagreement. A fold applied to both sides can only turn misses into agreements: over every record holding a comma, 83 psc-individuals records and 561 orcid-researchers records now agree, none move away, and 137 more have fewer fields wrong. The scorer's fingerprint moves, so both shared records are re-recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
local_agreement.folddropped periods but kept commas. A suffix typed into the surname box after a comma (Smith, Jr.) therefore scored as a miss, though the parser readsJohn Smith, Jr.correctly: familySmith, suffixJr.. The fold now reads a comma as a space, soSmith,Jr.reads likeSmith Jr.as well. The PatentsView probe found the gap: it was the commonest disagreement among US inventors (655 of 20,000 sampled names).Measured
A fold is applied to both sides of the comparison, so two values that matched before still match. The change can only turn a miss into an agreement, and it can only touch a record holding a comma. I scored every such record with the old fold and the new:
psc-individualsorcid-researchersThe 2 PSC records that stopped being set aside now agree, and a record is set aside only when its title disagrees. The re-recorded totals match: PSC moves from 1,005,742 to 1,005,825 agreeing, ORCID from 1,474,016 to 1,474,577.
Notes
--baserecords are re-scored on their next run.Roe Jr., PhD, passed under the old fold too, so it was swapped forRoe, Jr., PhD.🤖 Generated with Claude Code