csv.Sniffer ignores the dominant quotechar when candidates tie #159635

Bug report

csv.Sniffer.sniff() can choose the wrong quote character when different quote characters are plausible and one of them is used much more often than the other.

For instance, consider this CSV:

'a','b'
'a','b'
'a','b'
"k","l"

If the single quote is the dialect's quotechar, the first three records contain the values a and b; the last record contains the values "k" and "l", including the double quote characters as data. A quote character other than the dialect's quotechar is ordinary data.

In CPython 3.16.0a1+dev at revision 010bebbf9fd, sniff() chooses the double quote instead. Passing the returned dialect to csv.reader leaves the apostrophes in the first three records and removes the double quotes from the last record, without raising an error:

Actual:    [["'a'", "'b'"], ["'a'", "'b'"], ["'a'", "'b'"], ['k', 'l']]
Expected:  [['a', 'b'], ['a', 'b'], ['a', 'b'], ['"k"', '"l"']]

The input admits both interpretations, so the expected choice is a heuristic based on the predominant quote style. Repeating the single-quoted record 99 times still leaves sniff() choosing the double quote when just one double-quoted record is present.

Reproducer

The following reproducer uses exactly the four records shown above:

import csv
from io import StringIO

sample = """'a','b'
'a','b'
'a','b'
"k","l"
"""
dialect = csv.Sniffer().sniff(sample)
rows = list(csv.reader(StringIO(sample), dialect))

print(repr(dialect.quotechar))
print(rows)

Running the reproducer on CPython 3.16.0a1+dev at revision 010bebbf9fd selects the double-quote character and produces the “Actual” rows above. Setting quotechar to a single quote produces the “Expected” rows. Python 3.13.12 also selects the single quote for both examples.

Cause

For this example, using either quote character produces four records with two fields each. Both candidates also use the same comma delimiter and no escape character. The trial-parse score considers row-width consistency, record count, whether quoting is enabled, and delimiter preference, so these two candidates receive the same score. It does not consider how often each quote character starts a field: six fields start with a single quote, compared with only two that start with a double quote.

The double-quote candidate is generated first. The selection loop replaces its winner only when a later candidate has a strictly higher score, so the equally scoring single-quote candidate never replaces it. The generated order therefore decides the quote character even when single-quoted fields greatly outnumber double-quoted fields.

The expected improvement is to use field-start quote frequency to decide between quote candidates that otherwise tie, with the same delimiter and escape character.

Environment

Tested on Windows x64. Python 3.13.12 was tested for comparison.

Linked PRs

• gh-159635: Use quote frequency to break csv.Sniffer ties #159636