Repository navigation
Parse out-of-range floats as infinity on all platforms - #1731
Tyagiquamar wants to merge 1 commit into
Conversation
The reader relies on operator>> writing numeric_limits::max()/lowest() when a number overflows a double. That is libstdc++ behavior, not guaranteed by the standard: on MSVC the extraction fails and leaves the value untouched, so tokens like "-1e+9999" were rejected as "not a number". This made test/data/legacy_test_real_13.json fail on Windows (open-source-parsers#1446) and broke round-tripping infinity written with the default useSpecialFloats=false writer (open-source-parsers#1405). Detect overflow portably: when extraction fails, check whether the token is a well-formed JSON number; if so the value is out of range and maps to +/-infinity. Malformed tokens ("0e", "0e+") are still rejected. Applies to both the legacy Reader and OurReader. Adds regression tests covering +/-infinity parsing, the exact legacy_test_real_13.json input, and the writer round-trip.
Coverage Report for CI Build 41319884787Coverage decreased (-0.6%) to 89.525%Details
Uncovered Changes
Coverage RegressionsNo coverage regressions found. Coverage Stats💛 - Coveralls |
[Medium impact] The change reaches both JSON parsing APIs for numbers that overflow floating-point conversion. Fix the underflow-to-infinity conversion before merging. Findings
|
| else if (isWellFormedJsonNumber(token.start_, token.end_)) { | ||
| value = *token.start_ == '-' ? -std::numeric_limits<double>::infinity() | ||
| : std::numeric_limits<double>::infinity(); |
There was a problem hiding this comment.
If the standard library sets failbit and returns zero for a tiny number such as 1e-9999, zero skips the FP_SUBNORMAL branch. isWellFormedJsonNumber then makes the reader report success with infinity instead of zero or a parse error. Both decodeDouble implementations have this problem. Distinguish overflow from underflow before assigning infinity, and add positive and negative underflow tests.
Knowledge Base Used: Parser internals
| // An integer too large for any integral or floating-point type maps to | ||
| // infinity as well. | ||
| {"[9e99999999999999999999999999999999999999999999999999999999999999]", | ||
| std::numeric_limits<double>::infinity()}, |
There was a problem hiding this comment.
Integer overflow lacks coverage
The case described as an oversized integer uses 9e..., so it tests another exponent rather than an all-digit integer. Replace it with a number containing more than 309 digits and no exponent. That covers the integer-limit path through decodeNumber into decodeDouble, which the new tests currently miss.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Summary
Makes floating-point overflow handling deterministic across standard libraries. Out-of-range JSON numbers like
1e+9999and-1e+9999now parse to+infinity/-infinityon every platform, matching the behavior already expected bytest/data/legacy_test_real_13.json.Root Cause
Reader::decodeDoubleandOurReader::decodeDoubledetect overflow by checking whetheroperator>>leftvalueequal tostd::numeric_limits<double>::max()/lowest(). That is how libstdc++ reports an out-of-range conversion, but it is not guaranteed by the standard. On MSVC the extraction fails and leavesvalueuntouched (still0), so tokens like-1e+9999fall into the "is not a number" error path. This is whylegacy_test_real_13.jsonfails on Windows (#1446) and why negative infinity written with the defaultuseSpecialFloats=falsewriter no longer round-trips (#1405).Solution
On a failed extraction, if the token is a well-formed JSON number (checked with a small grammar validator), the conversion failure can only mean the magnitude is out of range, so it maps to
+/-infinity. Malformed tokens ("0e","0e+", non-numbers) are still rejected, preserving the existing behavior guarded by the subnormal handling from #1427. The change is applied identically to the legacyReaderandOurReader.Test Plan
Added regression tests:
CharReaderTest/parseOverflow:1e+9999,-1e+9999,1e309,-1e309, and an over-large integer all parse to the expected sign of infinity; the exactlegacy_test_real_13.jsoninput[-1e+9999, 1e+9999]parses to-inf/inf; and infinity written by the default writer (useSpecialFloats=false) round-trips.ReaderTest/parseOverflow(legacy Reader): same[-1e+9999, 1e+9999]case, plus[0e]still rejected.Local validation (native Windows, CMake 4.4.4 + Ninja + GCC 16.1.0 from MSYS2 UCRT64):
cmake --build buildpasses.jsoncpp_test: all 136 tests pass (134 baseline + 2 new).jsontestrunner_exe --json-writer StyledWriter test/data/legacy_test_real_13.jsonexits 0.clang-format(repo style) reports both changed files clean.jsoncpp_readerwriterctest harness fails on this machine for an unrelated reason (itsrunjsontests.pydoes not quote the executable path, so the "open source" space in this workspace path breaks every invocation); individual runner invocations work. GitHub Actions runs with a space-free checkout path, and the cmake CI matrix includeswindows-latest, where this fix is the relevant one.Fixes #1446