Skip to content

Parse out-of-range floats as infinity on all platforms - #1731

Open
Tyagiquamar wants to merge 1 commit into
open-source-parsers:masterfrom
Tyagiquamar:fix/overflow-float-parsing
Open

Tyagiquamar wants to merge 1 commit into
open-source-parsers:masterfrom
Tyagiquamar:fix/overflow-float-parsing

Conversation

@Tyagiquamar

Copy link
Copy Markdown
Contributor

Summary

Makes floating-point overflow handling deterministic across standard libraries. Out-of-range JSON numbers like 1e+9999 and -1e+9999 now parse to +infinity/-infinity on every platform, matching the behavior already expected by test/data/legacy_test_real_13.json.

Root Cause

Reader::decodeDouble and OurReader::decodeDouble detect overflow by checking whether operator>> left value equal to std::numeric_limits<double>::max()/lowest(). That is how libstdc++ reports an out-of-range conversion, but it is not guaranteed by the standard. On MSVC the extraction fails and leaves value untouched (still 0), so tokens like -1e+9999 fall into the "is not a number" error path. This is why legacy_test_real_13.json fails on Windows (#1446) and why negative infinity written with the default useSpecialFloats=false writer no longer round-trips (#1405).

Solution

On a failed extraction, if the token is a well-formed JSON number (checked with a small grammar validator), the conversion failure can only mean the magnitude is out of range, so it maps to +/-infinity. Malformed tokens ("0e", "0e+", non-numbers) are still rejected, preserving the existing behavior guarded by the subnormal handling from #1427. The change is applied identically to the legacy Reader and OurReader.

Test Plan

Added regression tests:

  • CharReaderTest/parseOverflow: 1e+9999, -1e+9999, 1e309, -1e309, and an over-large integer all parse to the expected sign of infinity; the exact legacy_test_real_13.json input [-1e+9999, 1e+9999] parses to -inf/inf; and infinity written by the default writer (useSpecialFloats=false) round-trips.
  • ReaderTest/parseOverflow (legacy Reader): same [-1e+9999, 1e+9999] case, plus [0e] still rejected.

Local validation (native Windows, CMake 4.4.4 + Ninja + GCC 16.1.0 from MSYS2 UCRT64):

  • cmake --build build passes.
  • jsoncpp_test: all 136 tests pass (134 baseline + 2 new).
  • jsontestrunner_exe --json-writer StyledWriter test/data/legacy_test_real_13.json exits 0.
  • clang-format (repo style) reports both changed files clean.
  • Note: the jsoncpp_readerwriter ctest harness fails on this machine for an unrelated reason (its runjsontests.py does not quote the executable path, so the "open source" space in this workspace path breaks every invocation); individual runner invocations work. GitHub Actions runs with a space-free checkout path, and the cmake CI matrix includes windows-latest, where this fix is the relevant one.

Fixes #1446

The reader relies on operator>> writing numeric_limits::max()/lowest()
when a number overflows a double. That is libstdc++ behavior, not
guaranteed by the standard: on MSVC the extraction fails and leaves the
value untouched, so tokens like "-1e+9999" were rejected as "not a
number". This made test/data/legacy_test_real_13.json fail on Windows
(open-source-parsers#1446) and broke round-tripping infinity written with the default
useSpecialFloats=false writer (open-source-parsers#1405).

Detect overflow portably: when extraction fails, check whether the token
is a well-formed JSON number; if so the value is out of range and maps to
+/-infinity. Malformed tokens ("0e", "0e+") are still rejected.

Applies to both the legacy Reader and OurReader. Adds regression tests
covering +/-infinity parsing, the exact legacy_test_real_13.json input,
and the writer round-trip.
@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 41319884787

Coverage decreased (-0.6%) to 89.525%

Details

  • Coverage decreased (-0.6%) from the base build.
  • Patch coverage: 10 uncovered changes across 1 file (13 of 23 lines covered, 56.52%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
src/lib_json/json_reader.cpp 23 13 56.52%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 2765
Covered Lines: 2640
Line Coverage: 95.48%
Relevant Branches: 2705
Covered Branches: 2257
Branch Coverage: 83.44%
Branches in Coverage %: Yes
Coverage Strength: 23630.45 hits per line

💛 - Coveralls

@greptile-apps

greptile-apps Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The change reaches both JSON parsing APIs for numbers that overflow floating-point conversion.

Fix the underflow-to-infinity conversion before merging.

Findings

  1. P1 Tiny numbers become infinity ▶
  2. P2 Integer overflow lacks coverage ▶

Summary

Both readers now use isWellFormedJsonNumber to turn failed conversions into signed infinity. New tests cover overflow and writer round-trips.

  • Both JSON readers parse out-of-range numbers as signed infinity.

Reviews (1) · Last reviewed commit: "Parse out-of-range floats as infinity on..." · Reviewed by Greptile

Comment on lines +1730 to +1732
else if (isWellFormedJsonNumber(token.start_, token.end_)) {
value = *token.start_ == '-' ? -std::numeric_limits<double>::infinity()
: std::numeric_limits<double>::infinity();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Tiny numbers become infinity

If the standard library sets failbit and returns zero for a tiny number such as 1e-9999, zero skips the FP_SUBNORMAL branch. isWellFormedJsonNumber then makes the reader report success with infinity instead of zero or a parse error. Both decodeDouble implementations have this problem. Distinguish overflow from underflow before assigning infinity, and add positive and negative underflow tests.

Knowledge Base Used: Parser internals

Comment on lines +3388 to +3391
// An integer too large for any integral or floating-point type maps to
// infinity as well.
{"[9e99999999999999999999999999999999999999999999999999999999999999]",
std::numeric_limits<double>::infinity()},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Integer overflow lacks coverage

The case described as an oversized integer uses 9e..., so it tests another exponent rather than an all-digit integer. Replace it with a number containing more than 309 digits and no exponent. That covers the integer-limit path through decodeNumber into decodeDouble, which the new tests currently miss.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TESTING: legacy_test_real13.json parsing failed.

2 participants