Repository navigation
Conversation
zzylol
force-pushed
the
fix/1307-relative-error-through-negation
branch
from
November 15, 2026 06:40
d099b0b to
5923dce
Compare
zzylol
force-pushed
the
test/sign-derivation
branch
from
November 15, 2026 15:32
22c7ef7 to
004cb3b
Compare
zzylol
force-pushed
the
fix/1307-relative-error-through-negation
branch
from
November 16, 2026 00:57
7c0e508 to
bc48168
Compare
zzylol
force-pushed
the
test/sign-derivation
branch
from
November 16, 2026 01:54
004cb3b to
03cd542
Compare
Tests for sign derivation by construction, written before the implementation. They fail today where the rules or the executor are unsound or incomplete. - types/tests/value_sign_rules.rs: one row per transfer rule (Scan, row operators, measures, arithmetic for Float64 and Int64, math and date functions, scalar expressions, binary operators, summary states, readouts and finalize), plus the regression rows and a source check for the module lint. - integration-tests value_sign_oracle.rs: every subtree of the fuzz and adversarial queries is planned and executed exactly, and every known sign is checked against the evaluated values. - integration-tests value_sign_function_keys.rs: every PromQL math or date function name from the parser parses to one typed key. - integration-tests value_sign_executor_zeros.rs: signed zeros through sum, sgn and clamp. - Consumer tests: error contract, Count-Min in Stage 2, the DDSketch relative guarantee, and the selected-sketch check. S2 cost tests are ignored. Non-test code only declares the functions the tests call, with todo!() bodies: ValueSign::is_one_sign, GroupProperties::state_column_sign, math_function_of_call, time_function_of_call and the test-only column_signs_of_candidate_expression. The test-support fixture memo_fixtures::binary_op is public and takes vector_match and return_bool, so the rule tables build every BinaryOp through it instead of a local copy.
zzylol
force-pushed
the
test/sign-derivation
branch
from
November 16, 2026 04:46
03cd542 to
f5d364f
Compare
zzylol
changed the base branch from
fix/1307-relative-error-through-negation
to
main
November 16, 2026 04:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GroupProperties::column_signs) are checked only against hand-picked expectations, and several rules are unsound under the sign-bit definition. Example:sum_over_time(neg[5m])over a window of only-0.0is labelledNonPositive, but the executor returns+0.0.Review guide
Base:
main. The former base, fix(accuracy): relative error through negation, abs, set operators and relabels; readouts compose only through error-carrying inputs; exact accumulators take non-finite values and reject stale markers; finalize sign of counters #1309, was squash-merged intomain; this PR was rebased ontomain863153e keeping only its own commit, and retargeted.Read first:
crates/types/tests/value_sign_rules.rs(T1, T2.3, T4);crates/integration-tests/tests/value_sign_oracle.rs(T3).Non-test code is five stubs with
todo!()bodies (see Interfaces). Rule changes, the lint and the executor fixes belong to the implementation PR.Every expected value comes from one of three places:
TransformFunctionlist (T2.1).None comes from the code's own derivation. A row that depends on a provisional answer is marked
// provisional answer QN.Plan summary
The spec defines
column_signs[c]as a claim about every non-NaN, non-NULL exact value of columnc:NonNegative: the sign bit is clear (+0.0, positives,+inf).NonPositive: the sign bit is set (-0.0, negatives,-inf).v as f64, so the int0is N.The invariants:
The test sections:
PhysicalASAPDAGNode.column_signs, which are PR S2 and ignored here.Provisional answers (the user reviews them)
-0.0and add the Kahan compensation only when it is nonzero. So Sum and Avg keep P.sgn(±0) = x. So Sgn keeps its input's sign.Max(+0,-0) = +0andMin(+0,-0) = -0.clamp(x, +0, -0)is U.state_column_sign.valueentry. Only the Scan rule changes for an undeclared_sum.arithmetic_signmatchesopfirst, and a source-text check catches tuple wildcards. EveryFieldDataTypeandOptionvariant is listed.atan2, which getsl's sign, and forCase, which takes what its branches agree on.add_candidategets no debug assert.#[ignore = "nightly"].Behavior: before → after
None. This PR adds tests and
todo!()stubs only, and no production path calls the stubs.Interfaces
column_properties.rsgroup/mod.rsimpl GroupProperties { + /// The sign of a state group's state column (its first non-plain field); `None` without one. + pub fn state_column_sign(&self) -> Option<ValueSign>; }scalar_type_rules.rstest_support/forwarders.rs(test-supportfeature only, Q11)Every body is
todo!(). Usage: the tests in this PR are the only callers.test_support/memo_fixtures.rs(test-supportfeature only)The rule tables build every
BinaryOpthrough this fixture (arithmeticandset_operatorcall it withNone, false);value_sign_rules.rshas no local builder.Design scope
types/tests/value_sign_rules.rsandvalue_sign_derivation_sites.rs: Stage 0 group properties.integration-tests/tests/value_sign_{oracle,consumers,function_keys,executor_zeros}.rs: end to end, from frontend through the executor.metricsql_parserandstrumforintegration-tests. They provide the independent function-name list for T2.1.Algorithm changes
None.
Verification
Each row was run locally with
cargo test -p <crate> <filter>; ignored tests were run with--ignored.Re-run on
main863153e (after the retarget): this head (f5d364f) and the previous head (03cd542) were run with the commands below, with--include-ignored, and compared by test name. The result is unchanged: 14 pass, 40 fail and 1 cannot run, each test with the same status as before. Two adaptations were needed for the newmain:crates/cost/src/node/usage/tests.rsconflicted withmain's new DDSketch magnitude tests; both sets are kept.mainnow reads a DDSketch's sign from the one source column the build folds, when its producer passes that column on unchanged. The fixturepriced_state_bytestherefore gives its producer the schema[ts, value]and lists both as unchanged for a time range (a rate computesvalue), and sets the newvalue_magnitude_range: None. Without this,ddsketch_over_a_declared_raw_scan_is_unchangedfailed andan_undeclared_histogram_sum_prices_two_storespassed, for the wrong reason (no sign was read at all); with it, both have their previous statuses. The cost filterddsketch_over_now also matches 4 ofmain's own tests, which pass and are not counted.Earlier re-run on the rebased base (bc48168, on
main3e434c7): this head (03cd542) and the previous head (004cb3b on 7c0e508) were run with the commands below, with--include-ignored, and compared by test name. The result is unchanged: 14 pass, 40 fail and 1 cannot run, and the 82 panic lines are identical. The oracle numbers are the same: 1683 nodes covered, 235 skipped, 67,911 counterexample values, and the 200-seed run (4026 covered, 510 skipped) still has 1,185 values. One fix was needed for the new base: the cost testpriced_state_bytessetsProducerOutput::chain_tumbling_window_width_ms: None, a fieldmainadded.Earlier re-run (base 7c0e508): every test was run with
--include-ignored, on both this head and the previous head (22c7ef7 on d099b0b), and the outputs were compared by test name. The result is unchanged: 14 pass, 40 fail and 1 cannot run, and every panic message is the same. The oracle numbers are the same too: 1683 nodes covered, 235 skipped and 67,911 counterexample values, and the same 122 counterexample lines. The 200-seed run still has 1,185 values. One line changed, and only in the nightly skip list. Twelvetopk(3, sum_over_time(m_gauge[1m]))Aggregates are still skipped, but the reason now readsaggregate intent has no native implementationinstead ofaggregate output names differ from measures, because the base now names and validates every aggregate measure.Commands (from the workspace root):
scan_signs_follow_declarations_and_metric_types_sum/_count/_bucketh_sumis N (B14)row_operators_copy_their_input_sign@ 100,@ end()), Subquery, SeriesSample, InfoEnrich, MaintainPopulation, Relabel, Values, window column, Join, SetOp, Concatvector_from_scalar_has_its_literals_signvector(-0)is P;vector(NaN)andvector(time())are Uevery_measure_has_a_sign_ruleAggIntentover N, P and UMath(f)except Abs is U; everyTimeFnis Uarithmetic_signs_for_every_operator_and_operand_sign-,%,^andatan2are U; IntN*P,P*N,N/PandP/Nare P (F5)every_math_function_has_a_sign_ruleMathFuncaspromql_<f>(value, literals); SQLabs/ABS; a non-literal bound; another functionevery_time_function_has_a_sign_ruleevery_scalar_expression_has_a_sign_ruleScalarExprvariantbinary_op_values_and_labelsbool,group_righta > bool bkeeps the left side's sign.neg > bool -1is labelled P, but its value is 0 or 1summary_states_and_readoutsExactKind, mergesvalue_sign_one_sign_and_state_column_signis_one_sign;state_column_signof a DDSketch build, a Scan and a chainevery_promql_function_name_parses_to_a_typed_keyevery_typed_key_is_reachable_from_a_nameTimestampcomes from some namea_non_literal_bound_has_no_typed_key,parse_functions_reject_other_callsNonethe_sign_module_denies_wildcard_enum_armsmatch (…)ending in_ =>arithmetic_signevery_known_sign_holds_on_evaluated_valuesevery_known_sign_holds_on_evaluated_values_over_200_fuzz_seeds(ignore = "nightly")exact_accumulator_realizations_keep_the_group_signpos,negand-0.0panesnegSum gives+0.0against P (F1)a_groups_sign_does_not_depend_on_its_queries[q1,q2],[q2,q1],[q2],[q3,q4],[q4]N - P, entropy,h_sum, an int cast, int negationb10_abs_spellings_parse_to_one_keyabs,ABSandpromql_absall giveMathFunc::Absb11_estimated_zero_divisor_needs_a_checked_division(ignore = "S3")todo!)b12_…,b13_…mixed; no derivation outside the sign modulef1_a_sum_of_negative_zeros_keeps_its_sign_bitsum_over_time,avg_over_time,sumandavgover-0.0+0.0f2_sgn_of_negative_zero_is_negative_zerosgn(-0.0) = -0.0+0.0f3_clamp_resolves_signed_zeros_as_prometheusrelative_sum_and_avg_contracts_need_one_signed_inputsis_one_sign, over declared and derived child signsq(pos) - q(neg)rowcount_min_is_offered_only_over_non_negative_weightsclamp_min(mixed, 0)row. HydraCms cannot run: it has no build parametersddsketch_relative_guarantee_needs_a_one_signed_state_columnpos/neg, none overmixedmixedis granted (#807)error_contract_and_sign_read_one_function_keykeeps_errorperMathFuncselected_sketches_satisfy_their_sign_conditionsb08_…,ddsketch_over_non_positive_values_prices_one_store,an_undeclared_histogram_sum_prices_two_stores,cost_derives_no_sign,histogram_metadata,physical_nodes_carry_their_groups_column_signs,column_signs_survive_plan_serialization(ignore = "PR S2: …")--ignored); the two field tests end intodo!ddsketch_over_mixed_values_prices_two_stores,ddsketch_over_a_declared_raw_scan_is_unchangedCounterexamples the oracle found
Each line gives the query, the data, the node and column, the derived sign, and the evaluated values. Evaluation times:
only-zeros: the window holds only
-0.0;ordinary: an ordinary window;
non-finite: the window holds
±infand NaN.F1, a sum of
-0.0is+0.0. Data:negdeclared P, 12 series. Derived P, evaluated+0.0:sum_over_time(neg[5m]), Aggregatevalue: 12 values at only-zeros, 1 at ordinary, 1 at non-finite.avg_over_time(neg[5m]): the same.sum by (job) (neg),sum: 3 at only-zeros.+0.0.SUM(-p), withpdeclared N: 6 at only-zeros.neg, finalized, gives+0.0for job j1 at 20 min, and for j0–j2 at 30 min.non_positivevariant:sum_over_time,avg_over_time,sum/max/count/quantile byover them,sum(m_gauge), and SQLSUM/AVG. Seeds 0, 1, 6, 7, 10, 12, 22, 24, 28, 29, 40, 41, 43, 44, 47.Sortand aLimitundertopk(3, avg_over_time(..)).F5, an int zero labelled P. Data:
flows.bytesInt64 declared N,bytes ∈ {0, 1, 5000}. Derived P, evaluated int0(non-finite / only-zeros / ordinary):SELECT -bytes: 1200 / 3600 / 1200.SELECT bytes * -1: 1200 / 3600 / 1200.SELECT bytes / -2: 2400 / 3600 / 2400. Here1 / -2is also0.SUM(bytes * -1): the same in its Project, plus 6 in its Aggregate and its output column.B14, the
_sumof a histogram with negative observations. Queryrate(foo_sum[5m]), datafoo_sumdecreasing.value, derived N: 30,240 negative values, for example-8.07.value, derived N: 3,600 negative values at each time, for example-4788.3.New: vector/vector
boolcompare. Derived P:neg > bool neg: 12 values of+0.0at each of three times.neg > bool (neg * 2):+0.0(3 / 12 / 1) and+1.0(9 / 11).Precision list (Unknown columns whose values all had one sign; reported, not failed): 24 entries. Examples:
clamp_min(neg, 0)is all clear;clamp_max(pos, -0)is all set;AVG(bytes - p).Skip list: 235 nodes, 25 kinds that the executor cannot run or lower. Examples:
Castto Int64, which is why B15 is not observed;abs;@ 100;without;andwith a matcher on the right side.T2.3 by hand
Done in a throwaway copy, not committed. Adding
#![deny(clippy::wildcard_enum_match_arm)]tocolumn_properties.rsmakescargo clippy -p asap-typesfail at four arms:_ => Unknown;project_lineage's_ => None, a lineage function in the same module;expression_sign's_ => Unknown;measure_sign's_ => Unknown.It does not flag the tuple match in
arithmetic_sign(F11 confirmed).Mismatches with the plan's T7 predictions
boolfinding.neg > bool -1is labelled P.CAST(x AS VARCHAR)keeps the numeric sign (predicted U), and the three Case rows fail (Q10 precision).Coverage(DefinitionMismatch)), so it was dropped.foo(value)" cannot be built, because the memo rejects unregistered functions. It is replaced bynullif(value, 1).Cast.TimeRangeof a subquery ofc_total, today's raw-producer check would already price one store.Decisions and open questions
Follow-ups
Next: the implementation, after review.