[mikenrafter]

Review updates

Inherited the review fixes from #102, including the spectral-clock and cosinor follow-ups in d059e6e. Dart 3.11.4 analysis clean; full initial fix pass 1,286 passed, 6 existing capture skips. All 58 current output-change cases pass after the follow-ups. This cumulative draft stays on main until #102 merges, then will be rebased to show its own changes and marked ready.

Depends on 00 / #102.

This is draft 01 of the split of #99. It targets main, so its diff is cumulative and includes the preceding branches. Review this area in its final commit; merge the preceding PRs first, then rebase this branch before merging.

Repeated calculations have no common way to retain partial work, merge adjacent slices or validate a saved summary. This adds slice-based aggregators, reusable summary structures and a law checker, with the arithmetic core they require.

What changes

• Add Batch, Aggregator, composition, checkpoint envelopes and JSON number codecs.
• Add counts, sums, moments, edges, grouping, bounded windows, suffixes, prefixes and affine recurrences.
• Add deterministic property generators, shrinking, reach checks and deliberately broken law-checker controls.
• Add incremental_core.dart. Moments imports its RunningMoments; the same existing file also defines the integer histogram, fixed-grid spectral state and calculation cache.
• Export only available implementations at this point in the stack.

Existing metric entry points retain their parent behavior. Methods remain pure Dart with zero runtime dependencies. Welford 1962 and Chan, Golub and LeVeque 1983 support the moment formulas; Hyndman and Fan 1996 support the histogram's type-7 percentile rule. Source comments retain the citations. Missing observations remain absent.

Algebraic breakdown

The contract consumes mutable states. A combine joins adjacent slices in chronological order. Laws compare finished observations on fresh states, rather than requiring identical private states. Floating-point associativity is approximate, not a mathematical identity of IEEE doubles. For the rows marked ordered monoid, associativity and empty identity apply only with fixed parameters and valid ordered input. Commutativity is not promised by the interface.

Structure	State and empty	Combine; lift; finish	Laws and evidence
Batch, ListBatch, DoubleColumn, RowCodec	Typed rows; zero rows	Ordered concatenation; lift copies a slice; finish exposes rows. Sequence monoid, not commutative. Codec transforms representation only.	Copy/join/range and codec cases in aggregator_core_test.dart; no numeric tolerance.
Aggregator, fold, run, .map, _MappedAggregator	User state S; empty()	Delegate combine and slice lift; map transforms finish only. An interface cannot make an arbitrary implementation lawful.	laws_harness_test.dart catches broken identities, associativity, seams, slicing and impure finish.
both, Pair, _BothAggregator, _CheckpointValidatedAggregator	Pair of child states; pair of identities	Componentwise combine and lift; finish returns a pair. Product monoid only if both children qualify. Validation adds no accumulation.	Core and law-checker tests exercise component composition and failures; equality is supplied by each caller.
Count	Integer; 0	Add counts; lift slice length; finish count. Commutative monoid.	Exact identity, split, associativity, singleton and slicing laws in primitive tests.
CountFinite	Integer; 0	Add; lift finite-value count; finish count. Commutative monoid.	Same exact laws, including missing values.
CountWhere	Integer; 0	Add; lift predicate count; finish count. Commutative for a fixed pure predicate.	Same exact laws.
Sum	_SumState sum and compensation; zeros	Merge Neumaier contributions; lift non-NaN values; finish compensated total. Approximate commutative numeric monoid.	primitives_test.dart and sum_of_test.dart; the primitive law comparator uses absolute 1e-6 plus relative 1e-12. Exceptional IEEE values use explicit semantic equality.
SumOf	Same compensated state; zeros	Sum fixed transform of non-NaN rows; same merge and finish. Approximate commutative numeric monoid.	Sum-of tests check transformed sums and chunk seams with absolute 1e-6 plus relative 1e-12.
SumSq	Double; 0	Add square totals; lift squares; finish total. Approximate commutative numeric monoid.	Primitive law tests, absolute 1e-6 plus relative 1e-12.
Min, Max	Nullable extreme; null	Select extreme; lift non-NaN values; finish nullable extreme. Commutative monoids over the accepted domain.	Primitive laws and empty/NaN cases, exact finite extrema.
RunningMoments, Moments	Count, origin, mean offset, centered M2; all zero	Chan merge; lift finite observations via Welford updates; finish count/mean/sample or population SD. Approximate commutative monoid. Removal is a separate partial inverse requiring an actually represented observation.	Primitive and running-moment tests check folds, add/remove and JSON continuation. Number comparisons use max(1e-9, abs(expected)*1e-8) in incremental_compare.dart; later property laws add commutativity tests.
First, Last	_EdgeState with optional one-row batch; empty	Leftmost or rightmost nonempty selection; lift edge row; finish row or null. Ordered monoids.	Exact ordered split/identity/associativity in primitive and core tests.
LastRows(k), _RowsState	Bounded suffix; no rows	Suffix of concatenation; lift last k rows; finish batch or null. Ordered monoid for fixed k.	last_rows_test.dart; exact row equality with NaN-aware comparisons where needed.
GroupBy	Map of keys to child states; empty map	Merge common keys using child combine in source order; lift equal-key runs; finish each group. Monoid conditional on child; commutative only for commutative children.	group_by_test.dart checks disjoint and recurring keys, identity and association with child equality.
Lookback(k), _LookbackState	Inner state, first/last k-1 rows, length; empty	Merge children and lift seam windows exactly once; lift complete local k-row windows; finish inner. Ordered monoid conditional on a lawful window inner.	lookback_test.dart checks empty/tiny slices, association, singleton and sliced paths; integer examples exact. Copies cost O(k) per seam.
Affine, Ewma	_AffineState count, scale, offset; identity map	Compose affine maps in order; lift each finite recurrence step; finish Affine. Ordered, noncommutative monoid algebraically.	ewma_prefix_test.dart checks composition and laws, affine relative 1e-12 plus absolute 1e-15; prefix numeric comparisons use their explicit 1e-12/1e-9 comparators.
Prefix(k, head, tail), _PrefixState	First k raw rows, length and tail state; empty	Concatenate bounded prefixes, feed overflow to tail; lift rows; finish head(prefix) and tail. Ordered monoid conditional on tail; head runs at finish.	Prefix tests cover chunks crossing k and both identities, with component comparisons.
IntHistogram	Sorted integer value/count bins and total; empty	Add bin counts; lift integral observations; finish exact type-7 ranks/median. Commutative monoid.	int_histogram_test.dart compares against expanded sorted inputs exactly, checks merging/removal and decoder refusal.
IncrementalLombScargle	Fixed grid, seven sums/frequency, centered moments, origins/extrema and retained input prefix; zero sums/empty prefix	No public combine. sync verifies a growing prefix and accumulates new points, rebuilding on edits; finish reconstructs PSD. Checkpointed ordered fold, not a monoid.	lomb_scargle_test.dart checks prefix parity, restart and work counters using max(1e-9, abs(expected)*1e-8); precision tests check time-origin stability.
CalculationCache, _CalculationEntry	Map of key to dependency snapshot/result plus counters; empty map	No combine. Lift is evaluate(key, dependencies, calculate); finish is returned result. Dependency memoization, not an accumulator monoid.	Mode-dependent cache tests arrive in area 6 with CalculationMode; the source is present here because it shares the core file. No new metric is defined.
Checkpoints, JSON codecs, LawViolation, ConversionFailure, _Lcg, law/property support	Envelope kind/version/state or generated test recipe; no accumulated metric state	Encode/decode or generate/shrink/check, not combine/lift/finish algorithms. Not monoids.	Core/property tests validate deterministic replay and law failures. Full cross-metric persistence tests arrive in area 6.

The common law set is left/right identity, split equivalence, associativity, singleton and sliced folding, pure finish and supported checkpoint round trips. The law checker does not assert commutativity generically. Commutativity of moments and histograms is checked explicitly by the later cross-area property file. Tolerances are fixture acceptance comparisons, not universal error bounds. Sum and SumOf require finite partial sums for chunk invariance; overflow can make IEEE results depend on chunking.

Where the speedup comes from

Edge calculates the growing night again at each roughly 15-minute sync: first 15 minutes, then 30, until the last sync covers eight hours. Each batch call costs more as the night grows, so 32 syncs replay 528 chunks of history. Retained state folds each new chunk once and revisits the unfinished tail, moving reusable calculation work from roughly quadratic to roughly linear over the night. Checking old inputs, copying results and rescoring a whole night still cost work. The app gets these paths when area 06 and Edge #657 adopt and retain the states.

I measured generated data only, using AOT on Linux aarch64 with Dart 3.11.4, at least five full schedules per case after warmup; these totals sum the family medians.

Workload	Main batch s	#102 batch s	Full retained stack s	Main / retained	#102 / retained
Night, all nine families	117.467207	90.730203	8.402044	13.98×	10.80×
Day, seven within-day families	322.685004	8.952278	0.586589	550.10×	15.26×
Cross-day histories, supplemental	0.207665	0.207174	0.199487	1.04×	1.04×
Night, six families used in #99	104.836146	77.907354	7.514821	13.95×	10.37×

The 30,000-beat micro-benchmark measures one HRV pass: its historical 4,956 µs batch, 1,281 µs composed, and 639 µs hand loop compare implementations of that one pass. The hand loop is cheaper than composition, but that comparison does not measure the repeated history that streams avoid across syncs. This framework supplies the operations; its own direct contribution in this workload is 0 s. Later PRs supply the retained calculations, and #102 already contains measured batch changes, so the #102 column keeps those inherited savings separate.

Per sync on the generated night (milliseconds, all nine calculations):

Sync	Night so far	Main	Retained stack
1	0.25 h	341	242
8	2 h	1,768	212
16	4 h	3,606	230
24	6 h	5,234	249
32	8 h	6,860	340

Full recalculation grows with the night; the retained stack stays flat. That growth, summed over 32 syncs, is where most of the saving comes from. Main here is 3a7ca34, the base of #102.

How each structure saves work

Counts, extremes and moments combine in O(1). Slice dispatch removes per-row generic calls. Affine recurrences combine in O(1); grouping costs the number of keys; bounded windows copy O(k) context. Integer histograms scale with distinct values, not history length. Fixed-grid Lomb arithmetic changes from O(nF) per pass to O(new points * F), plus O(F) rendering. Its retained-prefix validation still scans O(n) input, so the complete sync is not O(new points).

Fresh timings are in "Where the speedup comes from" above. Older numbers in the docs predate this split.

Tests

Dart 3.11.4, resolved matcher 0.12.20. dart analyze: no issues. Full timeout 3000 nice -n 10 dart test -j 2 --reporter json: 1,272 passed, 6 skipped, 0 failed. Counts include the inherited suite. Elapsed test time 35.979 seconds.

Logs: areas-logs/area-1.analyze.log, areas-logs/area-1.test.jsonl, exit status files and start/finish timestamps. Skips are optional external fixtures/corpora. No assertion was removed and no tolerance was widened. Dart 3.5.0 itself was not run; final test blobs retain the existing NaN-aware matcher fix.

Suggested review order

One commit: b5903fd9a9ee Repeated calculations need summaries that combine without replay. Review the state/empty/combine or append transitions first, the finish/publication rules next, then the area reference/law/checkpoint tests and tools.

Change size and stacking

Logic under lib/: +2,273/-0, 2,273 changed lines. Tests, tools and docs: +5,144/-0, 5,144 changed lines. Other root documentation: +7/-0.

Stacks on split/rr-hrv-outputs at c17280c2139a. This is branch 1 of 6, one compiled and fully tested commit per branch. Apply the stack in numerical order; the final branch must match origin/split/aggregators exactly.

Questions for the reviewer

• Is retaining the existing core file preferable to splitting its spectral/cache types solely to change branch ownership?
• Are observational laws and consumed mutable states the right contract for callers that save checkpoints?

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[mikenrafter]

@coderabbitai review

[coderabbitai]

Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.