Repository navigation
Decide which genes respond with a gate, and check it against noise - #84
Merged
Merged
Conversation
A dose series runs several tests of each gene, and nothing said which of them decides that a gene responds, at which false discovery rate, or how a fold change enters that decision. Toxicology's prefilters cut the estimated fold change after the test, which controls no error rate for the size of the change, and lowest_effective_doses took a screen and a level of its own, a second name for the same call. dose_response takes gate=, one test of any response at a stated FDR after the dose-blind expression filter: the spline by default, the dose groups, Dunnett's, the Williams-type or the MCP test, or, with a fold change, each dose against the vehicle by TREAT with Bonferroni over the doses and Benjamini-Hochberg across genes. The gate's test runs whether tests names it or not and is never left out as a repeat. Each cell type gains the table "gate" with passes_gate, the summary n_genes_passing_gate (NaN where the gate did not run), and the result keeps the gate, its cube and its design; the printout names the gate and its adjustments, and counts calls at most the FDR, as the gate does. lowest_effective_doses reads the gate: under a fold change it calls each dose by its TREAT p-value. The seed rule moves to the options' helpers. One recorded decision now says which genes respond, reproducibly, and later stages (roll-ups, points of departure, the permutation check) read it.
A Williams prefilter with a 1.5-fold cut gave a point of departure from vehicle-only zebrafish data in 93% of runs, and an unadjusted prefilter found responsive genes in every one of 100 label permutations (Lee et al. 2024; Bruns et al. 2026). With 2 replicates per dose, a moderated gate at 5% FDR still passes genes in 10% to 16% of null cell types, and nothing showed a user how often noise passes the gate at their own design. permutation_check(result) permutes each cell type's dose labels among the samples the design treats alike (samples in blocks of the same row of the design's terms; units of a random intercept whole when each holds one dose, within units otherwise), reruns dose_response with the result's settings and gate on each labelling, and reports per cell type the share of runs that pass genes, the genes per run and a permutation p-value (Phipson and Smyth 2010). Designs with few assignments run every one, which makes the p-value exact, and the summary gives the smallest p-value such runs can give: 6/90 for an order-free gate at three dose groups in duplicate. Each cell type first reruns its true doses and must reproduce the result's gate p-values bit for bit; each run's assignment is recorded. The options of an analysis read back from its record now come from one helper, which sumrank shares. Users see at their design whether the genes a gate passes stand out from noise.
The gate's p-values, TREAT at each dose, the lowest effective doses under a fold change and the assignments of dose labels that permutation_check runs needed references independent of CytePool, and the gates' error under the null needed rows in the calibration. validation/dose_gate.py compares, on series simulated from fixed seeds, each gate's p-values and calls with limma 3.62.1 fitted by R from the counts with its own knots (exact under voom's weights and with limma's standard errors), each dose's TREAT p-values with treat and topTreat, the fold-change gate's p-values and calls, and its lowest effective doses with stageR 1.28.0; the units, counts and enumerations of assignments with R's own reading of the sample table; the genes each of the 89 runs of the vehicle and two doses in duplicate passes with limma refitted by R on every assignment; and dream's F-statistics with variancePartition 1.36.2. The calibration script adds the gates' shares of null cell types with a call (the bound 7.76% a check at 3 replicates and a known limit at 2), the permutation check's agreement with them on 100 null datasets per design cell, the share of its p-values at most 0.05, the unadjusted Williams-type gate as a control, the 1.5-fold gate's power and false discovery proportion, and the vehicle-only design of Lee et al. (2024).
The check's printout said that noise passed genes in its runs, but where genes respond, their responses, scrambled over the permuted dose groups, still pass in some runs: in the dose guide's hepatocytes, 47 of the 51 genes that passed in the runs respond in truth. A coverage pass also found paths without a test: results that named their tests, whose runs must fit the same contrasts; blocks of units that each hold one dose; the printout's lines for a fold-change gate and for runs that cannot reach a small p-value; and the class's refusals. No test showed that the runs leave the batch out. The printout now says that the gate passed genes in so many runs, and the docstrings say that the share then overstates how often noise alone passes genes. New tests cover each of those paths: the batch test counts by hand the 42 of 89 assignments that would confound a dose with a lane, and the check of a result that named its tests reproduces its gate p-values bit for bit. dose_responses.py and permutation_checks.py are now covered in full.
The check's docstring cited Bourgon, Gentleman and Huber (2010) without the work in its References, the private module of reruns had no reason for citing nothing, and Sphinx read three property docstrings, whose first phrase ended in a colon, as types, which fails the API reference's build. The References list the work, the documentation tests give the module its reason, and the three docstrings open with a sentence. The check's docstring gives its measured cost: about 1 s per run of 20,000 genes and 24 samples with the spline gate on two threads, twice that with a fold change.
The dose chapter, the guide and the reference pages said nothing of which test decides that a gene responds, how a fold change enters that decision, or how often noise passes it at a series' design, and the lowest effective doses still spoke of a screen. The chapter gains "Which genes count as responding", with the gate's tests, its table and record, and a fold change by TREAT at each dose against a cut on the estimated changes, and "Could the calls come from noise?", with the check's units, its runs, the smallest p-value of small designs (6/90 at the vehicle and two doses in duplicate), the vehicle-only design of Lee et al. (2024) and the check's cost; its lowest-dose section, defaults, limits, checks and further reading follow the gate. Choosing a method gains a question, the glossary "Gate" and "Permutation check", the bibliography Lee et al. (2024) and Bruns et al. (2026), and the API reference a page for the check. The guide chooses a gate against the truth (the 1.5-fold gate passes 75 of the 215 hepatocyte genes beyond 1.5-fold and no other gene) and checks the calls against permuted doses, remaking the runs that passed genes: 47 of their 51 genes respond in truth. Tests run the chapter's examples and check its numbers against the simulator's truth and the check's records.
The Validation page had no section for the gate and the permutation check, the calibration's section no rows for the gates and the check, and no outcome for the rows at 2 replicates that print their bound and exceed it. The gate's validation compared counts of assignments above 2^53 that none of its series has, and its benchmark timed the ordinary least squares fit rather than the epic's weighted baseline. validation/dose_gate.py drops the empty row and stops if a count reaches 2^53, and times its benchmark against the weighted fit of 20,000 genes x 200 samples. The page gains the section "The gate and its permutation check" (28 checks, 28 pass), its summary row and its commands; the calibration's 139 checks (16 pass, 111 information, 4 known limits, 8 controls) with the gates' shares under the null, the check against them, the 1.5-fold gate's power and the vehicle-only design; the check's timings; and "known limit" as an outcome, which the documentation tests count. docs/validation_runs.toml records the three runs.
…nding The permutation check reported the smallest p-value its runs can give from a rule about renamings of dose groups, which missed the swap of the vehicle and a single treated dose under the fold-change, Dunnett and Williams-type gates, the reflection of doses that sit symmetrically on the axis, and renamings drawn among random runs. Its guard required the true doses to give the result's gate p-values bit for bit, which another CPU's arithmetic breaks by 1e-13, and its reruns fitted every test of the result to meet that guard. The check now counts as ties the runs whose gate gives the true doses' genes, p-values within a relative 1e-9 and calls, and reports the smallest possible p-value as (1 + ties) / (1 + runs). The guard compares the same way, each run fits the gate's test alone with the result's seed, and the check reuses the result's reasons, the cube's group checks, the gate's wording and the analysis map instead of copies. The calibration bounds each gate's share of null cell types by the 2,000 cell types it counts, which makes the dose-group gate's rows at 3 replicates known limits, and compares the check with the null datasets it did not run on; R counts the ties from its own refits of every assignment.
The gate's validation recomputed the spline's F-tests that the dose tests' validation already matches to limma, and its counts of ties recoded the rule they were meant to check. R now counts as ties the assignments whose refit gives the true doses' genes and p-values, refits every assignment of the sexes as well as of the duplicate design, and checks the spline gate's calls on CytePool's p-values; a covariate now splits the animals' units in R's comparison of units. The records follow: the gate's script printed 26 checks, 26 pass, and the fixture shrank from 480 to 406 KB; the contrasts' script printed its recorded table again; the benchmark puts 100 runs of the spline gate at 340 baselines, within the target of 350, now that each run fits the gate's test alone.
The documentation said the permutation check shows how often noise passes the gate, which holds only where no gene responds: in the guide's hepatocytes, 47 of the 51 genes that passed in the runs respond in truth. It now leads with the p-value, which tests whether the true doses pass more genes than permuted ones, and reads the share of runs as how often the gate passes genes with the samples' doses shuffled, which varies between series of one design. The chapter, the glossary and the docstring use one wording for the gate against BMDExpress's prefilter, the guide splits its two long cells, and calls are filtered at an adjusted p-value of at most 0.05 throughout. New tests pin the ties at one treated dose, under Dunnett's test of two doses, for a spline through unequal groups, for sexes with unequal counts and for symmetric doses, and that a Monte Carlo gate runs with the result's seed.
The gates' bound counted the 1,000 null datasets of a design cell, while each share counts its 2,000 cell types, whose standard error between datasets matches the binomial one, so the bound sat 5.7 standard errors above 5%. At 6.95%, the dose-group gate exceeds it at 3 replicates too (7.50% and 7.25%), and those rows join the known limits. The check's runs are compared with the 900 null datasets they did not run on, at 4 standard errors of the difference, and pass in every design cell. The run printed 139 checks: 14 pass, 111 information, 6 known limits and 8 controls, with every other row as the build's run printed it.
…target The Validation page timed permutation_check against "the baseline of the epic's targets" and said that its runs "lie within the target of 350", terms that the page defines nowhere. The page now names the baseline as it does for multiple_contrast_tests.py, the weighted fit of linear_models.py --benchmark, and says that a check of 100 runs costs about 100 analyses of the cell type. The script's docstring drops the same phrase; docstrings do not count in the recorded digest, so the run's record stays valid.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
A dose series runs several tests of each gene (#81, #83), and nothing said which of them decides that a gene responds, at which false discovery rate, or how a fold change enters that decision;
lowest_effective_dosestook a screen and a level of its own, a second name for the same call. Toxicology's workflows decide with a prefilter and cut the estimated fold change after the test, which controls no error rate for the size of a change, and two studies measured what that costs. With zebrafish embryos exposed to the vehicle alone and placed at random in 4 mock concentration groups of 6, BMDExpress's default prefilter, a Williams trend test with a 1.5-fold cut, gave a transcriptomic point of departure, the benchmark dose of the 25th most sensitive gene, in 93% of runs (Lee et al. 2024). With the concentration labels of a real series permuted 100 times, a workflow whose prefilter took an unadjusted Williams p-value found responsive genes in every permutation, where one with a false-discovery-rate-adjusted Williams test found none in nearly all (Bruns et al. 2026). #81's moderated tests control the false discovery rate within a cell type, but at 2 replicates per dose a cell type of 1,000 null genes still had a spline call in 10.2% to 15.5% of datasets, and nothing tested, at a user's own design, whether the genes their analysis calls could come from noise.The solution
gate=indose_response, one recorded test of any response at a stated FDR after the dose-blind expression filter, andpermutation_check, which reruns the analysis on permuted dose labels:gate=takes{"test": ..., "fdr": ...}with the spline (the default; the dose groups where fewer than two doses are on the axis), the dose groups, Dunnett's, the Williams-type or the MCP test, or{"fold_change": 1.5}: each dose against the vehicle by TREAT (McCarthy and Smyth 2009), so that the threshold is part of each dose's null hypothesis, the smallest of a gene's p-values times its doses (Bonferroni, valid whatever the doses' correlation), then Benjamini-Hochberg across the cell type's genes, recorded as the test"each_dose". A gene passes at an adjusted p-value at most the FDR. The gate's test runs whethertestsnames it or not and is never left out as a repeat;"linear_trend"and"nonlinearity"are refused, as they test a line or a departure from one. A fold change under dream is refused, as TREAT takes limma's moderated t-tests.table(g, "gate"), the summaryn_genes_passing_gate(NaN where the gate did not run),DoseResponseResult.gate, whichdose_response(..., gate=result.gate)takes back,provenance["gate_adjustment"], and the result'scubeanddesign, which the check reruns. The printout names the gate with its adjustments, as the OECD Omics Reporting Framework asks a report to name a prefilter's test, threshold, multiple-testing method and fold change (OECD 2023, Sec. 4.3.4), and counts calls at most the FDR, as the gate does.lowest_effective_doses(group)reads the gate. This PR replaces Test dose trends and candidate shapes, and find lowest effective doses #83'sscreen=andalpha=with the gate, which no release carried: another test or level is another gate, and the call column ispasses_gate, one name for one call. Under a fold-change gate each dose is called by its TREAT p-value, so that the lowest effective dose is the lowest dose whose change beyond the threshold is called.cp.permutation_checkandcp.PermutationCheck. In each cell type, the dose labels of the samples it uses are permuted among the samples that the design treats alike: samples in blocks of the same row of the design's other terms (so"~ sex"permutes within each sex), the units of one random intercept whole when each holds one dose, in blocks of units with the same rows, and the doses within units otherwise, so that every run keeps the dose groups' sizes, the knots and the filter's genes.dose_responseruns again with the result's options and gate, fitting the gate's test alone with the result's seed, and the summary gives the genes the true doses pass, the share of runs that pass genes, the genes per run, the permutation p-value (1 + b) / (1 + B) (Phipson and Smyth 2010), which answers whether the true doses pass more genes than permuted ones, andsmallest_possible_p_value. A design with at mostn_permutationsother assignments runs each of them, which makes p exact. Runs whose gate gives the true doses' p-values up to rounding tie, and the smallest possible p-value is (1 + ties) / (1 + runs): at the vehicle and two doses in duplicate, the 6 orders of the three groups' labels tie under the dose-group gate and the default spline, so p cannot fall below 6/90 = 0.067; at one treated dose every gate ties the swap of the dose and the vehicle; and the spline ties the reflection of doses that sit symmetrically on the axis. Before the runs, the true doses run once through the same path and must give the result's gate, the same genes, p-values within a relative 1e-9 and the same calls; each run's assignment is recorded, so any run can be made again.cytepool/_reruns.py, whichsumranknow uses too, and the recorded seed's rule moves to_options.recorded_seed; the check reusesdose_response's analysis map, its wording of the gate and its result's reasons, and the cube's checks ofgroups.The Dose response chapter gains "Which genes count as responding" and "Could the calls come from noise?", and its lowest-dose section, defaults, limits, checks and further reading follow the gate; Choosing a method gains a question and switches; the glossary gains "Gate" and "Permutation check"; the bibliography Lee et al. (2024) and Bruns et al. (2026); the API reference a page for the check. The guide "Dose response per cell type" gains two sections: on its hepatocytes the 1.5-fold gate passes 75 of the 215 genes whose true change exceeds 1.5-fold and no other gene, where the default gate with a 1.5-fold cut on the estimated changes keeps 216 genes, 38 of them never beyond 1.5-fold in truth; and the check finds no run that passes as many genes as the true doses in any cell type, so each p-value is 1/101, with 47 of the 51 genes that passed in the hepatocytes' runs responsive in truth, which is why the guide and the chapter read the p-value, not the share of runs, as the answer.
Why it matters
One recorded decision now says which genes respond, with its test, level, fold change and adjustments, and later stages read it.
permutation_checkanswers at the user's own design the question that Lee et al. and Bruns et al. asked of whole workflows: could the genes that the gate passes come from noise?The gate builds on #81's tests and expression filter, #83's multiple contrast tests and stage-wise calls,
cytepool.stats.empirical_bayes.treat, already validated against limma, and #79's simulator, which the validation and the guide use. PRs 10 and 11 can put the gate's call beside each curve and benchmark dose, whichever of them merges after this PR; PR 15's gene-set roll-ups and points of departure will admit only the genes the gate passes, and PR 15 adds benchmark doses and points of departure to the check's runs, a stage PR 11 left to it; PR 17 weighs the default gate on the full grid of designs; and PR 23 reruns the gate on resampled data.Validation
Each script ran twice in one Modal container (4 CPU, 8 GiB) and printed the same rows both times.
validation/dose_gate.py(new) printed 26 checks, 26 pass; the recorded run, at the final code on an AMD Zen 3 host as the Validation page's, took 1 min 45 s with a peak memory of 737 MiB. R sees the counts, cells, doses, the design's variables and the settings, and nothing CytePool computed; it fits limma 3.62.1 and edgeR 4.4.0 from the counts, stageR 1.28.0 calls the lowest effective doses, R builds the exchangeable units from the sample table itself, counts and enumerates the assignments, and refits limma on every assignment of two designs, counting the ties among its refits; variancePartition 1.36.2 fits the dream designs. The spline gate's p-values are #81's F-tests, whichdose_response_tests.pychecks against limma, so R checks that gate's calls on CytePool's p-values.lm.wfitunder voom's weights orclassifyTestsF,pfon each gene's dfp.adjust(p, "BH") <= fdron R's p-values, and on CytePool's for the splinetreat(fit, lfc = log2(fc)),topTreatpmin(1, m * apply(P, 1, min))ontreat's p-valuesp.adjust; stageRstageWiseAdjustment(method = "holm")on R's TREAT p-valueschoose()dose_responseon the permuted designdream,topTablevalidation/dose_contrasts.py, run again sincelowest_effective_dosesnow reads the gate, printed the same 80 checks as #83, 71 pass, 7 deliberate differences and 2 information, its rows of lowest effective doses now read through the spline and dose-group gates, with the same values: identical calls to stageR's, adjusted p-values within 5.2e-12, and every gene's lowest effective dose identical to the enumerated closure's (6,579 and 6,340 genes). Its recorded run, at the final code, printed the same table row for row; the second run took 3 min 26 s with a peak memory of 940 MiB.validation/dose_response_calibration.pyprinted 139 checks, 14 pass, 111 information, 6 known limits and 8 controls, 0 failed, #81's and #83's 101 rows as recorded; the second run took 1 h 2 min 26 s with a peak memory of 451 MiB, on an Intel host with the harness's AVX2 settings, where the earlier runs on AMD Zen 3 took 1 h 57 min. The gates' share of null cell types with a gene passing at 5% FDR, 1,000 datasets of 2 cell types per design cell, against 5% plus 4 binomial standard errors of the 2,000 cell types each share counts, 6.95%:The spline gate holds the bound at 3 replicates and exceeds it at 2, and the dose-group gate exceeds it at both, the tail that #81 measured for the moderated tests; those six rows print the bound and "known limit" without failing the script, and the default gate stays as it is until PR 17 weighs it on the full grid of designs. The bound counts the 2,000 cell types each share counts, whose between-dataset standard error matches the binomial one; a bound from the 1,000 datasets, 7.76%, would lie 5.7 standard errors above 5%.
permutation_checkof the default gate, 100 runs on each of the first 100 null datasets, passed genes as often as the other 900 null datasets did: a mean share of runs of 0.162, 0.072, 0.105 and 0.064 against 0.154, 0.068, 0.102 and 0.069, each within 4 standard errors of the difference (2 pass per design cell), and its p-value was at most 0.05 in 1.5% to 6.0% of null cell types, within 11.2%. Between series of one design the share varies: at 4 dose groups in duplicate its standard error over 100 series, 0.0076, gives a standard deviation of 0.08 around 0.16. On the power datasets, where a fifth of the genes respond, every check gave a p-value at most 0.05; the 1.5-fold gate found 0.24 to 0.45 of the genes of a 2 log2 change with a false discovery proportion of 0.000 to 0.011, a false call a gene never beyond 1.5-fold. In the vehicle-only design of Lee et al. (2024), 4 mock dose groups of 6 samples, 200 datasets, the default gate passed a gene in 17 of 400 cell types (4.3%), where an unadjusted Williams-type p-value below 0.05 with a 1.5-fold change at a dose passed one in all 400. The four new controls, the unadjusted Williams-type gate, passed a gene in all 400 null cell types of each design cell, above the 9.36% that 400 cell types allow.validation/dose_gate.py --benchmark, on 2 threads, the median of three checks after a first one, on one cell type of 20,000 genes x 24 profiles (the vehicle and 7 doses x 3), against a baseline, the weighted fit of 20,000 genes x 200 samples (0.207 s in the same run): 100 runs took 70.6 s with the spline gate, 340 baselines, within a target of 350 baselines per 100 runs, where runs that fitted every test of the result took 101.6 s (477); 140.6 s with the 1.5-fold gate (678), which fits each run's cell type once more for TREAT; and 24.1 s per run with the Williams-type gate, as information.Modal runs, each an ephemeral app through the reference harness (times CET; 17:58 to 22:55 on 20 November during the build, 00:29 to 01:43 on 21 November after code review, and the two documentation jobs of the final checks at 03:22 and 03:26):
make -C docs html, the guide's check-outputs)ruff format --check docs: one line of the guide's new cell; formatted, executed againmake -C docs htmlpassed (Sphinx with warnings as errors, MyST strict), the guide's outputs currentcytepool/_version.pywhile the worktree uploadedmake -C docs htmlpassed (Sphinx with warnings as errors, MyST strict), the guide's outputs currentmake -C docs htmlneeds Node.js, which this machine lacksmake -C docs htmlpassed, the guide's outputs current on AVX-512make -C docs htmlpassed, the guide's outputs currentdocs/changed_since_run.pylists 39 scripts on this branch: 34 of the 35 that master lists, all but the calibration, which this PR records, andanndata_conversion.py,distributions.py,normalization.py,pool_cells.pyandsmoothing.py, which reachcytepool/_options.py, whose only change is the newrecorded_seed. This PR recordsdose_gate.py,dose_contrasts.pyand the calibration, each run at the final code.Tests
New tests: 28 in
tests/test_permutation_checks.py(51 with their cases) and 15 intests/test_dose_responses.py, with the two new callables in the masked-input and top-level registries andgate_textamong the masked-input inventory's unexported callables; #83's tests now read the gate. They check against independent references: limma 3.62.1, edgeR 4.4.0 and stageR 1.28.0 in R, throughtests/data/dose_gate/limma.json(406 KB, written byvalidation/dose_gate.py --write-fixtures): the dose-group gate's p-values and calls under voom and trend, with exact and limma's standard errors, and R's adjustment of the spline gate's p-values; TREAT with robust and trend moderation, the fold-change gate's genes by name and stageR's stage-wise p-values; R's own units, counts and enumerations of assignments, read from the sample table; and the genes that every run of a duplicate design (89 runs) and of a design of two sexes (35) passes, refitted by R, with the ties R finds among its refits. Beside them: brute force over itertools for the assignments of nine block structures (sexes; animals nested in dose, with and without sexes; donors crossed with dose, with and without plates; a numeric covariate; units of unequal size); chi-squared tests of uniform assignments on one block and on two of the same size; ties against renamings, swaps and reflections worked out by hand (one treated dose under three gates, 2/6; Dunnett's test of two doses, 2/90; a spline through four groups of unequal size, 1/180; sexes with unequal counts, 1/9; 0, 5 and 10 on a linear axis, 2/90); counts put into the summary by hand; runs made again bydose_responsefrom their recorded assignments and the result's seed, every run that passes genes under nine option sets, Dunnett's and the MCP gate among them, a Monte Carlo gate and dream; the guard against a forged setting, another cube, p-values moved by 1e-6 and a flipped call, with p-values moved by 1e-12 accepted; the gate worked by hand from a cell type's fit; and counts derived by hand, such as the 42 of 89 assignments that would confound a dose with a lane.Mutations, each run on its own against the two test files and each caught; 21 ran during the build, and those on code that code review reshaped ran again on the reshaped code:
<instead of<=test_the_gate_by_handtest_the_printout_counts_calls_at_the_fdrtest_the_fold_change_gate_matches_treattest_treat_moderates_as_the_fittest_lowest_effective_doses_under_a_robust_fold_change_gatetest_the_summary_by_hand,test_each_run_matches_r_on_every_assignmenttest_units_and_assignments_match_rtest_assignments_by_brute_forcetest_units_of_unequal_size_keep_the_dose_groupstest_random_runs_are_uniformn_permutationstest_every_assignment_runs_when_the_runs_allow_ittest_one_treated_dose_ties_with_its_swap,test_designs_without_renamings_that_tieand four more tie teststest_a_result_its_doses_do_not_give_again_is_refused(bit for bit also the tie tests)test_a_monte_carlo_gate_runs_with_the_result_s_seed, added when the mutation survived the first pass, whose option sets had two treated doses, where the multiple contrast tests take no random numberstest_runs_fit_the_gate_s_test_alonetest_refusalstest_cell_types_that_cannot_be_checkedtest_a_run_is_dose_response_on_the_permuted_dosestest_runs_do_not_depend_on_threads_or_other_cell_typestest_the_gate_s_test_runs_when_tests_do_not_name_ittest_runs_leave_the_batch_outCoverage (coverage 7.13.1, branches) over the tests of dose responses, permutation checks, sumrank, masked inputs and the top level (1,918 tests): every new statement of
cytepool/permutation_checks.py(294, all branches),cytepool/dose_responses.py(156) andcytepool/_reruns.py(9) runs. The pass ran with numba's JIT on, since none of the new code is compiled.Checks at the final head, 77c380b, on master 9fa2cc1 (#83), which has not moved: the full suite 6,111 passed and 18 skipped (zstd needs Python 3.14) in 3 min 18 s;
ruff checkandruff format --checkoncytepool tests validation docsclean;mypy cytepoolclean (96 files);make -C docs check-outputs PAGES="guides/dose_response"found the guide's outputs current here;make -C docs htmlbuilt the Sphinx API reference here with warnings as errors and stopped at MyST, which needs Node.js, so CI's documentation job (Node 24, ruff on docs,make -C docs htmlwith MyST strict, the guide's check-outputs) ran on Modal and passed, at 77c380b on an AMD host and at 8686bb3, which differs only in two sentences of the Validation page and a docstring, on an Intel host with AVX-512 with the harness's AVX2 settings unset, as on GitHub's runners.docs/changed_since_run.pystill finds the three scripts this PR records current.Decisions
adjusted_p_valueis;adjusted_p_value_across_groupsstays in every table.gate, as PRs 10, 11 and 15 use it; the docstring, the chapter and the glossary say how it differs from NTP's ANOVA gate, a test of the data set, and from BMDExpress's prefilter.lowest_effective_dosesreads the gate and loses Test dose trends and candidate shapes, and find lowest effective doses #83'sscreen=andalpha=, which no release carried: stage-wise values hold only at the gate's level, so another level is another gate.respondsbecomespasses_gate.dose_response, with the options read back from the result's record, so that a run is the user's analysis. Each run fits the gate's test alone, with the result's seed, so that the count is a fixed function of the labels and the permutation p-value stays exact under a Monte Carlo gate; each cell type's true doses first run through the same path and must give the result's gate within rounding, the same genes, p-values within a relative 1e-9 and the same calls. A bit-for-bit guard, tried first, refused a result checked on another CPU, whose arithmetic moved the p-values by 1e-13 with the same calls, and forced each run to fit every test of the result.n_permutationsothers.permutation_check. The bound counts the 2,000 cell types each row's share counts, whose between-dataset standard error equals the binomial one; with it, the dose-group gate's two 3-replicate rows (7.50%, 7.25%) are known limits too, and Choosing a method states that gate's excess where it offers it.mode="vehicle"that places vehicle samples in mock dose groups (a vehicle-only series given mock doses in its design runs throughdose_responseand the check, and the chapter says so); a gate adjusted across cell types; TREAT under dream, for which variancePartition offers no reference to validate against, so a fold change under dream is refused with the reason; benchmark doses and points of departure in the check's runs, which PR 15 adds; and admitting a cell type's genes only when the check's p-value is at most the level, which could admit nothing at the vehicle and two doses in duplicate (smallest possible p-value 6/90) and which PR 17 weighs as a default if the default gate fails there.