Repository navigation
Test which genes respond to a dose in each cell type - #81
Merged
Merged
Conversation
Users who want to know which genes respond to a dose in each cell type build a dose factor or spline columns, a design and F-tests by hand for every cell type, and most single-cell dose studies compare the doses as unordered groups. dose_response checks the design with check_dose_design, fits one moderated model of the dose groups in each cell type that supports tests, with limma_voom, limma_trend or dream_voom, and runs four tests as contrasts of that fit: a natural spline of dose fitted to the dose groups' means, its departure from a straight line, the slope of a line along the axis and the dose groups against the vehicle. Every test of a gene uses its one moderated variance. DoseResponseResult holds a table per cell type and test, with FDR within and across cell types, each dose against the vehicle, and the fitted spline with a pointwise 95% band at any dose a cell type compares. The indicator of each dose group is named in _doses.py, where later dose tests find it.
The dose tests rest on contrasts of a model of the dose groups, so each needs a reference that shares nothing with CytePool but the counts and the settings. validation/dose_response_tests.py simulates seven dose series with simulate_dose_series and has R filter each cell type's samples, place the knots with its own rule and fit the spline to the dose groups' means with its own ns() and Hmisc columns. It compares the F-tests, slopes, each dose against the vehicle, adjusted p-values and curves with their 95% bands with limma 3.62.1 and edgeR 4.4.0 (exact), the exact default under voom's weights with each gene's lm.wfit (exact), the spline's sums of squares with rms 8.2-1 (exact) and the dream route with variancePartition 1.36.2 (close). DRomics, dreamlet and muscat run on the same profiles as information. The fixture keeps R's results for the series the tests read.
A dose test says which genes respond; reading how they respond needs the dose groups' estimates beside the fitted curve and its uncertainty, on the dose axis. plot_dose_response draws, for up to 8 genes of a cell type, each dose's log2 fold change against the vehicle, the fitted spline as a change from the vehicle on 100 doses evenly spaced along the axis, and its pointwise 95% band, with the vehicle one median step below the lowest dose beyond an axis break when it is a group of its own. validation/plotting.py checks the drawn curves, bands and points against limma-trend's spline computed in R from its own knots and ns() columns, with topTable(confint = TRUE).
Whether the dose tests hold their error rates at 2 or 3 replicates per dose, and how much each finds, can only be read from series whose truth is known. validation/dose_response_calibration.py runs dose_response on the core grid of simulated series, 4 and 9 dose groups by 2 and 3 replicates: on 1,000 null datasets per design cell, the distance of each test's p-values from uniform, their shares below 0.05 and 0.01 and the share of cell types with any call; on 10 datasets with responsive genes, each test's power by shape, the false discovery proportion within and across cell types, and the spline's power by its size. The rows are information, with seeds that later checks of a gate can share.
Readers need to know which of the four dose tests to read, what each asks, how a cell type's dose groups and the spline's size shape them, and how far the curves' bands can be trusted. The chapter gains a section on the tests, with a worked series, how to read each table and the adjustment within and across cell types, and one on curves with their pointwise 95% bands; its hand-built F-tests take the tests' names, and its simulated series runs dose_response. Choosing a method answers which genes respond to dose in each cell type. The Validation page reports the checks against limma, edgeR, rms and variancePartition and the error rates and power on simulated series, including the heavier tail of the null at 2 replicates per dose. The glossary, the bibliography and the defaults and limits follow.
A user starting a dose series has no page that runs the steps from the cube to the calls and the curves. The guide reads a dose series simulated from known curves, four cell types of 27 mice, places the doses on an axis, tests every cell type with dose_response, reads a test's table, compares each test's calls with the truth and draws the fitted curves with their bands over the true ones. docs/datasets.py prepares the series, the navigation gains a part for dose response, and the docs workflow checks the guide's outputs.
…cases The record keeps the analysis's options as given, so a value that JSON cannot write would stop dose_response only after every fit. The options are now checked before anything runs. Tests cover a cell type whose doses cannot place a slope, doses below a spline's second knot that leave its columns dependent, a design with one treated dose and no spline, a test named alone, numpy scalars among the options and a figure of one gene.
A masked group reached the figure's lookup of the cell type and ended in a bare StopIteration. The figure now refuses it with the standard message, looks the cell type up through the result, which says why a cell type has no curve, and joins the inventory of entry points that read arrays.
The table of each dose and the column adjusted across cell types rebuilt their tables from scratch, which checks every gene id of every table again; on 20,000 genes this took a tenth of dose_response's time. They are now built by take, with_columns and select, which keep the columns already checked. The properties of DoseResponseResult open without a colon, which the API reference read as a type.
The three scripts ran twice in one Modal container at the final code and printed the same rows both times. The Validation page gives the second runs' time and peak memory, and the timings add the benchmark: dose_response took 1.44 times limma_voom with the F-test of the dose groups on 20,000 genes, 24 profiles and 11 cell types.
dose_response failed on a series in molar units, whose indicator names such as dose_1e-09 the design quotes in backticks, and on a dose column such as "dose (nM)". Its test of a straight line was called slope, the name of the Hill slope in the simulator's truth and in the curves to come. The chapter said the curves' 95% bands hold the true curve, while on simulated series they hold it at 87% of responsive genes' doses. result.fits exposed analysis results whose tables had been removed, and the tests checked neither covariates in the design nor a dropped low or middle dose. The indicators are now looked up by their quoted names, and the test is linear_trend. DoseResponseResult.fit replaces fits; curve() takes genes by symbol and defaults to 100 doses along the axis; plot_dose_response takes genes by keyword and leaves out tick labels that would overlap. A check that only a falsified design check could trip is gone, with the raw copy of the options. The chapter, glossary and guide say that a band covers the spline fitted to the true dose-group means, and the Limits give the share of null cell types with a call at 2 replicates per dose, 6.4% to 18.3%, with its cause. validation/dose_response_calibration.py gains rows for the bands' coverage and for what the null calls share, the three dose scripts ran again twice on Modal with the same results, and new tests catch 16 deliberate bugs, among them a dose-group model without the design's other terms and contrasts placed on the wrong doses.
…ing line The Validation page said the dose tests' cost stays "under the target of 1.5 times" limma_voom, a target that readers of the page never saw; the comparison with limma_voom's time already gives the number its meaning. The validation script's description of R's contrasts broke a sentence after "tests". Neither change touches code that the run records digest.
… the CPU The docs job could not prepare the guide's dataset on GitHub's runner: the SHA-256 of simulated_dose_series_truth.json differed from the registry while the cube's matched. On CPUs with AVX-512, numpy computes float64 powers, exponentials and logarithms with its own vector code, whose last bits differ from those of glibc. Built on a host with AVX-512, 967 of the file's 72,009 floats differed from the build on a CPU without AVX-512, by at most 7e-16 relative, and the file had the runner's hash. The builder now writes the truth's floats to 6 significant digits, NaN kept, and the registry records the new hash. The rounded file is byte-identical with and without AVX-512, and DoseTruth reads it back: the parameters move by at most 5e-6 relative and the true curves on the guide's grid by at most 8e-6 log2 units, so the guide prints the same numbers. A test pins the written text.
This was referenced Nov 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Which genes respond to a dose in each cell type, and how? Until now a CytePool user answered with a factor of the dose groups or the spline columns of
DoseAxis, a design andf_testsbuilt by hand for every cell type, following the recipe in the Dose response chapter. Few studies get that far: in a survey of 130 published single-cell studies with three or more dose groups, 76 compared doses as unordered groups, each against a control, and the studies that fitted curves per gene did so in a single cell line, not per cell type within a tissue. Even the nine-dose TCDD liver series, with 2 or 3 mice per dose and 11 cell types, was tested with a Bayesian test of all dose groups at once, and with dose as a number only in a secondary likelihood-ratio test (Nault 2022; Nault 2023).The tests themselves are established. A joint F-test of a spline's terms tests for any effect of dose, and a test of its non-linear terms tests the departure from a straight line (Desquilbet and Mariotti 2010). The limma User's Guide fits a natural spline of time with a moderated F-test of its coefficients (section 9.6.2) but gives no dose variant. The genomic dose-response approach of the National Toxicology Program starts with a one-way ANOVA of the dose groups as a gate, which at least one gene must pass at a Benjamini-Hochberg FDR below 0.05 (NTP 2018), and Tukey's trend test regresses on log dose among its three dose scales (Tukey 1985). All of these tests are linear in their parameters, so moderated variances, voom's weights and donor random effects apply to them unchanged (Smyth 2004; Law 2014; Hoffman and Roussos 2021). Pseudobulk adds two problems. Cell types lose dose groups: stellate cells that fall to a tenth at the top dose keep no profile of 10 cells there, so their tests must leave that group out. And a spline and the dose groups fitted as two models per cell type, as first planned, give one gene two variances and would cost about 1.75 times
limma_voom(a design probe).The solution
dose_responsechecks the design withcheck_dose_design(PR 5), fits one moderated model of the dose groups in each cell type that supports tests, and runs every test as a contrast of that one fit, so that a gene's tests share its one moderated variance:splinenonlinearitylinear_trenddose_groupslimma_voom(the default),limma_trendordream_voomfits the design's own terms plus an intercept for the vehicle and an indicator for each treated dose, such asdose_0.03. The analyses' other options pass through (robust,prior_weights,quality_weights,contrast_standard_errorsand the rest).DoseAxis.spline_basis, so knots and curves compare across cell types. The default size is 2 columns when the axis holds 4 doses or more, and 1 otherwise.tests_not_run, with the reason in the record; when two tests ask the same question, as a spline of one column and the dose groups do at three dose groups, the default runs the first ofspline,dose_groupsandlinear_trend. Descriptive and refused cell types are not fitted, with the check's reason in the summary.summaryper cell type, one table per test withadjusted_p_value(Benjamini-Hochberg within the cell type) andadjusted_p_value_across_groups(over the cell types that ran the test), andeach_dose, each dose against the vehicle.curve()gives the fitted spline as a change from the vehicle's mean, by default at 100 doses evenly spaced along the axis over the doses a cell type compares, for genes named by feature id or symbol, with a pointwise 95% band on the t quantile of the gene's moderated degrees of freedom, as limma'stopTable(confint = TRUE)forms intervals; the standard errors use each gene's own covariance under voom's weights. The band covers the spline that least squares fits to the true means of the dose groups: on simulated series it held the true change at the doses for 95% of flat genes' doses but 87% of responsive genes', 80% for bells and 81% for steps, against 95% for each dose's own interval, soplot_dose_responsedraws each dose's estimate beside the curve and its band, the vehicle left of an axis break.fit()computes a cell type's fit again.The Dose response chapter gains "Which genes respond to dose" and "Curves with a band", with defaults, limits and checks; "Choosing a method" gains "Which genes respond to dose in each cell type?"; the glossary gains "Linear trend test", "Non-linearity test" and "Confidence band (pointwise)"; the bibliography gains NTP 2018 and Tukey 1985; the API reference gains a "Dose responses" page. A new guide, "Dose response per cell type", runs on a simulated series (the new dataset
simulated_dose_series) of four cell types: the spline finds about 6 in 10 of the hepatocytes' responses and calls 8 of their 1,491 flat genes, and the figure draws true curves over the fitted ones. The dataset's truth is written to 6 significant digits: numpy's float64 powers and logarithms differ in their last bits between CPUs with and without AVX-512, and at full precision the file's SHA-256 would depend on the CPU.Why it matters
The PR is the epic's first answer to its first question, which genes respond in each cell type, and the fit that later PRs extend. PR 8 adds Williams, Dunnett and MCP-Mod contrasts to the same fit and reads the dose-group estimates and their covariance from
_dose_group_estimates; PR 9 gates ontable(g, "spline")and adds its gate rows tovalidation/dose_response_calibration.py; PR 10 adds model curves and estimates for descriptive cell types; PR 19 runsdose_responseon pathway scores. The one fit costs 1.44 timeslimma_voomwith the F-test of the dose groups (9.19 s against 6.37 s on 20,000 genes x 24 profiles x 11 cell types), under the target of 1.5 times, where two fits would cost about 1.75 times. On simulated series of the vehicle and 8 doses, the default spline of 2 columns found more responsive genes than the dose-group test, which is a spline through every group: 0.466 against 0.406 with 3 replicates, 0.386 against 0.331 with 2.Validation
Each script ran twice in one Modal container (4 CPU, 8 GiB) at the final code and printed the same rows both times.
validation/dose_response_tests.pyprinted 35 checks: 21 pass, 13 information and 1 deliberate difference; the second run took 1 min 10 s with a peak memory of 1,299 MiB. It simulates seven dose series (seeds 20261208 to 20261215), among them a cell type that loses its top dose and a sample of the dose below, the vehicle on the axis, a linear axis,"~ sex"and donors crossed with dose. R sees only the counts, cells, doses and settings, places the knots withns(), builds its own contrasts and runs limma'svoom,lmFit,contrasts.fit,eBayes,topTableandclassifyTestsF. Reference: R 4.4.3, limma 3.62.1, edgeR 4.4.0, Hmisc 5.3-1, rms 8.2-1, variancePartition 1.36.2, lme4 1.1-38, dreamlet 1.4.1, DRomics 2.6.3, DESeq2 1.46.0, muscat 1.20.0.nonEstimable,filterByExprclassifyTestsFlm.wfitandeBayes;classifyTestsFpf,p.adjusttopTable;lm.wfittopTable;lm.wfittopTable(confint = TRUE);lm.wfitols,anovadream,topTabletopTabletopTable's p-values of dream's F-tests, on the first gene's dfpfns()rows against the restricted cubic rows, limma's standard errors, 19 testsclassifyTestsFitemselectpbDStopTable's over 474limma's approximate standard errors (
contrast_standard_errors="limma") combine each gene's weighted standard errors with the correlation of the unweighted design, so limma's F of a spline depends on the spline's columns: on R'sns()rows and on the restricted cubic rows of the same spline its F-statistics differed by up to 52%. That option is compared on the rows CytePool tests; the exact default and limma-trend, which do not depend on the basis, are compared on R's own rows.validation/dose_response_calibration.pyprinted 53 information rows; the second run took 5 min 50 s with a peak memory of 435 MiB. It runsdose_responsewith its defaults on a grid of designs: 4 and 9 dose groups, 2 and 3 replicates, a median of 100 cells per profile, 2 cell types of 1,000 genes, 1,000 null datasets and 10 with a fifth of the genes responsive per design cell, seedsnumpy.random.default_rng([20261201, kind, design_cell, dataset]); and, for the bands, 12 datasets of 9 dose groups x 3 with 3,000 cells per profile, so that counting compresses no change, and 600 genes of which 30% respond.The null p-values lie close to uniform (shares below 0.05 of 0.049 to 0.053, standard error 0.0002; Kolmogorov-Smirnov distances 0.004 to 0.009), and the mean false discovery proportion at 5% FDR was 0.028 to 0.062. The far tail is heavier with 2 replicates: a cell type of 1,000 null genes had a call in 6.4% to 18.3% of datasets, depending on the test and the dose groups, against 4.6% to 7.5% with 3. The calls fall on genes whose true variance between replicates is well above most genes': 75% to 81% of the dose-group test's null calls were among the tenth of genes with the largest true SD. With 4 residual degrees of freedom (4 groups x 2), moderation pulls each gene's variance most of the way to a common prior, which understates the variance of such genes; in 7.4% of those cell types the prior's degrees of freedom were infinite, and 66% of them had a call, against 23% of the others.
limma_voom's own F-test of a factor of the dose groups gives about the same share as the dose-group test (20.3% in 150 datasets at 4 groups and 2 replicates, a local probe, against 18.3%), robust moderation 25% and limma-trend 14.7%, so the excess comes from the moderation, not from the contrasts; the chapter's Limits say so, and PR 9's gate has to allow for it.At the 8 treated doses of the 12 band datasets, the bands held the true change at 0.951 of flat genes' doses and 0.872 of responsive genes' (0.928 for exponential curves down to 0.812 for steps and 0.798 for bells), while each dose's own 95% interval held it at 0.950 and 0.951; against their own target, the spline fitted to the true dose-group means, the bands of responsive genes held at 0.955. The band's standard errors are right, and what it misses is the spline's distance from a response that turns or jumps more sharply than two columns can follow, which the chapter now says. The spline's power by its columns at 9 dose groups, 2 and 3 replicates: 1 column 0.351 and 0.432, 2 columns 0.386 and 0.466, 3 columns 0.385 and 0.460, 7 (every group) 0.331 and 0.406, standard errors about 0.01; this is the evidence for the default of 2.
validation/plotting.pyprinted 30 checks (27 pass, 2 information, 1 deliberate difference); the second run took 29.0 s with a peak memory of 883 MiB. No R function drawsplot_dose_response, so R computes what it draws on a dose series of the vehicle and 5 half-log doses x 3 (seed 20261217): the 100 doses of the curves (1.6e-15), limma-trend's curves of 3 genes (1.1e-14), their 95% bands fromtopTable(confint = TRUE)(1.7e-14) and each dose'slogFC(1.1e-14), tolerance 1e-10.Modal runs, each an ephemeral app through the reference harness:
dose_response_tests.py, 6 runscpmmasked, dream's F of one contrast, then two real differencesdose_response_tests.pydose_response_tests.py --write-fixturestests/data/dose_response_tests/limma.jsonplotting.pydose_response_calibration.pydose_response_tests.py --benchmarklimma_voom, over target: the tables were checked again on every rebuilddose_response_tests.py --benchmarkdose_response_tests.py --write-fixtureslinear_trendplotting.pyran againplotting.pytwice in one containersimulated_dose_series_truth.jsondocs/validation_runs.tomlrecords the digests of the code that the final runs checked.docs/changed_since_run.pyalso lists 11 scripts whose checked code changed without a change of behaviour, so they were not run again: activity, combined_p_values_and_replication, gene_set_files_and_ora, gsea, pathway_scores, pca_and_dot_plots, sumrank and variance_partitioning importcytepool.plotting, which gains a function; dose_axes, dose_design and simulate_dose_series checkcytepool/_doses.py, which gainsdose_group_column, and dose_design alsocytepool/dose_designs.py, whose refusal of dose terms no longer namescheck_dose_design. Twenty further scripts were stale on master already.Tests
tests/test_dose_responses.py(51 tests, about 3 s) checks every test, linear trend, each-dose table, curve and band against limma 3.62.1 in R through the fixture, with R's prior degrees of freedom, posterior variances, total degrees of freedom and log-odds and rms's sums of squares, the default F under voom's weights against each gene'slm.wfit, and the spline without weights againstnumpy.linalg.lstsqon the samples; a saturated spline against the dose-group test; a dropped dose group againstlimma_voomon the remaining doses; the adjustment across cell types againstadjust_p_valueson the stacked p-values; on the axis, the curve as a change from the vehicle's mean against least squares on the samples; infinite prior df giving bands on limma's pooled df; the dream route with the linear trend's weights (-0.1, 0.1, 0.3) derived by hand; a design of"~ sex"againstlimma_voomon a dose-group model written by hand; a cell type that loses its lowest or a middle dose against least squares on its samples; doses a millionth apart as two dose groups; doses in M and a dose column named"dose (nM)", which the design quotes, against the same series in nM; genes by symbol; curves in the order of the genes and doses given, and on 100 doses by default; which tests run at 2 to 9 dose groups on both placements, and which repeat each other; descriptive and refused cell types; options passed through and refused;"~ 0","~ sex"aliasing a dose, NaN and negative doses; results invariant to sample order, dose units and threads; pickling, the record and the chapter's example.tests/test_plotting.pychecks the figure's artists againstcurve()andeach_doseon both placements, genes named by symbol, tick labels that would overlap on a linear axis, and its refusals;tests/test_masked_inputs.pyclassifies the new entry points. Coverage with numba's JIT off, measured again at the final commit over the 241 dose tests, is 100% of statements and branches forcytepool/dose_responses.py(559 statements, 172 branches) andcytepool/_doses.py(48 and 10), andplot_dose_responseand its tick-label helper are fully covered. The full suite passes (5,723 tests; 18 skipped, since zstd needs Python 3.14); ruff, the format check and mypy are clean; the guide's saved outputs are current.Each of 16 deliberate bugs, reverted after its run, makes at least one test fail: dose groups weighted equally (6 tests), the slope's line through a vehicle in its own group (8), the non-linearity test including the spline's first column (5), dropped dose groups kept (19), the adjustment across cell types done within each (3), the band on the residual df or the 0.95 quantile (4 each), the treated jump left out of the curve (4), knots placed on each cell type's doses (10), descriptive cell types fitted (2), the priority of repeated tests reversed (3), and, caught by one test each, no intercept under
"~ 0", limma's standard errors taken as one covariance for every gene, each gene's covariance ignored under voom, and doses between the vehicle and the lowest dose let into the curve. The check that the fit used the check's samples was removed after review, with its test, since both steps choose samples with the same code.Sixteen more, run after the review, each make a new or extended test fail: the dose-group model without the design's other terms (the
"~ sex"test), contrasts placed on the first indicator columns rather than the kept doses' own (the lost-dose tests, 5 in all), indicator names of 6 significant digits (the close doses), the analysis's member columns kept in the F tables and a table name given as the cell type not pointed toname=(the read-back test), the curve's genes sorted (2) and its doses sorted and deduplicated (the order test), the each-dose gene ids repeated rather than tiled,df_priorholding the residual df andn_dose_groups_testedwithout the vehicle (the tests against R), the slope's unit always per log10 dose (the dream route), indicator names looked up without the design's quoting (the molar doses), symbols not looked up (the symbol test), the default doses spaced in dose rather than along the axis (the default curve), the figure's tick labels never left out, and its points looked up by the name given rather than the feature id (the figure tests). Eight of the first sixteen, run again on the final code, were each caught again: equal group weights (6 tests), the adjustment across cell types done within each (3), the 0.95 quantile (4), the non-linearity test with the first column (4), the priority reversed (2), no intercept under"~ 0"(1), doses between the vehicle and the lowest dose let into the curve (2) and each gene's covariance ignored under voom (1).Decisions
limma_voomagainst about 1.75contrasts.fittests=None, the first ofspline,dose_groupsandlinear_trendwhen two test the same hypothesisdose_groupsfirstsplineis the test to read first and PR 9's default gate, so it exists in every cell type that supports a splinetopTable(confint = TRUE); dream's fit does not keep the df of each point of a curvecurve()only within the doses a cell type comparesanalysis_settingsanalyse_each_study's convention; one record of each optionlinear_trendslopenonlinearity;slopestays the Hill slope of the simulator's truth and of PR 12fit(group)computes a cell type's fit againfitsof the analysis's results without their tablesgroup_samples, so the state could only be reached by falsifying the check; a contrast the fit cannot estimate still fails loudly inresult.tablecurve()on 100 doses along the axis by default, genes by feature id or symbolatrequired; feature ids onlyLeft for later: the second copy of
differential_expression's adjustment across groups indose_responses.py, to be shared whendifferential_expression.pynext changes, since changing it now would change the run record of every script that imports it. Trimmed, and where it went: DRomics' rank-dose tests, since DRomics runs unweighted limma on values the user supplies and no test insidedose_response's pipeline reproduces its selections (an information row instead, where its limma path called 167 genes in the depleted cell type, 66 of them flat in truth); each dose against the vehicle with control over a gene's doses to PR 8 (Dunnett); a public accessor for the dose groups' means and covariance to PR 8; estimates for descriptive cell types and model curves to PR 10; bands fordream_voomfits to PR 10 or 21; simultaneous bands out. Pre-existing, outside this PR: withNUMBA_DISABLE_JIT=1,t_isf_scalarraises OverflowError where the compiled code returns infinity, on series without biological variation, so one test is deselected in the coverage pass.