Repository navigation
Add the fused_quant graph passes to backends/transforms/fused_quant (#24720) - #24720
Merged
Merged
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24720
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 017efc0 with merge base 558c071 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
mcremon-meta
had a problem deploying
to
cadence
November 6, 2026 05:52 — with 
GitHub Actions
Failure
GitHub Actions
Failure
mcremon-meta
had a problem deploying
to
cadence
November 6, 2026 05:52 — with
GitHub Actions
Failure
GitHub Actions
Failure
mcremon-meta
had a problem deploying
to
cadence
November 6, 2026 05:52 — with
GitHub Actions
Failure
GitHub Actions
Failure
mcremon-meta
had a problem deploying
to
cadence
November 6, 2026 05:52 — with
GitHub Actions
Failure
GitHub Actions
Failure
Contributor
|
@mcremon-meta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D129247543. |
digantdesai
approved these changes
Nov 6, 2026
digantdesai
left a comment
digantdesai
left a comment
Contributor
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
This PR needs a
|
meta-codesync Bot
pushed a commit
that referenced
this pull request
Nov 6, 2026
…24720) Summary: Pull Request resolved: #24720 Adds the passes that run on the exported graph before quantization and the optimization passes that run on the fused_quant graph, with their tests. Under `backends/transforms/fused_quant/pre_quantize_passes/`: - `fold_batch_norm`, `replace_batch_norm_with_conv` - `fold_input_free_subgraphs` - `fuse_add_softmax_into_masked_softmax` - `replace_adaptive_avg_pool_with_avg_pool` Under `backends/transforms/fused_quant/optimization_passes/`: - Layout: `to_channels_first`, `to_channels_last`, `to_convolution` - Folding into and reshaping linears: `fuse_add_into_linear`, `fuse_mul_into_linear`, `split_linear_at_slices`, `canonicalize_linear_input_rank`, `constant_fold` - Removing or merging quantize/dequantize steps: `quant_absorption`, `replace_dequant_quant_with_requantize`, `prequantize_embedding` - Constant and elementwise rewrites: `fold_scalar_mul`, `hoist_activations`, `replace_sub_constant_with_add`, `sink_constant_cat` Reviewed By: digantdesai Differential Revision: D129247543
meta-codesync
Bot
force-pushed
the
export-D129247543
branch
from
November 6, 2026 06:04
09c8f39 to
472b58c
Compare
meta-codesync
Bot
changed the title
meta-codesync Bot
pushed a commit
that referenced
this pull request
Nov 6, 2026
…24720) Summary: Pull Request resolved: #24720 Adds the passes that run on the exported graph before quantization and the optimization passes that run on the fused_quant graph, with their tests. Under `backends/transforms/fused_quant/pre_quantize_passes/`: - `fold_batch_norm`, `replace_batch_norm_with_conv` - `fold_input_free_subgraphs` - `fuse_add_softmax_into_masked_softmax` - `replace_adaptive_avg_pool_with_avg_pool` Under `backends/transforms/fused_quant/optimization_passes/`: - Layout: `to_channels_first`, `to_channels_last`, `to_convolution` - Folding into and reshaping linears: `fuse_add_into_linear`, `fuse_mul_into_linear`, `split_linear_at_slices`, `canonicalize_linear_input_rank`, `constant_fold` - Removing or merging quantize/dequantize steps: `quant_absorption`, `replace_dequant_quant_with_requantize`, `prequantize_embedding` - Constant and elementwise rewrites: `fold_scalar_mul`, `hoist_activations`, `replace_sub_constant_with_add`, `sink_constant_cat` Reviewed By: digantdesai Differential Revision: D129247543
meta-codesync
Bot
force-pushed
the
export-D129247543
branch
from
November 6, 2026 16:12
472b58c to
b019630
Compare
…24720) Summary: Pull Request resolved: #24720 Adds the passes that run on the exported graph before quantization and the optimization passes that run on the fused_quant graph, with their tests. Under `backends/transforms/fused_quant/pre_quantize_passes/`: - `fold_batch_norm`, `replace_batch_norm_with_conv` - `fold_input_free_subgraphs` - `fuse_add_softmax_into_masked_softmax` - `replace_adaptive_avg_pool_with_avg_pool` Under `backends/transforms/fused_quant/optimization_passes/`: - Layout: `to_channels_first`, `to_channels_last`, `to_convolution` - Folding into and reshaping linears: `fuse_add_into_linear`, `fuse_mul_into_linear`, `split_linear_at_slices`, `canonicalize_linear_input_rank`, `constant_fold` - Removing or merging quantize/dequantize steps: `quant_absorption`, `replace_dequant_quant_with_requantize`, `prequantize_embedding` - Constant and elementwise rewrites: `fold_scalar_mul`, `hoist_activations`, `replace_sub_constant_with_add`, `sink_constant_cat` Reviewed By: digantdesai Differential Revision: D129247543
meta-codesync
Bot
force-pushed
the
export-D129247543
branch
from
November 6, 2026 16:16
b019630 to
017efc0
Compare
meta-codesync Bot
pushed a commit
that referenced
this pull request
Nov 8, 2026
Summary: This should have been part of #24720, which added `QuantAbsorptionPass` with the other fused_quant graph passes. It was left out of that PR by mistake. By default, `QuantAbsorptionPass` now only runs the rewrite that is never less accurate. The docstring explains why the other two modes are opt-in. - Output side (unchanged, always on): the producer writes its output directly in the downstream quantize's qparams. The value still ends up on that grid, rounded once instead of twice, so each element is at least as close to the float result. It isn't bit-exact with the q/dq graph, though. - Input side (now opt-in, `absorb_into_inputs`): the consumer reads values on the dequantize's grid instead of the quantize's. Dropping the second rounding is more accurate on average, but not for every value. - Forked dequantize (already opt-in, `absorb_with_fork`): its float users see values on the quantize's grid instead of the producer's, which can be coarser. Only the docstring changes. With input-side absorption off, a pair that only the input side would remove is left to `ReplaceDequantQuantWithRequantize`, which turns it into a `fused_quant.requantize`. Differential Revision: D130466673
meta-codesync Bot
pushed a commit
that referenced
this pull request
Nov 9, 2026
Summary: This should have been part of #24720, which added `QuantAbsorptionPass` with the other fused_quant graph passes. It was left out of that PR by mistake. By default, `QuantAbsorptionPass` now only runs the rewrite that is never less accurate. The docstring explains why the other two modes are opt-in. - Output side (unchanged, always on): the producer writes its output directly in the downstream quantize's qparams. The value still ends up on that grid, rounded once instead of twice, so each element is at least as close to the float result. It isn't bit-exact with the q/dq graph, though. - Input side (now opt-in, `absorb_into_inputs`): the consumer reads values on the dequantize's grid instead of the quantize's. Dropping the second rounding is more accurate on average, but not for every value. - Forked dequantize (already opt-in, `absorb_with_fork`): its float users see values on the quantize's grid instead of the producer's, which can be coarser. Only the docstring changes. With input-side absorption off, a pair that only the input side would remove is left to `ReplaceDequantQuantWithRequantize`, which turns it into a `fused_quant.requantize`. Reviewed By: DrJessop Differential Revision: D130466673
meta-codesync Bot
pushed a commit
that referenced
this pull request
Nov 9, 2026
Summary: This should have been part of #24720, which added `QuantAbsorptionPass` with the other fused_quant graph passes. It was left out of that PR by mistake. By default, `QuantAbsorptionPass` now only runs the rewrite that is never less accurate. The docstring explains why the other two modes are opt-in. - Output side (unchanged, always on): the producer writes its output directly in the downstream quantize's qparams. The value still ends up on that grid, rounded once instead of twice, so each element is at least as close to the float result. It isn't bit-exact with the q/dq graph, though. - Input side (now opt-in, `absorb_into_inputs`): the consumer reads values on the dequantize's grid instead of the quantize's. Dropping the second rounding is more accurate on average, but not for every value. - Forked dequantize (already opt-in, `absorb_with_fork`): its float users see values on the quantize's grid instead of the producer's, which can be coarser. Only the docstring changes. With input-side absorption off, a pair that only the input side would remove is left to `ReplaceDequantQuantWithRequantize`, which turns it into a `fused_quant.requantize`. Reviewed By: DrJessop Differential Revision: D130466673
meta-codesync Bot
pushed a commit
that referenced
this pull request
Nov 9, 2026
Summary: This should have been part of #24720, which added `QuantAbsorptionPass` with the other fused_quant graph passes. It was left out of that PR by mistake. By default, `QuantAbsorptionPass` now only runs the rewrite that is never less accurate. The docstring explains why the other two modes are opt-in. - Output side (unchanged, always on): the producer writes its output directly in the downstream quantize's qparams. The value still ends up on that grid, rounded once instead of twice, so each element is at least as close to the float result. It isn't bit-exact with the q/dq graph, though. - Input side (now opt-in, `absorb_into_inputs`): the consumer reads values on the dequantize's grid instead of the quantize's. Dropping the second rounding is more accurate on average, but not for every value. - Forked dequantize (already opt-in, `absorb_with_fork`): its float users see values on the quantize's grid instead of the producer's, which can be coarser. Only the docstring changes. With input-side absorption off, a pair that only the input side would remove is left to `ReplaceDequantQuantWithRequantize`, which turns it into a `fused_quant.requantize`. Reviewed By: DrJessop Differential Revision: D130466673 Pull Request resolved: #24780
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Adds the passes that run on the exported graph before quantization and the
optimization passes that run on the fused_quant graph, with their tests.
Under
backends/transforms/fused_quant/pre_quantize_passes/:fold_batch_norm,replace_batch_norm_with_convfold_input_free_subgraphsfuse_add_softmax_into_masked_softmaxreplace_adaptive_avg_pool_with_avg_poolUnder
backends/transforms/fused_quant/optimization_passes/:to_channels_first,to_channels_last,to_convolutionfuse_add_into_linear,fuse_mul_into_linear,split_linear_at_slices,canonicalize_linear_input_rank,constant_foldquant_absorption,replace_dequant_quant_with_requantize,prequantize_embeddingfold_scalar_mul,hoist_activations,replace_sub_constant_with_add,sink_constant_catReviewed By: digantdesai
Differential Revision: D129247543