Skip to content

fix: do not drop GROUP BY keys using nullable UNIQUE constraints - #26220

Open
buraksenn wants to merge 3 commits into
apache:mainfrom
buraksenn:fix-nullable-unique-group-by
Open

buraksenn wants to merge 3 commits into
apache:mainfrom
buraksenn:fix-nullable-unique-group-by

Conversation

@buraksenn

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

A nullable UNIQUE column permits multiple NULL rows, so it cannot safely
determine other grouping columns. Dropping those columns can merge distinct
groups and return incorrect results.

What changes are included in this PR?

  • Only rewrite GROUP BY expressions using dependencies valid across NULLs.
  • Apply the same check to DISTINCT elimination to preserve correct results.
  • Add regression tests and an upgrade guide note.

What is the testing strategy for this PR?

SLT and plan-builder tests cover nullable and composite UNIQUE keys,
DISTINCT after GROUP BY, and valid PRIMARY KEY / NOT NULL UNIQUE cases.
The DISTINCT regressions fail without the fix.

Are there any user-facing changes?

Yes: incorrect results are fixed

…olumn

A nullable UNIQUE column permits multiple NULL rows that may differ in
other columns, so it does not functionally determine them once NULLs are
grouped together. Only use dependencies that are valid across NULLs when
removing redundant GROUP BY expressions.
   Only eliminate DISTINCT when the functional dependency remains valid
   across NULLs, preventing duplicate NULL values after GROUP BY.

   Add SQL regressions for grouped projections, aliases, subqueries,
   outer counts, and composite keys. Preserve NOT NULL UNIQUE
optimizations.
@github-actions github-actions Bot added documentation Improvements or additions to documentation logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) common Related to common crate labels Oct 12, 2026
@codecov-commenter

codecov-commenter commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.76923% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.93%. Comparing base (3ee3e39) to head (ad3dc85).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/expr/src/logical_plan/builder.rs 84.61% 0 Missing and 6 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26220      +/-   ##
==========================================
- Coverage   82.93%   82.93%   -0.01%     
==========================================
  Files        1154     1154              
  Lines      458002   458049      +47     
  Branches   458002   458049      +47     
==========================================
+ Hits       379856   379888      +32     
- Misses      55123    55131       +8     
- Partials    23023    23030       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jayzhan211 jayzhan211 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @buraksenn , LGTM

@jayzhan211
jayzhan211 enabled auto-merge October 23, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to common crate documentation Improvements or additions to documentation logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect results: GROUP BY column dropped based on a nullable UNIQUE constraint

3 participants