Skip to content

[python] Add regression coverage for sparse IVF-PQ searches - #10726

Merged
JingsongLi merged 4 commits into
apache:masterfrom
QuakeWang:ivfpq-search-fix
Oct 26, 2026
Merged

JingsongLi merged 4 commits into
apache:masterfrom
QuakeWang:ivfpq-search-fix

Conversation

@QuakeWang

@QuakeWang QuakeWang commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Purpose

The scalar FFI rejected IVF-PQ results shorter than K when filters included live NULL vectors. #10782 already contains the reader fix; this PR adds real-SDK regression coverage.

Cover sparse, NULL-only and empty filters across three metrics and single/batch queries, plus Python table searches with deletion vectors. The final diff contains tests only and does not require the Rust SDK.

Tests

  • Related suites: 435 passed, 29 skipped; prior CI failure cases: 58 passed.
  • All 29 new cases pass with Rust imports blocked and with an unusable Rust SDK stub present.
  • Restoring the pre-fix search method reproduces the length mismatch in 13 cases.
  • Flake8, license headers, Python 3.6 syntax, diff checks and 9 packaging tests passed.
  • Full Native CI was not run locally.

Use the existing Vindex batch API for single queries and singleton batches
so filters containing live NULL vectors can return fewer hits than requested.
Keep Python execution independent of the Native backend and preserve the
existing score conversion and invalid-ID filtering.

Add real SDK and deletion-vector regressions for sparse and empty results.

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
Sync merged CI baseline fixes and the current Native search API.
Keep the IVF-PQ fix and its regression coverage within the original scope.

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
Sync upstream CI baseline and Rust API alignment fixes.
Preserve the IVF-PQ reader change and existing regression scope.

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
Keep the reader and Native test identical to upstream after apache#10782.
Retain real-SDK sparse-filter and Python deletion-vector regression coverage.

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
@QuakeWang QuakeWang changed the title [python] Fix IVF-PQ searches with sparse filters [python] Add regression coverage for sparse IVF-PQ searches Oct 24, 2026

@Akash3121 Akash3121 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit ae2ea66 into apache:master Oct 26, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants