[dependabot]

Bumps polars from 1.44.2 to 2.1.0.

Release notes

Sourced from polars's releases.

Python Polars 2.1.0

💥 Breaking changes

• Disallow mixed string <=> numeric arithmetic in eager (#30071)

🚀 Performance improvements

• Limit the memory of buckets sorted ahead in the sort flush (#30297)
• Make file readers wait for spills while over the memory budget (#30296)
• Count unspilled frames in the memory estimate right away (#30295)
• Simplify more boolean expressions and drop redundant null checks (#30293)
• Wait for spills while over the out-of-core memory budget (#30294)
• Lower the default out-of-core prefetch fraction to 0.5 (#30292)
• Aggregate unique and n_unique in the streaming group-by itself (#30281)
• Return freed memory to the OS right away while out-of-core data is spilled (#30280)
• Gather List(Array) and Array columns without per-row slices (#30278)
• Evaluate the streaming window node out of core (#30282)
• Avoid deep copies when building a DataFrame from flat dataclasses (#30275)
• Sort the window partitions by each order key instead of their row encoding (#30277)
• Scan the haystack for a single needle in is_in (#30276)
• Prefer fixed-width columns when only the row count is needed (#30274)
• Native gather for List arrays (#30019)
• Evaluate windows in filter predicates in the streaming window node (#30272)
• Don't busy-wait in RAYON.block_on (#30259)
• Encode nested Parquet levels directly from the level buffers (#30251)
• Reduce memory of CSV scans on wide rows (#30165)
• Avoid quadratic JSON serialization of nested lists and maps (#30203)
• Skip the gather when exploding lists without nulls (#30234)
• Speed up writing nested Parquet columns (#30245)
• Don't keep whole parquet string dictionaries alive for a few selected rows (#30248)
• Set default spill compression level to 1 (#30237)
• Reuse zstd contexts in compression and avoid memcpys (#30233)
• Remove repeated and scalar ordering keys before the sort rewrites (#30231)
• Remove repeated keys from by's (#30095)
• Keep only one guessed unique key per join order leaf (#30191)
• Don't let extra join keys shrink an estimate once one key is unique (#30190)
• Reduce binary size of asof join with by keys (#30192)
• Reduce binary size of SQL visitors and parallel group-by collect (#30171)
• Push as-of join key filters to the right input (#30123)
• Drop null as-of right keys with a parallel filter before the join node (#30125)
• Keep as-of nearest join state small (#30126)
• Explode lists in group_by aggregations without a per-group apply (#30020)
• Streaming OOC group-by with shuffles (#29911)
• Drop scalar ordering keys in the optimizer (#30063)
• Merge union inputs that scan files of the same kind into one multi-file scan (#30056)
• Use the streaming scalar window for std, var and combined reductions in over (#30066)
• Limit the bytes of local CSV and NDJSON files read at once in multi-file scans (#30055)
• Check the matched flag of join keys in the table entry before the bitset (#30067)
• Track matched build rows of all join tables in a bitset (#30050)
• Add dedicated list gather kernels (#30049)

... (truncated)

Commits

• 7f902bd Python Polars 2.1.0 (#30124)
• dfb63c7 fix(rust): Include rolling index columns in expression metadata (#29245)
• a26073c perf: Limit the memory of buckets sorted ahead in the sort flush (#30297)
• 8d11086 test: Fix allocator-dependent failure in masked zero-copy to_numpy test (#3...
• a9a998b perf: Make file readers wait for spills while over the memory budget (#30296)
• 8b57b77 perf: Count unspilled frames in the memory estimate right away (#30295)
• 720d744 refactor: Simplify the out-of-core decay state and tests (#30283)
• f7a3227 perf: Simplify more boolean expressions and drop redundant null checks (#30293)
• 48de7ae perf: Wait for spills while over the out-of-core memory budget (#30294)
• 28cfd43 fix: Compare Array(Struct) rows field by field (#30290)
• Additional commits viewable in compare view

[ebyhr]

Looks like a transient infra failure and not a polars issue:

--------------------

  14 |     # limitations under the License.

  15 |     

  16 | >>> FROM apache/hive:4.2.1

  17 |     

  18 |     USER root

--------------------

target hive: failed to solve: apache/hive:4.2.1: failed to resolve source metadata for docker.io/apache/hive:4.2.1: unexpected status from HEAD request to https://registry-1.docker.io/v2/apache/hive/manifests/4.2.1: 502 Bad Gateway

make: *** [Makefile:114: test-integration-setup] Error 1

[Fokko]

@dependabot rebase

[Fokko]

LGTM!