You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 5cd7941
Browse filesBrowse the repository at this point in the historyBrowse files
perf(kernel): cap the pandas-free conversion by cost, not rows
to_pylist() costs more per value than pandas, so with only a 128-row cap
wide or string/decimal/date-heavy results were slower than the pandas
path (16 lineitem columns x 128 rows 0.92-1.15x, 64 strings x 128 rows
up to 2.2x, 32 dates x 128 rows up to 4.2x). A result now skips pandas
only while rows times the sum of per-type weights stays within 1024,
which kept every admitted shape at most 0.67x of the pandas CPU on
pandas 1.3-3.1.
date64, time32 and time64 now take the pandas path, since to_pylist()
rounds pre-1970 date64 values down and wraps out-of-range times where
pandas raises. A to_pylist() error (e.g. a date past 9999-12-31) also
falls back to pandas, so errors match the pandas path.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Copy file name to clipboardExpand all lines: CHANGELOG.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@
4
4
- Fix the kernel backend ignoring `use_cloud_fetch=False`, so complete results can be fetched without cloud-storage downloads.
5
5
- Fix inline parameters (`use_inline_params=True`) so a sequence directly after `IN` renders as a list again: `x IN %(ids)s` renders as `x IN (1,2,3)` instead of `x IN ARRAY(1,2,3)`, which Spark rejects. Inside the list, tuples render as rows for a multi-column `IN`, and a `set` or `frozenset` is accepted again. Sequences anywhere else still render as `ARRAY(...)` ([#629](https://github.com/databricks/databricks-sql-python/issues/629))
6
6
- Speed up Thrift response handling: read replies in 1 MiB chunks and decode them with Thrift's C-accelerated protocol, and stop stringifying every request/response for the default no-op debug logger. Wide result schemas, metadata calls with many rows, and inline Arrow results benefit most.
7
-
- Speed up fetching small results on the kernel backend (`use_kernel=True`): results of up to 128 rows are converted to rows without building a pandas DataFrame when that gives the same values, which cuts client CPU for a one-row query by about 40%.
7
+
- Speed up fetching small results on the kernel backend (`use_kernel=True`): they are converted to rows without building a pandas DataFrame when that is cheaper and gives the same values, which cut client CPU per query by 47% in a local benchmark of 200 concurrent sessions running one-row queries against a mock server.
8
8
9
9
# 4.9.0 (2026-11-04)
10
10
- Upgrade Databricks SQL Kernel to 1.4.0, which keeps Lakehouse Real-Time session affinity and flushes shared telemetry only after the final session closes.
0 commit comments