Skip to content

A long float list, or a float64 array in a named parameter, crosses as one buffer - #107

Merged
tae898 merged 1 commit into
mainfrom
fix/106-query-vector-buffer
Oct 28, 2026
Merged

tae898 merged 1 commit into
mainfrom
fix/106-query-vector-buffer

Conversation

@tae898

@tae898 tae898 commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

Closes #106

Draft: a timed A/B (ab.py) of the two input forms follows before this leaves draft. The numbers below are untimed encode costs on the laptop (cores 14,15, 2000 repetitions each), information and not a timing claim.

Reasoning

A list is not always a vector, so the change takes only what is unambiguous and leaves every other value on the path it had. The engine must read exactly the object it read before, not an approximation of it. That rules out turning a list into float32 on the JVM build (float[] is a different type from ArrayList<Double>, and float32 rounds doubles), and it rules out any change that depends on guessing from the SQL text whether a parameter is a vector.

What changed

JVM build (src/arcadedb/type_conversion.py, convert_python_to_java):

  • A list or tuple of 32 or more Python floats and nothing else (set(map(type, value)) == {float}) crosses as one double[], and Java boxes it with Arrays.stream(...).boxed().toList() into an ArrayList. That is the same ArrayList<Double>, value for value (NaN, infinities and -0.0 included), that the element-wise path built.
  • A one-dimensional native-order float64 array of 32 or more numbers takes the same route instead of value.tolist() and the boxed list.

Native build (src/arcadedb/_native/codec.py, native/src/main/java/org/humemai/nativelib/Json.java):

  • A list of 64 or more Python floats and nothing else, for which double -> float32 -> double is the identity and every number is finite, goes in the existing float32 side buffer under a new tag BL. Json.decodeTag reads it as a List<Double> (the float widens to the same double), the object the JSON text became. A list from a float32 model (ndarray.tolist()) passes; a list of other doubles, NaN or an infinity fails the check and stays JSON text.
  • _prep returns a list of plain scalars as it is instead of rebuilding it element by element (69 us for 1536 numbers, no change in the text).

The docs (databases-and-queries.adoc, "A query vector") name np.asarray(v, dtype=np.float32) as the fast path and state which forms are rounded to float32.

Encode cost, 1536-d, same laptop and entry points as the issue's second table

wheel form before us after us
jvm list, positional 544.84 76.44
jvm list, named 522.82 79.66
jvm f64 array, named 598.35 76.10
jvm f64 array, positional 12.81 11.22
native list, positional 643.69 61.44
native list, named 611.96 60.16

On the native build the list went from 31686 bytes of text to a 38 byte stub and 6144 bytes of float32.

What deliberately did not change, and why

  • A list with an int, bool, None, str or numpy scalar among its floats, an int list, and any list shorter than the threshold: the old path, so an int stays an int (Long) and every type survives. A list of numpy scalars (list(array)) also keeps the old path, a float subclass too.
  • A list to float32 on either build. It would change the stored and compared values (0.1 is not a float32), and float[] is not List<Double> for IN, CONTAINS, equality or an untyped list property.
  • On the native build, a list of doubles float32 would round: it stays JSON text. A float64 side buffer would remove that cost too, but it needs a new entry point in the library; left for a separate change.
  • Positional float64 arrays (rounded to float32 on both builds, native any float array) and float32 arrays: unchanged, they already cross as one buffer.
  • No new public API. A big-endian or two-dimensional array goes through tolist() as before (the rows then take the list path).

Tests

  • tests/test_query_vector_params.py (both builds): TestNativeCodec states the bytes handed over (stub text, side buffer, offsets with other arrays, fallbacks for the number cases above, text unchanged without the buffer). TestStatements states that the values and types come back unchanged and that IN, a list property (insert, read back, =), CONTAINS and vectorNeighbors (positional and named; list, float32 array, float64 array, literal text) give the same results, for float32-exact lists and for doubles.
  • tests/test_query_vector_params_jvm.py (JVM build; listed in NATIVE_NOT_APPLICABLE): counts the one-buffer crossings with a spy on _boxed_doubles and compares the Java objects (ArrayList, Double and Long classes, equals against the element-wise list).

Local results on cores 14,15 against the 2026-10-26 wheels (main e60a748, Python overlaid with this branch):

build files result
jvm both new files plus test_type_conversion, test_parameter_binding, test_numpy_support, test_conversion_fixes, test_positional_count, test_vector_sql, test_float_vector_bulk, test_native_result_transport 247 passed
native, old library new statement file plus the same files without test_parameter_binding 161 passed, 29 skipped (NA-JVM), 5 failed

The 5 native failures are the float32-exact list cases, which fail with unknown parameter tag BL because the local native library predates the tag. The Java side of the tag (Json.java) compiles against Graal SDK 25.0.2 and the engine jars but has not run: the native end-to-end check is this PR's native CI, which builds the library. Against the old code the 6 codec tests and all 24 JVM spy tests fail (the new function is absent), and the statement tests pass, so they assert the old results.

Static gate: pre-commit run --files (black 26.10.1, isort) clean, bandit over src and tests clean.

🤖 Generated with Claude Code

…s one buffer (#106)

JVM: a list of 32 or more Python floats and nothing else, or a native-order
float64 vector, becomes one double[] that Java boxes into the same ArrayList<Double>
the element-wise conversion built (505 us -> 44 us for 1536 numbers, 570 us -> 60 us
for the named float64 array). Mixed lists, ints, bools, None, strings, numpy scalars
and short lists keep the old path, so an int stays a Long.

Native: a list of 64 or more Python floats that float32 holds exactly (the check
double -> float32 -> double is the identity, finite numbers only) goes in the float32
side buffer under a new tag BL that Json.java reads as a List<Double>; every other list
stays JSON text. _prep also stops walking a list of plain scalars element by element.

Tests state the bytes (codec), the Java objects (spy on the one buffer crossing), and
that IN, list properties, equality, CONTAINS and vectorNeighbors give the same results.
The docs name the float32 array as the fast path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898
tae898 marked this pull request as ready for review October 27, 2026 15:22
@tae898

tae898 commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

Timed A/B done: base wheel (main 40ce107) against this PR's head (e4840cc453), on the CI wheels. For the JVM build the head arm is the base wheel with only this PR's two .py files overlaid, so both arms carry the same engine jars; the native wheels carry the same engine. ab.py --noise, ABBA, cores pinned, 15 to 19 clean pairs per figure (11 to 17 on the excess_p50 figures). Each run is one fresh process: 100 warm-up calls, then 1600 timed vectorNeighbors queries, the timer wrapping db.query(...).to_list() so parameter encoding is inside it. B/A < 1 means the head is faster.

query vector passed as 1536-d: p50 B/A 95% CI (A -> B, ms) 256-d: p50 B/A 95% CI
JVM, positional list [0.824, 0.843] (2.751 -> 2.298) [0.929, 0.959]
JVM, named list [0.826, 0.850] (2.744 -> 2.318) [0.931, 0.956]
JVM, named float64 array [0.807, 0.836] (2.764 -> 2.254) [0.928, 0.952]
native, positional float32-exact list [0.800, 0.809] (5.293 -> 4.266) [0.906, 0.928]
native, named float32-exact list [0.800, 0.807] (5.322 -> 4.261) [0.919, 0.928]
control: JVM positional float32 / float64 array [0.980, 1.014] / [0.980, 1.016] [0.979, 1.013] / [0.991, 1.014]
control: native float32 array [0.995, 1.006] [0.996, 1.004]
control: native plain-doubles list (stays JSON) [1.002, 1.010] [1.008, 1.014]

Every changed form is faster, about 16% (JVM) and 20% (native) per 1536-d query, and the cost on top of an unchanged float32 query (excess_p50) drops to 7 to 33% of what it was at 1536-d. Answers are identical: the 10 ids of the first 100 queries match between base and head in all 378 compared (unit, round) pairs, and the head takes the one-buffer path on the treated forms only (checked in every run). One small cost: a native list of plain doubles that float32 cannot hold exactly still goes as JSON text and is now 0.6 to 1.1% slower, from the float32-exactness check before the fallback (about 0.03 ms on a 5.3 ms query).

Merging on green.

@tae898
tae898 merged commit e02ab36 into main Oct 28, 2026
50 checks passed
@tae898
tae898 deleted the fix/106-query-vector-buffer branch October 28, 2026 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Query vector as a list, or a float64 array in a named parameter, costs 0.5 ms to convert

1 participant