Skip to content

native: build_graph_now() does not rebuild an existing vector graph #162

Closed

Description

@tae898

On the native build (arcadedb), VectorIndex.build_graph_now() does not rebuild a vector graph that already exists. The docstring says it triggers an immediate rebuild, and on arcadedb-jvm it does (it calls the engine's LSMVectorIndex.buildVectorGraphNow()). The native shim, _IndexShim.buildVectorGraphNow() in src/arcadedb/_native/schema_objects.py, runs one vectorNeighbors search instead. A search rebuilds synchronously only when there is no graph or the graph is under 1,000 nodes (the engine's ASYNC_REBUILD_MIN_GRAPH_SIZE), so on an index whose graph is built and at least that large the call returns at once and every vector written since stays in the delta.

Seen in a timed study (agent-memory, an agent that inserts one record per turn and searches after it): 10,000 vectors at the start, automatic rebuild off (mutations_before_rebuild very high, maxDeltaScanRatio 0), 3,000 inserts, and build_graph_now() after turn 1,000 and turn 2,000. Matched dev wheels 26.11.1.dev0 (bindings 4d012f3114, engine e353330c5c), Linux x86-64. Read from the run files of the explicit arm, every run:

arcadedb-jvm arcadedb (native)
build_graph_now() calls (runs) 26 (13) 26 (13)
seconds per call, min / median / max 9.876 / 11.050 / 12.966 0.002 / 0.002 / 0.002
graphNodeCount at open 10000 10000
graphNodeCount at end 12000 10000
graphRebuildCount at end 2 0
deltaVectorsCount at end 1000 3000
totalVectors at end 13000 13000

On the JVM wheel each call took seconds and the graph grew by the vectors written before it; on the native wheel each call took milliseconds, the graph kept its size, and no rebuild was counted.

Fix: a build op in arcadedb_n_vindex (NativeExtras.java) that calls LSMVectorIndex.buildVectorGraphNow() on every bucket index, as the JPype wrapper does, and the shim calls it. warm_up() keeps its one search (it only has to load a persisted graph, not rebuild it). Regression test in tests/test_vector_index_introspection.py, on both builds: an index built over 1,000 vectors, 100 more written with the automatic triggers off, then build_graph_now() must leave graphNodeCount at 1,100 and raise graphRebuildCount.

Edited 2026-11-01: the size condition (graphs under 1,000 nodes are rebuilt by the search, so the old shim worked on small indexes) and the test size (first written at 200 vectors, which passed on the pre-fix native wheel) were added after the first test run.

Activity

  1. added a commit that references this issue on Nov 1, 2026
  2. tae898 commented on Nov 1, 2026

    @tae898
    CollaboratorAuthor

    Edited the description: the old search shim does rebuild a graph under 1,000 nodes (the engine rebuilds small graphs synchronously on the next search), so the defect shows only from 1,000 nodes up. The regression test in #163 now starts at 1,000 vectors; the pre-fix native wheel fails it (graphNodeCount 1000, expected 1100) and the JVM wheel passes it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions