Skip to content

Compress PrefixTree into a radix tree to reduce memory - #4272

Open
Fryguy wants to merge 1 commit into
Shopify:mainfrom
Fryguy:reduce-prefix-tree-memory
Open

Fryguy wants to merge 1 commit into
Shopify:mainfrom
Fryguy:reduce-prefix-tree-memory

Conversation

@Fryguy

@Fryguy Fryguy commented Nov 16, 2026 •

Copy link
Copy Markdown

Motivation

On large projects the Ruby LSP uses a lot of memory. Indexing a ManageIQ checkout (43,691 files, 330,729 distinct names, including its bundled gems) grows the process by ~1.87 GB.

Profiling the index shows RubyIndexer::PrefixTree is the largest single cost. It allocates one Node for every character of every indexed name, plus an empty children Hash on every node. For ManageIQ that is ~3.46M nodes and ~3.46M hashes, about 810 MB, or over 40% of everything the indexer retains.

Implementation

PrefixTree is now a radix tree:

  • Runs of characters with no branching are stored as a single node. Nodes are split on insert when a key diverges in the middle of an existing edge.
  • Leaf nodes no longer allocate a children Hash unless they get a child.

The public API (insert, search, delete) and the order of search results are unchanged, so no caller needed to change.

Things to watch for in review:

  • find_node can end in the middle of an edge. In that case it returns the node at the end of the edge, because every key below it still matches the prefix.
  • delete removes the node returned by find_node, so a partial key still deletes everything under it (same as before).

Results indexing ManageIQ with its own bundle, Ruby 3.4.10, main vs. this branch, same machine:

main this PR Change
RSS growth during indexing 1867 MB 1117 MB -750 MB (-40%)
Total process RSS 1957 MB 1207 MB -37%
Live object slots 17.8M 8.6M -52%
PrefixTree::Node objects 3.46M 458k -87%
Index time 60 s 56 s no regression

These numbers cover the index only (measured with Index#index_all), not open documents. Remaining index memory is mostly String (~165 MB), Location (~129 MB) and Array (~90 MB), which could be follow-ups.

Automated Tests

I added tests to prefix_tree_test.rb for edge splitting, keys that end in the middle of an edge, searching prefixes that end mid-edge, and deleting after a split. They also pass against the previous implementation, so they check behavior rather than the new structure.

Manual Tests

  1. Save this as measure_index.rb:

    require "bundler/setup"
    require "prism"
    require "rbs"
    require "pathname"
    require "ruby_indexer/ruby_indexer"
    
    rss_mb = -> { `ps -o rss= -p #{Process.pid}`.to_i / 1024 }
    
    index = RubyIndexer::Index.new
    index.configuration.workspace_path = Dir.pwd
    uris = index.configuration.indexable_uris
    before = rss_mb.call
    index.index_all(uris: uris)
    GC.start
    puts "files: #{uris.size}, RSS growth: #{rss_mb.call - before} MB, live slots: #{GC.stat(:heap_live_slots)}"
  2. From the root of this repository, run it on main and then on this branch: bundle exec ruby -Ilib /path/to/measure_index.rb. On this repository (1,870 files) I get 119 MB / 1.01M live slots on main and 74 MB / 538k live slots on this branch. Larger projects show a bigger difference.

  3. To check completion in VS Code, open this repository's folder on this branch. The extension runs the local exe/ruby-lsp when the workspace folder is named ruby-lsp.

  4. Create a scratch Ruby file in this repo with # typed: ignore as the first line (the Ruby LSP leaves constant completion to Sorbet in other files). Once indexing finishes, type RubyIndexer::Pre and expect RubyIndexer::PrefixTree. Then type require "ruby_indexer/ and expect the files under lib/ruby_indexer/lib/ruby_indexer/. These use @entries_tree and @require_paths_tree.

  5. In another open file, define a new constant, then confirm it shows up in completion from the scratch file. This exercises PrefixTree#delete and #insert.


Possible follow-ups

Remaining index memory on ManageIQ after this change (1.1 GB growth) is mostly entry data, not the tree. None of these are part of this PR, and the savings are not measured.

  • Location objects (~129 MB, 1.7M objects): every Method entry holds two (location and name_location). They could be packed into a single Integer or computed lazily from the file when requested.
  • String objects (~165 MB, 2.5M objects): 330k distinct names are spread across a much larger number of entries. Deduplicating names when entries are created (-name) would avoid keeping separate copies.
  • Signature and parameter objects (~30 MB): a third of the methods in a smaller sample (25.6k of 77.5k) have no parameters and could share one frozen empty Signature.
  • RBS::Environment in index_ruby_core: it is built only for its side effects and then discarded, which costs about 20 MB of RSS on its own. Dropping it also drops StringIO, because Environment.from_loader adds the stringio library, so it would need loader.add(library: "stringio") instead.
  • Remaining tree overhead (~51 MB): about 203k children Hashes remain. Nodes with a small fan-out could use a small array of pairs.

The PrefixTree allocated one Node, and one empty children Hash, for
every character of every indexed name. On a large index (~53k names)
this accounted for about 1.4M objects and ~129MB, roughly half of the
memory retained by the indexer.

Store runs of non-branching characters as a single node and lazily
allocate the children hash. The public API and result ordering are
unchanged. In the same benchmark, the tree drops to ~13MB and total
indexer RSS growth from ~295MB to ~167MB.

Add tests for edge splitting and for searching and deleting keys that
end in the middle of an edge.
@Fryguy
Fryguy requested a review from a team as a code owner November 16, 2026 17:13
@@ -36,10 +36,7 @@ module RubyIndexer
class PrefixTree

@Fryguy Fryguy Nov 16, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note I did not rename this to RadixTree, and kept it as PrefixTree. I can rename it if you think that's more appropriate, but I didn't want to introduce that much churn.

@Fryguy Fryguy closed this Nov 16, 2026
@Fryguy Fryguy reopened this Nov 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant