Repository navigation
Conversation
WTinyLFU_evict added 1 to the frequency sketch for every object as it left the window, whether it was admitted to the main cache or dropped. Leaving the window is not an access, so an object read once was counted twice, and a main-cache victim read once tied a candidate read twice; a tie keeps the victim. The W-TinyLFU paper, Caffeine and mandreyel/w-tinylfu, which this file credits, do not count it. Remove the increment. test_WTinyLFU_counts_window_hits now reads object 102 twice instead of six times, so it must beat a victim read once: it needs both the window hit counted and the window exit not counted, and fails without this change. Follow-up to 1a1a11a#376, which lists this difference.
marcft
force-pushed
the
wtinylfu/match-paper
branch
3 times, most recently
from
November 7, 2026 22:23
f2b1449 to
430a5cf
Compare
The sketch's counters were unbounded, and the sketch was halved after 32 x the main cache's size accesses, with the access count restarting at 0. Stop counters at 15 (4 bits) and halve after 10 x the cache size accesses, as Caffeine does; this is the Caffeine configuration that the TinyLFU paper evaluates. Halve the access count along with the counters, as the paper's reset does (Sec. 3.3), so later halvings come every 5 x the cache size accesses. minimalIncrementCBF gains a max_count field. It defaults to UINT_MAX, so Merlin is unchanged. The file's preamble now lists what still differs from the paper: there is no doorkeeper, the sketch is sized and its period counted in bytes, an add increments all of an object's counters, and the cap is 15 rather than the paper's W/C. Follow-up to 1a1a11a#376, which lists these differences.
marcft
force-pushed
the
wtinylfu/match-paper
branch
from
November 7, 2026 22:28
430a5cf to
0e3eb1c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Hey, I saw your note on 1a1a11a#376 about the other spots where
WTinyLFU.c doesn't match the paper, and that fixing just one of them made things
worse. So I tried doing them together.
This PR fixes three of them:
request counter gets halved too instead of going back to 0
I left a few things out on purpose:
happens
that would change Merlin too)
Results are kind of mixed. It helps a lot on the prefix cache traces
(FreeInference goes from 0.85 to 0.93 hit ratio), barely moves anything with the
default SLRU main, and it's worse with LRU as main. All the numbers and commands
are below. If you'd rather not change the defaults I can put it behind a
parameter instead.
Follow-up to 1a1a11a#376. Related: 1a1a11a#21.
Type of change
How was it tested?
test_WTinyLFU_counts_window_hitsis tightened. Object 102 is now readtwice instead of six times, so it must beat a victim read once. It fails on
develop (
test_evictionAlgo.c:617, 102 is dropped) and passes with commit 1.test_minimalIncrementCBF_max_countis new. With a cap of 15, 20 addsgive an estimate of 15.
Merlin is unchanged. Same miss ratios on develop and here for
cachesim ../data/cloudPhysicsIO.vscsi vscsi merlin 0.01,0.1andcachesim Zipf_2000000_100000_1.0 syn merlin 0.01,0.1.CLI tests.
cd _build && bash ../test/test_cli.sh: 419 passed, 0 failed.Sanitizers. Both tests are clean under ASan and UBSan. LeakSanitizer
isn't available on macOS; CI's Linux job runs it.
ctest --test-dir _build --output-on-failurepasses (15/15, Release, macOS)Build is warning-free with the strict flags from
code-quality.yml. Theonly error locally is the existing
bin/debug/main.cpp:27-WmainfromApple clang, which develop has too.
Results
LLM prefix-cache traces (block hit ratio, higher is better)
The traces are FreeInference and Chutes from
harvardMadsys/2026_prefixcache
(rev ba89a34). The simulator is the prefixsim driver from
LauYeeYu/libCacheSim@0006aa5, the onecacheMon/PrefixSimships, builtagainst develop and against this PR.
Every row improves, and the FreeInference curve no longer dips from 96 GiB to
1 TiB. As in 1a1a11a#376, the window-exit fix alone is a bit worse; see "Which change
does what" below.
All runs have 0 unexpected evictions. prefixsim does warn that W-TinyLFU can
evict a block the arriving request has matched, which an engine that pins a
running request's blocks would not do. On FreeInference at 1 TiB,
n_self_evictionis 5,394,352 on develop and 1,281,730 with this PR.Classic traces (miss ratio, lower is better)
These are 1a1a11a#376's traces and sizes. As in 1a1a11a#376, the 0.1% Zipf size is left out:
it misses 100% in every version, because the 1% window rounds down to zero
objects.
Default main cache (SLRU): every difference is within −0.0031 / +0.0062.
main-cache=LRU: never better, and up to +0.0354 worse.Which change does what
"Cap only" and "10x only" are commit 1 plus one half of commit 2. Prefix rows are
at 1 TiB, as hit ratio; classic rows are miss ratio.
Reading these:
points over commit 1). There, sessions start and finish, so the popular set
keeps moving. It is also what costs on stationary Zipf with an LRU main cache,
where a longer sample estimates popularity better.
the prefix hit ratio (−1.7 / −7.1 points); on top of 10x it adds about 1
point. On twitter it is what raises miss ratios, by up to +2.2 points with an
LRU main.
above), the period never elapses within the trace, so commit 2's halving
changes nothing there.
A follow-up could count the period in objects, so it also elapses on traces
with large objects.
All sizes and variants (miss ratio)
Checklist
clang-formatdiffers from the paper)