[marcft]

Review copy in my fork, before anything goes upstream. Upstream issue 21 asks for a dynamic window size in W-TinyLFU, based on Adaptive Software Cache Management (Middleware '18).

What does this PR do?

It adds -e hill-climb=1 to WTinyLFU: the paper's window climber (WC-W-TinyLFU).

• After every 10 × cache size of requests, it compares the hit ratio with the previous period.
• It keeps moving the window 5% in the same direction unless the hit ratio dropped; then it turns around.
• The grid is the paper's 1%, 5%, 10%, …, 80%, and it starts at 1%.

It's off by default, and results without hill-climb=1 are unchanged. Only LRU and SLRU main caches are supported; anything else is rejected with an error.

How the resize works. WTinyLFU_evict already moves window victims into the main cache while the main cache has room, so the window holds whatever the main cache doesn't use. A resize therefore only sets the main cache's size:

• Window grows: the main cache evicts its own victims down to its new size right away. New misses then fill the window.
• Window shrinks: the next evictions move window victims into the main cache until it's full.
• SLRU main: SLRU_resize() keeps each segment's share of the cache. Otherwise the protected segment would keep its old size, and probation would vanish once the window passed about 20%. The shares are stored at init, so repeated resizes don't drift.
• Oversized objects in SLRU: after a resize, a resident object can be larger than the next segment. SLRU_find() now keeps such an object at the head of its own segment instead of promoting it. Promoting it cooled it straight back down and evicted it while SLRU_find() was still using it: a heap-use-after-free through the public SLRU_resize(). Through the climber this can't happen: for every cache size up to 500,000, the largest object W-TinyLFU can admit at any window size fits the smallest protected segment, but with zero margin. Objects that fit the next segment, which is every object in the default layouts, are promoted exactly as before.

Deviations from the paper (also written in the WTinyLFU.c header):

• No 0% window. Every object enters through the window, so 0% means every request misses.
• A tie keeps the direction, as Caffeine does. The paper's "otherwise turn around" traps the climber when no object fits in the 1% and 5% windows: every period has 0 hits, so it switches between them forever. That happened with 10 and 19 objects, and with small caches that count object metadata.
• The period counts bytes. It's 10 × cache size bytes of requests, which is the same thing as the paper's 10 × cache size requests when objects have unit size. Counted in requests, a 19 MiB cache would need about 2×10⁸ requests per decision, and cloudPhysics has 113,872 in total.
• Objects aren't moved between window and main when the window changes. Caffeine moves them; the paper doesn't say.

Type of change

• Bug fix
• New eviction / admission / prefetch algorithm
• New trace reader or trace format
• Performance improvement
• Documentation
• Build / CI
• Other: new option for an existing algorithm

How was it tested?

• ctest: ctest --test-dir _build --output-on-failure --parallel 4 (Release) passes 15/15.

• CLI tests: cd _build && bash ../test/test_cli.sh gives 428 passed, 0 failed. New cases:

  • hill-climb, hill-climb=, hill-climb=2 and hill-climb=yes are rejected cleanly;
  • hill-climb=0, hill-climb=1 and main-cache=LRU,hill-climb=1 are accepted;
  • hill-climb=1,print reports the setting;
  • main-cache=FIFO,hill-climb=1 is a clean error.

• New unit tests in test_evictionAlgo.c:

  • WTinyLFU_hill_climb_grows_window: after exactly 10 × C requests the window grows from 1% to 5%, and the main cache evicts down to its new size at once.
  • WTinyLFU_hill_climb_no_hits: with 19 objects, where the 1% and 5% windows hold nothing, the climber must keep growing until it gets hits. This fails with turn-around-on-tie.
  • SLRU_resize_keeps_large_object: a 20-byte object in a 100-byte 1:4 SLRU resized to 20 bytes must survive its next hit. It fails without the fix, and ASan reports a heap-use-after-free.
  • SLRU_resize: each segment keeps its configured share. Size 71 → 30 → 71 gives a first segment of 14 → 6 → 14. Deriving the share from the truncated initial size gave 13 back at 71.

• Strict Debug build: with scripts/debug.sh's C flags (-Wall -Wextra -Werror -Wpedantic …), cachesim and testEvictionAlgo build and the eviction tests pass. The Release build has one warning, bin/debug/main.cpp:27 (-Wmain), which is pre-existing and not touched here.

• ASan + UBSan Debug build: 0 reports and 0 assertions on:

  • the new tests;
  • hill-climb=1 on cloudPhysics with LRU and SLRU mains at 1000 and 5000 objects;
  • Zipf(200K requests, 10K objects) with LRU and SLRU mains, starting at window-size=0.5;
  • cloudPhysics in byte mode with an LRU main.

• Randomized stress test (an instrumented copy of the library plus a fuzz harness, not part of this PR):

  • Workloads: about 700 random configs, 1–2,000,000-byte caches, LRU and SLRU mains, starts from 0 to 0.99, unit, per-object and per-request sizes, zero-size objects, metadata on and off, Zipf with phase shifts, bursts and scans. Run in an ASan+UBSan Release build and an ASan+UBSan Debug build.
  • Checked after every request: window + main = cache size; the window matches the grid step; occupancy is within the cache size; find agrees with get; an admitted miss is resident; no object is in both window and main; SLRU lists match their counters; segment budgets add up after a resize; main fits its new size after every resize.
  • Result: 0 failures over 7,716 decisions. The only failures were develop's, reproduced on untouched develop with hill-climb off (see "Not in this PR").
  • Independent replay: a Python reimplementation of the rule re-derived all 1,160 decisions of one shard from the per-request log. When each decision happens, its period counts, hit ratio, direction, step, window and main size all match exactly.

• Leaks: macOS leaks --atExit reports 0 leaks with LRU and SLRU mains.

• Plain SLRU unchanged: cachesim … slru gives output identical to develop in 32 runs: 8 segment layouts (default, 1:4, 1:1, n-seg=2, n-seg=3, 1:2:3, 4:1, 3:1) × cloudPhysics in byte and unit mode, twitter_cluster52 and Zipf.

• Defaults unchanged: I compared printed results against develop f02f42e for cloudPhysics, twitter_cluster52 and Zipf(2M requests, 100K objects, α=1.0), with LRU, SLRU and FIFO mains at 1000 and 5000 objects (unit size), plus cloudPhysics byte mode at 0.01 and 0.1 with LRU and SLRU mains. All 11 are identical.

• ctest --test-dir _build --output-on-failure passes

• Build is warning-free (CI uses -Wall -Wextra -Werror): the changed files build with the strict flags; the one Release warning is the pre-existing bin/debug/main.cpp:27

Results

These numbers are a sanity check on the bundled samples, not an evaluation. The cache_dataset runs (fixed 1% vs climb vs the best fixed window per trace and size) are still to do.

cachesim <trace> wtinylfu 1000,5000 --ignore-obj-size=1 --hashpower=12 --num-thread=1 \
  -e main-cache=<LRU|SLRU>,hill-climb=1        # or window-size=<0.01|0.05|0.1|0.2|0.4|0.8>

trace	main	size	fixed 1%	climb	best fixed (window)
cloudPhysics	LRU	1000	0.8317	0.8277	0.8228 (0.4)
cloudPhysics	LRU	5000	0.7730	0.7720	0.7662 (0.05)
cloudPhysics	SLRU	1000	0.8333	0.8259	0.8253 (0.4)
cloudPhysics	SLRU	5000	0.7835	0.7822	0.7809 (0.05)
twitter_cluster52	LRU	1000	0.3700	0.3238	0.3208 (0.2)
twitter_cluster52	LRU	5000	0.2431	0.2335	0.2304 (0.1)
twitter_cluster52	SLRU	1000	0.3621	0.3221	0.3134 (0.2)
twitter_cluster52	SLRU	5000	0.2428	0.2332	0.2299 (0.1)
Zipf α=1.0	LRU	1000	0.4044	0.4141	0.4044 (0.01)
Zipf α=1.0	LRU	5000	0.2697	0.2756	0.2697 (0.01)
Zipf α=1.0	SLRU	1000	0.3971	0.4101	0.3971 (0.01)
Zipf α=1.0	SLRU	5000	0.2660	0.2754	0.2660 (0.01)

cloudPhysics and Twitter. The climber beats fixed 1% in all 8 cells, but doesn't reach the best fixed window. At 1000 objects, cloudPhysics only allows 10 decisions in 113,872 requests, so those rows say little.

Stationary Zipf: the climber is worse. Fixed 1% is the best window there, and the climber loses to it by +0.6 to +1.3 points. I traced the decisions with a temporary print (SLRU main, 1000 objects):

• 199 decisions, only 13% of them at 1%, going as high as 50%;
• after warm-up, the per-period hit ratio has a stdev of 0.0095, while the gap between the 1% and 5% windows is 0.0027.

So the noise is about 3.5× the signal, and the climber random-walks. The paper warns about exactly this ("a noisy process", "it never stops oscillating"). Caffeine's decaying step is the obvious follow-up to try.

Choices to review (what this PR picked)

1. Parameter name: hill-climb=1, not adaptive-window=1 or window-size=adaptive. It leaves room for an indicator variant later.
2. Period: 10 × cache size requested bytes, not 10 × cache size requests.
3. Shrinking the main cache: immediately at the resize, not lazily inside evict as S3FIFOd does. It measured within noise of lazy, and SLRU's own size assert stays true.
4. Grid: an integer step index over the paper's list, not window-size ± 5 points.
5. Ties: keep going, as Caffeine does, instead of the paper's "otherwise turn around". See the deviations above.
6. Hook: the climber runs in get(), as Cacheus and S3FIFOd do. Callers that drive find/insert directly don't adapt.
7. Edges: clamp at 1% and 80%. The next period that doesn't improve turns it around.

Limitations

• SLRU contents follow a shrink lazily. When main shrinks, its own victims are evicted at once, and they come from probation first. The protected segment's overflow moves down at the next promotion, which is how SLRU always enforces its segment sizes. Caffeine demotes it immediately.
• Tiny caches:
  • Below 25 bytes, an SLRU main's first segment can round to 0 at an 80% window, so nothing is admitted until the window shrinks again. SLRU_init already rejects SLRU mains under 5 bytes.
  • The 1% window rounds to 0 below 100 objects; see "Not in this PR".
• Pinned objects are best effort, as everywhere in the library. When the window grows, main evicts its own victims at once. to_evict skips pinned objects, so a pinned object is evicted only when every object in main is pinned. A strict guarantee would mean deferring the shrink, which leaves main over its size.
• Start window: window-size is rounded to the nearest multiple of 5%, and a result of 0% means 1%.
• Throughput: the per-request bookkeeping is not measurable. With 990K requests on a 100K-object cache, where no decision happens, climb and fixed run within noise of each other. Larger windows themselves are slower: on Zipf(5M requests, 1M objects, α=0.9) a fixed 10–30% window runs at 4.96–5.18 MQPS vs 5.55 at 1%, so a climbing run is slower when it spends time there.

Not in this PR

Pre-existing issues I found along the way. They're separate concerns, so none are touched here:

• Debug abort with an SLRU main and real object sizes, on develop.
  • Command: cachesim data/cloudPhysicsIO.vscsi vscsi wtinylfu 0.01,0.1 --hashpower=12 --num-thread=1 -e main-cache=SLRU
  • Result: Assertion failed: ((cache->occupied_byte <= cache->cache_size)), function SLRU_find, file SLRU.c, line 275.
  • Cause, from reading the code: the duel in WTinyLFU_evict evicts exactly one main victim, whatever the incoming object's size. Release builds compile the assert out.
• Segfault with --consider-obj-metadata.
  • Command: cachesim data/twitter_cluster52.csv csv wtinylfu 0.001 -t "time-col=1,obj-id-col=2,obj-size-col=3,obj-id-is-num=1" --consider-obj-metadata=1
  • On develop this exits 139, also with plain slru.
  • Cause: SLRU_cool moves obj_size between segment counters, while promotion and eviction move obj_size + obj_md_size. The counters drift until SLRU_to_evict returns NULL. Moving the metadata too made every such crash disappear in the stress test.
• W-TinyLFU misses every request below 100 objects. The 1% window rounds down to 0. With --ignore-obj-size=1, cloudPhysics at 50 and 99 objects gives a miss ratio of 1.0000, while LRU at 50 gives 0.9014.
• Cacheus.c:375. num_hit / update_interval is integer division, so Cacheus's learning-rate update sees a hit rate of 0. Found by reading only; not run.

Follow-ups: Caffeine's step decay and restart, the indicator variant (WI-W-TinyLFU), and an evaluation on cache_dataset traces.

Checklist

• Formatted with clang-format
• Added or updated tests
• Added or updated documentation (none: -e print shows hill-climb=, and the paper deviations are in the WTinyLFU.c header)
• New algorithms are registered in the CLI and listed in the README (not a new algorithm)

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.