Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/claude-plugin.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Claude plugin directory edition

on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: claude-plugin/ matches AAE.md, the knowledge base and the report template
run: python3 scripts/build_claude_plugin.py --check
- name: no symbolic links in claude-plugin/
run: test -z "$(find claude-plugin -type l)"
Binary file added claude-plugin/.claude-plugin/icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 20 additions & 0 deletions claude-plugin/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"name": "aae",
"displayName": "Agentic Algorithm Engineering",
"version": "1.12.0",
"description": "Agentic Algorithm Engineering: autonomous performance tuning through falsifiable hypotheses, benchmarks, numbered patches, and a PDF report.",
"author": {
"name": "Christian Schulz"
},
"homepage": "https://chszlab.github.io/AgenticAlgorithmEngineering/",
"repository": "https://github.com/CHSZLab/AgenticAlgorithmEngineering",
"license": "MIT",
"icon": ".claude-plugin/icon.png",
"keywords": [
"performance",
"optimization",
"algorithm engineering",
"benchmarking",
"profiling"
]
}
21 changes: 21 additions & 0 deletions claude-plugin/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Christian Schulz and contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
11 changes: 11 additions & 0 deletions claude-plugin/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# AAE for the Claude plugin directory

This folder is the edition of [Agentic Algorithm Engineering](https://github.com/CHSZLab/AgenticAlgorithmEngineering)
for Anthropic's plugin directory. It is generated by `scripts/build_claude_plugin.py` from `AAE.md`,
the knowledge base and the report template; do not edit it by hand.

Compared with the plugin at the repository root, this edition ships no programs: the agent writes
the LaTeX report itself from the template in `skills/aae/report-template.md`, and knowledge-base
contributions are prepared as a file for the user to submit instead of being opened as pull requests.

Homepage: https://chszlab.github.io/AgenticAlgorithmEngineering/
363 changes: 363 additions & 0 deletions claude-plugin/skills/aae/SKILL.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Cache-Friendly Blocked Recursive Sorting

## Problem

Sorting large integer arrays (10M+ elements) in C++. The metric was wall-clock time measured via `std::chrono::high_resolution_clock`. The baseline was `std::sort`, which uses introsort internally.

## What Worked

Replacing the default quicksort partitioning with a blocked approach that processes elements in cache-line-sized blocks (64 elements per block). Instead of scanning left-to-right and right-to-left with pointer swaps, the algorithm fills two buffers of indices that need swapping, then performs the swaps in bulk. This keeps memory accesses sequential within each block and reduces branch mispredictions because the buffer-fill loop has no data-dependent branches.

The improvement was ~20% over `std::sort` on uniformly random 64-bit integers (10M elements, averaged over 10 runs).

## Experiment Data

| Variant | Time (ms) | Relative |
|---------|-----------|----------|
| std::sort baseline | 842 | 1.00x |
| Blocked partition, block size 32 | 713 | 1.18x |
| Blocked partition, block size 64 | 678 | 1.24x |
| Blocked partition, block size 128 | 691 | 1.22x |

Block size 64 was the sweet spot, matching the L1 cache line width.

## What Didn't Work

- **Radix sort**: Faster in theory for integers, but the extra memory allocation and multiple passes made it slower for arrays that fit in L2 cache. Only won for 100M+ elements.
- **SIMD-based partitioning**: The overhead of gather/scatter instructions on the test hardware (Zen 3) negated the theoretical throughput gain. Might work better on Intel with AVX-512.

## Code Example

```cpp
// Core idea: buffer indices that need swapping, then swap in bulk
int buf_l[BLOCK], buf_r[BLOCK];
int nl = 0, nr = 0;
while (left + BLOCK <= right - BLOCK) {
if (nl == 0) {
for (int i = 0; i < BLOCK; i++)
buf_l[nl] = i + (arr[left + i] >= pivot); // branchless
nl += (arr[left + i] >= pivot);
}
if (nr == 0) {
for (int i = 0; i < BLOCK; i++)
buf_r[nr] = i + (arr[right - 1 - i] < pivot);
nr += (arr[right - 1 - i] < pivot);
}
int swaps = min(nl, nr);
for (int i = 0; i < swaps; i++)
std::swap(arr[left + buf_l[i]], arr[right - 1 - buf_r[i]]);
// advance pointers...
}
```

## Environment

C++17, GCC 12.2 with `-O3 -march=native`, AMD Ryzen 9 5900X, 64 GB DDR4-3600, Ubuntu 22.04. Array fits entirely in RAM, no disk I/O.
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Gradient Accumulation as Batch Size Proxy for Transformer Training

## Problem

Training a small transformer language model (125M parameters) on a single GPU with limited VRAM (24 GB). The metric was validation bits-per-byte (val_bpb), lower is better. Larger batch sizes are known to stabilize training, but the model with batch size > 32 caused OOM on the available hardware.

## What Worked

Using gradient accumulation to simulate large batch sizes without increasing memory usage. Instead of trying to fit batch size 128 into memory, accumulate gradients over 4 forward passes of batch size 32 before performing one optimizer step. This achieved the same effective batch size 128 while staying within VRAM limits.

The key insight: gradient accumulation is not just a memory workaround, it is functionally equivalent to a larger batch (assuming no batch normalization). The model trained with accumulation steps=4 achieved identical val_bpb (within noise) to a machine with enough VRAM for true batch size 128, and val_bpb improved from 1.42 (batch 32) to 1.31 (effective batch 128).

## Experiment Data

```
commit metric_value resource_usage status hypothesis
a1b2c3d 1.4200 22.1 keep baseline (batch_size=32, no accumulation)
b2c3d4e 1.3500 22.1 keep accumulation_steps=2 (effective batch 64) will reduce val_bpb by smoothing gradient noise
c3d4e5f 1.3100 22.3 keep accumulation_steps=4 (effective batch 128) will further reduce val_bpb
d4e5f6g 1.3150 22.3 discard accumulation_steps=8 (effective batch 256) will continue the trend
e5f6g7h 1.2950 22.3 keep accumulation_steps=4 with linear LR scaling (4x base LR) following the linear scaling rule
```

## What Didn't Work

- **Effective batch 256** (accumulation_steps=8): No further improvement; the model is small enough that batch 128 already provides sufficient gradient signal. Diminishing returns.
- **Square root LR scaling**: Scaling LR by sqrt(accumulation_steps) instead of linearly underperformed. Linear scaling rule (Goyal et al., 2017) worked better for this model size.

## Environment

PyTorch 2.1, single NVIDIA RTX 4090 (24 GB VRAM), 125M parameter GPT-2-style transformer, OpenWebText dataset, trained for 50k steps.
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Arena Allocation for JSON Parsing

## Problem

Parsing large JSON files (100 MB+) in a Rust service. The metric was throughput in MB/s. The baseline used `serde_json::from_str` which allocates individually for every string and object in the JSON tree, causing heavy allocator pressure.

## What Worked

Switching from per-node heap allocation to an arena allocator (`bumpalo`). All parsed nodes, strings, and intermediate structures are allocated from a single contiguous memory region that is freed in one shot after processing. This eliminated thousands of individual `malloc`/`free` calls per parse and improved cache locality because related nodes end up adjacent in memory.

Throughput improved from 380 MB/s to 620 MB/s (~63% improvement) on a 150 MB JSON file with deeply nested objects.

## Environment

Rust 1.74, `bumpalo` 3.14, AMD EPYC 7763 (server), 256 GB RAM, Linux 6.1. Input files are API response dumps with 3-5 levels of nesting and many small string values.
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Structure of Arrays vs Array of Structures for Cache Efficiency

## Problem

Processing large collections of objects (1M+ entities) in C++ where each operation touches only a subset of fields. The metric was throughput (operations per second). Baseline used a traditional AoS layout (`std::vector<Particle>` where `Particle` has position, velocity, mass, color, etc.) but the hot loop only read `x, y, z` positions.

## What Worked

Converting the data layout from Array of Structures (AoS) to Structure of Arrays (SoA). Instead of one `struct` with all fields packed together, store each field in its own contiguous array. This ensures that when the hot loop iterates over positions, every cache line contains only position data — no wasted bandwidth loading unused fields like color or material ID.

For a particle simulation update loop touching only `x, y, z` on 4M particles, SoA achieved 2.1x throughput over AoS because each 64-byte cache line carried 8 useful `double` values instead of 2 (the struct was 48 bytes, so only 1.3 structs fit per line, wasting ~60% of fetched memory bandwidth on cold fields).

## Experiment Data

| Layout | Throughput (Mops/s) | L1 Cache Miss Rate |
|--------|--------------------|--------------------|
| AoS (baseline) | 142 | 12.3% |
| SoA | 298 | 3.1% |
| SoA + AVX2 vectorization | 487 | 2.8% |

## What Didn't Work

- **Hybrid AoSoA** (blocking 8 structs into a mini-array): Marginal improvement over AoS (~15%) but added complexity. Only useful when multiple fields are always accessed together.
- **`__attribute__((packed))`** on the struct: Reduced struct size but caused unaligned access penalties that negated the benefit.

## Code Example

```cpp
// AoS — cold fields pollute cache lines
struct Particle { double x, y, z, vx, vy, vz; int material; float color[4]; };
std::vector<Particle> particles(N);
for (auto& p : particles) p.x += p.vx * dt; // loads 80 bytes per particle, uses 16

// SoA — only position data in cache
struct Particles {
std::vector<double> x, y, z, vx, vy, vz;
std::vector<int> material;
std::vector<std::array<float,4>> color;
};
Particles ps(N);
for (size_t i = 0; i < N; i++) ps.x[i] += ps.vx[i] * dt; // loads 16 bytes, uses 16
```

## Environment

C++17, GCC 13.1 with `-O3 -march=native`, Intel Core i7-13700K, 32 GB DDR5-5600, Ubuntu 23.04. Measured with `perf stat` for cache miss rates.
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Branchless Programming to Eliminate Branch Mispredictions

## Problem

Filtering or transforming large arrays in C++ where element-wise decisions depend on data values (e.g., clamping, conditional increment, partitioning). The metric was wall-clock time. Baseline used straightforward `if/else` branches, which suffer from branch misprediction when data is random or has low predictability.

## What Worked

Replacing conditional branches with arithmetic equivalents. Modern CPUs pipeline ~15-20 instructions ahead; a mispredicted branch flushes the entire pipeline (~12-15 cycle penalty on x86). For random data the mispredict rate approaches 50%, making branches extremely expensive in tight loops.

Key patterns:
1. **Conditional move via arithmetic**: `x = (cond) ? a : b` → compiler emits `cmov` if written carefully, but sometimes needs manual help.
2. **Branchless min/max**: Use subtraction + sign-bit masking instead of `std::min`/`std::max` when the compiler doesn't optimize.
3. **Predicated accumulation**: `sum += arr[i] * (arr[i] > threshold)` — the boolean converts to 0/1, no branch needed.

For a partitioning loop on 10M random integers, branchless code ran 2.8x faster than the branching version due to eliminating ~47% misprediction rate.

## Experiment Data

| Variant | Time (ms) | Branch Misses (perf) |
|---------|-----------|---------------------|
| if/else partition | 38.2 | 24.7M |
| Branchless (arithmetic) | 13.6 | 0.02M |
| Branchless + unrolled 4x | 11.1 | 0.01M |

## What Didn't Work

- **Branchless on already-sorted data**: When branch prediction accuracy is >95% (sorted/nearly-sorted input), the branch version is actually faster because `cmov` has a data dependency that serializes execution, while a correctly predicted branch allows speculative execution. Always profile with realistic data distributions.

## Code Example

```cpp
// Branching — ~50% mispredict on random data
for (int i = 0; i < N; i++) {
if (arr[i] < pivot) out[left++] = arr[i];
else out[right--] = arr[i];
}

// Branchless — zero mispredictions
for (int i = 0; i < N; i++) {
int goes_left = (arr[i] < pivot); // 0 or 1
out[left] = arr[i];
out[right] = arr[i];
left += goes_left;
right -= (1 - goes_left);
}
```

## Environment

C++17, GCC 12.2 with `-O3 -march=native`, AMD Ryzen 9 5900X, measured with `perf stat -e branch-misses`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Small Buffer Optimization to Avoid Heap Allocations

## Problem

High-frequency allocation and deallocation of small, variable-sized objects in C++ (e.g., short strings, small vectors, temporary buffers). The metric was throughput of an operation that creates and destroys thousands of small containers per call. Baseline used `std::string` and `std::vector` which heap-allocate even for tiny sizes, causing allocator contention and cache misses from pointer chasing.

## What Worked

Small Buffer Optimization (SBO) embeds a fixed-size buffer directly inside the object. If the data fits in the inline buffer, no heap allocation occurs. This is the technique behind `std::string`'s SSO (Short String Optimization, typically 15-22 bytes inline depending on implementation) and can be applied to any container.

For a tokenizer producing millions of short tokens (avg. 8 bytes), switching from `std::string` to a custom SBO string with 32-byte inline buffer eliminated 94% of heap allocations and improved throughput by 1.7x. The key insight: even when SSO is already active in `std::string`, the default inline size (typically 15 bytes on libstdc++) may be too small for the workload. A custom SBO with a tuned buffer size can outperform.

## Experiment Data

| Variant | Tokens/sec (M) | Heap Allocs (millions) |
|---------|----------------|----------------------|
| std::string (SSO=15) | 4.2 | 3.1 |
| SBO string (inline=32) | 7.1 | 0.19 |
| SBO string (inline=64) | 6.8 | 0.04 |

Inline 64 had slightly lower throughput than 32 because the larger object size reduced cache density for the token array itself.

## Code Example

```cpp
template<size_t InlineSize = 32>
class SmallString {
size_t size_;
union {
char inline_buf_[InlineSize];
char* heap_ptr_;
};
bool is_heap() const { return size_ >= InlineSize; }
public:
char* data() { return is_heap() ? heap_ptr_ : inline_buf_; }
// ... constructor allocates on heap only when size >= InlineSize
};
// Same pattern applies to SmallVector<T, N>: inline storage for N elements
```

## Environment

C++17, GCC 13.1 with `-O3`, libstdc++, Intel Core i7-12700K, Ubuntu 22.04.
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Move Semantics to Eliminate Expensive Deep Copies

## Problem

C++ programs that return or transfer ownership of large containers (vectors, strings, maps) from functions, causing unnecessary deep copies. The metric was function call overhead for routines returning `std::vector<double>` with 1M+ elements.

## What Worked

Three techniques at different levels:

1. **Return value optimization (RVO/NRVO)**: The compiler elides the copy entirely when a function returns a local variable by name. This is guaranteed in C++17 for prvalues (copy elision). Ensure the function has a single, named return variable — multiple return paths can prevent NRVO.

2. **`std::move` for transfers**: When moving an object to a new owner (e.g., pushing into a container or passing to a constructor), `std::move` converts the lvalue to an rvalue reference, triggering the move constructor. A moved `std::vector` transfers pointer ownership in O(1) instead of copying N elements.

3. **Emplace instead of insert**: `container.emplace_back(args...)` constructs the object in-place, avoiding both copy and move. Particularly impactful when the object is expensive to construct.

For a pipeline that passed `vector<double>(1M)` through 5 transformation stages, ensuring moves instead of copies reduced stage-transition overhead from 4.2ms to <0.001ms per transfer.

## What Didn't Work

- **Moving from `const` references**: `std::move(const_ref)` silently falls back to a copy because the move constructor requires a non-const rvalue reference. This is a common silent performance bug — no compiler warning by default.
- **Moving small types**: For types smaller than two pointers (e.g., `std::pair<int,int>`), move is identical to copy. The overhead of thinking about moves is wasted.

## Code Example

```cpp
// BAD: forces copy if NRVO fails (multiple return paths)
std::vector<double> compute(bool flag) {
std::vector<double> a = heavy_compute_a();
std::vector<double> b = heavy_compute_b();
if (flag) return a; // NRVO may fail — two candidates
return b;
}

// GOOD: single return variable, guaranteed NRVO
std::vector<double> compute(bool flag) {
std::vector<double> result;
if (flag) result = heavy_compute_a();
else result = heavy_compute_b();
return result; // NRVO: single named variable returned
}

// Transfer ownership explicitly
pipeline.add_stage(std::move(large_vector)); // O(1) pointer swap
```

## Environment

C++17 or later (mandatory copy elision for prvalues). Applies to all major compilers (GCC, Clang, MSVC).
Loading