Upgrade llama.cpp from b11815 to b11827

CHANGELOG.md

@@ -136,7 +136,7 @@ the first two items break every existing consumer at upgrade.
     fails the stream through `onError`.
 
 ### Added
-- **llama.cpp b11815** (from b11639, in eight steps on the branch: b11760, b11761, b11799, b11800, b11803, b11809, b11814, b11815). What the
+- **llama.cpp b11827** (from b11639, in ten steps: b11760, b11761, b11799, b11800, b11803, b11809, b11814, b11815, b11820, b11827). What the
   range brings to a consumer: `POST /tts` (llama.cpp b11781, #26603, speech from a TTS-capable model through the
   embedded server) and the realtime transcription WebSocket `/v1/realtime` (b11780, #30636), both served by
   `NativeServer` -- classic and attach mode -- and not by the Java transport or `LlamaModel` yet; a
@@ -157,6 +157,16 @@ the first two items break every existing consumer at upgrade.
   pre-tokenizer (#30960), KleidiAI v1.32.0 with SVE Q8_0 kernels (#30808, not built here) -- nothing
   here changed. The b11815 step (2026-10-22, one commit): the SYCL `CONV_2D` kernel accepts the
   channel-fastest CWHN layout instead of aborting (#30363), which reaches the two SYCL module jars.
+  The b11820 step (2026-10-22): llama.cpp's rerank route takes typed items (#29625) -- the query and each
+  document a string or `{"text": ..., "image": ...}`, the images ranked through the projector -- which
+  `NativeServer` serves; `OpenAiCompatServer` reads the text half (see "Reranking" under Fixed) and
+  `LlamaModel.rerank` stays text-only. Also Nemotron models with more Q heads than `n_embd / head_dim`,
+  the `domyn-small` pre-tokenizer and a `tokenizer.ggml.metaspace.prepend_scheme` GGUF key (#23343), and
+  `posix_memalign` in the Vulkan buffers (#30828). The b11827 step (2026-10-22): `--mlock` locks the
+  parts of a split host buffer too (a CUDA pinned host buffer above 4 GB, which it skipped before), SYCL
+  mat-vec kernels on the Intel XMX engines for short batches (#29864, `GGML_SYCL_ENABLE_XMX_ESIMD`), an
+  OpenCL depthwise `CONV_2D_DW` (#30900), the MiniCPM-OCR 1.0 projector (#30979), a DFlash drafter
+  fix for qwen4exp (#30927) and fewer graph inputs for glm5-next (#30932) -- nothing here changed.
   Two patches dropped (`0016`, `0019`), `0007` refreshed.
 - **`RouterModel.Status.INCOMPLETE`** (llama.cpp b11714): a router download that never finished is listed
   as `"incomplete"` without a worker; it was `UNKNOWN` here.
@@ -1136,7 +1146,12 @@ the first two items break every existing consumer at upgrade.
   negative one is `400` in both spellings; `"documents": null` next to `texts` is served from
   `texts`; and a native answer that is not a JSON array is a `500` with the reason instead of an
   empty result list. `LlamaModel.rerank(String, String...)` returns its map in the order of the
-  documents instead of hash order.
+  documents instead of hash order. The query and each document are read as llama.cpp reads them since
+  b11820 (#29625): a string or `{"text": string or [strings]}` (an object query is accepted, a text array
+  concatenated, a missing or `null` `text` empty), and an item with an `image` -- which upstream ranks
+  through the projector -- is `400` instead of being ranked by its text alone (a `null` or empty `image`
+  is none, there as here); `video` / `fps` / `max_frames` are refused, as upstream refuses them.
+  `NativeServer` serves upstream's route, images included.
 - **Java API.**
   - **`InferenceParameters.withMessages(List<ChatMessage>)` keeps `tool_calls` and `tool_call_id`**,
     which only `ChatRequest` added, so a tool round built through it sent an assistant turn without

CLAUDE.md

@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.
 
-Current llama.cpp pinned version: **b11815**
+Current llama.cpp pinned version: **b11827**
 
 ## Natives jars: library jars and GPU module jars (`.github/natives.csv`)
 
@@ -213,7 +213,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1
   a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static
   platforms (macOS, s390x) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own
   CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT`
-  and `GGML_CUDA_GRAPHS_DEFAULT` at b11815 (unchanged since b11639) -- because adding ggml first skips that block; re-check with
+  and `GGML_CUDA_GRAPHS_DEFAULT` at b11827 (unchanged since b11639) -- because adding ggml first skips that block; re-check with
   `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once:
   ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of
   its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which
@@ -1183,7 +1183,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
 ships no UI):
 ```bash
 # needs node/npm + network for the asset build; the embed step is plain cmake -P
-git clone --depth 1 --branch b11815 https://github.com/ggml-org/llama.cpp /tmp/lc
+git clone --depth 1 --branch b11827 https://github.com/ggml-org/llama.cpp /tmp/lc
 ( cd /tmp/lc/tools/ui && npm ci && npm run build )
 mkdir -p webui-generated /tmp/ui-gen
 cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \
@@ -1223,7 +1223,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
 - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
   as the repo secret **`DEPOT_TOKEN`**.
 
-Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11815`), the
+Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11827`), the
 ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
 *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
 per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
@@ -2671,7 +2671,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"
 
 #### Upstream source location (in CMake build tree)
 
-llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11815`.
+llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11827`.
 
 **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely
 by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the

README.md

@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b11815](https://img.shields.io/badge/llama.cpp-%23b11815-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11815)  
+[![llama.cpp b11827](https://img.shields.io/badge/llama.cpp-%23b11827-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11827)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev)  
@@ -1015,7 +1015,7 @@ built-in WebUI" below). It serves:
 | `POST /v1/chat/completions` | `LlamaModel.streamChatCompletion` (streaming SSE) / `chatComplete` (blocking) |
 | `POST /v1/completions` | `LlamaModel.streamCompletionsOai` (streaming SSE) / `handleCompletionsOai` (blocking) |
 | `POST /v1/embeddings` (requires `--embedding`) | `LlamaModel.handleEmbeddings` |
-| `POST /v1/rerank` (requires `--reranking`) | `LlamaModel.handleRerank` (reshaped to `results`/`data` with `usage`; a request with TEI's `texts` is answered as TEI's bare `[{index, score[, text]}]` array, `text` on `return_text`, as upstream) |
+| `POST /v1/rerank` (requires `--reranking`) | `LlamaModel.handleRerank` (reshaped to `results`/`data` with `usage`; a request with TEI's `texts` is answered as TEI's bare `[{index, score[, text]}]` array, `text` on `return_text`, as upstream; the query and each document are a string or `{"text": string or [strings]}`, as upstream reads them since llama.cpp b11820, and one with an `image` -- which upstream ranks through the projector -- is a `400` rather than its text alone; `NativeServer` serves upstream's route with images) |
 | `POST /infill` | `LlamaModel.handleInfill` (fill-in-the-middle autocomplete) |
 | `GET /v1/models` | the configured model id |
 | `GET /metrics` | native server and per-slot token/cache counters (JSON) |

TODO.md

@@ -575,8 +575,8 @@ stays a no-op -- callers repeat the stop until the serve returns, as `RpcServer`
 The bump itself is done (b11639 -> b11760 -> b11761 -> b11799 -> b11800, b11803 on 2026-10-21: Flint, SenseNova
 U1 and a jinja fix, nothing to wire; b11809 on 2026-10-22: LoRA on prism.hadamard folded weights, an OpenCL
 ARGMAX kernel, nothing to wire; b11814 on 2026-10-22: the `tokenizer.ggml.clean_spaces` key and the Pixtral
-pre-tokenizer change, nothing to wire; b11815 on 2026-10-22: a SYCL CONV_2D layout fix, nothing to wire; patches `0016` and `0019` dropped, `0007` refreshed;
-`docs/history/llama-cpp-breaking-changes.md` has the rows). Four things the range brings that this
+pre-tokenizer change, nothing to wire; b11815 on 2026-10-22: a SYCL CONV_2D layout fix, nothing to wire; b11820 on 2026-10-22: typed rerank items, `OpenAiCompatServer` reads their text and refuses images; b11827 on 2026-10-22: SYCL XMX mat-vec, OpenCL depthwise conv, MiniCPM-OCR, the `--mlock` split-buffer fix, nothing to wire; patches `0016` and `0019` dropped, `0007` refreshed;
+`docs/history/llama-cpp-breaking-changes.md` has the rows). Five things the range brings that this
 library does not expose, each a decision rather than an omission:
 
 - **Server-side TTS through `LlamaModel` / the Java transport.** `POST /tts` (b11781, #26603) runs
@@ -586,6 +586,14 @@ library does not expose, each a decision rather than an omission:
   `/v1/audio/speech`-shaped route on `OpenAiCompatServer` would make it reachable without the native
   frontend; `TextToSpeech` (its own context over `mtmd_helper::gen_audio`) stays the one-shot path. Not before
   the release.
+- **Multimodal rerank through `LlamaModel` / the Java transport.** Since b11820 (#29625) upstream's rerank
+  route takes `{"text": ..., "image": ...}` items and splices the images through `process_mtmd_prompt`
+  (`format_prompt_rerank(..., query_files, doc_files, ...)`, `handle_media()` for the decoding); `NativeServer`
+  serves it. `engine::rerank` passes empty file lists, `jllama_rerank` and `LlamaModel.rerank` take strings,
+  and `OpenAiCompatServer` refuses an `image` with a `400` naming `NativeServer` rather than ranking the text
+  alone. Wiring it means items in the C API (text plus base64 media, as the multimodal prompt of
+  `jllama_completions` -- `multimodal_data` -- already carries) and a typed overload on `LlamaModel`; it
+  needs a multimodal reranker in the CI model set to be tested. Not before the release.
 - **Realtime transcription** (`/v1/realtime`, WebSocket, b11780, #30636: `mtmd_audio_stream`,
   `SERVER_TASK_TYPE_TRANSCRIBE_RT`, `SERVER_CONTROL_ACTION_AUDIO_*`). Served by `NativeServer`; the JDK's
   `com.sun.net.httpserver` has no WebSocket, so the Java transport cannot follow without a dependency. Not