From 21285330e27c5bafa4142cfa7fea2ffe63eb3dab Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 28 Oct 2026 12:50:48 +0100 Subject: [PATCH 1/6] Upgrade llama.cpp from b11901 to b11929 28 upstream commits, compatible: no source change besides the pin. All 14 patches apply unchanged; built with .github/build.sh -DBUILD_TESTING=ON (GCC 13.3): 0 warnings, ctest 725/725. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DuAkLtHrCTx81vP5Mt4jad --- CLAUDE.md | 10 +++++----- README.md | 2 +- llama/CMakeLists.txt | 4 ++-- llama/legal/natives/NOTICE.md | 4 ++-- .../net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4fc7c3d50..9e8387dc5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11901** +Current llama.cpp pinned version: **b11929** ## Natives jars: library jars and GPU module jars (`.github/natives.csv`) @@ -215,7 +215,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1 a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static platform (macOS) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT` - and `GGML_CUDA_GRAPHS_DEFAULT` at b11901 (unchanged since b11639) -- because adding ggml first skips that block; re-check with + and `GGML_CUDA_GRAPHS_DEFAULT` at b11929 (unchanged since b11639) -- because adding ggml first skips that block; re-check with `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once: ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which @@ -1233,7 +1233,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11901 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11929 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -1273,7 +1273,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11901`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11929`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -2853,7 +2853,7 @@ the parameterized `CApiVocabOnly` case expands to 22 instances and `MoeRepack` t #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11901`. +llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11929`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 0a15e3fb0..a11226c39 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11901](https://img.shields.io/badge/llama.cpp-%23b11901-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11901) +[![llama.cpp b11929](https://img.shields.io/badge/llama.cpp-%23b11929-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11929) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index a98df2101..d78aa67ab 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -356,8 +356,8 @@ endif() # hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable # asserts that the fetched commit's count IS the number. Clear it once the tag exists; # `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b` prints the hash. -set(JLLAMA_GIT_TAG b11901) -set(JLLAMA_GIT_COMMIT "5320c87ff0d994845b44ce7976f10055ec697447") # the commit the tag b11901 names; kept as the pin (a tag name can move) +set(JLLAMA_GIT_TAG b11929) +set(JLLAMA_GIT_COMMIT "74a1fd3d039b5be814d54ee21d419cbee4cb00b8") # the commit the tag b11929 names; kept as the pin (a tag name can move) if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$") message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b, got '${JLLAMA_GIT_TAG}'") endif() diff --git a/llama/legal/natives/NOTICE.md b/llama/legal/natives/NOTICE.md index 47f83c76c..432f81fc3 100644 --- a/llama/legal/natives/NOTICE.md +++ b/llama/legal/natives/NOTICE.md @@ -22,7 +22,7 @@ texts are not needed here. | Component | Version | Compiled into | SPDX | Texts | Source | |---|---|---|---|---|---| -| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11901 5320c87ff0d994845b44ce7976f10055ec697447 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | +| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11929 74a1fd3d039b5be814d54ee21d419cbee4cb00b8 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | | nlohmann/json | 3.12.0 + vendor/nlohmann/json-deep-nesting.patch | library jars | MIT | `nlohmann-json` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/licenses/LICENSE-jsonhpp | | cpp-httplib | v0.62.0 | library jars | MIT | `cpp-httplib` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/vendor/cpp-httplib/LICENSE | | BoringSSL (ssl + crypto; static; OPENSSL_NO_ASM) | 0.20261020.0 = d3bf4795ad5b79d646cddf2fec687da5b9b502db | library jars except cpu-android-* and cpu-linux-s390x | Apache-2.0 | `boringssl` | https://boringssl.googlesource.com/boringssl/+/refs/tags/0.20261020.0/LICENSE | @@ -47,7 +47,7 @@ texts are not needed here. | WebUI vendored big-integer | in tree | library jars (embedded ui.cpp) | Unlicense | `webui-big-integer` | tools/ui/src/lib/vendors/big-integer/LICENSE | | WebUI vendored decimal.js | in tree | library jars (embedded ui.cpp) | MIT | `webui-decimal.js` | tools/ui/src/lib/vendors/decimal.js/LICENCE.md | | WebUI vendored nerdamer-prime | in tree | library jars (embedded ui.cpp) | MIT | `webui-nerdamer-prime` | tools/ui/src/lib/vendors/nerdamer-prime/LICENSE | -| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11901 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | +| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11929 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | | WebUI third-party logos (huggingface.ico / exa.ico / context7.png / github-*.png) | - | library jars (yes) | trademarks (no license file) | `(not shipped)` | tools/ui/static | | CCCL (CUB / Thrust / libcu++ headers) | v3.4.3 | cuda13-linux-x86-64 + cuda13-windows-x86-64 | Apache-2.0 WITH LLVM-exception AND Apache-2.0 AND BSD-3-Clause (CUB) | `cccl` | https://github.com/NVIDIA/cccl/blob/v3.4.3/LICENSE | | CUDA Toolkit headers / libdevice | CUDA 13.4 | cuda13 modules (yes) | LicenseRef-NVIDIA-CUDA-EULA | `(not shipped)` | - | diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 3f506ad18..f5d9f856c 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11901"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11929"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11901-"} — call + * plus the resolved upstream commit, e.g. {@code "b11929-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11901"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11929"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11901"; + public static final String LLAMA_CPP_VERSION = "b11929"; // Constants holder — not instantiable. private LlamaCppVersion() {} From 260bb5d9ed2873a083c09beea98cee84f6ee4205 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 28 Oct 2026 12:50:49 +0100 Subject: [PATCH 2/6] Upgrade llama.cpp from b11929 to b11930; compile server-models-sched.cpp #21231 moves the router's request queue into the new unit tools/server/server-models-sched.cpp (plus optional device-memory accounting, --models-memory-margin). server-models.cpp calls it on every spawn: without the unit jllama_test fails to link with undefined server_models_sched symbols (measured at 85cc432e2). It goes wherever server-models.cpp goes: the non-Android block of jllama and the jllama_test list. ctest 725/725; RouterModeIntegrationTest, NativeServerAttachIntegrationTest, NativeLibraryLoadSmokeTest green. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DuAkLtHrCTx81vP5Mt4jad --- CLAUDE.md | 10 +++++----- README.md | 2 +- llama/CMakeLists.txt | 10 ++++++++-- llama/legal/natives/NOTICE.md | 4 ++-- .../net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 20 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9e8387dc5..7745c31cc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11929** +Current llama.cpp pinned version: **b11930** ## Natives jars: library jars and GPU module jars (`.github/natives.csv`) @@ -215,7 +215,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1 a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static platform (macOS) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT` - and `GGML_CUDA_GRAPHS_DEFAULT` at b11929 (unchanged since b11639) -- because adding ggml first skips that block; re-check with + and `GGML_CUDA_GRAPHS_DEFAULT` at b11930 (unchanged since b11639) -- because adding ggml first skips that block; re-check with `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once: ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which @@ -1233,7 +1233,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11929 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11930 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -1273,7 +1273,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11929`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11930`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -2853,7 +2853,7 @@ the parameterized `CApiVocabOnly` case expands to 22 instances and `MoeRepack` t #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11929`. +llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11930`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index a11226c39..62433c162 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11929](https://img.shields.io/badge/llama.cpp-%23b11929-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11929) +[![llama.cpp b11930](https://img.shields.io/badge/llama.cpp-%23b11930-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11930) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index d78aa67ab..558ec43de 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -356,8 +356,8 @@ endif() # hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable # asserts that the fetched commit's count IS the number. Clear it once the tag exists; # `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b` prints the hash. -set(JLLAMA_GIT_TAG b11929) -set(JLLAMA_GIT_COMMIT "74a1fd3d039b5be814d54ee21d419cbee4cb00b8") # the commit the tag b11929 names; kept as the pin (a tag name can move) +set(JLLAMA_GIT_TAG b11930) +set(JLLAMA_GIT_COMMIT "85cc432e2c8b92925f17dfb8937742541312f1d4") # the commit the tag b11930 names; kept as the pin (a tag name can move) if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$") message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b, got '${JLLAMA_GIT_TAG}'") endif() @@ -935,9 +935,14 @@ target_sources(jllama PRIVATE ${llama.cpp_SOURCE_DIR}/tools/server/server-stream.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-decision.cpp ) +# +# server-models-sched.cpp (b11930, #21231) is the router's request queue and device-memory +# accounting (server_models_sched, --models-memory-margin); server-models.cpp holds one and calls it +# on every spawn, so it goes wherever server-models.cpp goes, under the same Android guard. if(NOT ANDROID_ABI AND NOT OS_NAME MATCHES "Android") target_sources(jllama PRIVATE ${llama.cpp_SOURCE_DIR}/tools/server/server-models.cpp + ${llama.cpp_SOURCE_DIR}/tools/server/server-models-sched.cpp ) endif() @@ -1386,6 +1391,7 @@ if(BUILD_TESTING) ${llama.cpp_SOURCE_DIR}/tools/server/server-task.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-schema.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-models.cpp + ${llama.cpp_SOURCE_DIR}/tools/server/server-models-sched.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-stream.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-decision.cpp ) diff --git a/llama/legal/natives/NOTICE.md b/llama/legal/natives/NOTICE.md index 432f81fc3..24104b904 100644 --- a/llama/legal/natives/NOTICE.md +++ b/llama/legal/natives/NOTICE.md @@ -22,7 +22,7 @@ texts are not needed here. | Component | Version | Compiled into | SPDX | Texts | Source | |---|---|---|---|---|---| -| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11929 74a1fd3d039b5be814d54ee21d419cbee4cb00b8 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | +| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11930 85cc432e2c8b92925f17dfb8937742541312f1d4 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | | nlohmann/json | 3.12.0 + vendor/nlohmann/json-deep-nesting.patch | library jars | MIT | `nlohmann-json` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/licenses/LICENSE-jsonhpp | | cpp-httplib | v0.62.0 | library jars | MIT | `cpp-httplib` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/vendor/cpp-httplib/LICENSE | | BoringSSL (ssl + crypto; static; OPENSSL_NO_ASM) | 0.20261020.0 = d3bf4795ad5b79d646cddf2fec687da5b9b502db | library jars except cpu-android-* and cpu-linux-s390x | Apache-2.0 | `boringssl` | https://boringssl.googlesource.com/boringssl/+/refs/tags/0.20261020.0/LICENSE | @@ -47,7 +47,7 @@ texts are not needed here. | WebUI vendored big-integer | in tree | library jars (embedded ui.cpp) | Unlicense | `webui-big-integer` | tools/ui/src/lib/vendors/big-integer/LICENSE | | WebUI vendored decimal.js | in tree | library jars (embedded ui.cpp) | MIT | `webui-decimal.js` | tools/ui/src/lib/vendors/decimal.js/LICENCE.md | | WebUI vendored nerdamer-prime | in tree | library jars (embedded ui.cpp) | MIT | `webui-nerdamer-prime` | tools/ui/src/lib/vendors/nerdamer-prime/LICENSE | -| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11929 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | +| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11930 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | | WebUI third-party logos (huggingface.ico / exa.ico / context7.png / github-*.png) | - | library jars (yes) | trademarks (no license file) | `(not shipped)` | tools/ui/static | | CCCL (CUB / Thrust / libcu++ headers) | v3.4.3 | cuda13-linux-x86-64 + cuda13-windows-x86-64 | Apache-2.0 WITH LLVM-exception AND Apache-2.0 AND BSD-3-Clause (CUB) | `cccl` | https://github.com/NVIDIA/cccl/blob/v3.4.3/LICENSE | | CUDA Toolkit headers / libdevice | CUDA 13.4 | cuda13 modules (yes) | LicenseRef-NVIDIA-CUDA-EULA | `(not shipped)` | - | diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index f5d9f856c..062bc22c7 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11929"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11930"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11929-"} — call + * plus the resolved upstream commit, e.g. {@code "b11930-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11929"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11930"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11929"; + public static final String LLAMA_CPP_VERSION = "b11930"; // Constants holder — not instantiable. private LlamaCppVersion() {} From ba03eed26c813d9a7b7ba377fae64dafd577bde2 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 28 Oct 2026 12:50:49 +0100 Subject: [PATCH 3/6] Upgrade llama.cpp from b11930 to b11945 15 upstream commits, compatible: LFM3 models, deferred tools, template reasoning-effort levels in /props, an RPC robustness fix. No source change besides the pin; 0 warnings, ctest 725/725. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DuAkLtHrCTx81vP5Mt4jad --- CLAUDE.md | 10 +++++----- README.md | 2 +- llama/CMakeLists.txt | 4 ++-- llama/legal/natives/NOTICE.md | 4 ++-- .../net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 7745c31cc..1fc1e0ba2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11930** +Current llama.cpp pinned version: **b11945** ## Natives jars: library jars and GPU module jars (`.github/natives.csv`) @@ -215,7 +215,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1 a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static platform (macOS) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT` - and `GGML_CUDA_GRAPHS_DEFAULT` at b11930 (unchanged since b11639) -- because adding ggml first skips that block; re-check with + and `GGML_CUDA_GRAPHS_DEFAULT` at b11945 (unchanged since b11639) -- because adding ggml first skips that block; re-check with `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once: ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which @@ -1233,7 +1233,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11930 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11945 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -1273,7 +1273,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11930`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11945`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -2853,7 +2853,7 @@ the parameterized `CApiVocabOnly` case expands to 22 instances and `MoeRepack` t #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11930`. +llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11945`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 62433c162..a50f2d6cc 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11930](https://img.shields.io/badge/llama.cpp-%23b11930-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11930) +[![llama.cpp b11945](https://img.shields.io/badge/llama.cpp-%23b11945-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11945) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 558ec43de..29a1bef4c 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -356,8 +356,8 @@ endif() # hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable # asserts that the fetched commit's count IS the number. Clear it once the tag exists; # `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b` prints the hash. -set(JLLAMA_GIT_TAG b11930) -set(JLLAMA_GIT_COMMIT "85cc432e2c8b92925f17dfb8937742541312f1d4") # the commit the tag b11930 names; kept as the pin (a tag name can move) +set(JLLAMA_GIT_TAG b11945) +set(JLLAMA_GIT_COMMIT "d05706b4f32a545814856e98ca518292585443b9") # the commit the tag b11945 names; kept as the pin (a tag name can move) if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$") message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b, got '${JLLAMA_GIT_TAG}'") endif() diff --git a/llama/legal/natives/NOTICE.md b/llama/legal/natives/NOTICE.md index 24104b904..c9cbe46f0 100644 --- a/llama/legal/natives/NOTICE.md +++ b/llama/legal/natives/NOTICE.md @@ -22,7 +22,7 @@ texts are not needed here. | Component | Version | Compiled into | SPDX | Texts | Source | |---|---|---|---|---|---| -| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11930 85cc432e2c8b92925f17dfb8937742541312f1d4 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | +| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11945 d05706b4f32a545814856e98ca518292585443b9 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | | nlohmann/json | 3.12.0 + vendor/nlohmann/json-deep-nesting.patch | library jars | MIT | `nlohmann-json` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/licenses/LICENSE-jsonhpp | | cpp-httplib | v0.62.0 | library jars | MIT | `cpp-httplib` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/vendor/cpp-httplib/LICENSE | | BoringSSL (ssl + crypto; static; OPENSSL_NO_ASM) | 0.20261020.0 = d3bf4795ad5b79d646cddf2fec687da5b9b502db | library jars except cpu-android-* and cpu-linux-s390x | Apache-2.0 | `boringssl` | https://boringssl.googlesource.com/boringssl/+/refs/tags/0.20261020.0/LICENSE | @@ -47,7 +47,7 @@ texts are not needed here. | WebUI vendored big-integer | in tree | library jars (embedded ui.cpp) | Unlicense | `webui-big-integer` | tools/ui/src/lib/vendors/big-integer/LICENSE | | WebUI vendored decimal.js | in tree | library jars (embedded ui.cpp) | MIT | `webui-decimal.js` | tools/ui/src/lib/vendors/decimal.js/LICENCE.md | | WebUI vendored nerdamer-prime | in tree | library jars (embedded ui.cpp) | MIT | `webui-nerdamer-prime` | tools/ui/src/lib/vendors/nerdamer-prime/LICENSE | -| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11930 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | +| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11945 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | | WebUI third-party logos (huggingface.ico / exa.ico / context7.png / github-*.png) | - | library jars (yes) | trademarks (no license file) | `(not shipped)` | tools/ui/static | | CCCL (CUB / Thrust / libcu++ headers) | v3.4.3 | cuda13-linux-x86-64 + cuda13-windows-x86-64 | Apache-2.0 WITH LLVM-exception AND Apache-2.0 AND BSD-3-Clause (CUB) | `cccl` | https://github.com/NVIDIA/cccl/blob/v3.4.3/LICENSE | | CUDA Toolkit headers / libdevice | CUDA 13.4 | cuda13 modules (yes) | LicenseRef-NVIDIA-CUDA-EULA | `(not shipped)` | - | diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 062bc22c7..1c0cdcfa2 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11930"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11945"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11930-"} — call + * plus the resolved upstream commit, e.g. {@code "b11945-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11930"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11945"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11930"; + public static final String LLAMA_CPP_VERSION = "b11945"; // Constants holder — not instantiable. private LlamaCppVersion() {} From 6ded747e661d05ca4793af1bfa2dc4b29f51edae Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 28 Oct 2026 12:50:50 +0100 Subject: [PATCH 4/6] Upgrade llama.cpp from b11945 to b11946; compile server-realtime.cpp #31269 serves realtime conversations on /v1/realtime from the new unit tools/server/server-realtime.cpp. server-context.cpp creates the slot and calls into it: without the unit jllama_test fails to link (measured at 8ca4060b7). It only uses the ws_conn interface and spawns nothing, so it joins server-decision.cpp outside the Android guard, and the jllama_test list. ctest 725/725. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DuAkLtHrCTx81vP5Mt4jad --- CLAUDE.md | 10 +++++----- README.md | 2 +- llama/CMakeLists.txt | 12 ++++++++++-- llama/legal/natives/NOTICE.md | 4 ++-- .../net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 22 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1fc1e0ba2..a1c7feaa4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11945** +Current llama.cpp pinned version: **b11946** ## Natives jars: library jars and GPU module jars (`.github/natives.csv`) @@ -215,7 +215,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1 a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static platform (macOS) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT` - and `GGML_CUDA_GRAPHS_DEFAULT` at b11945 (unchanged since b11639) -- because adding ggml first skips that block; re-check with + and `GGML_CUDA_GRAPHS_DEFAULT` at b11946 (unchanged since b11639) -- because adding ggml first skips that block; re-check with `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once: ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which @@ -1233,7 +1233,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11945 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11946 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -1273,7 +1273,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11945`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11946`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -2853,7 +2853,7 @@ the parameterized `CApiVocabOnly` case expands to 22 instances and `MoeRepack` t #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11945`. +llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11946`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index a50f2d6cc..81b4b112f 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11945](https://img.shields.io/badge/llama.cpp-%23b11945-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11945) +[![llama.cpp b11946](https://img.shields.io/badge/llama.cpp-%23b11946-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11946) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 29a1bef4c..c9d7b2bb3 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -356,8 +356,8 @@ endif() # hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable # asserts that the fetched commit's count IS the number. Clear it once the tag exists; # `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b` prints the hash. -set(JLLAMA_GIT_TAG b11945) -set(JLLAMA_GIT_COMMIT "d05706b4f32a545814856e98ca518292585443b9") # the commit the tag b11945 names; kept as the pin (a tag name can move) +set(JLLAMA_GIT_TAG b11946) +set(JLLAMA_GIT_COMMIT "8ca4060b7e30e76baaa5670ee1ec1b760b7f8552") # the commit the tag b11946 names; kept as the pin (a tag name can move) if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$") message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b, got '${JLLAMA_GIT_TAG}'") endif() @@ -927,6 +927,12 @@ add_library(jllama SHARED # server-stream.cpp -- a missing unit is an undefined reference, latent on Linux # but a hard link error on macOS/ld64 and Windows/MSVC. Platform-neutral, so it # stays outside the Android guard too. +# +# server-realtime.cpp (b11946, #31269) holds the realtime conversations over /v1/realtime +# (server_realtime_slot, server_realtime_conversation); server-context.cpp creates the slot and +# calls into it from its main loop and its websocket route, so it is mandatory for the same reason. +# It talks to the socket only through server_http_context::ws_conn, an interface, and spawns +# nothing, so it stays outside the Android guard as well. target_sources(jllama PRIVATE ${llama.cpp_SOURCE_DIR}/tools/server/server-context.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-queue.cpp @@ -934,6 +940,7 @@ target_sources(jllama PRIVATE ${llama.cpp_SOURCE_DIR}/tools/server/server-schema.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-stream.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-decision.cpp + ${llama.cpp_SOURCE_DIR}/tools/server/server-realtime.cpp ) # # server-models-sched.cpp (b11930, #21231) is the router's request queue and device-memory @@ -1394,6 +1401,7 @@ if(BUILD_TESTING) ${llama.cpp_SOURCE_DIR}/tools/server/server-models-sched.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-stream.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-decision.cpp + ${llama.cpp_SOURCE_DIR}/tools/server/server-realtime.cpp ) target_include_directories(jllama_test PRIVATE diff --git a/llama/legal/natives/NOTICE.md b/llama/legal/natives/NOTICE.md index c9cbe46f0..19a80bfa5 100644 --- a/llama/legal/natives/NOTICE.md +++ b/llama/legal/natives/NOTICE.md @@ -22,7 +22,7 @@ texts are not needed here. | Component | Version | Compiled into | SPDX | Texts | Source | |---|---|---|---|---|---| -| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11945 d05706b4f32a545814856e98ca518292585443b9 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | +| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11946 8ca4060b7e30e76baaa5670ee1ec1b760b7f8552 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | | nlohmann/json | 3.12.0 + vendor/nlohmann/json-deep-nesting.patch | library jars | MIT | `nlohmann-json` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/licenses/LICENSE-jsonhpp | | cpp-httplib | v0.62.0 | library jars | MIT | `cpp-httplib` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/vendor/cpp-httplib/LICENSE | | BoringSSL (ssl + crypto; static; OPENSSL_NO_ASM) | 0.20261020.0 = d3bf4795ad5b79d646cddf2fec687da5b9b502db | library jars except cpu-android-* and cpu-linux-s390x | Apache-2.0 | `boringssl` | https://boringssl.googlesource.com/boringssl/+/refs/tags/0.20261020.0/LICENSE | @@ -47,7 +47,7 @@ texts are not needed here. | WebUI vendored big-integer | in tree | library jars (embedded ui.cpp) | Unlicense | `webui-big-integer` | tools/ui/src/lib/vendors/big-integer/LICENSE | | WebUI vendored decimal.js | in tree | library jars (embedded ui.cpp) | MIT | `webui-decimal.js` | tools/ui/src/lib/vendors/decimal.js/LICENCE.md | | WebUI vendored nerdamer-prime | in tree | library jars (embedded ui.cpp) | MIT | `webui-nerdamer-prime` | tools/ui/src/lib/vendors/nerdamer-prime/LICENSE | -| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11945 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | +| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11946 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | | WebUI third-party logos (huggingface.ico / exa.ico / context7.png / github-*.png) | - | library jars (yes) | trademarks (no license file) | `(not shipped)` | tools/ui/static | | CCCL (CUB / Thrust / libcu++ headers) | v3.4.3 | cuda13-linux-x86-64 + cuda13-windows-x86-64 | Apache-2.0 WITH LLVM-exception AND Apache-2.0 AND BSD-3-Clause (CUB) | `cccl` | https://github.com/NVIDIA/cccl/blob/v3.4.3/LICENSE | | CUDA Toolkit headers / libdevice | CUDA 13.4 | cuda13 modules (yes) | LicenseRef-NVIDIA-CUDA-EULA | `(not shipped)` | - | diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 1c0cdcfa2..9e7bf0384 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11945"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11946"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11945-"} — call + * plus the resolved upstream commit, e.g. {@code "b11946-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11945"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11946"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11945"; + public static final String LLAMA_CPP_VERSION = "b11946"; // Constants holder — not instantiable. private LlamaCppVersion() {} From f5becf7d4b7a49e9dc6d7e2c01d8cd76cfe8c8f7 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 28 Oct 2026 12:51:02 +0100 Subject: [PATCH 5/6] Upgrade llama.cpp from b11946 to b11954 8 upstream commits, compatible: qwen3-coder parsing, a Vulkan and a CUDA kernel fix, WebUI work. No source change besides the pin; 0 warnings, ctest 725/725, the full Java suite (mvn -P jcstress test) 2395 tests, 0 failures, 2390 executed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DuAkLtHrCTx81vP5Mt4jad --- CLAUDE.md | 10 +++++----- README.md | 2 +- llama/CMakeLists.txt | 4 ++-- llama/legal/natives/NOTICE.md | 4 ++-- .../net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index a1c7feaa4..06a2d8769 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11946** +Current llama.cpp pinned version: **b11954** ## Natives jars: library jars and GPU module jars (`.github/natives.csv`) @@ -215,7 +215,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1 a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static platform (macOS) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT` - and `GGML_CUDA_GRAPHS_DEFAULT` at b11946 (unchanged since b11639) -- because adding ggml first skips that block; re-check with + and `GGML_CUDA_GRAPHS_DEFAULT` at b11954 (unchanged since b11639) -- because adding ggml first skips that block; re-check with `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once: ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which @@ -1233,7 +1233,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11946 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11954 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -1273,7 +1273,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11946`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11954`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -2853,7 +2853,7 @@ the parameterized `CApiVocabOnly` case expands to 22 instances and `MoeRepack` t #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11946`. +llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11954`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 81b4b112f..cae0350f6 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11946](https://img.shields.io/badge/llama.cpp-%23b11946-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11946) +[![llama.cpp b11954](https://img.shields.io/badge/llama.cpp-%23b11954-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11954) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index c9d7b2bb3..784930e43 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -356,8 +356,8 @@ endif() # hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable # asserts that the fetched commit's count IS the number. Clear it once the tag exists; # `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b` prints the hash. -set(JLLAMA_GIT_TAG b11946) -set(JLLAMA_GIT_COMMIT "8ca4060b7e30e76baaa5670ee1ec1b760b7f8552") # the commit the tag b11946 names; kept as the pin (a tag name can move) +set(JLLAMA_GIT_TAG b11954) +set(JLLAMA_GIT_COMMIT "f1a7dea99aae1623bad10c4ce31fe220033ab7f6") # the commit the tag b11954 names; kept as the pin (a tag name can move) if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$") message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b, got '${JLLAMA_GIT_TAG}'") endif() diff --git a/llama/legal/natives/NOTICE.md b/llama/legal/natives/NOTICE.md index 19a80bfa5..27ab3877a 100644 --- a/llama/legal/natives/NOTICE.md +++ b/llama/legal/natives/NOTICE.md @@ -22,7 +22,7 @@ texts are not needed here. | Component | Version | Compiled into | SPDX | Texts | Source | |---|---|---|---|---|---| -| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11946 8ca4060b7e30e76baaa5670ee1ec1b760b7f8552 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | +| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11954 f1a7dea99aae1623bad10c4ce31fe220033ab7f6 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | | nlohmann/json | 3.12.0 + vendor/nlohmann/json-deep-nesting.patch | library jars | MIT | `nlohmann-json` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/licenses/LICENSE-jsonhpp | | cpp-httplib | v0.62.0 | library jars | MIT | `cpp-httplib` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/vendor/cpp-httplib/LICENSE | | BoringSSL (ssl + crypto; static; OPENSSL_NO_ASM) | 0.20261020.0 = d3bf4795ad5b79d646cddf2fec687da5b9b502db | library jars except cpu-android-* and cpu-linux-s390x | Apache-2.0 | `boringssl` | https://boringssl.googlesource.com/boringssl/+/refs/tags/0.20261020.0/LICENSE | @@ -47,7 +47,7 @@ texts are not needed here. | WebUI vendored big-integer | in tree | library jars (embedded ui.cpp) | Unlicense | `webui-big-integer` | tools/ui/src/lib/vendors/big-integer/LICENSE | | WebUI vendored decimal.js | in tree | library jars (embedded ui.cpp) | MIT | `webui-decimal.js` | tools/ui/src/lib/vendors/decimal.js/LICENCE.md | | WebUI vendored nerdamer-prime | in tree | library jars (embedded ui.cpp) | MIT | `webui-nerdamer-prime` | tools/ui/src/lib/vendors/nerdamer-prime/LICENSE | -| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11946 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | +| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11954 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | | WebUI third-party logos (huggingface.ico / exa.ico / context7.png / github-*.png) | - | library jars (yes) | trademarks (no license file) | `(not shipped)` | tools/ui/static | | CCCL (CUB / Thrust / libcu++ headers) | v3.4.3 | cuda13-linux-x86-64 + cuda13-windows-x86-64 | Apache-2.0 WITH LLVM-exception AND Apache-2.0 AND BSD-3-Clause (CUB) | `cccl` | https://github.com/NVIDIA/cccl/blob/v3.4.3/LICENSE | | CUDA Toolkit headers / libdevice | CUDA 13.4 | cuda13 modules (yes) | LicenseRef-NVIDIA-CUDA-EULA | `(not shipped)` | - | diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 9e7bf0384..9eebc4b1e 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11946"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11954"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11946-"} — call + * plus the resolved upstream commit, e.g. {@code "b11954-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11946"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11954"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11946"; + public static final String LLAMA_CPP_VERSION = "b11954"; // Constants holder — not instantiable. private LlamaCppVersion() {} From 6cc9c8909a71722992ace73865b4fcd82d75874b Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Wed, 28 Oct 2026 12:51:02 +0100 Subject: [PATCH 6/6] docs: llama.cpp b11901 -> b11954 in the history, CHANGELOG and CLAUDE.md One row per step in docs/history/llama-cpp-breaking-changes.md plus the patches row (all 14 apply at every tag of the range, drop-checks re-read at f1a7dea99); the CHANGELOG entry; CLAUDE.md names the two server units the build compiles in now. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DuAkLtHrCTx81vP5Mt4jad --- CHANGELOG.md | 10 ++++++++++ CLAUDE.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 6 ++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b17e89444..27208902a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,16 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by ## [Unreleased] ### Changed +- **llama.cpp b11954** (from b11901, 53 upstream commits, in five steps, all tagged). Two upstream + server units are compiled into `libjllama` now: `server-models-sched.cpp` (#21231, the router's + request queue, with optional device-memory accounting through `--models-memory-margin`) next to + `server-models.cpp`, and `server-realtime.cpp` (#31269, realtime conversations and transcription on + `/v1/realtime`), which the server context needs on every platform. Upstream also brings LiquidAI + LFM3 (text, VL, Audio, Omni and the d1 decision models, #31252, #31257, #31275) and MiniCPM-OCR-Jev + (#30980), deferred tools and `tool_reference` content parts in the chat layer (#31256), the + reasoning effort levels of a chat template in `/props` (#31271), qwen3-coder parsing that stops + before the template separators (#30557); a parse without a model now fails with `--model or -hf is + required` (#31242, the text only). No patch changed; the request schema is the same. - **Dependencies and build plugins:** logback-classic 1.6.6, NullAway 0.14.3, maven-surefire-plugin 3.6.1, jacoco-maven-plugin 0.8.16, palantir-java-format 2.103.0 (no formatting change), and in the standalone `llama-atmosphere-agent` Atmosphere 4.0.76 (with surefire and palantir); the Android Gradle diff --git a/CLAUDE.md b/CLAUDE.md index 06a2d8769..3e41f5cf7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2340,7 +2340,7 @@ If the local check passes (`BUILD SUCCESS`), the `mvn package` job in 2. **`common_json` converts to `std::string` implicitly**, so it binds happily to a `const nlohmann::json &` parameter (via nlohmann's string-constructible converting constructor) and then throws `json::type_error 302` at runtime. Never declare a project helper as taking `nlohmann::json` when callers pass the `json` alias — `require_json_field_impl` is a template for exactly this reason. Other differences to know: no `get_ref`/`array_t`/`type_name()`; a braced list in *value* position does not build an array (write `json::array({...})`); `at(key)` needs an explicit `.get()`; errors are `common_json_error`; and `get()` is limited to the types explicitly specialised in `common/json.cpp`. `log_helpers.hpp` and `train_engine.cpp` keep their own `nlohmann::json` alias — they never touch the server's `json`. - Uses `nlohmann/json` for JSON deserialization of parameters in the two files named above; everything on the server path uses `common_json`. -- The upstream server library (`server-context.cpp`, `server-queue.cpp`, `server-task.cpp`, `server-schema.cpp`, `server-models.cpp`, and — since b9829 — `server-stream.cpp`) is compiled directly into `jllama` via CMake — there is no hand-ported `server.hpp` fork. **`server-stream.cpp` is mandatory, not optional:** it defines the resumable-streaming SSE replay buffer (`g_stream_sessions`, `stream_session_attach_pipe`, `stream_aware_should_stop`, `stream_conv_id_from_headers`, the `stream_pipe_*` types) that `server-context.cpp` / `server-http.cpp` / `server-models.cpp` now `#include "server-stream.h"` and call, so omitting it fails the link with undefined references. It is platform-neutral (threads + std mutex/condvar, no `subprocess.h`/`posix_spawn_*`), so it builds on Android too and sits outside the `server-models.cpp` Android guard. `jllama` wires its own JNI routes and never calls `g_stream_sessions.start_gc()` (only the excluded standalone `server.cpp` `main()` does), so its GC thread stays dormant. **Phase 2:** the upstream HTTP transport (`tools/server/server-http.cpp`) is compiled into `jllama` too, and `jllama` links upstream's `cpp-httplib` static target (`vendor/cpp-httplib/`) instead of compiling `httplib.cpp` a second time (it did until 6.0.0, without SSL -- see "HTTPS: BoringSSL, statically" for what that cost), so the OpenAI-compatible server can be driven natively from JNI *inside* `libjllama` — no separate `llama-server` executable (a JNI shared library loads anywhere a JVM runs, which a standalone binary does not). `server-http.cpp` does `#include "ui.h"` (the WebUI asset table that `tools/ui`/`llama-ui` normally generates); since the Svelte WebUI is not shipped, `src/main/cpp/webui_stub/ui.h` supplies the upstream **empty-asset** interface and leaves `LLAMA_UI_HAS_ASSETS` undefined (all static-asset-serving blocks compile out). `` already resolves through `llama-common` — since upstream #27304 (b10488) not from a `PUBLIC ../vendor` include dir of its own but transitively, via the `vendor::nlohmann` / `vendor::sheredom` INTERFACE targets it links PUBLIC, each of which exports the `vendor/` root -- and that root came **before** the then FetchContent `json-src` include dir on every `jllama` compile line (measured in `flags.make`), so even our own `#include ` in `log_helpers.hpp` / `train_engine.cpp` compiles against upstream's **vendored** copy: one copy per TU. Since b11556 that copy is 3.12.0 plus upstream's `json-deep-nesting.patch` (nlohmann/json#5387, applied by `scripts/sync_vendor.py`), so the FetchContent `v3.12.0` was not even the same code; it is gone, and `llama/CMakeLists.txt` ("json: upstream's vendored copy, and only that one") guards that upstream's copy is there. **HTTPS is on**: the `cpp-httplib` target carries `CPPHTTPLIB_OPENSSL_SUPPORT` as a PUBLIC define whenever an SSL library is configured, and BoringSSL is linked statically on every desktop platform, so `server-http.cpp` serves TLS with `--ssl-key-file`/`--ssl-cert-file` and `common/download.cpp` fetches `https://` URLs (not on Android and s390x, where `LLAMA_OPENSSL=OFF`). **`server.cpp`, `server-tools.cpp` and `server-mcp.cpp` are now compiled in too** (on non-Android — they pull in `subprocess.h`/`posix_spawn_*`, so they share `server-models.cpp`'s Android guard): b9870 exposes `server.cpp`'s entry as `int llama_server(int, char**)` (no `main` in the file), and `patches/0006` makes it embeddable (no process signal handlers, forwarded-argv parse, out-of-band shutdown). **`server-mcp.cpp` is new in b10154** (upstream MCP-server support): both `server.cpp` (`llama_server`'s `mcp_mgr` lifecycle) and `server-tools.cpp` (`tools.setup(..., mcp_mgr)` / `server_mcp::call_tool`) reference `server_mcp`, so it **must** be in the `target_sources` list or the link fails with undefined `server_mcp::{start,shutdown,call_tool,list_tools,~server_mcp}` — **latent on Linux** (a shared object tolerates undefined symbols) but a **hard link error on macOS/ld64 and Windows/MSVC**. It is compiled into `jllama` and — on non-Android — into `jllama_test` too, together with `server.cpp`, `server-tools.cpp` and `server-http.cpp` (and the linked `cpp-httplib` target), so `test_native_server_shutdown.cpp` can drive the real `llama_server()`. The `NativeServer` JNI bridge (`src/main/cpp/native_server.cpp`) calls `llama_server` on a worker thread, so the **full** upstream server — WebUI and all — runs inside `libjllama`. See "Two server modes" below. +- The upstream server library (`server-context.cpp`, `server-queue.cpp`, `server-task.cpp`, `server-schema.cpp`, `server-models.cpp`, and — since b9829 — `server-stream.cpp`) is compiled directly into `jllama` via CMake — there is no hand-ported `server.hpp` fork. **`server-stream.cpp` is mandatory, not optional:** it defines the resumable-streaming SSE replay buffer (`g_stream_sessions`, `stream_session_attach_pipe`, `stream_aware_should_stop`, `stream_conv_id_from_headers`, the `stream_pipe_*` types) that `server-context.cpp` / `server-http.cpp` / `server-models.cpp` now `#include "server-stream.h"` and call, so omitting it fails the link with undefined references. It is platform-neutral (threads + std mutex/condvar, no `subprocess.h`/`posix_spawn_*`), so it builds on Android too and sits outside the `server-models.cpp` Android guard. `jllama` wires its own JNI routes and never calls `g_stream_sessions.start_gc()` (only the excluded standalone `server.cpp` `main()` does), so its GC thread stays dormant. **Two more units are mandatory for the same reason:** `server-realtime.cpp` (b11946, #31269; `server-context.cpp` creates a `server_realtime_slot` and serves `/v1/realtime` conversations through it -- it uses only the `ws_conn` interface, so it builds on Android too) and, next to `server-models.cpp` under its Android guard, `server-models-sched.cpp` (b11930, #21231; the router's request queue and memory accounting, which `server-models.cpp` calls on every spawn). Both were measured as link failures of `jllama_test` without them. **Phase 2:** the upstream HTTP transport (`tools/server/server-http.cpp`) is compiled into `jllama` too, and `jllama` links upstream's `cpp-httplib` static target (`vendor/cpp-httplib/`) instead of compiling `httplib.cpp` a second time (it did until 6.0.0, without SSL -- see "HTTPS: BoringSSL, statically" for what that cost), so the OpenAI-compatible server can be driven natively from JNI *inside* `libjllama` — no separate `llama-server` executable (a JNI shared library loads anywhere a JVM runs, which a standalone binary does not). `server-http.cpp` does `#include "ui.h"` (the WebUI asset table that `tools/ui`/`llama-ui` normally generates); since the Svelte WebUI is not shipped, `src/main/cpp/webui_stub/ui.h` supplies the upstream **empty-asset** interface and leaves `LLAMA_UI_HAS_ASSETS` undefined (all static-asset-serving blocks compile out). `` already resolves through `llama-common` — since upstream #27304 (b10488) not from a `PUBLIC ../vendor` include dir of its own but transitively, via the `vendor::nlohmann` / `vendor::sheredom` INTERFACE targets it links PUBLIC, each of which exports the `vendor/` root -- and that root came **before** the then FetchContent `json-src` include dir on every `jllama` compile line (measured in `flags.make`), so even our own `#include ` in `log_helpers.hpp` / `train_engine.cpp` compiles against upstream's **vendored** copy: one copy per TU. Since b11556 that copy is 3.12.0 plus upstream's `json-deep-nesting.patch` (nlohmann/json#5387, applied by `scripts/sync_vendor.py`), so the FetchContent `v3.12.0` was not even the same code; it is gone, and `llama/CMakeLists.txt` ("json: upstream's vendored copy, and only that one") guards that upstream's copy is there. **HTTPS is on**: the `cpp-httplib` target carries `CPPHTTPLIB_OPENSSL_SUPPORT` as a PUBLIC define whenever an SSL library is configured, and BoringSSL is linked statically on every desktop platform, so `server-http.cpp` serves TLS with `--ssl-key-file`/`--ssl-cert-file` and `common/download.cpp` fetches `https://` URLs (not on Android and s390x, where `LLAMA_OPENSSL=OFF`). **`server.cpp`, `server-tools.cpp` and `server-mcp.cpp` are now compiled in too** (on non-Android — they pull in `subprocess.h`/`posix_spawn_*`, so they share `server-models.cpp`'s Android guard): b9870 exposes `server.cpp`'s entry as `int llama_server(int, char**)` (no `main` in the file), and `patches/0006` makes it embeddable (no process signal handlers, forwarded-argv parse, out-of-band shutdown). **`server-mcp.cpp` is new in b10154** (upstream MCP-server support): both `server.cpp` (`llama_server`'s `mcp_mgr` lifecycle) and `server-tools.cpp` (`tools.setup(..., mcp_mgr)` / `server_mcp::call_tool`) reference `server_mcp`, so it **must** be in the `target_sources` list or the link fails with undefined `server_mcp::{start,shutdown,call_tool,list_tools,~server_mcp}` — **latent on Linux** (a shared object tolerates undefined symbols) but a **hard link error on macOS/ld64 and Windows/MSVC**. It is compiled into `jllama` and — on non-Android — into `jllama_test` too, together with `server.cpp`, `server-tools.cpp` and `server-http.cpp` (and the linked `cpp-httplib` target), so `test_native_server_shutdown.cpp` can drive the real `llama_server()`. The `NativeServer` JNI bridge (`src/main/cpp/native_server.cpp`) calls `llama_server` on a worker thread, so the **full** upstream server — WebUI and all — runs inside `libjllama`. See "Two server modes" below. ### Two server modes (`OpenAiCompatServer` vs `NativeServer`) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index fd704f94b..704c9a8ef 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -921,3 +921,9 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11883–b11887 | `include/llama.h`, `src/llama-sampler.cpp`, `tools/server/server-schema.cpp`, `src/models/qwen4exp.cpp`, `src/llama-arch.{cpp,h}` | **None needed in the code; a contract moved.** #31035 adds `LLAMA_DRY_MAX_SEQ_BREAKERS` (64) to `llama.h`: the request schema drops duplicate `dry_sequence_breakers` and refuses more than 64 distinct ones (an invalid request), the sampler stops at 64 with a warning. Pinned by `test_server.cpp` (`DrySequenceBreakers_DuplicatesDropped`, `DrySequenceBreakers_DistinctLimit`), noted in `withDrySequenceBreakers`' Javadoc. #31028 merges Qwen4-exp tensors on conversion (new tensor names); #30977/#31157 touch upstream CI and tests only. All twelve patches apply unchanged (`git apply` on `6050b467c`), the commit count is 11887. | | b11887–b11901 | `common/{common.{cpp,h},json-schema-to-grammar.{cpp,h},json-schema.cpp,peg-parser.{cpp,h},jinja/{parser,runtime,value}.cpp}`, `src/llama-grammar.cpp`, `src/llama-vocab.cpp`, `tools/mtmd/mtmd-helper.{cpp,h}`, `tools/server/server-context.cpp`, `tools/server/server-models.cpp`, `src/llama-ext.h`, `src/llama-arch.cpp`, `src/llama-model-saver.{cpp,h}`, `src/llama-kv-cache-dsv4.{cpp,h}`, `src/models/{deepseek4,deepseek41,dflash}.cpp`, `ggml/src/{ggml-cuda/ggml-cuda.cu,ggml-sycl/ggml-sycl.cpp,ggml-metal/kernels/mul_mm.metal,ggml-webgpu/ggml-webgpu.cpp,ggml-cpu/arch/arm/quants.c}` (14 commits, `5320c87ff` = b11901, 2026-10-25 .. 2026-10-26, tagged) | **None needed.** #31153 bounds every recursion over input upstream walks with `COMMON_RECURSIVE_LIMIT` (128, new in `common.h`): the JSON-schema converter and the schema walks (a `$ref` chain counts too; a property name of 128 characters or more is refused where `additionalProperties` makes the converter build a trie of the names), Jinja templates and values (`JINJA_RECURSIVE_LIMIT` 128, macro calls included) and `common.cpp`'s glob matcher -- each an `std::invalid_argument` / `std::runtime_error` where the native stack used to overflow and take the JVM with it; on our request paths these are request-parse errors and stay `InvalidRequestException`. The PEG chat-output parser throws `common_peg_recursion_error` (an `std::invalid_argument`, new in `peg-parser.h`) when a rule is entered more than 64 levels deep -- in the parse of the **model's output**, so it surfaces from `task_result_state::update_chat_msg` on the reading thread and reaches Java as `LlamaException` (`run_guarded` maps everything but `jllama_invalid_request` to `JLLAMA_STATUS_ERROR`). The grammar's candidate rejection, the SPM resegmentation and the vocab trie no longer recurse per character (a long GGUF token cannot overflow the stack; upstream measures faster grammar sampling, same masks), and the rule-name escaping drops `std::regex` (`gbnf_format_rule_name`, now public). #31183 adds `mtmd_helper_get_capabilities()` (`mtmd_helper_cap`: chat, vision, audio, video, audio_stream, text_gen, audio_gen, audio_gen_spk_ref) and deprecates `mtmd_helper_model_can_chat` / `mtmd_helper_support_video`; the server takes its capabilities from it with the same values, and the project calls neither deprecated function (syntax-only compile of `jllama.cpp`, `jllama_api.cpp`, `jllama_engine.cpp`, `native_server_engine.cpp`, `tts_engine.cpp` against the patched tree with `-Wdeprecated-declarations`: no diagnostic). #30880 adds C++ architecture queries to `llama-ext.h` (not included here). #31137 is DeepSeek-V4.1 review cleanup (no GGUF format change), #30072 stops a Metal output-offset overflow in large matrix multiplications, #31062 bounds the SYCL reorder buffer to 32 MiB, #30463 / #31180 are CUDA (no VLA) and ARM warning fixes, #30548 adds tests only, #31030 / #31119 are WebUI (no new npm package, `package-lock.json` unchanged), #31162 / #31174 / #31175 upstream CI. The root `CMakeLists.txt` (`_DEFAULT`s), the ggml CMake files, `vendor/`, `licenses/` and the release workflow (OpenVINO, CCCL, ROCm, SYCL pins) are unchanged; the commit count is 11901. | | b11887–b11901 | patches | **All 12 apply unchanged** (the fail-loud applier on a fresh tree at `5320c87ff`; `check-patches.py`: 12 patches, 94 hunks, 0 problems). Three patch-target files are in the range: `tools/server/server-context.cpp` (`0002`, `0003`, `0021`; #31183 caches the capabilities in a member, away from all three), `tools/server/server-models.cpp` (`0008`; one comment in `update_caps`) and `tools/mtmd/mtmd-cli.cpp` (`0001`'s flip; the capability change is away from it). Drop-checks re-read at `5320c87ff`: `0001` -- `argv = utf8.ptrs.data();` at `common/arg.cpp:1296`, no `common_params_parse_main`; `0002` -- `params_base.load_progress_callback = load_progress_callback;` unguarded at `server-context.cpp:1287`; `0003` -- no `slot_prompt_similarity` in `server-context.h`; `0008` -- no `LLAMA_SERVER_WORKER_CMD` upstream; `0021` -- the `else` branch at `server-context.cpp:1945` takes `params_base.lora_adapters` without the clear. The files of `0014`, `0015`, `0017`, `0018` and `0020` are not in the range, so their answers of the b11827–b11881 row stand. | +| b11901–b11929 | `common/{arg.cpp,common.{cpp,h}}`, `src/llama-{arch,context,graph,hparams,model,kv-cache*,memory*,moe-cache}.*`, `src/models/*.cpp`, `tools/server/server-{common.cpp,context.cpp,decision.{cpp,h},task.h}`, `ggml/src/{ggml-cpu/ops.cpp,ggml-sycl,ggml-vulkan,ggml-opencl,ggml-hexagon,ggml-metal}` | **None needed.** 28 commits: SYCL/Vulkan/OpenCL/Hexagon/Metal kernels, the CPU `concat` rewrite (#31179), the model graph builders return structs instead of `ggml_tensor **` out-params (#31142), `no_alloc` out of `llama_hparams` (#31141), the obsolete `n_pad` of the memory instances gone (#31200), GLM5-next QSA (#31009), MiniCPM-OCR-Jev as a decision model (#30980, a new `COMMON_DECISION_TYPE_*` value -- nothing here switches over that enum), WebUI work. #31242 only rewords `common_params_parse_ex`'s error to `--model or -hf is required`; nothing here matches the old text. No header the project includes changed in a way it uses. Built at `74a1fd3d0` (`.github/build.sh -DBUILD_TESTING=ON`, GCC 13.3): 0 warnings, `ctest` 725/725. | +| b11929–b11930 | `tools/server/server-models.{cpp,h}`, **new** `tools/server/server-models-sched.{cpp,h}`, `tools/server/{server-common.h,server.cpp,CMakeLists.txt}`, `common/{arg.cpp,common.{cpp,h},fit.{cpp,h}}` | **`llama/CMakeLists.txt`: `server-models-sched.cpp` compiled in.** #21231 moves the router's request queue into a new unit and adds device-memory accounting (`--models-memory-margin`, off by default; a `measure` child role and state). `server-models.cpp` holds a `server_models_sched` and calls it on every spawn, so without the unit `jllama_test` fails to link with undefined `server_models_sched::{tick,try_claim,make_room,measure_unmeasured,...}` (measured: built at `85cc432e2` without the change) -- and `libjllama.so` would have carried the same undefined symbols, latent on Linux, a link error on macOS and Windows. It goes wherever `server-models.cpp` goes (the non-Android block of `jllama` and the `jllama_test` list). `0008` still covers the new child: the scheduler spawns a measure child with the instance's rendered `meta.args` plus `--offline`, which `0008` has already rewritten to the worker command, so a JVM router's measure child is a JVM too (only with `--models-memory-margin` > 0; not exercised). Checked: `RouterModeIntegrationTest` (two JVM workers), `NativeServerAttachIntegrationTest` and `NativeLibraryLoadSmokeTest` green, `ctest` 725/725. | +| b11930–b11945 | `common/{arg.cpp,chat.{cpp,h},common.{cpp,h},jinja/caps.{cpp,h}}`, `tools/mtmd/{mtmd.{cpp,h},mtmd-helper.{cpp,h},mtmd-helper-gen.cpp,mtmd-audio.*,mtmd-image.*,mtmd-cli.cpp,clip*,models/*}` (new `lfm3a.cpp`, `lfm3a-gen.cpp`, `lfm3vl.cpp`), `tools/server/server-{chat.cpp,common.cpp,context.{cpp,h},decision.{cpp,h},task.h}`, `ggml/src/ggml-rpc/ggml-rpc.cpp`, `src/models/lfm3*.cpp` | **None needed.** LiquidAI LFM3 (text, VL, Audio, Omni, d1 decision) support (#31252, #31257, #31275): new `mtmd_helper_gen_audio_inp` fields (`n_past`, `out_mode`, `voice`) behind `mtmd_helper_gen_audio_inp_default()`, which `tts_engine.cpp` starts from, so its request is unchanged; the new mtmd sources come in through upstream's own `tools/mtmd/CMakeLists.txt`. Deferred tools and `tool_reference` content parts (#31256: `common_chat_tool::defer_loading`, `common_chat_msg_content_part::name`). #31271 turns `common_chat_templates_get_caps` and `server_context_meta::chat_template_caps` from `map` into JSON so `/props` can list the template's `reasoning_effort_levels`; nothing here reads either (the Java side reads no `chat_template_caps`). #31204 catches `std::length_error` in the RPC server's `recv_msg`. Built at `d05706b4f`: 0 warnings, `ctest` 725/725. | +| b11945–b11946 | `tools/server/server-{context.cpp,task.{cpp,h},chat.{cpp,h}}`, **new** `tools/server/server-realtime.{cpp,h}`, `tools/server/CMakeLists.txt` | **`llama/CMakeLists.txt`: `server-realtime.cpp` compiled in.** #31269 serves realtime conversations and transcription on `/v1/realtime` (`SERVER_TASK_TYPE_REALTIME`, `server_task_result_realtime`); `server-context.cpp` creates a `server_realtime_slot` and calls `server_realtime_conversation` from its websocket route, so without the unit `jllama_test` fails to link (measured at `8ca4060b7`: undefined `server_realtime_slot::{update,push,cancel,is_idle}` and `server_realtime_conversation`). The unit uses the socket only through the `server_http_context::ws_conn` interface and spawns nothing, so it joins `server-decision.cpp` outside the Android guard (and the `jllama_test` list). New response keys exist only on the new realtime result (`cancelled`, `n_decoded`, `n_samples`, `response_done`); the request schema is unchanged across the whole b11901–b11954 range. Built: 0 warnings, `ctest` 725/725. | +| b11946–b11954 | `common/{chat.cpp,parsers/qwen3-coder.cpp,peg-parser.{cpp,h}}`, `ggml/src/{ggml-vulkan,ggml-cuda}`, `tools/ui/**`, `tests/**` | **None needed.** qwen3-coder stops reasoning and content before template separators (#30557), a Vulkan coopmat1 prefetch clamp (#30283), a CUDA top-k MoE fusion for 384 experts (#31288), WebUI (search dialog, OGG attachments, settings buttons), test constants. Built at `f1a7dea99`: 0 warnings, `ctest` 725/725; the full Java suite (`mvn -P jcstress test`) on this build, see the PR. | +| b11901–b11954 | patches | **All 14 apply unchanged at every tag of the range** (each published tag from b11901 to b11954 checked with the patches applied in order; the fail-loud applier on fresh trees at the five steps; `check-patches.py`: `docs/patches.md` is current; `verify-patches-applied.sh`: 14 applied). Patch-target files in the range: `common/arg.cpp` (`0001`, `0015`; #31242's message and #21231's option, away from both), `tools/server/server.cpp` (`0001`, `0006`, `0007`; #21231's measure role in the dispatch), `tools/server/server-models.cpp` (`0008`; #21231, see its row), `tools/server/server-context.{cpp,h}` (`0002`, `0003`, `0021`; #31271's caps and #31269's realtime slot, away from all three), `ggml/src/ggml-rpc/ggml-rpc.cpp` (`0015`; #31204), `tools/mtmd/mtmd-cli.cpp` and `tools/tts/tts.cpp` (`0001`'s flips). Drop-checks re-read at `f1a7dea99`: `0001` -- `argv = utf8.ptrs.data();` at `common/arg.cpp:1296`, no `common_params_parse_main`; `0002` -- `params_base.load_progress_callback = load_progress_callback;` unguarded at `server-context.cpp:1301`; `0003` -- no `slot_prompt_similarity` in `server-context.h`; `0008` -- no `LLAMA_SERVER_WORKER_CMD` upstream; `0015` -- still no way to stop the RPC server in `ggml-rpc.h`; `0021` -- `slot.lora = params_base.lora_adapters;` at `server-context.cpp:1966` without the clear. The files of `0014`, `0017`, `0018`, `0020`, `0022` and `0023` are not in the range. |