From 0a1ef866f83722a770c496e7052de4cee615730a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 9 Sep 2026 15:01:39 +0000 Subject: [PATCH] feat!: upgrade llama.cpp from b10870 to b10878 50.3 KiB / 8 commits / 25 files -- under the runbook's 100 KiB chunking threshold, so bumped straight through rather than stepped. Four pin sites moved (CMakeLists GIT_TAG, LlamaCppVersion.LLAMA_CPP_VERSION, CLAUDE.md, README badge), plus the two incidental b10870 mentions in CLAUDE.md's local-WebUI recipe and sccache section. Verified rather than assumed, because the range touches two patch targets: * 0001 (common/arg.cpp): b10878 still carries the #ifdef _WIN32 count-guarded `argv = utf8.ptrs.data()` override, and common_params_parse_main appears 0 times in b10878:common/arg.h -- upstream has still not adopted the fix. * 0012 (src/llama-model.{cpp,h}): b10878 still normalises with a bare `splits[i] /= split_sum;` and has no split_sum == 0 guard of its own, so CLAUDE.md's "drop rather than refresh" instruction does not fire. Its llama-model diff is only a new LLM_TYPE_1B_A400M enumerator, far from the patch hunks. Fresh configure through the real FetchContent path: stamp written at head 4850c7727fa73bbe3098e10ee369fbc3467c445f (= b10878) with all nine hashes. Release build clean. ctest 527/527, including the four LlamaModelSplits.* cases that are the only place 0012's extracted functions are linked. 40 Java_* exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` 4/4, including nativeBuildInfoMatchesPinnedVersionConstant -- the end-to-end proof that the pin sites and the linked binary agree. (clean is required: javac inlines the constant into the test class and Maven's incremental compile cannot see that dependency.) NOT fixed here, and the reason this commit is marked breaking: upstream deleted the deprecated --mlock, --mmap, --no-mmap, -dio/--direct-io and -ndio/--no-direct-io options in favour of --load-mode. Two of them are public Java API here -- ModelFlag.MLOCK and ModelFlag.NO_MMAP, emitted by ModelParameters.enableMlock() and disableMmap() -- and loadModel() hands that argv straight to common_params_parse, where an unknown option is a hard error. So both builder methods now produce a failing model load. No test catches it: ModelFlagTest and ModelParametersExtendedTest assert only the string mapping, never that llama.cpp still accepts the flag, so they stay green while the flag is dead. The whole deprecation window opened and closed inside this 8-tag range -- --load-mode already existed at b10870. The faithful replacement is --mlock -> --load-mode mlock and --no-mmap -> --load-mode none, but choosing between re-pointing the two builders, adding a LoadMode value-taking setter, or removing the constants outright is a public-API decision and is left to its own change. docs/history records the full analysis. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index c1a7ae35..b5b074d5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10870** +Current llama.cpp pinned version: **b10878** ## Upgrading CUDA Version @@ -502,7 +502,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b10870 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10878 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -542,7 +542,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10870`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10878`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1465,7 +1465,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10870`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10878`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 4a296595..e66a677c 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10870](https://img.shields.io/badge/llama.cpp-%23b10870-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10870) +[![llama.cpp b10878](https://img.shields.io/badge/llama.cpp-%23b10878-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10878) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 83656e00..02e7f9a1 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -700,3 +700,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | **macOS-15 failure — root cause + fix, 2026-09-08** | `src/llama-model.cpp` (upstream) ← `ggml/src/ggml-metal/ggml-metal-device.m` `8c0b9cd04` ([#27701](https://github.com/ggml-org/llama.cpp/pull/27701)); carried as `llama/patches/0012` | **Found by reading, not by bisecting — the bisect was prepared and then not needed.** The chain, each link checked against the source rather than inferred: **(1)** `8c0b9cd04` "metal : fix memory query under low-memory conditions" lies inside the b10618→b10797 window and rewrote `ggml_metal_device_get_memory` to `*free = *total > cur ? *total - cur : 0`. **(2)** Both the green and the red run log `current allocated size is greater than the recommended max working set size`, i.e. `cur > total` — so that condition is the **precondition, not the discriminator**; it held on both sides of the regression. **(3)** Before the clamp, `*total - cur` **underflowed** to a huge `size_t`, which normalised harmlessly; after it, the device reports exactly `free == 0`. **(4)** `load_tensors`' `if (free == 0 && total == 0)` host-memory fallback does not fire, because `total` is `recommendedMaxWorkingSetSize` and is non-zero. **(5)** `splits[0] = 0` → `split_sum = 0` → `splits[i] /= split_sum` = **NaN**. **(6)** `std::upper_bound(…, NaN)` — every comparison false — returns the end iterator, so `layer_gpu == 1`. **(7)** `devices.at(1)` on a one-element vector throws `std::out_of_range`, whose libc++ `what()` is the bare string `"vector"`; `llama.cpp`'s `catch (const std::exception & err)` prints it verbatim. **(8)** The **discriminator** is `const int act_gpu_layers = devices.empty() ? 0 : …`: without a GPU backend `devices` is empty, every layer returns early on `cpu_dev`, and the `.at()` line is unreachable — which is exactly why only the Metal job failed. This accounts for every observation the two investigation rows above collected: Metal-only; only on *repeat* loads in a long-lived JVM (that is merely how `currentAllocatedSize` grew past the recommended set size, not a precondition of its own); `what() == "vector"`; ~30 ms in, right after the vocab warnings; and the absence of a `hyperparameters:`/`vocabulary:` prefix, since upstream's own rethrows would have added one. **The fix** is `patches/0012`: `llama_model_splits_normalize()` falls back to an even split when the weights sum to zero, and `llama_model_splits_select_device()` bounds-checks the lookup and throws a message naming the function, the layer, the device index and the split points. Both are lifted out of `load_tensors` into free functions **for testability** — the failing state needs a real over-committed GPU and cannot be arranged through any API — with an upstream `tests/test-model-split.cpp` and, because a FetchContent subproject sets `LLAMA_BUILD_TESTS=OFF`, a project-side runnable guard `src/test/cpp/test_model_split.cpp` that links the same two functions into `jllama_test` on every platform. **A second trigger, found while writing the fix up and verified against the unfixed library:** `--tensor-split` is parsed with `std::stof` and never range-checked, so `-ts 1,-1` makes the weights cancel, `split_sum` is 0 again, the split points become `[inf, -nan]`, and every layer maps one past the last device — on CUDA, Vulkan or ROCm as much as on Metal, in a fresh process with no memory pressure. The macOS failure is therefore one *instance* of a general defect, not a Metal edge case, which is what settles the question of upstream-submittability. **Two lessons worth carrying.** The `ggml/src/**` "safe to skip" rule in the review list is sound for compile/link breaks and blind to runtime ones; this is the first entry where it cost something, and the cost was ~180 builds of bisect window. And a `catch (…) { log(err.what()); }` over a library that throws `std::out_of_range` is a **diagnostic dead end** on libc++, which reports it as `"vector"` and nothing else — the second half of `0012` exists for that reason alone, and is why the message a future occurrence produces will name its own cause. | | b10850–b10870 | `common/chat.cpp` (**−2524 lines**: every model-specific chat parser split out into a new `common/parsers/` directory — 19 new files, wired in via `common/parsers/sources.cmake` + `common/CMakeLists.txt`), `common/arg.cpp` (**behaviour change, not a signature change** — see below), `common/speculative.cpp` (**behaviour change** — see below), `ggml/include/ggml.h` (**additive + one deprecation**: `ggml_prec` gains `GGML_PREC_UNDEFINED`/`BF16`/`F16`/`Q8`/`Q4`, `GGML_PREC_DEFAULT` kept as a same-value alias marked deprecated; two new `GGML_API` functions `ggml_prec_set_acc` / `ggml_prec_set_src`, and **`ggml_mul_mat_set_prec` + `ggml_flash_attn_ext_set_prec` are now `GGML_DEPRECATED`**), `tools/server/server-context.cpp` (checkpoint eviction), `tools/mtmd/clip.cpp` + two model files, `src/llama-model.cpp` (**#28160**, lazy-mode AUTO), `tests/CMakeLists.txt`. **`common/chat.h` is byte-identical in the range** | **No project-source change.** The headline number is misleading: `common/chat.cpp` losing 2524 lines is a pure **internal reorganisation** — `common/chat.h`, which `jllama.cpp` includes directly and which is #2 on the priority review list, does not change at all, so nothing the project compiles against moved. The `ggml.h` change is additive plus deprecations that break nothing: `GGML_PREC_DEFAULT` keeps its value, and the two newly-deprecated functions still exist. The project source was grepped for all of it — `GGML_PREC`, `ggml_prec_set_*`, `ggml_mul_mat_set_prec`, `ggml_flash_attn_ext_set_prec` across `src/main/cpp/**` and `src/test/cpp/**` — with **zero** references, so none of it can reach us. (A deprecation is worth naming anyway: it is the shape that becomes a removal two bumps later, and a removal is the one change that breaks a build with no diff hunk to notice.) Server contract re-checked mechanically and **byte-identical in all three dimensions** (request-field set, `set_hard_limits` bounds, response keys in both emit forms). **Two behaviour changes that a header diff cannot see, and both reach every entry point that parses argv** (`NativeServer` in both modes, `LlamaModel`'s own parameter parse): **(1)** `--mmproj-device` now **defaults to `--device`** instead of auto-selecting (`common_params_parse` assigns `params.mmproj_device = params.devices.front()` when `mmproj_use_gpu` is set and `-mmdev` was not) — a caller that sets `--device` but not `-mmdev` now pins the multimodal projector to the same device rather than letting it choose, which is the surface `MultimodalIntegrationTest` and every vision user exercises. **(2)** the **draft model inherits the global device list** the same way, and `common_speculative_init` now only overwrites `result.devices` **when the spec device list is non-empty** (it previously assigned unconditionally), plus forces `LLAMA_SPLIT_MODE_LAYER` when the draft is pinned to exactly one device. That is the speculative-decoding path `LlamaModelTest#testSpeculativeDecoding` drives — the same test that was red on macOS before `patches/0012`, so a failure there after this bump needs to be attributed carefully between the two. **Chunking, with the figures recorded rather than a verdict asserted:** the full diff is **393 KB over 20 commits**, over the runbook's 100 KiB threshold; the **review surface proper** (`common/`, `include/`, `tools/server/`, `tools/mtmd/`, `ggml/include/`, top-level `CMakeLists.txt`) is **28 files, +2686 / −2480**, but ~2500 of those lines on each side are the one mechanical parser move, so the material change is a few dozen lines. Bumped straight rather than chunked on that basis, with the raw numbers here so the call is auditable. | | b10850–b10870 | patches + upstream verification | **The intersection was NOT empty, and `0012` was the patch at risk.** The range touches `src/llama-model.cpp` and `tests/CMakeLists.txt` — both files `patches/0012` modifies, one week after that patch landed — plus `common/arg.cpp` (`0001`) and `tools/server/server-context.cpp` (`0002`/`0003`/`0010`). So this bump could not be waved through on a disjoint file list. **The `0012`-specific check `CLAUDE.md` mandates was run by hand first**, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `git show b10870:src/llama-model.cpp | grep -A3 split_sum` still shows the bare `splits[i] /= split_sum` with **no zero-sum guard**, so upstream has not adopted the fix and the patch stays rather than being dropped. Upstream's own change to that file (#28160, resolving `LLAMA_LAZY_MODE_AUTO` to `OFF` on devices without mmap support) sits ~60 lines above the patched region and is unrelated. Then the applier was run for real: fresh `rm -rf llama/build && cmake -B build -DBUILD_TESTING=ON`, configure clean, stamp written at head `1945e092030f8668ff93382799502d01490e564d` (= `b10870`), **all nine hashes recorded**. | +| b10870–b10878 | `common/arg.cpp` (**REMOVAL, and it reaches this project's public Java API**: the deprecated `--mlock`, `--mmap`, `--no-mmap`, `-dio`/`--direct-io`, `-ndio`/`--no-direct-io` options are deleted in favour of `-lm`/`--load-mode `; nothing was added — `--load-mode` already existed at b10870, so the whole deprecation window opened and closed inside a single 8-tag range), `include/llama.h` (`llama_sampler_chain_n` returns `int32_t` instead of `int` — **unreachable here**, no project TU calls it, and the two are the same type on every platform this builds for), `src/llama-model.{cpp,h}` (**additive only**: a new `LLM_TYPE_1B_A400M` enumerator for Granite3 MoE, far from `patches/0012`'s hunks), `tools/mtmd/mtmd-helper.cpp` (internal video frame-id propagation; **`mtmd-helper.h` is untouched**, so `mtmd_helper::gen_audio` and therefore `TextToSpeech` are unaffected), `src/llama-sampler.cpp`, `common/jinja/runtime.cpp`, `src/models/granite-moe.cpp`, and a ggml build-system change (`GGML_CUDA_FA_ALL_QUANTS` deprecated in favour of `GGML_CUDA_FA_QUANTS` — **not set anywhere in this repo**, so it cannot reach the CUDA/HIP jobs). 25 files, 336 insertions, 227 deletions, 50.3 KiB — under the runbook's 100 KiB chunking threshold, so bumped straight through. **The one row that needs project action is the first.** `ModelFlag.MLOCK` (`"--mlock"`) and `ModelFlag.NO_MMAP` (`"--no-mmap"`) are public constants emitted by `ModelParameters.enableMlock()` / `disableMmap()`, and `LlamaModel.loadModel(parameters.toArray())` hands that argv straight to `common_params_parse`, where an unknown option is a hard error rather than a warning — so both builder methods now produce a model load that fails. The other three removed options are not exposed here. **No test can catch this**: `ModelFlagTest` and `ModelParametersExtendedTest#testEnableMlock`/`#testDisableMmap` assert only the string mapping (`hasKey("--mlock")`), never that llama.cpp still accepts it, so they stay green while the flag is dead — the same "pins the mapping, not the contract" shape as the `getMetrics()` payload drift at b10408. Faithful replacement is `--mlock` → `--load-mode mlock` and `--no-mmap` → `--load-mode none`; deciding between re-pointing the two builders, adding a `LoadMode` value-taking setter, or removing the constants outright is a public-API call and is deliberately **not** made in the bump commit. | +| b10870–b10878 | patches + upstream verification | **All nine patches still apply, and `0012` is still required.** The range touches two patch targets — `common/arg.cpp` (`0001`) and `src/llama-model.{cpp,h}` (`0012`) — so both were checked against the pristine tag rather than assumed. `0001`: `b10878:common/arg.cpp` still carries the `#ifdef _WIN32` count-guarded `argv = utf8.ptrs.data()` override, and `common_params_parse_main` appears **0 times** in `b10878:common/arg.h`, so upstream has still not adopted the fix. `0012`: `b10878:src/llama-model.cpp` still normalises with a bare `splits[i] /= split_sum;` and has **no `split_sum == 0` guard** of its own — the CLAUDE.md instruction to *drop rather than refresh* this patch does not fire, and its `llama-model` diff is only the new enumerator. Verified for real: fresh `cmake -S llama -B /tmp/b10878-build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `4850c7727fa73bbe3098e10ee369fbc3467c445f` (= `b10878`) with **all nine hashes recorded**; full `cmake --build --config Release` clean; `ctest` **527/527**, including the four `LlamaModelSplits.*` cases that are the only place `0012`'s two extracted functions are linked in CI. `nm -D` on the fresh `libjllama.so` reports **40** `Java_*` exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree. Run with `clean`: `LLAMA_CPP_VERSION` is a compile-time constant javac inlines into the test class, and Maven's incremental compilation cannot see that dependency. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 3609f725..90040f83 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10870 + GIT_TAG b10878 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 17b07180..7b61b5fe 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10870"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10878"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10870-"} — call + * plus the resolved upstream commit, e.g. {@code "b10878-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10870"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10878"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10870"; + public static final String LLAMA_CPP_VERSION = "b10878"; // Constants holder — not instantiable. private LlamaCppVersion() {}