diff --git a/CLAUDE.md b/CLAUDE.md index 158d58b7..1a1121a4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10878** +Current llama.cpp pinned version: **b10883** ## Upgrading CUDA Version @@ -502,7 +502,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b10878 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10883 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -542,7 +542,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10878`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10883`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1466,7 +1466,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10878`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10883`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index e66a677c..1913a099 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10878](https://img.shields.io/badge/llama.cpp-%23b10878-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10878) +[![llama.cpp b10883](https://img.shields.io/badge/llama.cpp-%23b10883-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10883) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/TODO.md b/TODO.md index 382ce06f..6242a605 100644 --- a/TODO.md +++ b/TODO.md @@ -122,6 +122,30 @@ upstream PR #22393 — it drops automatically when that merges.) These are JNI plumbing items for upstream API additions. Policy: add only after a real user request — they are mostly relevant to specific model families or specialized workflows. +- **Three upstream flags found by the b10878 flag audit, deliberately NOT implemented there.** The + audit that produced `test_model_flags.cpp` swept every option `common/arg.cpp` registers for + `LLAMA_EXAMPLE_SERVER` against what `ModelParameters`/`ModelFlag` emit. Beyond the seven dead + flags it retired, it found ten option groups upstream had added since b10456 that the Java API + does not expose. Seven were already covered (`--kv-unified-per-slot`, `--mmproj-device`/`-mmdev`, + `--video-fps`, `--video-timestamp-interval`, `--video-ffmpeg-dir`, `--lazy-mode`/`-lzm`, + `--n-cpu-ffn`/`-ncffn`). These three are the remainder, left out of the correction PR on purpose + — it was a *fix* for an unloadable-model bug, and adding surface would have widened it: + + - **`--log-jsonl` / `--no-log-jsonl`** (a positive/negative flag pair, so it would fit `ModelFlag` + directly). The only one of the three with real consumer value, but it is **not a free addition**: + it flips `common_log_set_jsonl(common_log_main(), …)`, i.e. the process-wide llama.cpp logger, + whose output for this library goes through the JNI log callback. The project already has its own + JSON logging at the Java level — the `args.LogFormat` enum plus `log_helpers.hpp`'s + `format_log_as_json` — so the two would overlap and could contradict each other on the same + stream. Deciding which layer owns the format is a **feature decision**, not a correctness fix, + and needs its own change with its own tests. + - **`--spec-synth-len` and `--spec-synth-rates`** — upstream's own help text marks both as + benchmarking-only knobs for synthetic speculative-decoding measurements. No consumer use case + here; listed so a future audit does not re-discover them as an oversight. + + Nothing is broken by leaving these out: `NativeServer` forwards raw llama-server argv verbatim, so + all three remain reachable that way. The gap is only in the typed `ModelParameters` surface. + - **Video input (`ContentPart.videoFile(...)`).** `mtmd` has had an end-to-end video path since llama.cpp **b9562** (#24269) — `mtmd_helper_video_init_params` was already present at the previous pin, b10456. What **b10647** (#24318, commit `f29551215`) added is the surfacing: a fourth diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 851f6918..6b10c432 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -702,3 +702,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10850–b10870 | patches + upstream verification | **The intersection was NOT empty, and `0012` was the patch at risk.** The range touches `src/llama-model.cpp` and `tests/CMakeLists.txt` — both files `patches/0012` modifies, one week after that patch landed — plus `common/arg.cpp` (`0001`) and `tools/server/server-context.cpp` (`0002`/`0003`/`0010`). So this bump could not be waved through on a disjoint file list. **The `0012`-specific check `CLAUDE.md` mandates was run by hand first**, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `git show b10870:src/llama-model.cpp | grep -A3 split_sum` still shows the bare `splits[i] /= split_sum` with **no zero-sum guard**, so upstream has not adopted the fix and the patch stays rather than being dropped. Upstream's own change to that file (#28160, resolving `LLAMA_LAZY_MODE_AUTO` to `OFF` on devices without mmap support) sits ~60 lines above the patched region and is unrelated. Then the applier was run for real: fresh `rm -rf llama/build && cmake -B build -DBUILD_TESTING=ON`, configure clean, stamp written at head `1945e092030f8668ff93382799502d01490e564d` (= `b10870`), **all nine hashes recorded**. | | b10870–b10878 | `common/arg.cpp` (**REMOVAL, and it reaches this project's public Java API**: the deprecated `--mlock`, `--mmap`, `--no-mmap`, `-dio`/`--direct-io`, `-ndio`/`--no-direct-io` options are deleted in favour of `-lm`/`--load-mode `; nothing was added — `--load-mode` already existed at b10870, so the whole deprecation window opened and closed inside a single 8-tag range), `include/llama.h` (`llama_sampler_chain_n` returns `int32_t` instead of `int` — **unreachable here**, no project TU calls it, and the two are the same type on every platform this builds for), `src/llama-model.{cpp,h}` (**additive only**: a new `LLM_TYPE_1B_A400M` enumerator for Granite3 MoE, far from `patches/0012`'s hunks), `tools/mtmd/mtmd-helper.cpp` (internal video frame-id propagation; **`mtmd-helper.h` is untouched**, so `mtmd_helper::gen_audio` and therefore `TextToSpeech` are unaffected), `src/llama-sampler.cpp`, `common/jinja/runtime.cpp`, `src/models/granite-moe.cpp`, and a ggml build-system change (`GGML_CUDA_FA_ALL_QUANTS` deprecated in favour of `GGML_CUDA_FA_QUANTS` — **not set anywhere in this repo**, so it cannot reach the CUDA/HIP jobs). 25 files, 336 insertions, 227 deletions, 50.3 KiB — under the runbook's 100 KiB chunking threshold, so bumped straight through. **The one row that needs project action is the first.** `ModelFlag.MLOCK` (`"--mlock"`) and `ModelFlag.NO_MMAP` (`"--no-mmap"`) are public constants emitted by `ModelParameters.enableMlock()` / `disableMmap()`, and `LlamaModel.loadModel(parameters.toArray())` hands that argv straight to `common_params_parse`, where an unknown option is a hard error rather than a warning — so both builder methods now produce a model load that fails. The other three removed options are not exposed here. **No test can catch this**: `ModelFlagTest` and `ModelParametersExtendedTest#testEnableMlock`/`#testDisableMmap` assert only the string mapping (`hasKey("--mlock")`), never that llama.cpp still accepts it, so they stay green while the flag is dead — the same "pins the mapping, not the contract" shape as the `getMetrics()` payload drift at b10408. Faithful replacement is `--mlock` → `--load-mode mlock` and `--no-mmap` → `--load-mode none`. **Resolved in the follow-up PR, and it did all three:** a new `args.LoadMode` enum + `ModelParameters.setLoadMode(LoadMode)` expose the replacement option properly; `enableMlock()` / `disableMmap()` are kept and `@Deprecated`, re-pointed to `LoadMode.MLOCK` / `LoadMode.NONE` — upstream's own deprecation-shim mapping, so behaviour is unchanged and no API is lost; and `ModelFlag.MLOCK` / `NO_MMAP` are removed from the enum, because leaving them would keep the broken argv reachable through `setFlag` (the same reasoning that already excluded `FLASH_ATTN`). The “no test can catch this” half was closed at the same time and generalised: `src/test/cpp/test_model_flags.cpp` drives every flag the Java layer can emit — the list generated at configure time from the Java sources by `cmake/extract-java-cli-flags.cmake` — through the real `common_params_parser_init(params, LLAMA_EXAMPLE_SERVER)` option table. Run against the pre-fix sources it named **seven** dead flags, not two: the long-dead `--dump-kv-cache` / `--hf-repo-v` / `--hf-file-v`, and `--grp-attn-n` / `--grp-attn-w`, which are *present* in `arg.cpp` at every pinned tag but `set_examples()`-scoped away from `LLAMA_EXAMPLE_SERVER` — a case a textual sweep of upstream sources is structurally blind to. | | b10870–b10878 | patches + upstream verification | **All nine patches still apply, and `0012` is still required.** The range touches two patch targets — `common/arg.cpp` (`0001`) and `src/llama-model.{cpp,h}` (`0012`) — so both were checked against the pristine tag rather than assumed. `0001`: `b10878:common/arg.cpp` still carries the `#ifdef _WIN32` count-guarded `argv = utf8.ptrs.data()` override, and `common_params_parse_main` appears **0 times** in `b10878:common/arg.h`, so upstream has still not adopted the fix. `0012`: `b10878:src/llama-model.cpp` still normalises with a bare `splits[i] /= split_sum;` and has **no `split_sum == 0` guard** of its own — the CLAUDE.md instruction to *drop rather than refresh* this patch does not fire, and its `llama-model` diff is only the new enumerator. Verified for real: fresh `cmake -S llama -B /tmp/b10878-build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `4850c7727fa73bbe3098e10ee369fbc3467c445f` (= `b10878`) with **all nine hashes recorded**; full `cmake --build --config Release` clean; `ctest` **527/527**, including the four `LlamaModelSplits.*` cases that are the only place `0012`'s two extracted functions are linked in CI. `nm -D` on the fresh `libjllama.so` reports **40** `Java_*` exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree. Run with `clean`: `LLAMA_CPP_VERSION` is a compile-time constant javac inlines into the test class, and Maven's incremental compilation cannot see that dependency. | +| b10878–b10883 | **Nothing on the review surface.** The raw diff is large — 29 files, 2360 insertions, 2276 deletions, **426 KiB, well over the runbook's 100 KiB chunking threshold** — and was bumped straight through anyway, for a reason that is recorded here rather than asserted: the byte count is entirely GPU backend internals and Python pins. Restricting `git diff --stat` to the paths this project actually compiles, links or includes (`common/`, `include/`, `tools/server/`, `tools/mtmd/`, `ggml/include/`, `src/`, the top-level `CMakeLists.txt`) leaves **one file, one line**: `tools/server/tests/requirements.txt`, a Python test-requirement pin that is neither compiled nor linked. The remainder is `ggml/src/ggml-vulkan/**` (1804 lines in `ggml-vulkan.cpp` plus ~15 shader files), `ggml/src/ggml-hexagon/**`, `tests/test-backend-ops.cpp`, and six `requirements*.txt` / `pyproject.toml` version pins. **Zero** priority-8 headers moved; `common/arg.h`, `common/chat.h`, `include/llama.h`, `tools/mtmd/mtmd-helper.h` are all byte-identical. This is the same shape as the b10819–b10850 row: a headline number dominated by backends the project builds but whose internals it never calls. The one thing the size *does* imply is CI cost — the Vulkan rewrite is upstream-compiled code the `vulkan-linux-*` and `vulkan-windows-*` classifier jobs must still build, so a compile break there would surface in those jobs rather than in any project source. | +| b10878–b10883 | patches + upstream verification | **All nine patches apply, and not one needed refreshing — every patch-target file is byte-unchanged in the range.** Checked file by file rather than inferred from the aggregate: `common/arg.cpp`, `common/arg.h`, `common/peg-parser.cpp`, `tools/server/server.cpp`, `tools/server/server-context.{cpp,h}`, `tools/server/server-models.cpp`, `src/llama-model.{cpp,h}` and `tests/CMakeLists.txt` all report no diff between the two tags. The two standing drop-checks were still run against the pristine tag, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `0001` — `common_params_parse_main` appears **0 times** in `b10883:common/arg.h` and the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override is still at `common/arg.cpp:1282`, so it stays; `0012` — `b10883:src/llama-model.cpp:1489` still normalises with a bare `splits[i] /= split_sum;` and has **no `split_sum == 0` guard**, so the CLAUDE.md instruction to *drop rather than refresh* does not fire. Verified for real: `rm -rf build && cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `91f6a6cf361385700bbe15981f0f39909df77498` (= `b10883`) with **all nine hashes recorded**; full `cmake --build --config Release` clean; `ctest` **531/531**. That total is 4 up from b10878 because this is the first bump after `test_model_flags.cpp` landed — and it is the first bump whose **flag contract** was machine-checked rather than reasoned about: `JavaCliFlagContract` re-derives the 138 flags the Java layer emits and re-runs them through `common_params_parser_init(params, LLAMA_EXAMPLE_SERVER)` at the new tag, so "`arg.cpp` is unchanged, therefore the registered option set is unchanged" is now an assertion the build makes, not an inference a reviewer makes. `nm -D` on the fresh `libjllama.so` reports **40** `Java_*` exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree (`clean` is required: javac inlines `LLAMA_CPP_VERSION` into the test class and Maven's incremental compile cannot see that dependency). Full `mvn test`: **1759 tests, 0 failures, 0 errors**. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 2a6d368b..75a76716 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10878 + GIT_TAG b10883 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 7b61b5fe..7dea5d85 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10878"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10883"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10878-"} — call + * plus the resolved upstream commit, e.g. {@code "b10883-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10878"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10883"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10878"; + public static final String LLAMA_CPP_VERSION = "b10883"; // Constants holder — not instantiable. private LlamaCppVersion() {}