ROCm: compress the embedded GPU code (--offload-compress)

The Windows ROCm jllama.dll was ~1 GB: every HIP translation unit embeds
one code object per GPU target, i.e. the whole kernel set once per
architecture, and clang stores those bundles uncompressed by default.
The jar hid it (234 MB zipped), but LlamaLoader extracts the library to
the temp dir on every start, and in the all-backends fat jar ROCm is
tried right after CUDA - on nearly every machine without an NVIDIA card.

ggml-hip is now compiled with --offload-compress, so each bundle is
stored zstd-compressed (CCOB) and inflated by the HIP runtime at module
load. The option is scoped to the ggml-hip target and to its source
language (HIP on Linux, CXX on Windows, where upstream compiles HIP as
C++).

Both ROCm jobs run .github/verify-hip-offload-compressed.py after the
build: it prints the library size and bundle counts (also into the job
summary) and fails on any uncompressed bundle, or on none compressed,
so a toolchain or upstream change cannot quietly bring the 1 GB library
back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtfgoazykGcQSCR3TYBmTQ

Author: claude

.github/verify-hip-offload-compressed.py

@@ -0,0 +1,86 @@
+# SPDX-FileCopyrightText: 2026 Bernard Ladenthin <bernard.ladenthin@gmail.com>
+#
+# SPDX-License-Identifier: MIT
+
+"""Fail when a ROCm/HIP build ships its GPU code uncompressed.
+
+llama/CMakeLists.txt compiles ggml-hip with clang's --offload-compress, so every embedded
+device-code bundle is a compressed "CCOB" bundle. An uncompressed bundle starts with the magic
+"__CLANG_OFFLOAD_BUNDLE__"; one of those in the shipped library means the flag was lost and the
+library is back to carrying every GPU target's code uncompressed (~1 GB on Windows).
+
+Usage: verify-hip-offload-compressed.py <dir-or-file>...
+
+Scans every jllama.dll / libjllama.so found, prints its size and the bundle counts, and exits
+non-zero if a library has an uncompressed bundle, has no compressed bundle at all, or if no
+library was found. Only the standard library, so it runs on any runner.
+"""
+
+import os
+import sys
+
+UNCOMPRESSED = b"__CLANG_OFFLOAD_BUNDLE__"
+COMPRESSED = b"CCOB"
+NAMES = ("jllama.dll", "libjllama.so")
+CHUNK = 64 * 1024 * 1024
+
+
+def count(path):
+    """Counts both magics in one streaming pass (the file can be ~1 GB)."""
+    overlap = len(UNCOMPRESSED) - 1
+    uncompressed = compressed = 0
+    tail = b""
+    with open(path, "rb") as f:
+        while True:
+            block = f.read(CHUNK)
+            if not block:
+                break
+            data = tail + block
+            # a match lying entirely inside `tail` was counted in the previous round
+            uncompressed += data.count(UNCOMPRESSED) - tail.count(UNCOMPRESSED)
+            compressed += data.count(COMPRESSED) - tail.count(COMPRESSED)
+            tail = data[-overlap:]
+    return uncompressed, compressed
+
+
+def libraries(paths):
+    for p in paths:
+        if os.path.isfile(p):
+            yield p
+        for root, _, files in os.walk(p):
+            for name in files:
+                if name in NAMES:
+                    yield os.path.join(root, name)
+
+
+def main():
+    if len(sys.argv) < 2:
+        print("usage: verify-hip-offload-compressed.py <dir-or-file>...", file=sys.stderr)
+        return 2
+    found = failed = 0
+    summary = []
+    for lib in libraries(sys.argv[1:]):
+        found += 1
+        size = os.path.getsize(lib)
+        uncompressed, compressed = count(lib)
+        line = f"{lib}: {size / 1024 / 1024:.0f} MiB, {compressed} compressed / {uncompressed} uncompressed offload bundle(s)"
+        print(line)
+        summary.append(line)
+        if uncompressed:
+            print(f"::error::{lib} embeds {uncompressed} uncompressed GPU code bundle(s); is --offload-compress still set on ggml-hip?")
+            failed += 1
+        elif not compressed:
+            print(f"::error::{lib} embeds no compressed GPU code bundle; was it built with GGML_HIP=ON?")
+            failed += 1
+    if not found:
+        print(f"::error::no {' / '.join(NAMES)} found under {sys.argv[1:]}")
+        return 2
+    step_summary = os.environ.get("GITHUB_STEP_SUMMARY")
+    if step_summary:
+        with open(step_summary, "a", encoding="utf-8") as f:
+            f.write("### ROCm/HIP device code\n\n" + "\n".join(f"- `{s}`" for s in summary) + "\n")
+    return 1 if failed else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

.github/workflows/publish.yml

@@ -2164,6 +2164,10 @@ jobs:
         run: |
           mvn --no-transfer-progress -f llama/pom.xml compile
           .github/build.sh "-DGGML_HIP=ON -DCMAKE_HIP_COMPILER=$(hipconfig -l)/clang -DGPU_TARGETS=gfx900;gfx906;gfx908;gfx90a;gfx90c;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201 -DGGML_NATIVE=OFF -DOS_NAME=Linux -DOS_ARCH=x86_64"
+      - name: Verify the GPU code is compressed
+        # ggml-hip is compiled with --offload-compress (llama/CMakeLists.txt); without it the
+        # library carries every target's kernels uncompressed. Fails on an uncompressed bundle.
+        run: python3 .github/verify-hip-offload-compressed.py llama/src/main/resources_linux_rocm
       - name: Upload artifacts
         uses: actions/upload-artifact@v7
         with:
@@ -2245,6 +2249,10 @@ jobs:
         # support for. Same rule for the extras upstream omits as on Linux.
         run: |
           .github\build.bat -G "Ninja Multi-Config" -DGGML_HIP=ON -DGPU_TARGETS=gfx900;gfx906;gfx90c;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201 -DCMAKE_PREFIX_PATH="%HIP_PATH%" -DHIP_PATH="%HIP_PATH%" -DCMAKE_C_COMPILER="%HIP_PATH%\lib\llvm\bin\clang.exe" -DCMAKE_CXX_COMPILER="%HIP_PATH%\lib\llvm\bin\clang++.exe" -DCMAKE_HIP_COMPILER="%HIP_PATH%\lib\llvm\bin\clang.exe" -DCMAKE_C_FLAGS="-Wno-error=incompatible-pointer-types" -DOS_NAME=Windows -DOS_ARCH=x86_64
+      - name: Verify the GPU code is compressed
+        # Same check as the Linux ROCm job: an uncompressed bundle means ~1 GB jllama.dll again.
+        shell: pwsh
+        run: python .github/verify-hip-offload-compressed.py llama/src/main/resources_windows_rocm
       - name: Upload artifacts
         uses: actions/upload-artifact@v7
         with:

CLAUDE.md

@@ -355,6 +355,22 @@ without problems and without local patches; the moment one needs a patch or hold
 ROCm/llama.cpp, drop it. The two lists differ **only** by the Instinct parts
 (gfx908/gfx90a/gfx942/gfx950), which ROCm supports on Linux alone.
 
+**The ROCm GPU code is compressed (`--offload-compress`), and CI enforces it.** Each HIP
+translation unit embeds one code object per GPU target, i.e. the whole kernel set (flash attention,
+mmq per quant type, …) once per architecture — stored **uncompressed** by default, which made the
+Windows `jllama.dll` ~1 GB for its 23 targets (234 MB zipped, so the jar never showed it). That size
+also lands on disk: `LlamaLoader` extracts the library to the temp dir on every start, and in the
+all-backends fat jar ROCm is tried right after CUDA, i.e. on nearly every machine without an NVIDIA
+card. `llama/CMakeLists.txt` therefore adds `--offload-compress` to the `ggml-hip` target only
+(`$<COMPILE_LANGUAGE:HIP,CXX>`: its sources are HIP on Linux and CXX on Windows, where upstream
+compiles HIP as C++), so clang stores every bundle zstd-compressed (a `CCOB` bundle) and the HIP
+runtime inflates it when the module loads. Upstream llama.cpp does **not** do this; its
+`ggml-hip.dll` carries the same uncompressed code (for 20 targets). `.github/verify-hip-offload-compressed.py` runs after the build in
+both ROCm jobs, prints the library size and bundle counts (also into the job summary), and fails on
+any uncompressed bundle (`__CLANG_OFFLOAD_BUNDLE__`) or on none compressed — a toolchain or upstream
+change that drops the flag reds the job instead of quietly shipping the 1 GB library again. The
+jar barely shrinks (zip already compressed the code); what shrinks is the extracted library.
+
 Two routing notes mirror existing precedent: **Linux SYCL** ships two precision variants at the *same*
 arch, so `CMakeLists.txt` routes them to two *distinct* trees by `GGML_SYCL_F16` (fp16 vs fp32).
 **Windows OpenCL** now holds both `x86_64` (desktop ICD) and `aarch64` (Snapdragon/Adreno) in the one

llama/CMakeLists.txt

@@ -196,6 +196,19 @@ FetchContent_Declare(
 )
 FetchContent_MakeAvailable(llama.cpp)
 
+# ROCm/HIP: compress the embedded GPU code. Without it every HIP translation unit carries one
+# UNcompressed code object per GPU target, so the shipped library holds the complete kernel set
+# (flash attention, mmq per quant type, ...) once per architecture: ~1 GB for the 23 Windows
+# targets, which LlamaLoader then extracts to the temp dir on every start. --offload-compress
+# makes clang store each bundle zstd-compressed (a "CCOB" bundle); the HIP runtime inflates it
+# when the module is loaded. Scoped to the ggml-hip target (the only one with device code):
+# its sources are LANGUAGE HIP on Linux and CXX on Windows (upstream compiles HIP as C++ there).
+# CI (.github/verify-hip-offload-compressed.py) fails the ROCm jobs if an uncompressed bundle
+# ever reappears.
+if(TARGET ggml-hip)
+    target_compile_options(ggml-hip PRIVATE $<$<COMPILE_LANGUAGE:HIP,CXX>:--offload-compress>)
+endif()
+
 # b8831 added ggml_graph_next_uid() which calls _InterlockedIncrement64 via
 # <intrin.h> on x86. The intrinsic only exists on x64; provide the
 # implementation in a compat TU so the linker resolves __InterlockedIncrement64.