kernels: implement kernels cache ls

docs/source/_toctree.yml

@@ -33,6 +33,8 @@
 - sections:
     - local: cli
       title: Overview
+    - local: cli-cache
+      title: kernels cache ls
     - local: cli-info
       title: kernels info
     - local: cli-describe-api

docs/source/basic-usage.md

@@ -25,16 +25,30 @@ This fetches version `1` of the kernel `kernels-community/activation`.
 Kernels are versioned using a major version number. Using `version=1` will
 get the latest kernel build from the `v1` branch.
 
-You can check whether a kernel was downloaded with `hf cache ls`:
+You can check which kernels and build variants are cached with
+[`kernels cache ls`](cli-cache.md):
 
 ```bash
-# pip install hf
-$ hf cache ls
-ID                                  SIZE LAST_ACCESSED  LAST_MODIFIED  REFS
------------------------------------ ---- -------------- -------------- ----
-kernel/kernels-community/activation 2.5M 45 minutes ago 45 minutes ago     
+kernels cache ls
 ```
 
+Example output (the values depend on your cache):
+
+```text
+ID                                  SIZE LAST_ACCESSED  LAST_MODIFIED  VERSION VARIANTS
+----------------------------------- ---- -------------- -------------- ------- --------------------------------
+kernel/kernels-community/activation 2.5M 45 minutes ago 45 minutes ago v1      torch28-cxx11-cu126-x86_64-linux
+                                                                               torch28-cxx11-cu128-x86_64-linux
+                                                                               torch28-cxx11-cu129-x86_64-linux
+kernel/kernels-community/activation 2.6M 23 minutes ago 23 minutes ago v2      torch28-cxx11-cu126-x86_64-linux
+                                                                               torch28-cxx11-cu128-x86_64-linux
+                                                                               torch28-cxx11-cu129-x86_64-linux
+```
+
+Each version has its own row, with its cached variants listed on separate lines.
+Use `kernels cache ls --no-truncate` or `--json` to see full values when the
+table is shortened to fit your terminal.
+
 Kernels within a version branch must never break the API or remove builds
 for older PyTorch versions. This ensures that your code will continue to work.
 

docs/source/cli-cache.md

@@ -0,0 +1,83 @@
+# kernels cache ls
+
+List kernel versions stored in the local Hugging Face cache.
+The command runs locally and does not download or load kernel code.
+
+## Usage
+
+```bash
+kernels cache ls [--cache-dir PATH] [--json] [--no-truncate] [--show-warnings]
+```
+
+| Option | Description |
+| --- | --- |
+| `--cache-dir PATH` | Inspect a specific cache directory. |
+| `--json` | Print cached kernel information as JSON. |
+| `--no-truncate` | Display full table values regardless of terminal width. |
+| `--show-warnings` | Show individual cache inconsistencies instead of a short summary. |
+
+By default, the command uses the same cache as `get_kernel`: `KERNELS_CACHE`
+when set, otherwise the Hugging Face Hub cache configured by `HF_HUB_CACHE`
+or `HF_HOME`. An explicit `--cache-dir` takes precedence over these settings.
+
+## Output
+
+Example output in a wide terminal:
+
+```text
+ID                                  SIZE LAST_ACCESSED  LAST_MODIFIED  VERSION VARIANTS
+----------------------------------- ---- -------------- -------------- ------- --------------------------------
+kernel/kernels-community/activation 2.5M 45 minutes ago 45 minutes ago v1      torch28-cxx11-cu126-x86_64-linux
+                                                                               torch28-cxx11-cu128-x86_64-linux
+                                                                               torch28-cxx11-cu129-x86_64-linux
+kernel/kernels-community/activation 2.6M 23 minutes ago 23 minutes ago v2      torch28-cxx11-cu126-x86_64-linux
+                                                                               torch28-cxx11-cu128-x86_64-linux
+                                                                               torch28-cxx11-cu129-x86_64-linux
+```
+
+`VERSION` shows version labels such as `v1` and `v2`, in ascending numeric order.
+The listing includes only cached `vN` versions. Other branches, tags, and detached
+snapshots are omitted. Each version is listed separately, even when multiple
+versions point to the same cached snapshot.
+
+`VARIANTS` lists the builds present in that version's cached snapshot, one per line.
+Continuation lines leave the other columns blank. A variant cached for several
+versions appears under each of those versions.
+Variants are included regardless of compatibility with the current environment.
+Only recognized build directories with cached files are listed; empty build
+directories are excluded. If there are no cached variants, the column shows `-`.
+
+`SIZE`, `LAST_ACCESSED`, and `LAST_MODIFIED` describe the cached snapshot for that
+version. See the [Hugging Face Hub cache reference](https://huggingface.co/docs/huggingface_hub/package_reference/cache#data-structures)
+for details on sizes and timestamps.
+
+Like `hf cache ls`, the table shortens long values to fit the terminal width.
+Use `--no-truncate` or `--json` to see all values in full. Cache inconsistencies
+produce a short summary by default; use `--show-warnings` to see their details.
+
+## JSON output
+
+```bash
+kernels cache ls --json
+```
+
+The output contains `cache_dir` and a `repos` list. Each repository contains:
+
+- `repo_id`: the kernel repository ID.
+- `size_on_disk`: the repository's cached size in bytes.
+- `last_accessed`: the repository's last access time as a Unix timestamp.
+- `last_modified`: the repository's last modification time as a Unix timestamp.
+- `versions`: one entry per cached `vN` version. Each entry contains:
+  - `version`: the version label, such as `"v1"`.
+  - `size_on_disk`, `last_accessed`, and `last_modified`: the size in bytes and
+    Unix timestamps for that version's cached snapshot.
+  - `variants`: the sorted build variant names in that snapshot.
+
+Repository-level sizes and timestamps describe all its cached snapshots;
+version-level values describe just the corresponding snapshot.
+
+## See also
+
+- [kernels variants](cli-variants.md) - Inspect build variants available on the Hub
+- [kernels download](cli-download.md) - Download locked kernels
+- [Environment variables](env.md) - Configure cache locations

docs/source/cli.md

@@ -7,6 +7,7 @@ The `kernels` CLI provides commands for managing compute kernels.
 | Command                                     | Description                        |
 | ------------------------------------------- | ---------------------------------- |
 | [benchmark](cli-benchmark.md)               | Run benchmark results for a kernel |
+| [cache ls](cli-cache.md)                    | List cached kernels and build variants |
 | [download](cli-download.md)                 | Download a kernel                  |
 | [describe-api](cli-describe-api.md)         | List public functions and layers   |
 | [info](cli-info.md)                         | Describe a kernel                  |

kernels/pyproject.toml

@@ -10,7 +10,7 @@ license = { text = "Apache-2.0" }
 readme = "README.md"
 requires-python = ">= 3.10"
 dependencies = [
-  "huggingface-hub>=1.31.0",
+  "huggingface-hub>=1.32.0",
   "packaging>=20.0",
   "pyyaml>=6",
   "sigstore>=4,<5",

kernels/src/kernels/cli/__init__.py

@@ -3,6 +3,7 @@
 from pathlib import Path
 from typing import Literal
 
+from kernels.cli.cache import print_cached_kernels
 from kernels.cli.describe_api import print_kernel_api
 from kernels.cli.download import download_kernels
 from kernels.cli.info import print_kernel_info
@@ -16,6 +17,17 @@ def main():
     parser = argparse.ArgumentParser(prog="kernel", description="Manage compute kernels")
     subparsers = parser.add_subparsers(required=True)
 
+    cache_parser = subparsers.add_parser("cache", help="Inspect cached kernels")
+    cache_subparsers = cache_parser.add_subparsers(required=True)
+    cache_ls_parser = cache_subparsers.add_parser("ls", help="List cached kernel versions and their build variants")
+    cache_ls_parser.add_argument("--cache-dir", type=Path, help="Inspect a specific cache directory")
+    cache_ls_parser.add_argument("--json", action="store_true", help="Print cached kernel information as JSON")
+    cache_ls_parser.add_argument("--show-warnings", action="store_true", help="Show details of cache inconsistencies")
+    cache_ls_parser.add_argument(
+        "--no-truncate", action="store_true", help="Display full values regardless of terminal width"
+    )
+    cache_ls_parser.set_defaults(func=kernel_cache_ls)
+
     check_parser = subparsers.add_parser("check", help="Check a kernel for compliance")
     check_parser.add_argument("repo_id", type=str, nargs="?")
     check_parser.set_defaults(func=_check_moved)
@@ -179,6 +191,15 @@ def kernel_info(args):
     )
 
 
+def kernel_cache_ls(args):
+    print_cached_kernels(
+        cache_dir=args.cache_dir,
+        json_output=args.json,
+        show_warnings=args.show_warnings,
+        no_truncate=args.no_truncate,
+    )
+
+
 def kernel_api(args):
     print_kernel_api(
         args.kernel,

kernels/src/kernels/cli/cache.py

@@ -0,0 +1,123 @@
+import json
+import sys
+from pathlib import Path
+from typing import cast
+
+from huggingface_hub import CachedRevisionInfo, constants, scan_cache_dir
+from huggingface_hub.utils import CacheNotFound, tabulate
+from huggingface_hub.utils._parsing import format_timesince
+
+from kernels.hf_hub import _get_cache_dir
+from kernels.variants import get_variants_local
+
+
+def print_cached_kernels(
+    *,
+    cache_dir: Path | None = None,
+    json_output: bool = False,
+    show_warnings: bool = False,
+    no_truncate: bool = False,
+) -> None:
+    """List cached kernel versions with their own build variants."""
+    cache_dir = Path(cache_dir or _get_cache_dir() or constants.HF_HUB_CACHE).expanduser().resolve()
+    try:
+        cache = scan_cache_dir(cache_dir)
+    except CacheNotFound:
+        cache = None
+    except (OSError, ValueError) as error:
+        print(f"Cannot scan kernel cache: {error}", file=sys.stderr)
+        sys.exit(1)
+
+    repos = []
+    rows: list[list[str]] = []
+    if cache is not None:
+        for repo in sorted(cache.repos, key=lambda repo: repo.repo_id):
+            if repo.repo_type != "kernel":
+                continue
+            version_refs = [
+                (number, ref, revision)
+                for ref, revision in repo.refs.items()
+                if (number := _version_number(ref)) is not None
+            ]
+            if not version_refs:
+                continue
+            versions = []
+            for _, version, revision in sorted(version_refs, key=lambda entry: (entry[0], entry[1])):
+                variants = _cached_variants(revision)
+                last_accessed = max(
+                    (file.blob_last_accessed for file in revision.files),
+                    default=revision.snapshot_path.stat().st_atime,
+                )
+                versions.append(
+                    {
+                        "version": version,
+                        "size_on_disk": revision.size_on_disk,
+                        "last_accessed": last_accessed,
+                        "last_modified": revision.last_modified,
+                        "variants": variants,
+                    }
+                )
+                rows.append(
+                    [
+                        f"kernel/{repo.repo_id}",
+                        revision.size_on_disk_str,
+                        format_timesince(last_accessed),
+                        revision.last_modified_str,
+                        version,
+                        variants[0] if variants else "-",
+                    ]
+                )
+                rows.extend(["", "", "", "", "", variant] for variant in variants[1:])
+            repos.append(
+                {
+                    "repo_id": repo.repo_id,
+                    "size_on_disk": repo.size_on_disk,
+                    "last_accessed": repo.last_accessed,
+                    "last_modified": repo.last_modified,
+                    "versions": versions,
+                }
+            )
+
+    if json_output:
+        print(json.dumps({"cache_dir": str(cache_dir), "repos": repos}, indent=2))
+    elif not rows:
+        print(f"No cached kernel versions found in {cache_dir}.")
+    else:
+        # Use the same terminal-width handling as `hf cache ls`.
+        from huggingface_hub.cli._output import _truncate_columns
+
+        headers = ["ID", "SIZE", "LAST_ACCESSED", "LAST_MODIFIED", "VERSION", "VARIANTS"]
+        truncated = _truncate_columns(headers, rows, no_truncate=no_truncate)
+        print(tabulate(cast(list[list[str | int]], rows), headers=headers, alignments={"SIZE": "right"}))
+        if truncated:
+            print("Hint: Use `--no-truncate` or `--json` to display full values.", file=sys.stderr)
+
+    if cache is not None and cache.warnings:
+        if show_warnings:
+            for warning in cache.warnings:
+                print(f"Warning: {str(warning).rstrip('.')}. Repo ignored.", file=sys.stderr)
+        else:
+            print(
+                f"Warning: Found {len(cache.warnings)} cache inconsistencies. "
+                "Re-run with `--show-warnings` to display them.",
+                file=sys.stderr,
+            )
+
+
+def _version_number(ref: str) -> int | None:
+    if ref.startswith("v") and ref[1:].isdecimal():
+        return int(ref[1:])
+    return None
+
+
+def _cached_variants(revision: CachedRevisionInfo) -> list[str]:
+    build_path = revision.snapshot_path / "build"
+    # Empty build directories do not contain a cached variant.
+    cached_dirs = {
+        file.file_path.relative_to(build_path).parts[0]
+        for file in revision.files
+        if build_path in file.file_path.parents
+    }
+    return sorted(
+        variant.variant_str for variant in get_variants_local(build_path) if variant.variant_str in cached_dirs
+    )