Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,11 @@ Code: [`openai_vision/pipeline.py`](./openai_vision/pipeline.py)
download, declared fault injection, local processing, Airflow scheduling,
inspection with Foxglove and DuckDB, and SQL curation.

**Prerequisites:** accepted Hugging Face dataset terms, `hf` authentication,
network access, and about 2.5 GB of disk (a ~1 GB pinned download plus the
generated corpus). Docker is needed only for the Airflow path.
**Prerequisites:** accepted Hugging Face dataset terms, the `hf` CLI
authenticated (`uv tool install -U huggingface_hub`, then
`hf auth login`), network access, and about 2.5 GB of disk (a ~1 GB pinned
download plus the generated corpus). Docker is needed only for the Airflow
path.

```bash
uv run python examples/egocentric/prepare.py
Expand Down
5 changes: 3 additions & 2 deletions examples/egocentric/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,9 @@ declared data you can edit (see [inject your own faults](#inject-your-own-faults

## Prerequisites

- A Hugging Face account that has accepted the dataset's access terms, and an
authenticated `hf` CLI (`hf auth login`).
- A Hugging Face account that has accepted the dataset's access terms, and the
`hf` CLI, authenticated: `uv tool install -U huggingface_hub` then
`hf auth login`.
- Network access and disk: a ~1 GB download, about 2.5 GB total once the
sources are extracted and the 96 episodes (~640 MB) are written.
- Docker with Compose v2 (only for the [Airflow path](#run-it-under-airflow)).
Expand Down
6 changes: 6 additions & 0 deletions examples/egocentric/prepare.py
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,12 @@ def _ensure_source_archive(manifest: CorpusManifest, data_root: Path) -> Path:
download_root = data_root / "huggingface"
archive_path = download_root / manifest.archive.path
if not archive_path.is_file():
if shutil.which("hf") is None:
raise RuntimeError(
"the `hf` CLI is required to download this dataset but is not on PATH. "
"Install it with `uv tool install -U huggingface_hub`, then "
"`hf auth login`."
)
command = [
"hf",
"download",
Expand Down
Loading