diff --git a/docs/hub/jobs-overview.md b/docs/hub/jobs-overview.md index 089f92b4a5..bf2fdbf624 100644 --- a/docs/hub/jobs-overview.md +++ b/docs/hub/jobs-overview.md @@ -1,18 +1,16 @@ # Jobs Overview -Run compute jobs on Hugging Face infrastructure with a familiar UV & Docker-like interface! +Hugging Face Jobs runs your code on remote CPUs and GPUs. Use it to fine-tune models, run inference over datasets, or process data. + +A Job runs a command in an environment on the hardware you choose. You can follow its status and logs from the terminal or on the Hub. + +[Start with the Quickstart](./jobs-quickstart) to run your first CPU and GPU Jobs, or [browse examples](./jobs-examples) for a workload to adapt.
-

UV & Docker-like CLI

uv,run,ps,logs,stats,inspect

Any Hardware

CPUs to A100s & TPUs

Run Anything

UV, Docker, HF Spaces & more

Pay-as-you-go

Pay only for seconds used

- -The Hugging Face Hub provides compute for AI and data workflows via Jobs. - -Jobs runs on Hugging Face infrastructure and aim at providing AI builders, Data engineers, developers and AI agents an easy access to cloud infrastructure to run their workloads. They are ideal to fine tune AI models and run inference with GPUs, but also for data ingestion and processing as well. - -A job is defined with a command to run (e.g. a UV or python command), a hardware flavor (CPU, GPU, TPU), and optionally a Docker image from Hugging Face Spaces or Docker Hub. Many jobs can run in parallel, which is useful e.g. for parameter tuning or parallel inference and data processing. +

UV & Docker-like CLI

uv,run,ps,logs,stats,inspect

CPUs & GPUs

Choose hardware for your workload

Run your code

Python scripts & Docker images

Pay-as-you-go

Pay for the compute you use

## Run Jobs from anywhere @@ -24,20 +22,26 @@ There are multiple tools you can use to run jobs: ## Run any workload -The `hf` Jobs CLI and the `huggingface_hub` Python client offer a UV-like interface to run Python workloads. UV installs the required Python dependencies and run the Python script in one single command. Python dependencies may also be defined in a self-contained UV script, and in this case there is no need to specify anything but the UV script to run the Job. +### Python scripts + +Use `hf jobs uv run` to run a Python script remotely. Specify its dependencies with `--with` or in a [script header](https://docs.astral.sh/uv/guides/scripts/#declaring-script-dependencies). Jobs uses uv to install those dependencies before running your code. ```diff - uv run + hf jobs uv run ``` -More generally, Hugging Face Jobs supports any workload based on Docker and a command. Jobs offers a Docker-like interface to rub Jobs, where you can specify a Docker image from Hugging Face Spaces or Docker Hub, as well as the command to run. Docker provides the ability to package ready-to-use environments as Docker images that are shared by the community or custom made. Therefore you may choose or define your Docker image based on what your workloads need (e.g. python, torch, vllm) and run any command. This is more advanced than using UV but provides more flexibility. +### Docker images + +Use `hf jobs run` with a Docker image and the command to run. The image provides the tools and libraries your workload needs, whether you're using Python or another language. Choose an [existing image](./jobs-popular-images), use one built from a [Docker Space](./spaces-sdks-docker), or build your own. ```diff - docker run + hf jobs run ``` +Many Jobs can run in parallel, for tasks such as parameter tuning, inference and data processing. + ## Automate Jobs Trigger Jobs automatically with a schedule or using webhooks. diff --git a/docs/hub/jobs-quickstart.md b/docs/hub/jobs-quickstart.md index 1454f2daba..57c5d2a2e5 100644 --- a/docs/hub/jobs-quickstart.md +++ b/docs/hub/jobs-quickstart.md @@ -1,162 +1,113 @@ # Quickstart -In this guide you will run a Job to fine-tune an open source model on Hugging Face infrastructure in only a few minutes. -Make sure you are logged in to Hugging Face and have [pre-paid credits](https://huggingface.co/settings/billing) available on your account or organization. You can then access your [Jobs page](https://huggingface.co/settings/jobs) to create and manage Jobs. +Run your Python code on Hugging Face CPUs and GPUs. In this guide, you'll run a simple command on a CPU, then generate text with a small language model on a GPU. -
- - -
+You'll need a Hugging Face account with [pre-paid credits](https://huggingface.co/settings/billing). See [Pricing and Billing](./jobs-pricing) for compute costs. -## Getting started +## 1. Set up the CLI -First install the Hugging Face CLI: - -### 1. Install the CLI - - -Recommended approach: - -```bash ->>> curl -LsSf https://hf.co/cli/install.sh | bash -``` - -Or using Homebrew: - -```bash ->>> brew install hf -``` - -Or using uv: +[Install the Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/en/guides/cli#getting-started), then log in to your account: ```bash ->>> uv tool install hf +>>> hf auth login ``` -### 2. Login to your Hugging Face account +## 2. Run Hello World -Login +Run this command in your terminal: ```bash ->>> hf auth login +>>> hf jobs uv run python -c 'print("Hello from the cloud!")' ``` -### 3. Create your first jobs using the `hf jobs` command +`hf jobs uv run` runs the command in a Python environment on Hugging Face infrastructure. It uses a CPU by default and streams the Job's logs to your terminal. After startup, you'll see: -Run a UV command or script - -```bash ->>> hf jobs uv run python -c 'print("Hello from the cloud!")' -Job started with ID: 693aef401a39f67af5a41c0e -View at: https://huggingface.co/jobs/lhoestq/693aef401a39f67af5a41c0e +```text Hello from the cloud! ``` -```bash ->>> echo "print('Hello from uv script!')" > script.py ->>> hf jobs uv run script.py -Job started with ID: 695f6cd8d2f3efac77e8cf7f -View at: https://huggingface.co/jobs/lhoestq/695f6cd8d2f3efac77e8cf7f -Hello from uv script! -``` +The CLI also prints your Job's ID and a link to its page. Open the link to view its status and logs in your browser. You can find your Jobs again on your [Jobs page](https://huggingface.co/settings/jobs), or use the ID with the CLI commands below. + +## 3. Run a model on a GPU -Run a Docker command +Run this prepared script to generate a robot name. You can [view it on GitHub](https://github.com/huggingface/hub-docs/blob/main/examples/jobs/hello_gpu.py) or read the code below. ```bash ->>> hf jobs run ubuntu echo 'Hello from the cloud!' -Job started with ID: 693aee76c67c9f186cfe233e -View at: https://huggingface.co/jobs/lhoestq/693aee76c67c9f186cfe233e -Hello from the cloud! +hf jobs uv run \ + --flavor t4-small \ + --timeout 5m \ + https://raw.githubusercontent.com/huggingface/hub-docs/main/examples/jobs/hello_gpu.py ``` -### 4. Check your first jobs - -The job logs appear in your terminal, but you can also see them in your jobs page. Open the job page to see the job information, status and logs: - -
- - -
+- `--flavor t4-small` selects a machine with an NVIDIA T4 GPU. +- `--timeout 5m` sets a five-minute limit on the Job. +The Job downloads the model and prints its answer in the logs. For example: -## The training script +```text +RoboLearnbot +``` -Here is a simple training script to fine-tune a base model to a conversational model using Supervised Fine-Tuning (SFT). It uses the [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) model and the [trl-lib/Capybara](https://huggingface.co/datasets/trl-lib/Capybara) dataset, and the [TRL](https://huggingface.co/docs/trl/en/index) library, and saves the resulting model to your Hugging Face account under the name `"Qwen2.5-0.5B-SFT"`: +Here is the complete script: ```python -from datasets import load_dataset -from trl import SFTTrainer +# /// script +# dependencies = ["torch", "transformers"] +# /// + +from transformers import pipeline -dataset = load_dataset("trl-lib/Capybara", split="train") -trainer = SFTTrainer( - model="Qwen/Qwen2.5-0.5B", - train_dataset=dataset, +generator = pipeline( + "text-generation", + model="HuggingFaceTB/SmolLM2-360M-Instruct", + dtype="float16", ) -trainer.train() -trainer.push_to_hub("Qwen2.5-0.5B-SFT") +messages = [{ + "role": "user", + "content": "Suggest a name for a robot that helps people learn Python. Answer with only the name.", +}] +outputs = generator(messages, max_new_tokens=48, do_sample=False, return_full_text=False) +print(outputs[0]["generated_text"]) ``` -Save this script as `train.py`, and we can now run it with UV on Hugging Face Jobs. +The [dependency header](https://docs.astral.sh/uv/guides/scripts/#declaring-script-dependencies) tells uv to install `torch` and `transformers` in the Job. You can also specify dependencies with `--with`. You only need the `hf` CLI locally. -## Run the training job +The script runs [SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct) on the Job's GPU. `max_new_tokens` caps the answer length. -`hf jobs` takes several arguments: select the hardware with `--flavor`, choose a maximum duration with `--timeout`, and pass environment variable with `--env` and `--secrets`. Here we use the A100 Large GPU flavor with `--flavor a100-large` and pass your Hugging Face token as a secret with `--secrets HF_TOKEN` in order to be able to push the resulting model to your account. See [Persist your results](./jobs-manage#persist-your-results) for how to make sure your Job's outputs survive after it finishes. +Startup time varies with hardware availability, dependency installation and model downloads. -Moreover, UV accepts the `--with` argument to define python dependencies, so we use `--with trl` to have the `trl` library available. +> [!TIP] +> Pressing Ctrl+C stops streaming logs; the Job keeps running. To stop the Job, use `hf jobs cancel JOB_ID`, replacing `JOB_ID` with the ID printed by the CLI. -You can now run the final command which looks like this: +## 4. Check your result + +Use the GPU Job's ID to check its status and read its logs again: ```bash -hf jobs uv run \ - --flavor a100-large \ - --timeout 6h \ - --with trl \ - --secrets HF_TOKEN \ - train.py +>>> hf jobs inspect JOB_ID +>>> hf jobs logs JOB_ID ``` -The logs appear in your terminal, and you can safely Ctrl+C to stop streaming the logs, the job will keep running. - -``` -... -Downloaded nvidia-cudnn-cu12 -Downloaded torch -Installed 66 packages in 233ms -Generating train split: 100%|██████████| 15806/15806 [00:00<00:00, 76686.50 examples/s] -Generating test split: 100%|██████████| 200/200 [00:00<00:00, 43880.36 examples/s] -Tokenizing train dataset: 100%|██████████| 15806/15806 [00:41<00:00, 384.97 examples/s] -Truncating train dataset: 100%|██████████| 15806/15806 [00:00<00:00, 212272.92 examples/s] -The model is already on multiple devices. Skipping the move to device specified in `args`. -The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. -{'loss': 1.7357, 'grad_norm': 4.8733229637146, 'learning_rate': 1.9969635627530365e-05, 'entropy': 1.7238958358764649, 'num_tokens': 59528.0, 'mean_token_accuracy': 0.6124177813529968, 'epoch': 0.01} -{'loss': 1.6239, 'grad_norm': 6.200186729431152, 'learning_rate': 1.9935897435897437e-05, 'entropy': 1.644005584716797, 'num_tokens': 115219.0, 'mean_token_accuracy': 0.6259662985801697, 'epoch': 0.01} -{'loss': 1.4449, 'grad_norm': 6.167325496673584, 'learning_rate': 1.990215924426451e-05, 'entropy': 1.5156117916107177, 'num_tokens': 171787.0, 'mean_token_accuracy': 0.6586395859718323, 'epoch': 0.02} -{'loss': 1.6023, 'grad_norm': 5.133708953857422, 'learning_rate': 1.986842105263158e-05, 'entropy': 1.6885507702827454, 'num_tokens': 226067.0, 'mean_token_accuracy': 0.6271904468536377, 'epoch': 0.02} -``` +A successful run has the status `COMPLETED`, and its logs contain the generated answer. -Follow the Job advancements on the job page on Hugging Face: +The answer remains available in the Job's logs after it finishes. When you adapt the script to produce files, [save those results to a bucket or Hub repository](./jobs-manage#persist-your-results) so they survive the Job. -
- - -
+## Try your own script (optional) -Monitor GPU usage and other metrics in the CLI or use the [MacOS menu bar](./jobs-manage#macos-menu-bar). Here with the CLI you get: +Copy the code above into `hello_gpu.py`, edit the prompt in `messages`, and run your local file: ```bash ->>> hf jobs stats -JOB ID CPU % NUM CPU MEM % MEM USAGE NET I/O GPU UTIL % GPU MEM % GPU MEM USAGE ------------------------- ----- ------- ----- ---------------- --------------- ---------- --------- --------------- -695e83c5d2f3efac77e8cf18 8% 12.0 7.18% 10.9GB / 152.5GB 0.0bps / 0.0bps 100% 31.92% 25.9GB / 81.2GB +hf jobs uv run --flavor t4-small --timeout 5m hello_gpu.py ``` -Once the job is done, find your model on your account: +The CLI uploads your edited script automatically. Replace `hello_gpu.py` with its path if you saved it elsewhere. -
- - -
+## Next steps -Congrats ! You just run your first Job to fine-tune an open source model 🔥 +Build on this example with a larger workload: -Feel free to try out your model locally and evaluate it using e.g. [transformers](https://huggingface.co/docs/transformers) by clicking on "Use this model", or deploy it to [Inference Endpoints](https://huggingface.co/docs/inference-endpoints) in one click using the "Deploy" button. +- [Annotate a dataset with OCR, classification or batch inference](./jobs-examples#uv-scripts). +- [Fine-tune and save a model](./jobs-examples#guides-to-train-with-jobs) using TRL or Unsloth. +- [Read datasets or buckets and save processed results](./jobs-large-datasets). +- [Run commands in Docker images](./jobs-configuration#docker-jobs). +- [Use Jobs from a coding agent](./jobs-examples#coding-agent-skills). diff --git a/docs/hub/jobs.md b/docs/hub/jobs.md index d28c23d36c..c838533d7b 100644 --- a/docs/hub/jobs.md +++ b/docs/hub/jobs.md @@ -1,8 +1,10 @@ # Jobs -`Hugging Face Jobs` provide compute for AI and data workflows, allowing you to run workloads on Hugging Face infrastructure with a familiar UV & Docker-like interface. Jobs are ideal for fine-tuning AI models, running inference with GPUs, and data ingestion and processing. +Hugging Face Jobs runs your code on CPUs and GPUs in the cloud. Run Python scripts directly, or use a Docker image for other workloads and custom environments. -You can run jobs using the `hf` CLI, the `huggingface_hub` Python client, or the Jobs HTTP API. Jobs support any hardware from CPUs to A100s & TPUs, with pay-as-you-go pricing where you only pay for seconds used. +Use Jobs to fine-tune models, run inference over datasets, or process data without setting up your own compute infrastructure. + +[Run your first Job](./jobs-quickstart) with the `hf` CLI, or [browse examples](./jobs-examples) to start from an existing script. ## Contents diff --git a/examples/jobs/hello_gpu.py b/examples/jobs/hello_gpu.py new file mode 100644 index 0000000000..a009fedbe7 --- /dev/null +++ b/examples/jobs/hello_gpu.py @@ -0,0 +1,17 @@ +# /// script +# dependencies = ["torch", "transformers"] +# /// + +from transformers import pipeline + +generator = pipeline( + "text-generation", + model="HuggingFaceTB/SmolLM2-360M-Instruct", + dtype="float16", +) +messages = [{ + "role": "user", + "content": "Suggest a name for a robot that helps people learn Python. Answer with only the name.", +}] +outputs = generator(messages, max_new_tokens=48, do_sample=False, return_full_text=False) +print(outputs[0]["generated_text"])