Enhance API with healthcheck and profiling features

.dockerignore

@@ -0,0 +1 @@
+demo/

.github/workflows/ci.yml

@@ -0,0 +1,51 @@
+name: CI
+
+on:
+  push:
+    branches: ["**"]
+  pull_request:
+    branches: ["**"]
+
+jobs:
+  lint-and-type-check:
+    name: Pre-commit hooks (lint, sort, type-check, docker)
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Install dependencies
+        run: uv sync --all-extras
+
+      - name: Run pre-commit on all files
+        uses: pre-commit/action@v3.0.1
+        with:
+          extra_args: --all-files # TODO ymu : maybe run only on changed files
+
+  test:
+    name: Tests
+    runs-on: ubuntu-latest
+    needs: lint-and-type-check
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Install dependencies
+        run: uv sync --all-extras

.github/workflows/release.yml

@@ -0,0 +1,50 @@
+name: Release
+
+on:
+  push:
+    tags: ["v*"]
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_PREFIX: ghcr.io/${{ github.repository_owner }}
+
+jobs:
+  build-and-push:
+    name: Build and push image
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Log in to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Generate tags
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.IMAGE_PREFIX }}/mlprofiler
+          tags: |
+            type=raw,value=latest
+            type=sha,prefix=,format=short
+            type=semver,pattern={{version}}
+
+      - name: Build and push mlprofiler
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          file: docker/Dockerfile.prod
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max

.gitignore

@@ -182,9 +182,9 @@ cython_debug/
 .abstra/
 
 # Visual Studio Code
-#  Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore 
+#  Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
 #  that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
-#  and can be added to the global gitignore or merged into this file. However, if you prefer, 
+#  and can be added to the global gitignore or merged into this file. However, if you prefer,
 #  you could uncomment the following to ignore the entire vscode folder
 # .vscode/
 
@@ -209,3 +209,5 @@ __marimo__/
 # Others
 out/
 data/
+.idea/
+docker/.env

.pre-commit-config.yaml

@@ -0,0 +1,34 @@
+repos:
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  rev: v0.4.4
+  hooks:
+  - id: ruff
+    args: [--fix, --select, "I"]
+  #- id: ruff-format
+
+- repo: https://github.com/pre-commit/mirrors-mypy
+  rev: v1.10.0
+  hooks:
+  - id: mypy
+    additional_dependencies:
+    - fastapi
+    - pydantic
+    - types-ujson
+
+- repo: https://github.com/hadolint/hadolint
+  rev: v2.12.0
+  hooks:
+  - id: hadolint-docker
+    args: [--failure-threshold, warning]
+
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v4.6.0
+  hooks:
+  - id: trailing-whitespace
+  - id: end-of-file-fixer
+  - id: check-yaml
+  - id: check-toml
+  - id: check-merge-conflict
+  - id: check-added-large-files
+    args: [--maxkb=1000]
+  - id: debug-statements

.python-version

@@ -0,0 +1 @@
+3.12

CHANGELOG.md

@@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Parser metadata in JSON ML profile.
+- "dspipelines" parser
+- External taxonomies support
+- Distinction between context and code to classify
+- Removed the CLI
+
 ## [0.3] - 2025-07-16
 
 ### Added
@@ -19,7 +27,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 - Taxonomy metadata in JSON ML profile.
-- "dspipelines" taxonomy (from *The Art and Practice of Data Science Pipelines*, 2022 ICSE) with some improvements (Library Loading, Others)
+- "dspipelines" taxonomy (from *The Art and Practice of Data Science Pipelines*, 2022 ICSE) with some improvements (
+  Library Loading, Others)
 
 ## [0.1] - 2025-06-01
 

README.md

@@ -1,51 +1,98 @@
-# mlprofiler - The ML pipelines profiler
+# ML Profiler
+
+Profiles ML pipelines by analyzing source code and classifying pipeline steps.
 
 ## Requirements
 
-* Docker (tested on version 28.5.1, build e180ab8)
-* Docker Compose (tested on version v2.40.0)
+- Docker ≥ 28.5.1
+- Docker Compose ≥ 2.40.0
+- _(Advanced)_ [Docker Model Runner](https://docs.docker.com/ai/model-runner/get-started/) ≥ 1.1.37 required for the LLM profiler function
+
+## Quick start
+
+```bash
+cd docker
+cp .env.sample .env # edit .env if needed, the defaults work for basic setup
+docker compose --env-file .env up --build
+```
+
+## Advanced setup (LLM profiler)
+
+The LLM profiler function requires Docker Model Runner (DMR). Setup differs by platform.
 
-## Deployment
+### Mac DMR
 
-The following docker compose commands deploy the mlprofiler API and vLLM instance:
+No extra installation needed. DMR uses llama.cpp natively via Metal (Apple Silicon).
 
 ```bash
-$ docker compose --env-file .env build
-$ docker compose --env-file .env up
+docker compose -f docker-compose.yml -f docker-compose.mac.yml --env-file .env up --build
 ```
 
-It is also possible to deploy the LLM monitoring platform (Langfuse) as follows:
+### Linux DMR
+
+[Set up the vLLM inference backend](https://docs.docker.com/ai/model-runner/inference-engines/#setting-up-vllm):
 
 ```bash
-$ docker compose --file docker-compose-monitor-local.yml
+docker model install-runner --backend vllm --gpu cuda
 ```
 
-## Troubleshooting
+[Configure the model](https://docs.docker.com/ai/model-runner/inference-engines/#vllm-configuration):
 
-### Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock
+```bash
+docker model configure --hf_overrides '{
+  "max_model_len": 8192,
+  "max_num_seqs": 10,
+  "gpu_memory_utilization": 0.8,
+  "enforce_eager": true
+}' hf.co/Qwen/Qwen2.5-Coder-7B-Instruct-AWQ
+```
 
-Allow your user to use the socket:
 ```bash
-sudo usermod -a -G docker $USER
-newgrp docker
-reboot
+docker compose -f docker-compose.yml -f docker-compose.linux.dmr.yml --env-file .env up --build
 ```
 
-### Failed to solve: process "/bin/sh -c uv sync --locked --no-install-project" did not complete successfully: exit code: 1
+### Linux vLLM (standalone)
+
+Set `VLLM_FORCE=1` in `.env`, then:
 
-Regenerate the lock file by executing:
 ```bash
-uv lock
+docker compose -f docker-compose.yml -f docker-compose.linux.dmr.yml -f docker-compose.linux.vllm.yml --env-file .env up --build
+```
+
+## Monitoring (optional)
+
+| Stack | Command |
+|---|---|
+| Langfuse ([docs](https://langfuse.com/docs)) | `docker compose -f docker-compose-monitor-langfuse.yml up` |
+| Grafana/Prometheus ([docs](https://github.com/vllm-project/vllm/tree/main/examples/online_serving/prometheus_grafana)) | `docker compose -f docker-compose-monitor-grafana.yml up` |
+
+## Development
+
+Install pre-commit hooks:
+
+```bash
+uv run --with pre-commit pre-commit install
+```
+
+## Troubleshooting
+
+**`permission denied` connecting to Docker socket**
+```bash
+sudo usermod -a -G docker $USER && newgrp docker
 ```
+Then reboot.
 
-### Network colombus-dev_network declared as external, but could not be found
+---
 
-Create the missing network:
+**`uv sync` fails during build**
+
+Regenerate the lockfile:
 ```bash
-docker network create "colombus-dev_network"
+uv lock
 ```
 
-### Error response from daemon: could not select device driver "nvidia" with capabilities: [[gpu]]
+---
+
+**`could not select device driver "nvidia"`**
 
-Install the nvidia toolkit and restart docker using this link:
-https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
+Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#with-apt-ubuntu-debian) and [configure Docker to use the NVIDIA runtime](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#configuring-docker).