Skip to content

api-contract.md

Latest commit

 

History

History
344 lines (284 loc) · 12.6 KB

api-contract.md

File metadata and controls

344 lines (284 loc) · 12.6 KB
Raw

HTTP API Contract

English 中文

The machine-readable contract is docs/openapi.json (also served at GET /openapi.json on every model process). This page is the human summary. Do not copy the removed {req_id, code, msg, data} envelope or the img_data field from older blog posts — those requests answer 422.

Topology note: mortred-gateway

Production traffic goes through mortred-gateway (default :8080). The gateway looks up the catalog id first (/v1/models/{id}/…), then the legacy server_uri, and forwards to the model's loopback port. The gateway enforces the external Bearer token (MORTRED_GATEWAY_AUTH_TOKEN or MORTRED_API_TOKEN), maps a dead upstream to 503 and transport failures to 502; all model-server status codes below pass through unchanged. GET /healthz is public on the gateway. GET /metrics requires MORTRED_METRICS_TOKEN on every listen, including loopback. A gateway refuses to start without a distinct scrape Bearer (not the inference token). Model ports are loopback-only and must not be exposed. Mortred itself is plain HTTP; TLS is terminated by Nginx on the host network (mortredctl init-edge). Fail-closed startup refuses a listener with no auth, a missing metrics token, and a wildcard bind unless MORTRED_EXPOSE=docker or unsafe.

After one successful inference, gateway GET /metrics carries mortred_gateway_stage_duration_ms for the five stages gw_in, route, auth, fwd, and upstream. server_time_ms stays in the inference response body. Scrape authentication for GET /metrics is unchanged and still requires MORTRED_METRICS_TOKEN, including loopback.

The supervisor (:8787) exposes the management REST API under /api/v1/ (health/catalog/status/lifecycle/logs/metrics) and the embedded web UI; POST /api/v1/servers/{id}/start|stop|restart returns 200 with {"ok":true} on success and 500 with {"ok":false,"error":...} on failure (same contract as graceful_restart). Clients may rely on the HTTP status or the body ok field. mortredctl is its CLI client. Inference smoke tests (the Web UI send button and mortredctl infer) POST the data-plane envelope to POST /v1/models/{id}/infer on the gateway, with the same Bearer token as the management API (MORTRED_API_TOKEN). The supervisor is management only (catalog / lifecycle / logs / UI). Inference and async jobs go through the gateway. Legacy {server_uri} on :8080 is still accepted.

Any change to status codes, endpoints or response schemas must update docs/openapi.json (regenerate with python scripts/gen_openapi.py).

Authentication

Model inference endpoints require an Authorization header when the server is configured with auth_token:

Authorization: Bearer <token>
  • Missing or invalid token: 401 + WWW-Authenticate: Bearer realm="Mortred".
  • Request gate order on model ports (SME-07): public /healthz /ready /openapi.json first (no rate_limit_qps); then Bearer auth (401 if missing/invalid); then per-IP rate_limit_qps (429). Unauthenticated traffic does not consume the IP QPS budget.
  • Health/metadata endpoints (/healthz, /ready, /openapi.json) are public. Supervisor GET /api/v1/metrics requires the management token. Gateway GET /metrics requires MORTRED_METRICS_TOKEN on every listen, including loopback. Model GET /metrics requires the process auth token (supervisor children always have MORTRED_AUTH_TOKEN); empty token yields 401, not a public scrape. /healthz stays public. Do not reuse the inference token as the scrape secret.
  • When auth_token is empty, model inference and /metrics return 401. Health/metadata stay public. The server still refuses a non-loopback listen without a token, and refuses a wildcard bind unless MORTRED_EXPOSE=docker|unsafe.

Request rules

  • Model endpoints accept POST only; any other method returns 405 with Allow: POST.
  • Content-Type must be application/json (a ; charset= parameter is allowed); a missing or different media type returns 415.
  • The request body is limited to request_size_limit MB; an explicit Content-Length above the limit returns 413.
  • Allowed JSON keys are only req_id, images, params, options. Unknown keys and the removed img_data field answer 422.

Common response envelope

{
  "status": 0,
  "status_str": "OK",
  "task_id": "client-provided-or-server-generated",
  "model": { "name": "mobilenetv2", "version": "" },
  "results": [
    {
      "status": 0,
      "data": {
        "class_id": 123,
        "category": "tabby cat",
        "scores": [0.1, 0.8, 0.1]
      }
    }
  ],
  "server_time_ms": 41.2,
  "partial": false
}

On a contract violation (HTTP 422):

{
  "status": 66,
  "status_str": "invalid request parameter",
  "task_id": "",
  "results": [],
  "server_time_ms": 0.0,
  "partial": false,
  "errors": [
    {
      "pointer": "/img_data",
      "message": "field 'img_data' was removed; use images: [\"<base64>\"] (migration: img_data -> images[0])"
    }
  ]
}

Read HTTP status first, then top-level status, then each results[i].status. results[i].data is null when that item failed. Ignore unknown response fields. Per-task payloads live under results[].data. Human-readable shapes are below; the full schema remains in docs/openapi.json components.schemas (src/server/response_serializers.h).

Per-task results[].data

Recommended inference path through the gateway is POST /v1/models/{id}/infer (catalog {id} examples: yolov8s, mobilenetv2). Legacy {server_uri} paths such as /mortred_ai_server_v1/... are still accepted on :8080 but are not the recommended client entry.

Each block below is the value of one results[i].data (not the outer envelope). Detection and OCR detection are top-level JSON arrays.

Classification (mobilenetv2):

{
  "class_id": 281,
  "category": "tabby cat",
  "scores": [0.02, 0.91, 0.03]
}

Detection (yolov8s) — top-level array; items have bbox [x1,y1,x2,y2] and detail_infos, no polygon:

[
  {
    "class_id": 0,
    "score": 0.93,
    "category": "person",
    "bbox": [12.0, 40.0, 210.5, 480.0],
    "detail_infos": {}
  }
]

Segmentation (pphuman_server) — top-level object with image and colorized_mask (not matting, which only returns image):

{
  "image": "<base64-mask>",
  "colorized_mask": "<base64-colorized-mask>"
}

OCR text detection (dbnet) — top-level array; items have polygon and no text key (detection regions only, not recognition):

[
  {
    "score": 0.97,
    "bbox": [100.0, 200.0, 340.0, 260.0],
    "polygon": [[100.0, 200.0], [340.0, 200.0], [340.0, 260.0], [100.0, 260.0]],
    "detail_infos": {}
  }
]

Matting (modnet) — top-level object; the only key is image (base64 matting result). This is not segmentation and has no colorized_mask:

{
  "image": "<base64-matting-result>"
}

Enhancement (realesrgan) — top-level object; the only key is image (base64 enhanced image):

{
  "image": "<base64-enhanced-image>"
}

Depth (metric3d_512) — top-level object; the only key is image (base64 colorized depth map):

{
  "image": "<base64-colorized-depth-map>"
}

HTTP status mapping

The JSON field is status (not code). Mapping is src/server/http_status.h.

status Meaning HTTP
0 OK 200
68 Deadline exceeded, partial results 200
50 JSON decode error 400
3 Empty input image 400
66 Invalid request (img_data, unknown key, bad params) 422
60 Unsupported media type 415
61 Request entity too large 413
67 Too many items in one request 413
62 Method not allowed 405
63 Not found 404
65 Service not ready 503
4 Model run timeout 504
6 Model output contract failed 500
401 Unauthorized 401
429 Rate limited or queue full (Retry-After) 429
others Server error 500

Admitted model requests always return results[] of length N = images[]. Batch path (max_batch_size > 1, SME-09): shares the same request deadline / HTTP timer; past-deadline entries time out without a new worker checkout (in-flight run_batch is not cooperatively cancelled). HTTP 504 (status 4) means zero items had been published when the deadline fired; each slot is timeout with data: null. HTTP 200 + status 68 + partial: true means at least one item completed in time — clients must not retry that response as a 5xx. Rejection envelopes (401/422/…) may still use empty results[].

Common headers

Content-Type: application/json; charset=utf-8
X-Request-ID: <task_id>
Cache-Control: no-store

Common endpoints

Endpoint Method Description
/healthz GET Liveness probe
/ready GET Readiness probe
/metrics GET Prometheus metrics (Bearer as above)
/openapi.json GET OpenAPI document (served from the embedded copy)

Unknown paths, including the removed /welcome and /hello_world HTML probes, answer 404 with process-level UnifiedResponse.

Model inference request

{
  "req_id": "optional",
  "images": ["base64 encoded image"],
  "params": {},
  "options": {}
}

images is required and always an array (≥1). params / options are optional objects. Model-specific knobs (thresholds, DDPM timesteps, …) go in params, not at the root. Generative models still require images[] (≥1); the pixels are ignored, so a dummy base64 string is enough.

Removed field: img_data (HTTP 422)

The following body is not a success request. It is rejected even if images is also present:

{
  "req_id": "legacy",
  "img_data": "base64 encoded image"
}

Overload behaviour

When max_queue_depth > 0 and the waiting queue is full, the model server rejects immediately with 429 and a Retry-After header (estimated drain time from queue depth, run-time EWMA and worker count, clamped to 1-60s). The gateway forwards both verbatim. New optional server keys: max_queue_depth (0 = unlimited), max_batch_size (product servers set 1; a value above 1 collects requests for max_batch_delay_ms), plus the mortred_queue_rejected_total / mortred_batch_size / mortred_batch_window_wait_ms metrics.

Per-item failure isolation: within a batch, a failing item (bad image, decode error) returns its own error status while its batch mates keep their results; only session-level failures (engine errors) fail every participating item.

Async jobs

Long-running inference can be submitted asynchronously when the server enables async_enabled. On the model port the paths are unchanged (POST /jobs, GET /jobs/{id}, …). Through the gateway the same handlers are reached with a catalog-id prefix; the gateway is stateless and rewrites Location / poll_url / result_url onto that prefix:

Gateway Method Upstream on the model port
/v1/models/{id}/infer POST {server_uri}
/v1/models/{id}/jobs POST /jobs
/v1/models/{id}/jobs/{job} GET /jobs/{job}
/v1/models/{id}/jobs/{job}/wait GET /jobs/{job}/wait + query
/v1/models/{id}/jobs/{job}/result GET /jobs/{job}/result
{server_uri} POST {server_uri} (legacy)

GET /v1/models/{id}/infer and GET {server_uri} return 405. Unknown {id} is the same 404 envelope as an unknown server_uri. If the model has async_enabled off, upstream 404 is passed through.

Endpoint (model port) Success Errors
POST /jobs 202 at admission with job_id, state: pending, poll_url, result_url 429 when the admission queue is full
GET /jobs/{id} 200 with state (pending/running/done/failed/timeout) 404 unknown id
GET /jobs/{id}/wait?timeout=N 200 when the job is terminal, or when the wait budget expires (timeout is milliseconds; default 30000, cap 300000). State may still be pending/running on expiry 404 unknown id
GET /jobs/{id}/result 200 standard envelope (repeatable) 404 unknown id, 409 not finished (including pending/running/failed/timeout)

202 means the server accepted the job, not that inference has finished. A correct client never treats POST /jobs like a blocking /infer. Submit the same JSON envelope as /infer (images[]). Step-by-step customer verification is in async-jobs-customer-test.md.

The job ledger is in-memory (lost on restart). The component design, concurrency contract and verification gates are documented in async-jobs-customer-test.md.