Skip to content

Commit 6461e29

Browse files
committed
Say where /api/generate's load answer deviates from Ollama
The isLoadOnly Javadoc attributed the "and without a suffix" condition to Ollama, and toOllamaLoadResponse called its answer "as Ollama gives it". Ollama's GenerateHandler reads the prompt alone and answers a keep_alive of 0 with done_reason "unload". Both are deliberate choices here: an empty prompt with a suffix stays fill-in-the-middle at the start of a file (also for an empty suffix), and keep_alive 0 keeps the load answer because nothing is unloaded. The Javadoc, the README row of the Ollama routes, TODO.md and the F205 entry now say so. Documentation only, no behaviour change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBFFnrhtvXiUeyxjmUc4R2
1 parent 43f2f3b commit 6461e29

4 files changed

Lines changed: 19 additions & 6 deletions

File tree

‎BUGHUNT-FOUND.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1705,6 +1705,7 @@ Repos plus eins. Geprüfte Klassen ohne Fund stehen in [`BUGHUNT-CHECKED.md`](BU
17051705
- **Rot-Grün-Nachweis:** Ohne den Fix der HTTP-Test rot (das Backend generierte); mit ihm grün (64 + 37).
17061706
- **Status:** gefixt
17071707
- **Commit:** java-llama.cpp `200cc90`
1708+
- **Nachtrag (Branch-Review 2026-10-20, Funde J1-4, J3-3, J16-2):** Beide Regeln sind bewusste Abweichungen von Ollama, nicht Ollamas Verhalten: Ollamas `GenerateHandler` prüft nur `req.Prompt == ""` (der `suffix` fügt dort nur `CapabilityInsert` hinzu), beantwortet also einen leeren Prompt mit `suffix` mit der Ladeantwort, und antwortet bei `keep_alive: 0` ohne Prompt mit `done_reason "unload"`. Hier bleibt der leere Prompt mit `suffix` FIM am Dateianfang (auch bei `"suffix":""`, `hasSuffix` fragt nur nach einem String), und `keep_alive: 0` bekommt die Ladeantwort, weil nichts entladen wird — ein „unload“ wäre eine Meldung über eine Aktion, die nicht stattfand. Die Javadoc von `isLoadOnly`/`toOllamaLoadResponse`, die README-Zeile der Ollama-Routen und `TODO.md` sagen das jetzt; vorher schrieb die Javadoc die `suffix`-Bedingung Ollama zu. Reine Doku, kein Test.
17081709

17091710
### F213 – java-llama.cpp: Ollama `/api/chat` mit einem `messages`-Element, das kein Objekt ist, schließt die Verbindung ohne Antwort
17101711
- **Zeit:** 2026-10-20 19:55 UTC

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -998,7 +998,7 @@ path), all supporting tools and streaming:
998998

999999
| Surface | Routes | For |
10001000
|---|---|---|
1001-
| **Ollama-native** | `GET /api/version`, `GET /api/tags`, `POST /api/show`, `POST /api/chat` (NDJSON streaming), `POST /api/generate` (through the chat template with `system`, as Ollama does; the bare prompt with `raw: true`; FIM with `suffix`; without a prompt Ollama's "model loaded" answer) | Copilot's built-in **Ollama** provider; Ollama-hardcoded tools |
1001+
| **Ollama-native** | `GET /api/version`, `GET /api/tags`, `POST /api/show`, `POST /api/chat` (NDJSON streaming), `POST /api/generate` (through the chat template with `system`, as Ollama does; the bare prompt with `raw: true`; FIM with `suffix`; without a prompt and without a suffix Ollama's "model loaded" answer; unlike Ollama, a request with only a suffix stays fill-in-the-middle at the start of the file, and `keep_alive: 0` does not unload and is answered "load") | Copilot's built-in **Ollama** provider; Ollama-hardcoded tools |
10021002
| **Anthropic Messages** | `POST /v1/messages` (SSE event stream) | Claude-shaped clients (Claude Code); Copilot `messages` apiType |
10031003
| **OpenAI Responses** | `POST /v1/responses` (SSE event stream) | Copilot `responses` apiType; Responses-API clients |
10041004

‎TODO.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -520,6 +520,8 @@ they are not rediscovered:
520520
until a client needs the exact count.
521521
- **`OpenAiCompatServer` cannot unload a model**: an Ollama `/api/generate` with `keep_alive: 0`
522522
and no prompt gets the "load" answer (F205), the model stays resident (one model per server).
523+
Ollama itself unloads there and answers `done_reason "unload"`; this server does not claim an
524+
unload it did not do.
523525

524526
### The release: 6.0.0, with an "Upgrading from 5.1" section (decided 2026-10-15; roadmap item 9)
525527

‎llama/src/main/java/net/ladenthin/llama/server/OllamaApiSupport.java‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -542,8 +542,16 @@ static boolean hasSuffix(JsonNode request) {
542542

543543
/**
544544
* Whether the {@code /api/generate} request only asks for the model to be loaded: Ollama answers a
545-
* request without a prompt (and without a {@code suffix}) with {@code done} at once, which is how
546-
* its clients preload a model. The model of this server is always loaded, so there is nothing to do.
545+
* request without a prompt with {@code done} at once ({@code done_reason "load"}), which is how its
546+
* clients preload a model. The model of this server is always loaded, so there is nothing to do.
547+
*
548+
* <p>Two deliberate differences from Ollama, whose check reads the prompt alone. A request with an
549+
* empty prompt and a {@code suffix} is not load-only here but fill-in-the-middle at the start of a
550+
* file, a request editors send; Ollama answers it with the load answer. That includes an empty
551+
* {@code suffix} ({@link #hasSuffix} only asks for a string), which Ollama would not treat as
552+
* fill-in-the-middle either. And {@code keep_alive} is not read: Ollama unloads the model on a
553+
* {@code keep_alive} of {@code 0} without a prompt and answers {@code done_reason "unload"}, while
554+
* this server keeps its one model loaded and gives the load answer.
547555
*
548556
* @param request the parsed Ollama generate request
549557
* @return {@code true} when the request carries no prompt and no suffix
@@ -553,9 +561,11 @@ static boolean isLoadOnly(JsonNode request) {
553561
}
554562

555563
/**
556-
* The answer to a load-only {@code /api/generate} request, as Ollama gives it: one object, empty
557-
* {@code response}, {@code done} and {@code done_reason "load"}, whether streaming was asked for
558-
* or not.
564+
* The answer to a load-only {@code /api/generate} request, in the form of Ollama's load answer: one
565+
* object, empty {@code response}, {@code done} and {@code done_reason "load"}, whether streaming was
566+
* asked for or not. It is also the answer to a {@code keep_alive} of {@code 0}, which Ollama answers
567+
* with {@code done_reason "unload"}: nothing is unloaded here, so the server does not claim it (see
568+
* {@link #isLoadOnly}).
559569
*
560570
* @param model the model id to echo
561571
* @return the answer

0 commit comments

Comments
 (0)