Skip to content

Commit 707da0e

Browse files
committed
Document the one-prompt, one-completion rule on every entry point
F184 and F197 refuse several prompts in one request and "n" > 1 with InvalidRequestException, unlike upstream's server, which answers both. Only the C header of jllama_completions/jllama_completions_oai said so, and streamCompletionsOai named the prompts but not "n". The @throws of handleCompletions, handleCompletionsOai and streamCompletionsOai now name both, handleChatCompletions and handleInfill only "n" (a chat request renders one prompt from its messages, infill builds its own); the eight request methods of FfmLlamaModel carry the same split, and the chat, infill and streaming functions of jllama_api.h say it too. Phrased as an unconditional refusal, not tied to the slot count. Documentation only; javadoc builds clean for the core and llama-ffm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBFFnrhtvXiUeyxjmUc4R2
1 parent a520c36 commit 707da0e

4 files changed

Lines changed: 42 additions & 10 deletions

File tree

‎BUGHUNT-FOUND.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1612,6 +1612,7 @@ Repos plus eins. Geprüfte Klassen ohne Fund stehen in [`BUGHUNT-CHECKED.md`](BU
16121612
- **Rot-Grün-Nachweis:** Der C-API-Test gegen den alten Code rot (alle vier Wege `OK` statt `INVALID_REQUEST`), mit dem Fix grün; `ctest` grün. Die beiden Java-Tests lokal mit dem Zufallsmodell grün (die zwei Infill-Tests der Klasse scheitern dort erwartungsgemäß, das Zufallsmodell hat keine FIM-Tokens); in CI mit dem echten Modell.
16131613
- **Status:** gefixt
16141614
- **Commit:** java-llama.cpp `7a51bbc`
1615+
- **Nachtrag (Branch-Review 2026-10-20, Fund XJ-5):** Die Ablehnung mehrerer Prompts (F184) und von `"n"` > 1 stand im C-Header nur bei `jllama_completions`/`jllama_completions_oai` und in der Javadoc von `streamCompletionsOai` nur zur Hälfte. Jetzt nennen sie die `@throws InvalidRequestException` von `LlamaModel#handleCompletions`, `#handleCompletionsOai` und `#streamCompletionsOai` (beides), `#handleChatCompletions` und `#handleInfill` (nur `"n"`: eine Chat-Anfrage rendert genau einen Prompt aus `messages`, Infill baut ihn selbst), alle acht Anfrage-Methoden von `FfmLlamaModel` mit derselben Aufteilung, und im C-Header die Chat-, Infill- und Stream-Funktionen. Formuliert als unbedingte Ablehnung (ein Prompt und eine Completion je Anfrage), nicht über die Slot-Zahl. Nur Doku.
16151616

16161617
<a id="f198"></a>
16171618
### F198 – java-llama.cpp: `FfmTextToSpeech.close()` gibt die Engine frei, während `synthesize()` auf einem anderen Thread in ihr läuft (FFM-Hälfte von F16)

‎llama-ffm/src/main/java/net/ladenthin/llama/ffm/FfmLlamaModel.java‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,8 @@ public String meta() {
146146
*
147147
* @param requestJson the OpenAI chat body ({@code {"messages": [...], ...}})
148148
* @return the {@code chat.completion} object
149+
* @throws InvalidRequestException if the server refuses the body, including {@code "n"} greater than 1
150+
* (one completion per request; send separate requests)
149151
*/
150152
public String chatCompletions(String requestJson) {
151153
return stringOut(JllamaNative.CHAT_COMPLETIONS, requireNonNull(requestJson, "requestJson"));
@@ -157,6 +159,8 @@ public String chatCompletions(String requestJson) {
157159
*
158160
* @param requestJson the OpenAI chat body
159161
* @return the stream; close it to release the task early
162+
* @throws InvalidRequestException if the server refuses the body, including {@code "n"} greater than 1
163+
* (one completion per request; send separate requests)
160164
*/
161165
public FfmTaskStream chatCompletionsStream(String requestJson) {
162166
int task = taskOut(JllamaNative.CHAT_COMPLETIONS_STREAM_REQUEST, requireNonNull(requestJson, "requestJson"));
@@ -169,6 +173,8 @@ public FfmTaskStream chatCompletionsStream(String requestJson) {
169173
*
170174
* @param requestJson the OpenAI chat body
171175
* @return the stream of raw result objects
176+
* @throws InvalidRequestException if the server refuses the body, including {@code "n"} greater than 1
177+
* (one completion per request; send separate requests)
172178
*/
173179
public FfmTaskStream chatCompletionsRaw(String requestJson) {
174180
int task = taskOut(JllamaNative.CHAT_COMPLETIONS_REQUEST, requireNonNull(requestJson, "requestJson"));
@@ -180,6 +186,9 @@ public FfmTaskStream chatCompletionsRaw(String requestJson) {
180186
*
181187
* @param requestJson the request object ({@code {"prompt": ..., ...}})
182188
* @return the result object
189+
* @throws InvalidRequestException if the server refuses the body, including several prompts in one
190+
* request or {@code "n"} greater than 1 (one prompt and one completion per request; send separate
191+
* requests)
183192
*/
184193
public String completions(String requestJson) {
185194
return stringOut(JllamaNative.COMPLETIONS, requireNonNull(requestJson, "requestJson"));
@@ -190,6 +199,9 @@ public String completions(String requestJson) {
190199
*
191200
* @param requestJson the request object
192201
* @return the {@code text_completion} object
202+
* @throws InvalidRequestException if the server refuses the body, including several prompts in one
203+
* request or {@code "n"} greater than 1 (one prompt and one completion per request; send separate
204+
* requests)
193205
*/
194206
public String completionsOai(String requestJson) {
195207
return stringOut(JllamaNative.COMPLETIONS_OAI, requireNonNull(requestJson, "requestJson"));
@@ -202,6 +214,9 @@ public String completionsOai(String requestJson) {
202214
*
203215
* @param requestJson the request object, parsed as {@link #completionsOai(String)} parses it
204216
* @return the stream; close it to release the task early
217+
* @throws InvalidRequestException if the server refuses the body, including several prompts in one
218+
* request or {@code "n"} greater than 1 (one prompt and one completion per request; send separate
219+
* requests)
205220
*/
206221
public FfmTaskStream completionsOaiStream(String requestJson) {
207222
int task = taskOut(JllamaNative.COMPLETIONS_OAI_STREAM_REQUEST, requireNonNull(requestJson, "requestJson"));
@@ -213,6 +228,8 @@ public FfmTaskStream completionsOaiStream(String requestJson) {
213228
*
214229
* @param requestJson the request object ({@code input_prefix}, {@code input_suffix}, ...)
215230
* @return the result object
231+
* @throws InvalidRequestException if the server refuses the body, including {@code "n"} greater than 1
232+
* (one completion per request)
216233
*/
217234
public String infill(String requestJson) {
218235
return stringOut(JllamaNative.INFILL, requireNonNull(requestJson, "requestJson"));
@@ -223,6 +240,9 @@ public String infill(String requestJson) {
223240
*
224241
* @param requestJson the request object
225242
* @return the stream
243+
* @throws InvalidRequestException if the server refuses the body, including several prompts in one
244+
* request or {@code "n"} greater than 1 (one prompt and one completion per request; send separate
245+
* requests)
226246
*/
227247
public FfmTaskStream completionsStream(String requestJson) {
228248
int task = taskOut(JllamaNative.COMPLETIONS_STREAM_REQUEST, requireNonNull(requestJson, "requestJson"));

‎llama/src/main/cpp/jllama_api.h‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -123,15 +123,17 @@ JLLAMA_API void jllama_model_free(jllama_model *model);
123123
/**
124124
* An OpenAI chat completion, blocking: `request_json` is the request body
125125
* (`{"messages":[...], "max_tokens": ..., ...}`), *response_json the chat.completion object
126-
* (jllama_string_free). JLLAMA_STATUS_INVALID_REQUEST for a body the server refuses.
126+
* (jllama_string_free). JLLAMA_STATUS_INVALID_REQUEST for a body the server refuses, including
127+
* "n" > 1: a request takes one completion (send separate requests).
127128
*/
128129
JLLAMA_API jllama_status jllama_chat_completions(jllama_model *model, const char *request_json, char **response_json);
129130

130131
/**
131132
* Starts a streaming OpenAI chat completion (`"stream": true` in the body makes the server emit
132133
* chat.completion.chunk deltas) and returns its task id in *task_id. Receive with
133134
* jllama_chat_completions_stream_receive until it reports stop; a task not received to its end is
134-
* released with jllama_task_release.
135+
* released with jllama_task_release. "n" > 1 is JLLAMA_STATUS_INVALID_REQUEST, as for
136+
* jllama_chat_completions.
135137
*/
136138
JLLAMA_API jllama_status jllama_chat_completions_stream_request(jllama_model *model, const char *request_json,
137139
int *task_id);
@@ -192,27 +194,29 @@ JLLAMA_API jllama_status jllama_completions_oai(jllama_model *model, const char
192194
* Starts a streaming OpenAI /v1/completions request (the body as for jllama_completions_oai) and
193195
* returns its task id in *task_id. Receive with jllama_chat_completions_stream_receive, whose
194196
* envelopes then carry text_completion chunks, the last with usage; release an unfinished task with
195-
* jllama_task_release.
197+
* jllama_task_release. One prompt and one choice, as for jllama_completions.
196198
*/
197199
JLLAMA_API jllama_status jllama_completions_oai_stream_request(jllama_model *model, const char *request_json,
198200
int *task_id);
199201

200202
/**
201203
* A fill-in-the-middle request (`input_prefix`, `input_suffix`, optional `input_extra` and
202-
* `prompt`), blocking. JLLAMA_STATUS_ERROR when the model has no FIM tokens.
204+
* `prompt`), blocking. JLLAMA_STATUS_ERROR when the model has no FIM tokens; "n" > 1 is
205+
* JLLAMA_STATUS_INVALID_REQUEST, as for jllama_completions.
203206
*/
204207
JLLAMA_API jllama_status jllama_infill(jllama_model *model, const char *request_json, char **response_json);
205208

206209
/**
207210
* Starts a raw streaming completion (`"stream": true`; an infill one when the body carries
208211
* `input_prefix`/`input_suffix`) and returns its task id; receive with
209-
* jllama_completions_stream_receive.
212+
* jllama_completions_stream_receive. One prompt and one completion, as for jllama_completions.
210213
*/
211214
JLLAMA_API jllama_status jllama_completions_stream_request(jllama_model *model, const char *request_json, int *task_id);
212215

213216
/**
214217
* Starts a streaming chat completion whose results are llama.cpp's raw result objects (the chat
215218
* template applied, no OpenAI chunk formatting); receive with jllama_completions_stream_receive.
219+
* "n" > 1 is JLLAMA_STATUS_INVALID_REQUEST, as for jllama_chat_completions.
216220
*/
217221
JLLAMA_API jllama_status jllama_chat_completions_request(jllama_model *model, const char *request_json, int *task_id);
218222

‎llama/src/main/java/net/ladenthin/llama/LlamaModel.java‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -800,7 +800,8 @@ public void streamChatCompletion(InferenceParameters parameters, Consumer<String
800800
* @param requestJson the OpenAI completion body ({@code {"prompt": ..., "max_tokens": ..., ...}})
801801
* @param chunkSink receiver for each {@code text_completion} chunk JSON string, in order
802802
* @throws net.ladenthin.llama.exception.InvalidRequestException if the request is rejected (no
803-
* {@code prompt}, a parameter upstream refuses, several prompts, a prompt over the context size)
803+
* {@code prompt}, a parameter upstream refuses, several prompts, {@code "n"} greater than 1, a prompt
804+
* over the context size; see {@link #handleCompletionsOai(String)})
804805
* @throws net.ladenthin.llama.exception.LlamaException if inference fails
805806
*/
806807
public void streamCompletionsOai(String requestJson, Consumer<String> chunkSink) {
@@ -826,7 +827,9 @@ public void streamCompletionsOai(String requestJson, Consumer<String> chunkSink)
826827
* @param paramsJson JSON string with at least a "prompt" field
827828
* @return JSON response from the server
828829
* @throws net.ladenthin.llama.exception.InvalidRequestException if the request is rejected: not valid JSON, no
829-
* {@code "prompt"}, a parameter outside its limits, a prompt over the context size
830+
* {@code "prompt"}, a parameter outside its limits, a prompt over the context size, several prompts in
831+
* one request (an array of strings or of token arrays) or {@code "n"} greater than 1 -- one prompt and
832+
* one completion per request, unlike upstream's server; send separate requests
830833
* @throws net.ladenthin.llama.exception.LlamaException if inference fails
831834
*/
832835
public native String handleCompletions(String paramsJson);
@@ -838,7 +841,9 @@ public void streamCompletionsOai(String requestJson, Consumer<String> chunkSink)
838841
* @param paramsJson JSON string with OAI-compatible completion parameters
839842
* @return JSON response in OAI format
840843
* @throws net.ladenthin.llama.exception.InvalidRequestException if the request is rejected: not valid JSON, no
841-
* {@code "prompt"}, a parameter outside its limits, a prompt over the context size
844+
* {@code "prompt"}, a parameter outside its limits, a prompt over the context size, several prompts in
845+
* one request (an array of strings or of token arrays) or {@code "n"} greater than 1 -- one prompt and
846+
* one completion per request, unlike upstream's server; send separate requests
842847
* @throws net.ladenthin.llama.exception.LlamaException if inference fails
843848
*/
844849
public native String handleCompletionsOai(String paramsJson);
@@ -850,7 +855,8 @@ public void streamCompletionsOai(String requestJson, Consumer<String> chunkSink)
850855
* @param paramsJson JSON string with infill parameters
851856
* @return JSON response from the server
852857
* @throws net.ladenthin.llama.exception.InvalidRequestException if the request is rejected: not valid JSON, a missing
853-
* {@code "input_prefix"} or {@code "input_suffix"}, a parameter outside its limits
858+
* {@code "input_prefix"} or {@code "input_suffix"}, a parameter outside its limits, {@code "n"} greater
859+
* than 1 (one completion per request)
854860
* @throws net.ladenthin.llama.exception.LlamaException if the model has no fill-in-the-middle tokens or inference fails
855861
*/
856862
public native String handleInfill(String paramsJson);
@@ -1180,7 +1186,8 @@ public void setLoraAdapter(int adapterId, float scale) {
11801186
* @return JSON response in OAI chat-completion format
11811187
* @throws net.ladenthin.llama.exception.InvalidRequestException if the request is rejected: not valid JSON, no
11821188
* {@code "messages"}, a message the chat template cannot take, a parameter outside its limits, a
1183-
* prompt over the context size
1189+
* prompt over the context size, {@code "n"} greater than 1 (one completion per request, unlike
1190+
* upstream's server; send separate requests)
11841191
* @throws net.ladenthin.llama.exception.LlamaException if the model was loaded in embedding mode or inference fails
11851192
*/
11861193
public native String handleChatCompletions(String params);

0 commit comments

Comments
 (0)