Skip to content

Commit d8297df

Browse files
committed
Pin the output indices and sequence numbers of the Responses stream
Responses clients key the items of a stream by output_index (the OpenAI SDK raises on a duplicate) and order the events by sequence_number, yet no test read either field, none saw response.reasoning_text.done, and failed() was tested only through the server and only by substring. One test parses every data line of a stream with reasoning, text and a tool call: exactly 15 events numbered 0..14, the reasoning, message and function_call items at output_index 0, 1 and 2 in both their added and done events, the reasoning_text.done event with its full field set, and the item order of response.completed. A second one checks that response.failed continues the numbering. Red under a mutation of each property (e.g. "Expected: is <4> but: was <0>" with the counter reset in failed()), green on the code. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBFFnrhtvXiUeyxjmUc4R2
1 parent 6b75aa0 commit d8297df

2 files changed

Lines changed: 101 additions & 0 deletions

File tree

‎BUGHUNT-FOUND.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -757,6 +757,7 @@ Repos plus eins. Geprüfte Klassen ohne Fund stehen in [`BUGHUNT-CHECKED.md`](BU
757757
- **Rot-Grün-Nachweis:** ohne Fix rot („Expected: a string containing "event: response.output_item.added" but: was ""“), mit Fix grün (5/5 der Klasse, skipped 0); Gegenprobe per `git stash` rot, nach `pop` grün. `ResponsesApiSupportTest` 14/14, `OpenAiCompatServerHttpTest` 45/45, `LlamaArchitectureTest` 13/13, `spotbugs:check` grün (erst nach dem Helfer).
758758
- **Status:** gefixt
759759
- **Commit:** java-llama.cpp `f327156`
760+
- **Nachtrag (Branch-Review 2026-10-20, Fund J5-6):** `output_index`, die Fortlaufendheit der `sequence_number` und das Ereignis `response.reasoning_text.done` hatten keinen Test; die Clients ordnen die Items nach `output_index` (das OpenAI-SDK wirft bei einem doppelten). `ResponsesStreamTranslatorTest#reasoningTextAndToolCallGetConsecutiveOutputIndicesAndSequenceNumbers` (begin, Denken, Text, Werkzeugaufruf, end) zerlegt die `data:`-Zeilen und verlangt genau 15 Ereignisse mit `sequence_number` 0..14, `output_item.added`/`.done` für `reasoning`/`message`/`function_call` an den Indizes 0/1/2, `response.reasoning_text.done` mit `item_id` `rs_resp_1`, `content_index` 0, vollem Text und `sequence_number` 6 sowie die Reihenfolge der Items in `response.completed`. Gegenprobe rot: Nachricht ohne eigenen Index („no response.output_item.added at output_index 2“), `sequence = 0` am Anfang von `end()` („Expected: is <[0, 1, …, 14]> but: was <[0, 1, 2, 3, 4, 5, 0, 1, …, 8]>“), ohne `reasoning_text.done` („Expected: is <15> but: was <14>“); mit dem Code grün (9/9). upstream (b11639) gibt weder `output_index` noch `sequence_number` aus; der Maßstab ist die OpenAI-Form.
760761

761762
<a id="f84"></a>
762763
### F84 – java-llama.cpp: Anthropic-Verlauf verwirft `thinking`-Blöcke früherer Runden
@@ -1137,6 +1138,7 @@ Repos plus eins. Geprüfte Klassen ohne Fund stehen in [`BUGHUNT-CHECKED.md`](BU
11371138
- **Rot-Grün-Nachweis:** ohne Fix beide rot („Expected: a string containing "event: response.failed" but: was "event: response.created …"“), mit Fix grün (51/51 der Klasse, skipped 0); Gegenprobe per `git stash` beider Produktionsdateien rot (2), nach `pop` grün. Breite Liste (Server-Tests inkl. `ResponsesStreamTranslatorTest`, `LlamaArchitectureTest`) grün, `spotless:apply` und `spotbugs:check` grün.
11381139
- **Status:** gefixt
11391140
- **Commit:** java-llama.cpp `bd3db6a`
1141+
- **Nachtrag (Branch-Review 2026-10-20, Fund J5-6):** Die laufende `sequence_number` des `response.failed` war nur über den Server und nur per Teilstring geprüft. `ResponsesStreamTranslatorTest#failedContinuesTheSequenceNumber` (begin, ein Textchunk, `failed("boom", "server_error")`) verlangt fünf Ereignisse, das letzte `response.failed` mit `sequence_number` 4, `status` `failed` und `error.code`/`error.message`. Gegenprobe mit `sequence = 0` in `failed()` rot („Expected: is <4> but: was <0>“), mit dem Code grün (`ResponsesStreamTranslatorTest` 9/9).
11401142

11411143
<a id="f118"></a>
11421144
### F118 – java-llama.cpp: Anthropic-Serverfehler heißen `invalid_request_error`

‎llama/src/test/java/net/ladenthin/llama/server/ResponsesStreamTranslatorTest.java‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,16 @@
55
package net.ladenthin.llama.server;
66

77
import static org.hamcrest.MatcherAssert.assertThat;
8+
import static org.hamcrest.Matchers.contains;
89
import static org.hamcrest.Matchers.containsString;
910
import static org.hamcrest.Matchers.is;
1011
import static org.hamcrest.Matchers.not;
1112

13+
import com.fasterxml.jackson.databind.JsonNode;
14+
import com.fasterxml.jackson.databind.ObjectMapper;
15+
import java.io.IOException;
16+
import java.util.ArrayList;
17+
import java.util.List;
1218
import org.junit.jupiter.api.Test;
1319

1420
/**
@@ -18,6 +24,30 @@
1824
*/
1925
public class ResponsesStreamTranslatorTest {
2026

27+
private static final ObjectMapper MAPPER = new ObjectMapper();
28+
29+
/** The {@code data:} payloads of the framed SSE events, in order. */
30+
private static List<JsonNode> events(String sse) throws IOException {
31+
List<JsonNode> events = new ArrayList<>();
32+
for (String line : sse.split("\n")) {
33+
if (line.startsWith("data: ")) {
34+
events.add(MAPPER.readTree(line.substring("data: ".length())));
35+
}
36+
}
37+
return events;
38+
}
39+
40+
/** The first event of {@code type} whose {@code output_index} is {@code outputIndex}. */
41+
private static JsonNode eventAt(List<JsonNode> events, String type, int outputIndex) {
42+
for (JsonNode event : events) {
43+
if (event.path("type").asText().equals(type)
44+
&& event.path("output_index").asInt(-1) == outputIndex) {
45+
return event;
46+
}
47+
}
48+
throw new AssertionError("no " + type + " at output_index " + outputIndex);
49+
}
50+
2151
@Test
2252
public void beginEmitsResponseCreated() {
2353
ResponsesStreamTranslator translator = new ResponsesStreamTranslator("m", "resp_1");
@@ -77,6 +107,75 @@ public void reasoningDeltasStreamAsAReasoningItemLikeUpstream() {
77107
assertThat(completed, containsString("\"type\":\"reasoning_text\",\"text\":\"6 times 7\""));
78108
}
79109

110+
@Test
111+
public void reasoningTextAndToolCallGetConsecutiveOutputIndicesAndSequenceNumbers() throws IOException {
112+
// Responses clients key the items by output_index (the OpenAI SDK raises on a duplicate) and
113+
// order the events by sequence_number; reasoning, message and function_call are items 0, 1, 2.
114+
ResponsesStreamTranslator translator = new ResponsesStreamTranslator("m", "resp_1");
115+
List<JsonNode> events = events(translator.begin()
116+
+ translator.onChunk("{\"choices\":[{\"delta\":{\"reasoning_content\":\"6 times 7\"}}]}")
117+
+ translator.onChunk("{\"choices\":[{\"delta\":{\"content\":\"42\"}}]}")
118+
+ translator.onChunk("{\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"c1\","
119+
+ "\"function\":{\"name\":\"f\",\"arguments\":\"{}\"}}]},\"finish_reason\":\"tool_calls\"}]}")
120+
+ translator.end());
121+
122+
assertThat(events.size(), is(15));
123+
List<Integer> sequence = new ArrayList<>();
124+
List<Integer> expected = new ArrayList<>();
125+
for (int i = 0; i < events.size(); i++) {
126+
sequence.add(events.get(i).path("sequence_number").asInt(-1));
127+
expected.add(i);
128+
}
129+
assertThat(sequence, is(expected));
130+
131+
String[] itemTypes = {"reasoning", "message", "function_call"};
132+
for (int index = 0; index < itemTypes.length; index++) {
133+
assertThat(
134+
eventAt(events, "response.output_item.added", index)
135+
.path("item")
136+
.path("type")
137+
.asText(),
138+
is(itemTypes[index]));
139+
assertThat(
140+
eventAt(events, "response.output_item.done", index)
141+
.path("item")
142+
.path("type")
143+
.asText(),
144+
is(itemTypes[index]));
145+
}
146+
147+
JsonNode reasoningDone = eventAt(events, "response.reasoning_text.done", 0);
148+
assertThat(reasoningDone.path("item_id").asText(), is("rs_resp_1"));
149+
assertThat(reasoningDone.path("content_index").asInt(-1), is(0));
150+
assertThat(reasoningDone.path("text").asText(), is("6 times 7"));
151+
assertThat(reasoningDone.path("sequence_number").asInt(-1), is(6));
152+
153+
JsonNode completed = events.get(events.size() - 1);
154+
assertThat(completed.path("type").asText(), is("response.completed"));
155+
List<String> outputTypes = new ArrayList<>();
156+
for (JsonNode item : completed.path("response").path("output")) {
157+
outputTypes.add(item.path("type").asText());
158+
}
159+
assertThat(outputTypes, contains("reasoning", "message", "function_call"));
160+
}
161+
162+
@Test
163+
public void failedContinuesTheSequenceNumber() throws IOException {
164+
// response.failed ends a stream that already emitted events; its sequence_number follows them.
165+
ResponsesStreamTranslator translator = new ResponsesStreamTranslator("m", "resp_1");
166+
List<JsonNode> events = events(translator.begin()
167+
+ translator.onChunk("{\"choices\":[{\"delta\":{\"content\":\"he\"}}]}")
168+
+ translator.failed("boom", "server_error"));
169+
170+
assertThat(events.size(), is(5));
171+
JsonNode failed = events.get(4);
172+
assertThat(failed.path("type").asText(), is("response.failed"));
173+
assertThat(failed.path("sequence_number").asInt(-1), is(4));
174+
assertThat(failed.path("response").path("status").asText(), is("failed"));
175+
assertThat(failed.path("response").path("error").path("code").asText(), is("server_error"));
176+
assertThat(failed.path("response").path("error").path("message").asText(), is("boom"));
177+
}
178+
80179
@Test
81180
public void toolCallsBecomeFunctionCallItemsBeforeCompleted() {
82181
ResponsesStreamTranslator translator = new ResponsesStreamTranslator("m", "resp_1");

0 commit comments

Comments
 (0)