Is there any way to make ovms keep special tokens in openai response?

[notsyncing]

Hello, I'm serving the opendatalab/MinerU2.5-Pro-2604-1.2B model with OVMS 2026.1, and access it from the OpenAI endpoints. The mineru expects some special tokens in the output (<|box_start|>, <|box_end|>, etc), but OVMS stripped them in the output:

[2026-04-28 12:31:57.107][235][llm_calculator][trace][openai_completions.cpp:1013] Generated tokens: [151648, 16, 19, 24, 220, 15, 19, 19, 220, 17, 24, 22, 220, 15, 20, 23, 151649, 151657, 2708, 151658, 151668, 198, 151648, 22, 18, 20, 220, 15, 18, 24, 220, 22, 22, 16, 220, 15, 21, 15, 151649, 151657, 2708, 151658, 151668, 198, 151648, 22, 22, 19, 220, 15, 19, 15, 220, 23, 18, 17, 220, 15, 20, 16, 151649, 151657, 2708, 151658, 151668, 198, 151648, 22, 22, 21, 220, 15, 20, 17, 220, 23, 17, 24, 220, 15, 20, 24, 151649, 151657, 2708, 151658, 151668, 198, 151648, 16, 19, 15, 220, 15, 23, 18, 220, 23, 21, 15, 220, 16, 22, 24, 151649, 151657, 2005, 151658, 151668, 198, 151648, 16, 24, 16, 220, 16, 24, 23, 220, 19, 22, 16, 220, 17, 16, 19, 151649, 151657, 2102, 151658, 151668, 198, 151648, 16, 23, 19, 220, 17, 16, 24, 220, 20, 15, 24, 220, 17, 18, 19, 151649, 151657, 2005, 61705, 151658, 151668, 198, 151648, 22, 16, 24, 220, 17, 17, 15, 220, 23, 20, 15, 220, 17, 18, 19, 151649, 151657, 2005, 61705, 151658, 151668, 198, 151648, 16, 19, 15, 220, 17, 18, 22, 220, 23, 21, 15, 220, 21, 16, 20, 151649, 151657, 2005, 151658, 151668, 198, 151648, 16, 19, 15, 220, 21, 18, 15, 220, 23, 21, 15, 220, 23, 22, 16, 151649, 151657, 2005, 151658, 151668, 198, 151648, 19, 24, 23, 220, 24, 16, 22, 220, 20, 15, 24, 220, 24, 17, 21, 151649, 151657, 2893, 5500, 151658, 151668, 151645]
[2026-04-28 12:31:57.107][235][llm_calculator][debug][servable.cpp:240] Complete unary response: {"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"149 044 297 058header\n735 039 771 060header\n774 040 832 051header\n776 052 829 059header\n140 083 860 179table\n191 198 471 214title\n184 219 509 234table_caption\n719 220 850 234table_caption\n140 237 860 615table\n140 630 860 871table\n498 917 509 926page_number","role":"assistant","tool_calls":[]}}],"created":1777379439,"model":"MinerU2.5-Pro-2604-1.2B","object":"chat.completion","usage":{"prompt_tokens":30,"completion_tokens":245,"total_tokens":275}}

The model correctly generated these special tokens (151648 => <|box_start|>, 151649 => <|box_end|>, 151668 => <|rotate_up|>, 151657 => <|ref_start|>, etc), but in the response, they are all missing, so mineru cannot parse the response.

I tried to modify the skip_special_tokens option to False in the openvino_tokenizer.xml and openvino_detokenizer.xml of the model, but it still does not output these tokens.

Is there any way to make OVMS keep special tokens in openai response? Thanks!

[mzegla]

Hi, no, this is not a configurable setting now.
Is it a standard flow for MinerU to operate on special tokens?

[notsyncing]

Is it a standard flow for MinerU to operate on special tokens?

Yes, in its code it requires these special tokens:

in https://github.com/opendatalab/mineru-vl-utils/blob/main/mineru_vl_utils/mineru_client.py , line 26-31:

_layout_re = (
    r"<\|box_start\|>(\d+)\s+(\d+)\s+(\d+)\s+(\d+)"
    r"<\|box_end\|><\|ref_start\|>(\w+?)<\|ref_end\|>"
    r"(?:(<\|rotate_(?:up|right|down|left)\|>))?"
    r"(.*?)(?=<\|box_start\|>|$)"
)

[mzegla]

I believe, we could expose option to control whether special tokens should be included in the output.