Skip to content

feat: reviewed endpoint reasoning, verifier judge keys and job filtering - #241

Closed
osolmaz wants to merge 7 commits into
mainfrom
feat/reviewed-verifier-judge
Closed

osolmaz wants to merge 7 commits into
mainfrom
feat/reviewed-verifier-judge

Conversation

@osolmaz

@osolmaz osolmaz commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

This PR brings the Harbor-HF changes made during a recent benchmark campaign onto main. The first six commits are already running on the control Space, which was deployed from this branch plus #238, so merging them makes main match production. A later deploy from main without them would remove behavior that live runs depend on.

Nothing here is specific to one provider. The Pi changes let a reviewed OpenAI-compatible endpoint pass its model details (context window, reply limit, reasoning support and thinking-level map) to Harbor's Pi agent, so a run can use the highest reasoning level that endpoint accepts. The first endpoint to use this was a hosted model behind an inference binding.

The commits, in order:

  • 21fba17 fix(presets): admit reviewed ACP endpoint connections
  • 5c0a8c4 fix(pi): preserve reviewed endpoint reasoning metadata
  • e3ffba8 fix(pi): register reviewed endpoint metadata with Harbor preflight
  • 44ca670 fix(pi): preserve xhigh reasoning and reviewed token budget
  • 222867e feat(inference): deliver scoped verifier judge keys to reviewed runs
  • 5a0b33c fix(jobs): filter Harbor jobs before pagination
  • c450f03 fix(runs): scope submission projection refresh (not deployed yet)

The judge-key change delivers a scoped key to the verifier for LLM-judged tasks. The jobs fix filters Harbor jobs before pagination, so a run's jobs are not lost among unrelated ones.

These commits were written by an agent session during the campaign and deployed without a PR, so they have not had a review. They need one before merge. The branch merges cleanly with main.

🤖 Generated with Claude Code

osolmaz and others added 7 commits September 26, 2026 00:22
Co-Authored-By: GPT-6 Sol <noreply@pi.dev>
Generated-By: pi 0.87.1 (https://pi.dev)
Co-Authored-By: GPT-6 Sol <noreply@pi.dev>
Generated-By: pi 0.87.1 (https://pi.dev)
Co-Authored-By: GPT-6 Sol <noreply@pi.dev>
Generated-By: pi 0.87.1 (https://pi.dev)
Co-Authored-By: GPT-6 Sol <noreply@pi.dev>
Generated-By: pi 0.87.1 (https://pi.dev)
Co-Authored-By: GPT-6 Sol <noreply@pi.dev>
Generated-By: pi 0.87.1 (https://pi.dev)
Co-Authored-By: GPT-6 Sol <noreply@pi.dev>
Generated-By: pi 0.87.1 (https://pi.dev)
@osolmaz osolmaz changed the title feat: reviewed NIM endpoints, Luna judge keys and job filtering from the Nemotron campaign feat: reviewed endpoint reasoning, verifier judge keys and job filtering Sep 27, 2026
@osolmaz

osolmaz commented Sep 27, 2026

Copy link
Copy Markdown
Member Author

Replaced by #242. The Pi custom-endpoint commits (5c0a8c4, e3ffba8, 44ca670) duplicated behavior that upstream Harbor already has in 9faf4883 (#3352), so they are dropped. The remaining commits moved to #242 unchanged.

@osolmaz osolmaz closed this Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant