merge: Batch B — SSH mux (#77), LLM key reconcile (#76), tok/s chart (#75) - #1
Open
vincenzopalazzo wants to merge 40 commits into
Open
merge: Batch B — SSH mux (#77), LLM key reconcile (#76), tok/s chart (#75)#1vincenzopalazzo wants to merge 40 commits into
vincenzopalazzo wants to merge 40 commits into
Conversation
- Update Hermes header button is now a permanent neutral control: warning color +commit-count badge only when an update is available; no more toast notifications - Overview "Update Hermes" (was Update All): neutral by default, pending badge, live x/y settled progress bar driven by WS per-Spark update status - Remove toast system (useToasts/useHermesAlerts/Toaster were never tracked) and CpuPanel+CpuIcon as part of resources layout rework (GPU spans full left column, Storage+Network stack right) - Server: HermesProbe, HermesReleases + unit tests; /api/sparks/:id/hermes/updates|check|update and update-all endpoints; SparkMonitor hermes domain; config adds POLL_INTERVAL_HERMES (600s) and HERMES_UPDATE_TIMEOUT_MS - Frontend: HermesUpdateDialog + useHermesUpdateDialog hook, header/overview wiring, EditSparkDialog hermes fields, types/client/api additions - gitignore HANDOFF-HERMES.md; includes stray root bench.png (unreferenced)
Checkpoint active decode jobs and finalize them on SIGTERM/--watch reload so mid-run polls keep a job instead of 404. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
- Non-Spark units (kind host): add dedicated GPU hosts via the + button, monitored with SSH + nvidia-smi, hardware auto-detected (GPU model, driver, CPU, RAM) instead of fixed GB10 specs, CX7 hidden for hosts - RAM vs VRAM separation for hosts: VRAM straight from nvidia-smi, system RAM from /proc/meminfo; RAM panel + Overview RAM bar for hosts; host Resources layout = GPU left, RAM -> Network -> Storage right - Prefill tok/s, decode-bench Prefill column, live prefill (vLLM engine step tokens, ds4 computed diffs), bench checkpoint recovery - Update Hermes: permanent neutral button + live batch progress bar, toasts replaced by header control (from unpushed work) - Version bump to 1.8.0 (package.json, lockfile, README, CHANGELOG)
…helper text - README: new 'Hermes Agent monitoring' section (enable steps, status badges, one-click update dialog, batch update on Overview, side requirements, config field, related API, env), TOC entry, Features row, and env-variable docs - EditSparkDialog: helper text no longer says 'toasts' (toasts were replaced by the header Update Hermes button)
Key auth failed in Docker because OpenSSH looks in /root/.ssh, not the host user's keys. Document the bind-mount (including non-default names), honor SSH_IDENTITY_FILE, and hint in the add/edit dialogs. Co-authored-by: Cursor <cursoragent@cursor.com>
Persist busy-sample daily rollups on the LLM card and split cached/uncached prefill for ds4, llama.cpp, and SGLang. Release 1.8.1. Co-authored-by: Cursor <cursoragent@cursor.com>
Bare-metal npm start no longer exposes unauthenticated SSH/power on the LAN; Docker still binds 0.0.0.0. Decode bench now rejects ports that are not configured, matching showcase. Thanks MiaAI-Lab#35, MiaAI-Lab#45, MiaAI-Lab#39. Co-authored-by: Cursor <cursoragent@cursor.com>
The loopback default would have bound the dev API to 127.0.0.1 because the dev compose file never set BIND_HOST. Co-authored-by: Cursor <cursoragent@cursor.com>
Remote hosts now read hwmon/thermal over SSH; Sparks keep the old remote path and still have no CPU temp UI. Co-authored-by: Cursor <cursoragent@cursor.com>
Read-only tailscale status --json per unit so LAN-healthy boxes that have fallen off the tailnet show up. Probe and card stay dark until Edit Spark enables them. Co-authored-by: Cursor <cursoragent@cursor.com>
SGLang without --enable-metrics has no lifetime token counter and no engine_sleep_state gauge, so the Engine tile stayed blank and a steady last_gen_throughput expired to 0 tok/s while the server was busy. Probe /v1/loads (fallback /get_load) for running/waiting counts. Treat a reachable SGLang process as Active — sleep_on_idle is a launch flag, not live sleep. Keep last_gen_throughput visible while requests are in flight. Total Generated still requires real counters (older server_info totals or Prometheus). Do not invent a lifetime total from a rate gauge.
EXL3 OpenAI serving was detected as vLLM with no /metrics, so tok/s stayed 0. Probe /health counters instead, and pull Node from public.ecr.aws so Spark image builds survive IPv6 Docker Hub failures. Co-authored-by: Cursor <cursoragent@cursor.com>
…lectable before Run.
…-accept helper pattern. The LRU-with-comments prompt was prose-speed; clamp_00…clamp_49 is the code analog of count 1→200. Opening the sheet no longer restores the last type onto the picker.
Remote Sparks were hardcoded to 0°C; they now use the same SSH hwmon allowlist as hosts, and Overview plus the GPU panel actually render the reading. Co-authored-by: Cursor <cursoragent@cursor.com>
…he table Copy results rounded aggregate/stream tok/s to integers, so clipboard text disagreed with the visible table. Also use mean TTFT to match the table. Co-authored-by: Cursor <cursoragent@cursor.com>
Deprecated /get_server_info and /get_model_info log a warning on every hit, so the 2s poll spammed SGLang hosts. Keep the old paths as fallback for older servers. Co-authored-by: Cursor <cursoragent@cursor.com>
…like the table Co-authored-by: Cursor <cursoragent@cursor.com>
…del_info first Co-authored-by: Cursor <cursoragent@cursor.com>
Sequential unique-prefix prefills measure prompt_tokens ÷ TTFT per size; timeouts scale up to 45 minutes so 256k runs are not cut off at 12 minutes. Co-authored-by: Cursor <cursoragent@cursor.com>
The 'DecodeBench uses Showcase structural prompts' test still asserts
pickShowcasePrompts("structural") and withFillToMaxInstruction, both of
which 1.8.3 removed when DecodeBench moved onto the lab structured protocol
(count 1 -> 200), and 1.8.4 replaced with the output-type picker.
The test therefore fails on a clean main. Retargeted at the current API
(pickDecodeBenchPrompts / decodeBenchPromptForType / normalizeDecodeBenchType)
and added top_p: 1 plus a negative assertion on withFillToMaxInstruction, so
the protocol is pinned in the direction it actually moved.
The remaining assertions are unchanged and still pass.
Kernel journal scan is cached off the 2s GPU loop. Count is since boot; the row is hidden at zero. No CPU panel. Co-authored-by: Cursor <cursoragent@cursor.com>
Settings can hide Worker-role Sparks from Overview and the tab bar. Decode and prefill can use an on-demand Remote host or SSH-forward onto loopback; long prefills no longer die at undici's 5-minute idle timeout. Co-authored-by: Cursor <cursoragent@cursor.com>
# Conflicts: # CHANGELOG.md # server/collectors/ssh.js
- Add a ~1h in-memory trend chart for generation/prefill tok/s and TTFT - Show per-phase averages over busy (>0) samples only - Server: report live mean TTFT (seconds) from vLLM histogram sum/count
TTFT history only appends while vLLM is serving, so the series is sparse while tps/prefill are tick-dense. The chart's x-axis normalises by array index, so after any idle gap the TTFT polyline was stretched across the full width and mis-placed in time vs the other series. Take the reviewer's second option: no overlay. The busy-sample TTFT average badge is kept — it reads the sparse series directly (no x-axis) and is the non-redundant signal; the line itself tracked the prefill spikes it sat under.
…tion The trend chart's x-axis was normalised by the current sample count, so every new tick re-stretched the whole polyline leftward — the chart visibly rewrote its own history as it filled. Anchor x to a constant DISPLAY_WINDOW (900 samples = 30 min at the 2 s poll) instead: the line grows left-to-right while filling, then scrolls with the newest sample pinned at the right edge. Only the window is drawn; y-max is computed over the window so an old spike can't permanently squash recent detail. Retention is raised and configurable via VITE_HISTORY_HOURS (default 8, wired through the Dockerfile ARG + compose build arg). The extra hours feed the busy-sample average badges below the chart (and future long-horizon views); the chart itself stays at 30 min. Memory cost is ~112 KB per series per hour of Float64 samples — ~10 MB worst case at 8 h for the whole tab. Header label now states both windows: 'chart ~30m · avgs ~8h · 2s samples'.
…oute + load-time) MiaAI-Lab#25/MiaAI-Lab#26 added per-port LLM API keys and syncLlmApiKeysToPorts, but the sync is only called from the two llm-ports PUT routes. Two paths still rename llmPorts underneath the encrypted secrets store: - PATCH /api/sparks/:id carrying llmPorts - a direct sparks.json edit (config edited in the container / on disk) Either leaves the key stored on the old port while the port the probe now uses has none, so LLM probes hit the backend keyless and 401 until someone re-enters the key by hand. - PATCH /api/sparks/:id: capture prev llmPorts when the body carries llmPorts (validated as in the PUT route) and run syncLlmApiKeysToPorts after the update, then restart the monitor from the re-read spark. - SparkRegistry: reconcile once at load. Only the unambiguous shape — exactly one orphaned keyed port and exactly one configured port without a key — moves the key; every other mismatch warns and changes nothing. Load never prunes stored key material. - tests: load reconcile move / retain / aligned shapes plus the syncLlmApiKeysToPorts rails, with a synthetic key and temp SPARKS_JSON_PATH / SPARKS_SECRETS_PATH. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…-normalize ports Review (vincenzopalazzo): the PATCH sync was gated on the request body normalizing to a non-empty port list, but SparkRegistry.updateSpark() applies llmPorts: [] (-> [LLM_PORT]) and the legacy scalar shape too, so those paths could still orphan a key under the old port. Move the logic into SparkRegistry.patchSpark(): armed on hasOwnProperty(body, 'llmPorts') regardless of shape, and synced against the ports the stored spark actually ends up with. The route now delegates to it. Tests cover [] -> default port, legacy scalar, and a body without llmPorts (keys untouched). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>
Bring in ControlMaster multiplexing + post-probe invalidation on top of MiaAI-Lab#76. Co-authored-by: Vincenzo Palazzo <vincenzopalazzo@member.fsf.org>
Busy-sample averages, live TTFT metric, fixed 30-min chart window, VITE_HISTORY_HOURS. Co-authored-by: Vincenzo Palazzo <vincenzopalazzo@member.fsf.org>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8ca8ac7a1e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+266
to
+268
| <GpuPanel | ||
| gpu={metrics.gpu} | ||
| cpu={metrics.cpu} |
There was a problem hiding this comment.
For every kind !== "host" unit, this branch now renders only GpuPanel, Storage, Network, and optional Tailscale. GpuPanel exposes GPU data and CPU temperature only, so the previously displayed CPU usage/power, RAM usage, available memory, and unified-memory OOM risk disappear from all DGX Spark detail pages even though those metrics are still collected.
Useful? React with 👍 / 👎.
Comment on lines
+150
to
+153
| const ok = | ||
| !timedOut && | ||
| !result.error && | ||
| (result.prefillTokens > 0 || result.ttftMs > 0); |
There was a problem hiding this comment.
When an endpoint returns a successful but empty/non-SSE response, runStreamingRequest has no error and estimates prefillTokens from the request prompt, so this condition marks the level successful even though ttftMs and prefillTps are both zero. Since prefill throughput cannot be measured without observing a first token, require a positive TTFT rather than allowing the prompt-token fallback to make the row look valid.
Useful? React with 👍 / 👎.
Comment on lines
+108
to
+110
| if (spark?.kind === "host") { | ||
| void this.collector | ||
| .detectHardware() |
There was a problem hiding this comment.
If a dedicated GPU host is offline or still booting when its monitor is constructed, this sole detectHardware() call fails and the rejection is discarded. Nothing invokes detection after _checkOnline() later succeeds, so the header remains permanently populated with the null fallback hardware fields until the monitor is recreated or the server restarts.
Useful? React with 👍 / 👎.
Comment on lines
+363
to
+364
| const user = this.spark.ssh?.user || "root"; | ||
| const repo = `/home/${user}/.hermes/hermes-agent`; |
There was a problem hiding this comment.
For the default SSH user root, the checkout is under /root/.hermes, not /home/root/.hermes; the same problem occurs for any account with a nonstandard home. All git errors are suppressed, so pendingCommits() silently returns an empty result and the update dialog shows unrelated release information instead of the commits actually pending on that Spark. Resolve the user's home rather than constructing /home/<user>.
Useful? React with 👍 / 👎.
Comment on lines
+281
to
+283
| if (totalMB <= 0 && memTotalMB > 0) totalMB = memTotalMB; | ||
| else if (totalMB <= 0) totalMB = DGX_SPARK.MEMORY_HBM_SIZE_GB * 1024; // Convert to MB | ||
| if (totalMB > 0 && usedMB > 0) availableMB = Math.max(0, totalMB - usedMB); |
There was a problem hiding this comment.
On an idle dedicated GPU where nvidia-smi reports exactly 0 MiB used, this guard leaves availableMB set to the host's system MemAvailable value. The GPU panel can consequently report more available VRAM than the GPU's total capacity; once totalMB is known, host VRAM availability should always be totalMB - usedMB, including the zero-usage case. The remote-host path repeats the same guard.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebase/merge of the three Batch B PRs onto current
mainafter Concept ACK / follow-up fixes.Included
patchSparkarms onllmPortsown-property)VITE_HISTORY_HOURS)Verification (this branch)
npm test— 208/208 pass (includes new SSH multiplex + SparkRegistry key sync tests)npm run typecheck— passnpm run build— passRepo has no GitHub Actions checks configured; local suite is the gate.
After merge
Please close the originals (MiaAI-Lab#76, MiaAI-Lab#77, MiaAI-Lab#75) as superseded by this combined merge (same approach as MiaAI-Lab#78 / Batch A).
— Vincent