fix(settings): keep hand-set Claude settings when the settings tabs save

[pjdoland]

Summary

jupyter_ui_tools_external (#398) has no control in the Settings dialog, so it is set by hand under claude_settings in ~/.jupyter/nbi/config.json. Opening the Claude settings tab erased it. The tab saves the ten keys it renders when it mounts, and ConfigHandler.post stored that object in place of the old one. A deployment that set the key to serve the Jupyter UI tools through the relay lost it the first time a user opened Settings, and Claude mode went back to the in-process server without any notice. The same applied to any other claude_settings key the tab does not show.

Solution

• Merge, then clamp. ConfigHandler.post merges a posted claude_settings dict onto the stored value, then runs the admin policies, string overrides, inline-model clamp, and ANTHROPIC_API_KEY scrub on the merged result. The order matters: apply_claude_policies always writes tools and setting_sources, so merging after it would empty both on a partial POST. The merge is shallow, so a posted list still replaces the stored list and unchecking a tool sticks.
• Reload before saving. The handler saves the whole in-memory config but, unlike the GET handlers, never reloaded it. A hand edit made while JupyterLab runs was overwritten by the next settings POST from any tab, including the General tab's mount POST, which sends no claude_settings. It now calls nbi_config.load() before applying the POST. This also keeps settings saves from rewriting a hand-edited mcp.json from memory.
• Change checks keep the running baseline. The Claude and ACP enabled flags and the stored acp_settings are captured before the reload. The exclusivity check and the ACP restart decision therefore compare against the state the server is running with, as they did before, not against the disk. The handler also disconnects the Claude client whenever Claude was on before the POST and is off after it. Without that, a Claude disable picked up by the reload would leave the client connected while ACP took over.
• Docs and changelog. The feat: Add Proxied Jupyter UI Tools MCP Server #398 section of the admin guide now says to add the key alongside the other claude_settings keys in the user config and not to create an object holding only this key. It also says the key needs the Jupyter UI tools enabled and takes effect after a restart. A 5.4.1 entry is added under Fixed.

Testing

• Unit tests. tests/test_config_handler_claude_settings.py has 18 tests. They drive ConfigHandler.post against a config whose files all live in tmp_path and read the results back from disk.
  • Merge: a hand-set key survives the panel payload; posted keys and lists override stored ones; a partial POST keeps stored lists.
  • Policies: a forced tool policy, the continue_conversation policy, and the API-key scrub all apply to stored keys.
  • Other sources: an environment-wide key is copied into the user config; an ACP takeover keeps the key; null stored and posted values are handled.
  • Reload: a hand edit on disk survives both a Claude tab POST and a POST without claude_settings, including an unrelated top-level key and mcp.json; a malformed config.json fails without being overwritten.
  • Change checks: an unchanged ACP payload does not restart the agent, and a changed one does even when the disk already matches; enabling ACP wins over an ACP enable found only on disk; a Claude disable found on disk disconnects the live client.
• Mutation checks. Each of these edits to the handler fails at least one test:
  • no reload, reload before the prior-state capture, reload at the end of the handler, or reload only for claude_settings;
  • ACP change compared against the disk;
  • no Claude disconnect;
  • no merge, merge after the clamps, a union of lists, or stored values winning over posted ones.
• Suites. pytest tests/ --ignore=tests/test_claude_client.py (1771 passed), jlpm tsc --noEmit, stylelint, eslint, prettier on the changed docs, and jlpm jest (423 passed).
• Live check. Run in JupyterLab 4.6 with an isolated home directory and a config seeded with jupyter_ui_tools_external: true, against this branch and against main as a negative control. No console errors and no server tracebacks, and every config POST returned 200.

Step	This branch	main
Open Settings, then the Claude tab	Key kept	Key erased
Hand-add the key after the page loaded, then open the Claude tab	Key kept	Not run (already erased above)
Hand-add a top-level key and the flag, then POST only inline_completion_debouncer_delay	Both kept, new value saved	Both erased

Risks / follow-ups

• A malformed config file now fails the save. If config.json or mcp.json does not parse, a settings POST returns 500 and leaves the file alone. Before, the POST rewrote the broken file from memory, which discarded the edit. GET /capabilities already fails the same way. The frontend only logs a failed POST to the console, so a clearer error could be a follow-up.
• Environment-wide keys are copied. When a user has no claude_settings of their own, the first save copies the environment-wide object into the user config, merged keys included, as it already did for the keys the tab sends. A later change to the environment-wide value does not reach that user. Layering user settings over environment settings key by key on read would address this more generally.
• Other writers still save without reloading. The MCP config editor save and the rule toggle endpoint also write the whole in-memory config. Moving the reload into NBIConfig would cover every writer.
• Keys the tabs render can still revert. The tabs still post their mount-time snapshot, so a hand edit to a key a tab shows (for example tools), made while the page is open, is reverted when that tab opens. Now that the server merges, posting only changed fields from the panel would cover that and would let the ACP tab stop relying on change detection to avoid restarts. The Claude client also still reconnects on every Claude tab mount.
• acp_settings is not merged. The ACP tab replaces its object the same way. It has no hand-set keys today.
• A null claude_settings POST still replaces the stored object. Only a scripted client can send one.
• MCP Servers tab. Its mount POST looks like it can drop disabled_tools for servers that have not connected yet. That is separate, and this PR does not change it.
• Changelog heading. This adds a 5.4.1 section above 5.4.0, whose heading still reads "unreleased". Happy to adjust if you would rather manage the release headings.

No issue was filed for this. It came up while checking the 5.4 release notes against the code, and the details are above.