Conversation
plmbr#431) The capabilities response filtered `llm_providers` through the enablement predicate but returned the model lists straight from AIServiceManager, whose `chat_model_ids` and siblings walk every registered provider with no notion of enablement. With `disabled_providers = ["ollama"]` the provider correctly vanished from the picker while every Ollama model stayed in the same response. Two consequences. The payload advertised models an admin had switched off, and anything reading `capabilities.chat_models` without cross-checking `llm_providers` treated them as available. Worse, enumerating a provider's models is not free for all of them: the Ollama list is built by calling the Ollama host, so the response performed work on behalf of a provider that was disabled. That matters more once the Ollama enumeration moves off server startup and onto the capabilities path, so this stacks behind the branch for plmbr#428. The filter is a small pure helper beside the existing provider-enablement utilities, applied at the three payload sites. It takes the predicate rather than the denylist, because the caller's version already folds in the per-pod re-enable env var and that resolution should not be duplicated. An entry whose provider cannot be read is kept deliberately: every entry is built in-process with a `provider` key, so that only matters if the shape changes, and hiding a model for a reason the admin did not ask for would be the worse and quieter failure. Filtering cannot orphan a selection that was previously reachable. A denylisted provider is already absent from `llm_providers`, so it cannot be the selected provider, and the settings panel scopes the model list to the selected provider before rendering it. Tests cover the helper: a disabled provider's models dropped, an enabled provider untouched, a permissive predicate leaving the list identical, a re-enabled provider keeping its models, the fail-open entries, non-list input degrading to empty, and the input list left unmutated. The three call sites are one-liners around it and are not exercised, because the capabilities handler has no test harness here; the same reproduce-in-isolation limitation is already documented in tests/test_config_integration.py.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
disabled_providersremoved a provider from the picker but not its models from the capabilities response. Withdisabled_providers = ["ollama"],llm_providerscorrectly omitted Ollama whilechat_modelsstill carried every Ollama model in the same payload.Two consequences. The response advertised models the admin had switched off, and anything reading
capabilities.chat_modelswithout cross-checkingllm_providers(src/api.ts:331,:335) treated them as available. Worse, enumerating a provider's models is not free for all of them: the Ollama list is built by calling the Ollama host, so the response performed work on behalf of a provider that was disabled.Solution
GetCapabilitiesHandler.getalready filters the provider list through anis_provider_enabledclosure (extension.py:659-663, applied at:673). The three model lists came straight fromAIServiceManager, whosechat_model_idsand siblings walk every registered provider with no notion of enablement (ai_service_manager.py:492-508). They now pass through a small pure helper using that same predicate, so provider and model filtering cannot disagree.Two deliberate choices worth review:
allow_enabling_providers_with_envplusis_provider_enabled_in_env), and duplicating that resolution inutil.pywould be a second source of truth for it.providerkey, so the branch only matters if that shape changes, and silently hiding a model for a reason the admin did not ask for is the worse and quieter failure.Gating construction inside
AIServiceManagerwas considered and rejected: the traitlet is attached only to the handler (extension.py:4317) and is deliberately absent from the manager's options (:4219), so the manager has no denylist to consult and giving it one is a much larger change than this warrants.This does not orphan a previously reachable selection. A denylisted provider is already missing from
llm_providers, so it cannot be the selected provider, and the settings panel scopes the model list to the selected provider before rendering (settings-panel.tsx:447-449). I checked this specifically because filtering a model list is exactly how a persisted selection can silently change under a user.Testing
pytest1760 passed, including 7 new cases intests/test_disabled_provider_models.py: a disabled provider's models dropped, an enabled provider untouched, a permissive predicate leaving the list byte-identical, a re-enabled provider keeping its models, the fail-open entries (missing key, null provider, non-dict), non-list input degrading to empty, and the input list left unmutated.tsc --noEmitclean, jest 423 passed,eslint,stylelint,prettierclean. No TypeScript changed.Risks and follow-ups
tests/test_config_integration.pydocuments the same reproduce-in-isolation limitation for the adjacent_setup_handlersexpression. A handler fixture would retire both gaps and is worth doing separately.is_provider_enabled_in_env(util.py:379-381) splitsNBI_ENABLED_PROVIDERSon,without stripping, so"github-copilot, ollama"silently fails to re-enableollama. Separate issue, separate fix, andsplit_csvin the same module is the ready-made remedy.Closes #431