feat(embedding): declare a model's width without sending dimensions - #1857
feat(embedding): declare a model's width without sending dimensions#1857NullSense wants to merge 1 commit into
Conversation
|
✅ Health of changed files: 6.5 → 6.9 (+0.4) 📋 At a glance Files & modules (2)
✅ Health gate: passed 📌 Before you merge
🔎 More signals (2)🗺️ Change map flowchart LR
subgraph PR ["Changed in this PR (1 with dependents)"]
f_packages_core_src_repowise_core_providers_embedding_openai_py[".../embedding/openai.py 🔥"]:::changed
end
f_packages_core_src_repowise_core_providers_embedding_edenai_py[".../embedding/edenai.py"]
f_packages_core_src_repowise_core_providers_embedding_openai_py --> f_packages_core_src_repowise_core_providers_embedding_edenai_py
f_packages_core_src_repowise_core_providers_embedding_openrouter_py[".../embedding/openrouter.py"]
f_packages_core_src_repowise_core_providers_embedding_openai_py --> f_packages_core_src_repowise_core_providers_embedding_openrouter_py
f_packages_core_src_repowise_core_providers_embedding_registry_py[".../embedding/registry.py"]
f_packages_core_src_repowise_core_providers_embedding_openai_py --> f_packages_core_src_repowise_core_providers_embedding_registry_py
f_packages_core_src_repowise_core_providers_llm_deepseek_py[".../llm/deepseek.py"]
f_packages_core_src_repowise_core_providers_embedding_openai_py --> f_packages_core_src_repowise_core_providers_llm_deepseek_py
more(["+6 more dependents"])
PR --> more
t_tests_unit_server_mcp_test_embedder_resolution_py(["✅ .../mcp/test_embedder_resolution.py"]):::guard
t_tests_unit_server_mcp_test_embedder_resolution_py -.-> f_packages_core_src_repowise_core_providers_embedding_openai_py
classDef changed fill:#dbeafe,stroke:#1d4ed8,color:#1e3a5f
classDef warn fill:#fef3c7,stroke:#b45309,color:#78350f
classDef guard fill:#dcfce7,stroke:#15803d,color:#14532d
Solid arrows: code that imports the changed files (10 direct dependents, from the last indexed snapshot). Dashed: history/tests. 🔥 Hotspots touched (2)
👀 Suggested reviewers @RaghavChamadiya 📊 See the full report for this PR |
Ayush7614
left a comment
There was a problem hiding this comment.
Verified: pytest tests/unit/test_persistence/test_openai_embedder.py -q 52 passed. Correctly adds declared_dimensions/REPOWISE_EMBEDDING_DECLARED_DIMS as declare-only width (never sent as dimensions param) for non-Matryoshka models like Nemotron-3-Embed-1B that 400 on dimensions. Precedence dimensions > REPOWISE_EMBEDDING_DIMS > declared_dimensions > REPOWISE_EMBEDDING_DECLARED_DIMS > _DIMS > 1536 matches spec, error messages now name the actual source, and the strict-endpoint regression test proves the param is never sent. LGTM.
Summary
OpenAIEmbeddercan only override an embedding model's width by also sending that width to the API as thedimensionsparameter. That's correct for a Matryoshka model, but it breaks a model of non-default width that isn't Matryoshka-capable: the API rejects the request outright (e.g. nvidia/Nemotron-3-Embed-1B, 2048-wide, served OpenAI-compatible, errors with "does not support Matryoshka embeddings; dimensions must be unset").declared_dimensions/REPOWISE_EMBEDDING_DECLARED_DIMS, a second override that sets the declared width without ever sending it. Precedence:dimensions=>REPOWISE_EMBEDDING_DIMS>declared_dimensions=>REPOWISE_EMBEDDING_DECLARED_DIMS> the_DIMStable > 1536. Only the first two are ever sent on the wire._DIMSorREPOWISE_EMBEDDING_DIMS.Related Issues
None.
Test Plan
uv run pytest tests/unit/test_persistence/test_openai_embedder.py -q— 52 passeduv run pytest tests/unit/test_persistence/ tests/unit/test_providers/ tests/unit/server/mcp/test_embedder_resolution.py tests/unit/cli/test_embedder_key_resolution.py -q— 416 passeduv run ruff check .(the actual CI lint gate;ruff format --checkisn't run in CI, and this PR's files are clean under it anyway)uv run repowise risk main..HEAD:impacted-tests/health --fileneed a completedrepowise initagainst this clone; that indexing run did not finish in time and is omitted rather than faked.)Why this shape, not a
_DIMStable entryThe straightforward fix is hardcoding
"nemotron-embed-1b": 2048in_DIMS. That covers exactly one deployment; every other non-Matryoshka model of non-default width hits the identical wall. The existingdimensions=design already treats width as something the operator declares for whatever model they're running, not something the code enumerates by name —declared_dimensionsextends that same idea to the one case it didn't cover.Checklist
Written with AI assistance; measurements and tests were run and verified locally.