LCORE-1426: BYOK Config refactoring

[are-ces]

Summary

Unifies all RAG-related configuration under a single rag section in lightspeed-stack.yaml, replacing the three separate top-level sections (byok_rag, rag, okp). Hardcoded chunk limit constants are now user-configurable fields with sensible defaults.

Key changes:

• byok_rag list → rag.byok.stores (with rag.byok.max_chunks)
• rag.inline / rag.tool lists → rag.retrieval.inline.sources / rag.retrieval.tool.sources (each with max_chunks)
• okp section → rag.okp (with max_chunks)
• rag_type field renamed to backend (e.g. faiss instead of inline::faiss)
• BACKEND_TO_LLAMA_STACK_PROVIDER mapping validates backends during enrichment
• Chunk limit constants (BYOK_RAG_MAX_CHUNKS, OKP_RAG_MAX_CHUNKS, etc.) replaced with config defaults
• Removed redundant max_chunks parameter from _fetch_byok_rag (was always passed the config value)
• Renamed _fetch_solr_rag → _fetch_okp_rag for consistency with the OKP naming convention
• Updated BYOK and RAG guides with new config structure, chunk flow diagram, and prioritization docs

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change
• Unit tests improvement
• Integration tests improvement
• End to end tests improvement
• Benchmarks improvement

Tools used to create PR

• Assisted-by: Claude Code (Claude Opus 4.6)
• Generated by: Claude Opus 4.6

Related Tickets & Documents

• Closes LCORE-1426

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• All unit tests updated and passing (uv run make test-unit)
• All integration test fixtures updated to new config shape
• Linters and type checkers pass (uv run make verify)
• OpenAPI schema regenerated (uv run make schema)
• YAML example configs validated against new Pydantic models

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• New Features

  • Added nested RAG configuration structure for improved organization of BYOK stores, OKP settings, and retrieval strategies (inline and tool).
  • Introduced configurable chunk limits per RAG strategy and source type.

• Documentation

  • Enhanced BYOK content prioritization guidance with flow diagrams.
  • Updated RAG configuration guide with new nested structure and chunk flow documentation.
  • Clarified pgvector configuration requirements.

• Configuration Changes

  • RAG BYOK stores now use backend field for consistency.
  • Updated OpenAPI specification to reflect unified RAG configuration model.