Skip to content

feat: bridge DeepSeek V4 Python caches and metadata. - #2228

Open
0moyi0-2024 wants to merge 4 commits into
xLLM-AI:mainfrom
0moyi0-2024:dsv4_0813
Open

feat: bridge DeepSeek V4 Python caches and metadata.#2228
0moyi0-2024 wants to merge 4 commits into
xLLM-AI:mainfrom
0moyi0-2024:dsv4_0813

Conversation

@0moyi0-2024

Copy link
Copy Markdown

Summary

  • extend the Python layer-cache contract with the six DeepSeek-V4 DSA cache slots while preserving the existing five generic slots
  • expose DeepSeek-V4 forward metadata through the standalone Python attention metadata view
  • bridge all eleven KV-cache entries from the C++ Python executor and preserve existing expanded-decode and MLA index interfaces
  • add compatibility tests for legacy five-slot and DeepSeek-V4 eleven-slot cache tuples

Test plan

  • PYTHONPATH=$PWD pytest -q tests/python/test_model_executor.py -k 'NormalizeLayerCaches or BindKvCaches' (6 passed)
  • PYTHONPATH=$PWD pytest -q tests/python/test_model_executor.py -k 'not data_parallel_rejects_non_cuda_graph_backend' (42 passed, 1 deselected; the excluded baseline test expects the pre-ACLGraph error text)
  • NPU/Torch C++ syntax checks for py_attention_metadata.cpp and py_executor_impl.cpp with -Werror -fsyntax-only
  • full python setup.py build --device npu is currently running in the local workspace

Scope

This PR provides the cache and metadata bridge required by the DeepSeek-V4 Python model. The model implementation, DSA backend, custom kernels, and graph integration will be submitted separately.

@yingxudeng yingxudeng changed the title feat: bridge DeepSeek V4 Python caches and metadata feat: bridge DeepSeek V4 Python caches and metadata. Aug 13, 2026
@yingxudeng

Copy link
Copy Markdown
Collaborator

python setup.py build test --device npu 推荐本地单测也要跑下,加test 参数

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants