# DeepSeek-V4-Flash \u6607\u817e 910B PD \u5206\u79bb\u90e8\u7f72：8 \u5361\u5168\u6302\u4e4b\u540e，\u6211\u7528\u201c\u627e rank 0\u201d\u5b9a\u4f4d\u4e86 6 \u4e2a\u8fde\u73af\u5751<\/p>\n<\/p>\n
> \u4e24\u53f0 Atlas 910B-4\u30018 \u5f20\u5361\u3001vLLM-Ascend v0.30.0rc2\u30011P1D \u5206\u79bb\u90e8\u7f72\u3002<\/p>\n
> \u65e5\u5fd7\u91cc 6 \u4e2a rank \u9f50\u58f0\u558a\u201c\u8fde\u63a5\u5df2\u65ad\u5f00\u201d，\u4f46**\u771f\u6b63\u7684\u539f\u56e0\u53ea\u6709\u4e00\u4e2a rank**\u3002<\/p>\n
> \u8fd9\u7bc7\u6587\u7ae0\u628a 6 \u4e2a\u5751\u548c\u4e00\u5957\u6392\u67e5\u65b9\u6cd5\u8bba\u4e00\u6b21\u8bb2\u6e05\u3002<\/p>\n
><\/p>\n
> \u8bf4\u660e：\u6587\u4e2d IP \u5730\u5740\u3001\u7f51\u5361\u540d\u3001\u6a21\u578b\u8def\u5f84\u5747\u5df2\u8131\u654f\u5904\u7406，\u4e0d\u5f71\u54cd\u590d\u73b0\u4e0e\u7406\u89e3\u3002<\/p>\n<\/p>\n
## \u4e00\u3001\u73af\u5883\u4e0e\u76ee\u6807<\/p>\n<\/p>\n
| \u9879 | \u914d\u7f6e |<\/p>\n
|—|—|<\/p>\n
| \u786c\u4ef6 | 2 \u00d7 Atlas 910B-4（\u6bcf\u53f0 8 \u5361），\u5355\u673a 8 \u5361\u5168\u7528 |<\/p>\n
| \u955c\u50cf | `quay.io\/ascend\/vllm-ascend:v0.30.0rc2` |<\/p>\n
| \u5e76\u884c | DP4 \u00d7 TP2（\u6bcf\u8282\u70b9 4 \u4e2a engine，\u7aef\u53e3 7100\u20137103） |<\/p>\n
| \u90e8\u7f72 | 1P1D：P \u8282\u70b9 `kv_producer:36000`，D \u8282\u70b9 `kv_consumer:36100` |<\/p>\n
| KV \u4f20\u8f93 | `MooncakeHybridConnector` + \u8d1f\u8f7d\u5747\u8861\u4ee3\u7406（1999） |<\/p>\n
| \u6a21\u578b | DeepSeek-V4-Flash-w8a8（\u91cf\u5316\u7248） |<\/p>\n<\/p>\n
\u76ee\u6807\u5f88\u6734\u7d20：**\u8ba9 PD \u5206\u79bb\u7684\u4e24\u53f0\u673a\u5668\u628a\u670d\u52a1\u62c9\u8d77\u6765**\u3002\u7ed3\u679c\u662f\u8fde\u7eed 6 \u6b21\u201c\u542f\u52a8\u5373\u5168\u6302\u201d\u3002<\/p>\n<\/p>\n
## \u4e8c\u3001\u5148\u8bf4\u7ed3\u8bba：\u8fd9\u5957\u65e5\u5fd7\u7684\u201c\u966a\u846c\u6548\u5e94\u201d<\/p>\n<\/p>\n
\u8fd9\u662f\u5168\u6587\u6700\u503c\u94b1\u7684\u4e00\u6761\u7ecf\u9a8c\u3002<\/p>\n<\/p>\n
8 \u4e2a rank \u4e00\u8d77\u542f\u52a8，\u6700\u540e\u6302\u5728\u540c\u4e00\u4e2a\u5730\u65b9\u65f6，\u65e5\u5fd7\u662f\u8fd9\u6837\u7684：<\/p>\n<\/p>\n
```<\/p>\n
(Worker_DP1_TP0_EP2) RuntimeError: [2] is setting up HCCL communicator and<\/p>\n
retrieving hcclUniqueId from [0] via c10d key-value store by key '0',<\/p>\n
but store->get('0') got error: Failed to recv, got 0 bytes.<\/p>\n
Connection was likely closed. Did the remote server shutdown or crash?<\/p>\n<\/p>\n
(EngineCore_DP0) WARNING [shutdown] Executor: workers still running after<\/p>\n
grace period; sending SIGTERM count=2<\/p>\n
```<\/p>\n<\/p>\n
\u770b\u5230 6 \u4e2a rank \u90fd\u5728\u62a5 `store->get('0') failed`，\u7b2c\u4e00\u53cd\u5e94\u5bb9\u6613\u53bb\u67e5\u7f51\u7edc\u3001\u67e5 MoE\u3001\u67e5\u5b58\u50a8\u3002<\/p>\n<\/p>\n
**\u5168\u9519\u3002** \u8fd9\u51e0\u884c\u662f\u201c\u966a\u846c\u201d：<\/p>\n<\/p>\n
– `dp_rank 0` \u662f DP \u7ec4\u7684 head，\u670d\u52a1\u7aef\u7684 TCPStore \u5c31\u662f\u5b83\u8d77\u7684；<\/p>\n
– \u5b83**\u5148**\u6302\u4e86 \u2192 Store \u5173\u95ed \u2192 \u5176\u4f59 6 \u4e2a rank \u5361\u5728 `store->get('0')` \u4e0a\u96c6\u4f53\u62a5\u201c\u8fde\u63a5\u65ad\u5f00\u201d；<\/p>\n
– \u6240\u4ee5\u62a5\u9519\u6570\u91cf\u6700\u591a\u7684\u90a3\u4e2a\u4f4d\u7f6e，\u5f80\u5f80**\u4e0d\u662f**\u6839\u56e0\u6240\u5728\u3002<\/p>\n<\/p>\n
**\u5224\u636e（\u8bb0\u4f4f\u8fd9\u4e09\u6761\u5c31\u80fd\u4e00\u773c\u770b\u51fa\u662f\u7ea7\u8054）：**<\/p>\n<\/p>\n
1. \u62a5\u9519\u5185\u5bb9\u662f `Failed to recv, got 0 bytes \/ Connection was likely closed`\u2014\u2014\u8fd9\u662f\u201c\u5bf9\u7aef\u6ca1\u4e86\u201d，\u4e0d\u662f\u201c\u6211\u9519\u4e86\u201d；<\/p>\n
2. \u51fa\u73b0 `[shutdown] workers still running after grace period` \u2014\u2014\u8bf4\u660e\u6709\u8fdb\u7a0b\u5728**\u4e3b\u52a8**\u5173\u95ed；<\/p>\n
3. \u62a5\u9519\u7684\u662f `EP2~EP7`，\u800c `EP0 \/ EP1` **\u4e00\u884c\u62a5\u9519\u90fd\u6ca1\u6709**\u2014\u2014\u4e0d\u62a5\u9519\u7684\u90a3\u4e2a\u624d\u662f\u51f6\u624b\u3002<\/p>\n<\/p>\n
**\u5bf9\u5e94\u52a8\u4f5c**：\u65e5\u5fd7\u6c38\u8fdc\u4e0d\u8981\u53ea\u770b\u5c3e\u90e8\u3002\u53d6\u6839\u56e0\u8981\u7528\u5e26\u4e0a\u4e0b\u6587\u7684 grep：<\/p>\n<\/p>\n
```bash<\/p>\n
grep -n -m1 -B80 -A40 "Worker_DP0_TP0_EP0" logs\/serve.log<\/p>\n
```<\/p>\n<\/p>\n
> \u6211\u5728\u8fd9\u4e2a\u95ee\u9898\u4e0a\u88ab\u5751\u4e86\u4e00\u6574\u8f6e：\u540c\u4e8b\u53d1\u6765\u7684\u65e5\u5fd7\u4e09\u6b21\u90fd\u662f\u4ece\u201c\u966a\u846c\u533a\u201d\u5f00\u59cb\u622a\u7684，<\/p>\n
> \u6211\u4e24\u6b21\u57fa\u4e8e\u6b8b\u7f3a\u65e5\u5fd7\u7ed9\u51fa\u4e86\u9519\u8bef\u7ed3\u8bba\u3002**\u5b8c\u6574\u65e5\u5fd7 + rank 0 \u7684 traceback，\u662f\u8fd9\u4e2a\u573a\u666f\u4e0b\u552f\u4e00\u6709\u6548\u7684\u4fe1\u606f\u3002**<\/p>\n<\/p>\n
## \u4e09\u30016 \u4e2a\u5751，\u9010\u4e2a\u62c6<\/p>\n<\/p>\n
### \u5751 1：Windows \u6362\u884c\u7b26 + `sh` \u6267\u884c<\/p>\n<\/p>\n
**\u62a5\u9519**<\/p>\n<\/p>\n
```<\/p>\n
.\/start_serve.sh: 10: Syntax error: word unexpected (expecting "in")<\/p>\n
: not found<\/p>\n
```<\/p>\n<\/p>\n
**\u771f\u76f8**：\u811a\u672c\u662f Windows CRLF \u6362\u884c\u3002`\\\\r` \u88ab\u5f53\u6210\u8bcd\u7684\u4e00\u90e8\u5206，`in\\\\r` \u4e0d\u518d\u7b49\u4e8e `in`，`bash` \u8bed\u6cd5\u89e3\u6790\u76f4\u63a5\u5d29\u3002<\/p>\n<\/p>\n
**\u4fee\u590d**<\/p>\n<\/p>\n
```bash<\/p>\n
sed -i 's\/\\\\r$\/\/' *.sh<\/p>\n
# \u5e76\u4e14\u7528 bash \u800c\u4e0d\u662f sh：\u811a\u672c\u91cc\u7684 `==`\u3001`kill -TERM -$PID` \u90fd\u662f bash \u4e13\u6709<\/p>\n
bash .\/start_serve.sh<\/p>\n
```<\/p>\n<\/p>\n
### \u5751 2：\u4ee3\u7406\u7684 hosts \u6570\u91cf\u5fc5\u987b\u7b49\u4e8e ports \u6570\u91cf<\/p>\n<\/p>\n
**\u62a5\u9519**<\/p>\n<\/p>\n
```<\/p>\n
ValueError: Number of prefiller hosts must match number of prefiller ports<\/p>\n
```<\/p>\n<\/p>\n
**\u771f\u76f8**：`load_balance_proxy_server_example.py` \u5f3a\u6821\u9a8c\u4e24\u8005\u957f\u5ea6\u76f8\u7b49\u30021 \u4e2a host \u914d 4 \u4e2a port \u662f**\u4e0d\u884c**\u7684\u3002<\/p>\n<\/p>\n
**\u4fee\u590d**（host \u91cd\u590d 4 \u6b21）：<\/p>\n<\/p>\n
```bash<\/p>\n
–prefiller-hosts 10.10.10.1 10.10.10.1 10.10.10.1 10.10.10.1 \\\\<\/p>\n
–prefiller-ports 7100 7101 7102 7103 \\\\<\/p>\n
–decoder-hosts 10.10.10.2 10.10.10.2 10.10.10.2 10.10.10.2 \\\\<\/p>\n
–decoder-ports 7100 7101 7102 7103<\/p>\n
```<\/p>\n<\/p>\n
### \u5751 3：\u4e00\u4e2a\u5e03\u5c14\u503c\u5199\u6210\u5b57\u7b26\u4e32，8 \u5f20\u5361\u5168\u6302<\/p>\n<\/p>\n
**\u62a5\u9519**<\/p>\n<\/p>\n
```<\/p>\n
ValueError: enable_multithread_load must be a bool, got str<\/p>\n
subprocess.CalledProcessError: Command '['bash','.\/run_dp_template.sh','0,1','7100',…]'<\/p>\n
returned non-zero exit status 1<\/p>\n
```<\/p>\n<\/p>\n
**\u771f\u76f8**：JSON \u91cc\u628a\u5e03\u5c14\u5199\u6210\u4e86\u5b57\u7b26\u4e32\u3002<\/p>\n<\/p>\n
```bash<\/p>\n
# \u9519<\/p>\n
–model-loader-extra-config='{"enable_multithread_load": "true", "num_threads": 128}'<\/p>\n
# \u5bf9<\/p>\n
–model-loader-extra-config='{"enable_multithread_load": true, "num_threads": 128}'<\/p>\n
```<\/p>\n<\/p>\n
**\u6ce8\u610f**：\u8fd9\u4e2a\u53c2\u6570\u662f\u201c\u6bcf\u4e2a engine \u7684 8 \u4e2a Worker \u540c\u65f6\u5931\u8d25\u201d，4 \u4e2a engine \u5168\u9000\u3002<\/p>\n
`CalledProcessError … exit status 1` \u53ea\u662f\u8868\u8c61，**\u771f\u6b63\u7684\u7c7b\u578b\u9519\u8bef\u5728\u5b83\u4e0a\u9762 20 \u884c**\u3002<\/p>\n<\/p>\n
### \u5751 4：\u591a\u7ebf\u7a0b\u52a0\u8f7d\u4e0e prefetch \u7b56\u7565\u4e92\u65a5<\/p>\n<\/p>\n
**\u62a5\u9519**<\/p>\n<\/p>\n
```<\/p>\n
ValueError: enable_multithread_load does not support<\/p>\n
safetensors_load_strategy='prefetch'; the multi-thread loader only<\/p>\n
implements the default lazy strategy.<\/p>\n
```<\/p>\n<\/p>\n
**\u771f\u76f8**：\u8fd9\u662f\u5751 3 \u4fee\u5b8c\u4e4b\u540e\u201c\u6d6e\u51fa\u6765\u201d\u7684\u7b2c\u4e8c\u4e2a\u51b2\u7a81\u3002\u4e24\u4e2a\u5f00\u5173\u5728\u540c\u4e00\u4e2a\u914d\u7f6e\u91cc\u4e92\u65a5：<\/p>\n<\/p>\n
– `enable_multithread_load: true` \u2192 \u53ea\u5b9e\u73b0 lazy \u7b56\u7565<\/p>\n
– `–safetensors-load-strategy 'prefetch'` \u2192 \u9884\u53d6\u7b56\u7565<\/p>\n<\/p>\n
**\u4fee\u590d**：\u4e8c\u9009\u4e00\u3002\u63a8\u8350\u4fdd\u7559\u591a\u7ebf\u7a0b\u52a0\u8f7d（\u6743\u91cd\u52a0\u8f7d\u4ece\u201c\u5206\u949f\u7ea7\u201d\u964d\u5230\u201c\u79d2\u7ea7\/\u5206\u7247\u201d），\u5220\u6389 prefetch \u90a3\u884c\u3002<\/p>\n<\/p>\n
### \u5751 5（\u6838\u5fc3）：HCCL \u7f51\u7edc\u914d\u7f6e\u201c\u4e09\u65b9\u6253\u67b6\u201d<\/p>\n<\/p>\n
**\u62a5\u9519**<\/p>\n<\/p>\n
```<\/p>\n
RuntimeError: createHCCLCommOrigin: ProcessGroupHCCL.cpp:3090<\/p>\n
HCCL function error: hcclGetRootInfo(&hcclID), error code is 19<\/p>\n
(call network api fail)<\/p>\n
ERR02200 DIST call hccl api failed.<\/p>\n
```<\/p>\n<\/p>\n
\u53d1\u751f\u5728 `w8a8_dynamic.py` \u5efa MoE \u4e13\u5bb6\u5e76\u884c\u7684 all-to-all \u901a\u4fe1\u57df\u65f6，**rank 0 \/ Device:0 \u76f4\u63a5\u5931\u8d25**\u3002<\/p>\n<\/p>\n
**\u771f\u76f8**：HCCL \u7684\u7f51\u5361\u914d\u7f6e\u6765\u81ea\u4e09\u4e2a\u5730\u65b9，\u800c\u5b83\u4eec\u6307\u7684\u4e0d\u662f\u540c\u4e00\u5f20\u7f51\u5361：<\/p>\n<\/p>\n
| \u6765\u6e90 | `HCCL_SOCKET_IFNAME` | `HCCL_COMM_DEFAULT_IF` | `HCCL_IF_IP` |<\/p>\n
|—|—|—|—|<\/p>\n
| \u955c\u50cf\/\u5bb9\u5668\u6ce8\u5165 | `lo` | `lo` | \u672a\u8bbe |<\/p>\n
| \u811a\u672c export | `bond1` | **\u672a\u8986\u76d6 \u2192 \u4ecd\u662f lo** | `hostname -I` \u7684**\u7b2c\u4e00\u4e2a** IP |<\/p>\n<\/p>\n
\u4e09\u5904\u4e0d\u4e00\u81f4 \u2192 `hcclGetRootInfo` \u5931\u8d25\u3002**\u8fd9\u4e2a API \u662f\u7eaf\u672c\u5730\u7684**（\u53ea\u679a\u4e3e\u672c\u673a\u7f51\u5361\/\u5efa root），\u6240\u4ee5\u5b83\u5931\u8d25\u8ddf\u8de8\u673a\u3001\u8ddf PD\u3001\u8ddf Mooncake \u5168\u65e0\u5173\u7cfb\u2014\u2014\u522b\u88ab\u201c\u5206\u5e03\u5f0f\u201d\u4e09\u4e2a\u5b57\u5e26\u504f\u3002<\/p>\n<\/p>\n
**\u53e6\u4e00\u4e2a\u9690\u853d\u9677\u9631**：<\/p>\n<\/p>\n
```bash<\/p>\n
local_ip="`hostname -I | awk -F ' ' '{print$1}'`"<\/p>\n
```<\/p>\n<\/p>\n
`hostname -I` \u7684**\u987a\u5e8f\u4e0d\u53ef\u63a7**\u3002\u88c5\u6709 docker0 \u7684\u673a\u5668\u4e0a，`172.17.0.1` \u7ecf\u5e38\u6392\u5728\u6700\u524d\u9762 \u2192 `HCCL_IF_IP` \u88ab\u8bbe\u6210 `172.17.0.1`，\u800c\u5b83\u6839\u672c\u4e0d\u5728 `bond1` \u4e0a\u3002<\/p>\n<\/p>\n
**\u4fee\u590d\u539f\u5219（\u6bd4\u5177\u4f53\u547d\u4ee4\u91cd\u8981）**：\u8ba9 `HCCL_SOCKET_IFNAME`\u3001`HCCL_COMM_DEFAULT_IF`\u3001`HCCL_IF_IP` **\u4e25\u683c\u6307\u5411\u540c\u4e00\u5f20\u7f51\u5361\u7684\u540c\u4e00\u4e2a IP**，\u6216\u8005**\u5168\u90e8\u4e0d\u8bbe**（\u4ea4\u56de\u955c\u50cf\u57fa\u7ebf）\u3002\u6700\u5fcc\u8bb3\u201c\u5bb9\u5668\u7ed9 A\u3001\u811a\u672c\u7ed9 B\u201d\u3002<\/p>\n<\/p>\n
**\u4e00\u4e2a 5 \u79d2\u949f\u5b9a\u4f4d HCCL \u914d\u7f6e\u662f\u5426\u53ef\u7528\u7684\u63a2\u9488**（\u4e0d\u7528\u6bcf\u6b21\u7b49 3 \u5206\u949f\u8d77\u670d\u52a1）：<\/p>\n<\/p>\n
```bash<\/p>\n
cat > \/tmp\/hccl_probe.py <<'PY'<\/p>\n
import os, torch, torch_npu<\/p>\n
import torch.distributed as dist<\/p>\n
dist.init_process_group("hccl", rank=0, world_size=1)<\/p>\n
x = torch.ones(8, dtype=torch.float16).npu()<\/p>\n
dist.all_reduce(x)<\/p>\n
print("HCCL OK | IFNAME=%s | IF_IP=%s" % (<\/p>\n
os.environ.get("HCCL_SOCKET_IFNAME"), os.environ.get("HCCL_IF_IP")))<\/p>\n
PY<\/p>\n<\/p>\n
export MASTER_ADDR=127.0.0.1 MASTER_PORT=29507<\/p>\n
HCCL_SOCKET_IFNAME=lo python3 \/tmp\/hccl_probe.py<\/p>\n
HCCL_SOCKET_IFNAME=bond1 HCCL_IF_IP=<\u8be5\u7f51\u5361\u4e0a\u7684\u771f\u5b9eIP> python3 \/tmp\/hccl_probe.py<\/p>\n
```<\/p>\n<\/p>\n
\u5b83\u548c\u62a5\u9519\u70b9\u8d70**\u540c\u4e00\u6bb5 C++ \u4ee3\u7801**（`HcclCommInitRootInfo`），\u901a\u8fc7\/\u5931\u8d25\u51e0\u79d2\u5c31\u80fd\u770b\u5230，\u662f\u4e8c\u5206\u5b9a\u4f4d HCCL \u914d\u7f6e\u6700\u5feb\u7684\u529e\u6cd5\u3002<\/p>\n<\/p>\n
**\u987a\u5e26\u4e00\u4e2a\u5bb9\u5668\u7ec6\u8282**：\u8fd9\u4e2a\u955c\u50cf\u91cc**\u6ca1\u6709 `ip` \u547d\u4ee4**（\u6ca1\u88c5 iproute2）\u3002<\/p>\n<\/p>\n
```bash<\/p>\n
ip a # bash: ip: command not found<\/p>\n
```<\/p>\n<\/p>\n
\u6240\u4ee5\u6392\u67e5\u7f51\u5361\u53ea\u80fd\u7528：<\/p>\n<\/p>\n
```bash<\/p>\n
ls \/sys\/class\/net\/ # \u63a5\u53e3\u540d<\/p>\n
hostname -I # \u6240\u6709 IP<\/p>\n
cat \/proc\/net\/dev<\/p>\n
```<\/p>\n<\/p>\n
\u6216\u8005\u4e00\u6bb5\u65e0\u4f9d\u8d56\u7684 Python（\u53d6\u6307\u5b9a\u7f51\u5361\u7684 IPv4）：<\/p>\n<\/p>\n
```bash<\/p>\n
python3 -c "<\/p>\n
import socket,fcntl,struct,sys<\/p>\n
n=sys.argv[1]; s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)<\/p>\n
a=fcntl.ioctl(s,0x8915,struct.pack('256s',n[:15].encode()))<\/p>\n
print(n,'->',socket.inet_ntoa(a[20:24]))<\/p>\n
" bond1<\/p>\n
```<\/p>\n<\/p>\n
### \u5751 6：MTP \u8349\u7a3f\u6a21\u578b\u6743\u91cd\u5bf9\u4e0d\u4e0a，`assert` \u5931\u8d25<\/p>\n<\/p>\n
HCCL \u6253\u901a\u4e4b\u540e，\u65e5\u5fd7\u7ec8\u4e8e\u63a8\u8fdb\u5230\u4e86\u201c\u52a0\u8f7d\u4e2d\u201d：<\/p>\n<\/p>\n
```<\/p>\n
(Worker_DP0_TP0_EP0) Multi-thread loading shards: 69% Completed | 52\/75 [1.85it\/s]<\/p>\n
```<\/p>\n<\/p>\n
\u4e3b\u6a21\u578b\u5efa\u8d77\u6765\u4e86\u3001\u591a\u7ebf\u7a0b\u52a0\u8f7d\u6b63\u5e38\u8dd1，\u4f46 8 \u4e2a rank \u53c8\u7edf\u4e00\u6302\u5728\u4e00\u4e2a\u65b0\u7684 `AssertionError` \u4e0a：<\/p>\n<\/p>\n
```<\/p>\n
File "…\/vllm_ascend\/models\/deepseek_v4\/mtp.py", line 335, in load_weights<\/p>\n
assert "mtp.0." in name<\/p>\n
AssertionError<\/p>\n
```<\/p>\n<\/p>\n
**\u771f\u76f8**：\u8c03\u7528\u94fe\u662f<\/p>\n<\/p>\n
```<\/p>\n
model_runner_v1.py:4424 self.drafter.load_model(self.model)<\/p>\n
\u2192 llm_base_proposer.py:379 self._get_model()<\/p>\n
\u2192 default_loader.load_weights<\/p>\n
\u2192 deepseek_v4\/mtp.py:335 assert "mtp.0." in name<\/p>\n
```<\/p>\n<\/p>\n
\u5373：**MTP（\u591a token \u9884\u6d4b \/ \u6295\u673a\u89e3\u7801）\u7684\u8349\u7a3f\u6a21\u578b**\u5728\u52a0\u8f7d\u6743\u91cd\u65f6，\u8981\u6c42\u6743\u91cd\u540d\u5fc5\u987b\u542b `mtp.0.`，\u800c\u5f53\u524d checkpoint \u91cc\u53d6\u51fa\u6765\u7684\u540d\u5b57\u4e0d\u6ee1\u8db3\u3002<\/p>\n<\/p>\n
\u5178\u578b\u539f\u56e0：**W8A8 \u91cf\u5316\u6743\u91cd\u5728\u91cf\u5316\u8fc7\u7a0b\u4e2d\u4e22\u6389\u4e86 MTP \u5c42**（DeepSeek-V4 \u7684 nextn \u9884\u6d4b\u5c42）\u3002<\/p>\n<\/p>\n
**\u5148\u786e\u8ba4**：<\/p>\n<\/p>\n
```bash<\/p>\n
python3 – <<'PY'<\/p>\n
import json, os<\/p>\n
d='\/data\/models\/DeepSeek-V4-Flash-w8a8'<\/p>\n
cfg=json.load(open(os.path.join(d,'config.json')))<\/p>\n
print("num_nextn_predict_layers =", cfg.get('num_nextn_predict_layers'))<\/p>\n
wm=json.load(open(os.path.join(d,'model.safetensors.index.json')))['weight_map']<\/p>\n
mtp=[k for k in wm if 'mtp' in k.lower()]<\/p>\n
print("\u542b mtp \u7684\u6743\u91cd\u6761\u6570:", len(mtp))<\/p>\n
for k in mtp[:5]: print(" ", k)<\/p>\n
PY<\/p>\n
```<\/p>\n<\/p>\n
**\u5904\u7f6e**：<\/p>\n<\/p>\n
– `mtp \u6761\u6570 = 0` \u2192 \u8be5\u6743\u91cd\u6ca1\u6709 MTP \u5c42，**\u5220\u6389 `–speculative-config`** \u5148\u628a\u670d\u52a1\u8dd1\u8d77\u6765；<\/p>\n
– \u6709 MTP \u6743\u91cd\u4f46\u547d\u540d\u4e0d\u662f `mtp.0.` \u2192 \u9700\u8981\u6362\u6210\u4e0e vllm-ascend \u8be5\u7248\u672c\u5bf9\u9f50\u7684 checkpoint；<\/p>\n
– \u547d\u540d\u6b63\u5e38 \u2192 \u95ee\u9898\u5728\u52a0\u8f7d\u8fc7\u6ee4\u903b\u8f91，\u9700\u8981\u770b\u5177\u4f53\u6743\u91cd\u540d\u518d\u6df1\u6316\u3002<\/p>\n<\/p>\n
```bash<\/p>\n
# \u7ed5\u884c：\u6682\u65f6\u5173\u6389 MTP，\u5148\u9a8c\u8bc1 PD \u94fe\u8def<\/p>\n
sed -i "\/–speculative-config\/d" run_dp_template.sh<\/p>\n
```<\/p>\n<\/p>\n
## \u56db\u3001\u62a5\u9519 \u2192 \u6839\u56e0 \u2192 \u4fee\u590d \u603b\u8868<\/p>\n<\/p>\n
| # | \u62a5\u9519\u5173\u952e\u8bcd | \u6839\u56e0 | \u4fee\u590d |<\/p>\n
|—|—|—|—|<\/p>\n
| 1 | `Syntax error: word unexpected` | CRLF \u6362\u884c + \u7528 `sh` \u6267\u884c | `sed -i 's\/\\\\r$\/\/'`，\u6539\u7528 `bash` |<\/p>\n
| 2 | `Number of prefiller hosts must match` | hosts \u4e0e ports \u6570\u91cf\u4e0d\u7b49 | host \u6309\u7aef\u53e3\u6570\u91cd\u590d |<\/p>\n
| 3 | `enable_multithread_load must be a bool` | JSON \u91cc `"true"` \u662f\u5b57\u7b26\u4e32 | \u6539\u6210\u88f8 `true` |<\/p>\n
| 4 | `does not support safetensors_load_strategy='prefetch'` | \u591a\u7ebf\u7a0b\u52a0\u8f7d\u4e0e prefetch \u4e92\u65a5 | \u5220 `–safetensors-load-strategy` |<\/p>\n
| 5 | `hcclGetRootInfo … error code is 19 (call network api fail)` | \u5bb9\u5668\/\u811a\u672c HCCL \u7f51\u5361\u4e0e IP \u4e09\u65b9\u4e0d\u4e00\u81f4 | \u4e09\u5904\u7edf\u4e00\u5230\u540c\u4e00\u7f51\u5361\u540c\u4e00 IP |<\/p>\n
| 6 | `assert "mtp.0." in name` | \u91cf\u5316\u6743\u91cd\u65e0 MTP \u5c42 | \u5220 `–speculative-config` \u6216\u6362\u6743\u91cd |<\/p>\n<\/p>\n
## \u4e94\u3001\u51e0\u6761\u771f\u6b63\u7701\u65f6\u95f4\u7684\u7ecf\u9a8c<\/p>\n<\/p>\n
1. **\u5148\u627e rank 0**\u3002`DP0 \/ Device:0` \u7684 traceback \u624d\u662f\u6839\u56e0，`EP2~EP7` \u7684\u201c\u8fde\u63a5\u65ad\u5f00\u201d\u90fd\u662f\u966a\u846c\u3002<\/p>\n
2. **\u65e5\u5fd7\u522b\u53ea\u770b\u5c3e\u90e8**\u3002\u7528 `grep -n -m1 -B80 -A40` \u628a\u4e0a\u4e0b\u6587\u4e00\u8d77\u635e\u51fa\u6765，\u5426\u5219\u4f1a\u5f97\u51fa\u5b8c\u5168\u76f8\u53cd\u7684\u7ed3\u8bba\u3002<\/p>\n
3. **`hcclGetRootInfo` \u662f\u672c\u5730\u8c03\u7528**\u3002\u5b83\u5931\u8d25\u5c31\u53bb\u67e5\u672c\u673a\u7f51\u5361\/IP，\u522b\u5f80\u8de8\u673a\u7f51\u7edc\u4e0a\u60f3\u3002<\/p>\n
4. **\u914d\u7f6e\u53ea\u7559\u4e00\u5904\u59fb\u6e90**\u3002\u5bb9\u5668\u6ce8\u5165\u4e86\u5c31\u522b\u5728\u811a\u672c\u91cc\u518d\u8986\u76d6\u4e00\u534a\u2014\u2014\u201c\u534a\u8986\u76d6\u201d\u662f\u6700\u96be\u67e5\u7684\u5f62\u6001\u3002<\/p>\n
5. **\u522b\u7528 3 \u5206\u949f\u7684\u670d\u52a1\u542f\u52a8\u53bb\u9a8c\u8bc1\u4e00\u4e2a 5 \u79d2\u5c31\u80fd\u9a8c\u8bc1\u7684\u914d\u7f6e**\u3002HCCL \u63a2\u9488\u811a\u672c\u80fd\u7701\u6389\u5927\u91cf\u7b49\u5f85\u3002<\/p>\n
6. **\u62a5\u9519\u4f1a\u201c\u6392\u961f\u6d6e\u51fa\u201d**\u3002\u4fee\u6389\u4e00\u4e2a（\u5982 bool \u7c7b\u578b）\u4e4b\u540e，\u4e0b\u4e00\u4e2a\u51b2\u7a81\u624d\u4f1a\u66b4\u9732（\u5982 prefetch \u4e92\u65a5）\u3002\u4e00\u6b21\u53ea\u6539\u4e00\u5904\u3001\u53ea\u91cd\u8dd1\u4e00\u8f6e，\u6bd4\u4e00\u6b21\u6539\u4e94\u5904\u9ad8\u6548\u5f97\u591a\u3002<\/p>\n
—<\/p>\n<\/p>\n
**\u4e0b\u4e00\u7bc7\u9884\u544a**：PD \u5206\u79bb\u771f\u6b63\u8dd1\u8d77\u6765\u4e4b\u540e，\u8fd8\u6709\u66f4\u786c\u7684\u4e09\u5173\u2014\u2014Mooncake KV \u8de8\u673a\u4f20\u8f93\u7684 RoCE \u9009\u5361\u3001\u4ee3\u7406\u5c42\u7684\u5e76\u53d1\u5206\u53d1\u4e0e\u538b\u6d4b\u3001\u4ee5\u53ca MTP \u5728\u6607\u817e\u4e0a\u7684\u6b63\u786e\u6253\u5f00\u65b9\u5f0f\u3002\u611f\u5174\u8da3\u53ef\u4ee5\u5148\u5173\u6ce8\u3002<\/p><\/p>\n","protected":false},"excerpt":{"rendered":"
# DeepSeek-V4-Flash \u6607\u817e 910B PD \u5206\u79bb\u90e8\u7f72：8 \u5361\u5168\u6302\u4e4b\u540e，\u6211\u7528\u201c\u627e rank 0\u201d\u5b9a\u4f4d\u4e86 6 \u4e2a\u8fde\u73af\u5751> \u4e24\u53f0 Atlas 910B-4\u30018 \u5f20\u5361\u3001vLLM-Ascend v0.30.0rc2\u30011P1D \u5206\u79bb\u90e8\u7f72\u3002> \u65e5\u5fd7\u91cc 6 \u4e2a rank \u9f50\u58f0\u558a\u201c\u8fde\u63a5\u5df2\u65ad\u5f00\u201d，\u4f46**\u771f\u6b63\u7684\u539f\u56e0\u53ea\u6709\u4e00\u4e2a rank**\u3002> \u8fd9\u7bc7\u6587\u7ae0\u628a 6 \u4e2a\u5751\u548c\u4e00\u5957\u6392\u67e5\u65b9\u6cd5\u8bba\u4e00\u6b21\u8bb2\u6e05\u3002>> \u8bf4\u660e：\u6587\u4e2d IP \u5730\u5740\u3001\u7f51\u5361\u540d\u3001<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[14020,11820,347,69,2681],"topic":[],"class_list":["post-122771","post","type-post","status-publish","format-standard","hentry","category-server","tag-ascend","tag-atomgit","tag-llama","tag-npu","tag-vllm"],"yoast_head":"\n