PD分离部署：基于Atlas 800T A3服务器+SGLang 1P2D部署Qwen3-235B指南

作者：昇腾实战派 Sglang知识地图： https://blog.csdn.net/weixin_41406651/article/details/156754353?spm=1001.2014.3001.5502

背景概述

Qwen3-235B模型用Atlas 800I A3或Atlas 800T A3均可部署，本文档以Atlas 800T A3为例，记录了使用SGLang框架部署Qwen3-235B模型的1P2D（一个Prefill节点和两个Decode节点）配置方案。该配置针对大规模语言模型推理场景进行了优化，特别适用于需要高吞吐量的生产环境。

1. 版本与环境配置

• 机器型号： Atlas 800T A3 * Cann版本 8.2.RC1
• Sglang分支（此分支有别于主干，是开发中的分支暂未合入 & 只有对qwen系列的优化）git clone https://github.com/ping1jing2/sglang.git
  cd sglang
  git checkout -b main_qwen origin/main_qwen​
• Docker镜像版本：# docker.io
  docker pull lmsysorg/sglang:main-cann8.3.rc1-910b # a2
  docker pull lmsysorg/sglang:main-cann8.3.rc1-a3 # a3
  # 国内镜像站
  docker pull swr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/sglang:main-cann8.3.rc1-910b # a2
  docker pull swr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/sglang:main-cann8.3.rc1-a3 # a3
  ​
• triton-ascend & torch_npu更新：https://wiki.huawei.com/domains/119266/wiki/243400/WIKI202511259151404?title=_fde87425
• sgl-kernel-npu需要手动编译git clone https://github.com/sgl-project/sgl-kernel-npu
  cd sgl-kernel-npu
  bash build.sh
  pip install output/sgl_kernel_npu*.whl​
• set_env.sh 注意：启动router时，操作前手动 source source /usr/local/Ascend/ascend-toolkit/set_env.shexport LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64/common:/usr/local/Ascend/driver/lib64/driver:$LD_LIBRARY_PATH
  export ASCEND_TOOLKIT_HOME=/usr/local/Ascend/ascend-toolkit/latest
  export LD_LIBRARY_PATH=${ASCEND_TOOLKIT_HOME}/lib64:${ASCEND_TOOLKIT_HOME}/lib64/plugin/opskernel:${ASCEND_TOOLKIT_HOME}/lib64/plugin/nnengine:${ASCEND_TOOLKIT_HOME}/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/$(arch):$LD_LIBRARY_PATH
  export LD_LIBRARY_PATH=${ASCEND_TOOLKIT_HOME}/tools/aml/lib64:${ASCEND_TOOLKIT_HOME}/tools/aml/lib64/plugin:$LD_LIBRARY_PATH
  export PYTHONPATH=${ASCEND_TOOLKIT_HOME}/python/site-packages:${ASCEND_TOOLKIT_HOME}/opp/built-in/op_impl/ai_core/tbe:$PYTHONPATH
  export PATH=${ASCEND_TOOLKIT_HOME}/bin:${ASCEND_TOOLKIT_HOME}/compiler/ccec_compiler/bin:${ASCEND_TOOLKIT_HOME}/tools/ccec_compiler/bin:$PATH
  export ASCEND_AICPU_PATH=${ASCEND_TOOLKIT_HOME}
  export ASCEND_OPP_PATH=${ASCEND_TOOLKIT_HOME}/opp
  export TOOLCHAIN_HOME=${ASCEND_TOOLKIT_HOME}/toolkit
  export ASCEND_HOME_PATH=${ASCEND_TOOLKIT_HOME}​

2 服务启动脚本

服务启动配置

以下脚本支持P节点、D节点和sgl-router共用，只需修改P节点、D节点相关参数即可。

# 235b_run.sh
# docker exec -it sglang_perf_b150 bash
pkill -9 python | pkill -9 sglang
pkill -9 sglang
echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
sysctl -w vm.swappiness=0
sysctl -w kernel.numa_balancing=0
sysctl -w kernel.sched_migration_cost_ns=50000

export SGLANG_SET_CPU_AFFINITY=1
# 设置PYTHONPATH

cd /home/sglang_ascend
export PYTHONPATH=${PWD}/python:$PYTHONPATH
unset https_proxy
unset http_proxy
unset HTTPS_PROXY
unset HTTP_PROXY
unset ASCEND_LAUNCH_BLOCKING

source /usr/local/Ascend/ascend-toolkit/latest/opp/vendors/customize/bin/set_env.bash
#export SGLANG_EXPERT_DISTRIBUTION_RECORDER_DIR=/home///sglang_ascend_1111/sglang_ascend/hot_map
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True

export SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=16

MODEL_PATH=/mnt/share/weights/Qwen3-235B-A22B-W8A8
# pd传输, IP设置为p节点首节点
export ASCEND_MF_STORE_URL="tcp://x.x.x.x:24667"
# p节点IP
P_IP=('x.x.x.x')
# D节点IP
#D_IP=('x.x.x.x')
D_IP=('x.x.x.x' 'x.x.x.x')
#export SGLANG_ENABLE_TORCH_COMPILE=1
export SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT=600

LOCAL_HOST1=`hostname -I|awk -F " " '{print$1}'`
LOCAL_HOST2=`hostname -I|awk -F " " '{print$2}'`

echo "${LOCAL_HOST1}"
echo "${LOCAL_HOST2}"

for i in "${!P_IP[@]}";
do
if [[ "$LOCAL_HOST1" == "${P_IP[$i]}" || "$LOCAL_HOST2" == "${P_IP[$i]}" ]];
then
echo "${P_IP[$i]}"
source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
export HCCL_BUFFSIZE=3000
export TASK_QUEUE_ENABLE=2
export HCCL_SOCKET_IFNAME=lo
export GLOO_SOCKET_IFNAME=lo
export STREAMS_PER_DEVICE=32
export ENABLE_ASCEND_MOE_NZ=1
export DEEP_NORMAL_MODE_USE_INT8_QUANT=1
# export ENABLE_PROFILING=1

# P节点
python -m sglang.launch_server –model-path ${MODEL_PATH} –disaggregation-mode prefill \\
–host ${P_IP[$i]} –port 8000 –disaggregation-bootstrap-port 8995 –trust-remote-code \\
–nnodes 1 –node-rank $i –tp-size 16 –dp-size 8 –mem-fraction-static 0.6 \\
–disable-radix-cache \\
–ep-dispatch-algorithm static –init-expert-location /homesglang_ascend/hot_map/expert_distribution_recorder_1763480391.7582676.pt \\
–attention-backend ascend –device npu –quantization w8a8_int8 –disaggregation-transfer-backend ascend \\
–max-running-requests 128 –chunked-prefill-size 114688 –max-prefill-tokens 458880 \\
–disable-overlap-schedule –enable-dp-attention –tokenizer-worker-num 4 \\
–moe-a2a-backend deepep –deepep-mode normal –dtype bfloat16
NODE_RANK=$i
break
fi
done

for i in "${!D_IP[@]}";
do
if [[ "$LOCAL_HOST1" == "${D_IP[$i]}" || "$LOCAL_HOST2" == "${D_IP[$i]}" ]];
then
echo "${D_IP[$i]}"
source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
export DP_ROUND_ROBIN=1
export SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=60
export HCCL_BUFFSIZE=512
export HCCL_SOCKET_IFNAME=data0.3001
export GLOO_SOCKET_IFNAME=data0.3001
export STREAMS_PER_DEVICE=32

# export ENABLE_ASCEND_MOE_NZ=1
# export ENABLE_PROFILING=1
# D节点
python -m sglang.launch_server –model-path ${MODEL_PATH} –disaggregation-mode decode \\
–host ${D_IP[$i]} –port 8001 –trust-remote-code \\
–nnodes 2 –node-rank $i –tp-size 32 –dp-size 16 –mem-fraction-static 0.83 –max-running-requests 960 \\
–attention-backend ascend –device npu –quantization w8a8_int8 –enable-dp-attention \\
–moe-a2a-backend ascend_fuseep –cuda-graph-bs 6 8 12 15 18 20 22 24 30 36 42 48 54 56 58 60 \\
–dist-init-addr 172.27.1.143:5000 \\
–disaggregation-transfer-backend ascend –watchdog-timeout 9000 –context-length 8192 \\
–prefill-round-robin-balance –enable-dp-lm-head –tokenizer-worker-num 4 –dtype bfloat16
NODE_RANK=$i
break
fi
done

exit 1
python -m sglang_router.launch_router \\
–pd-disaggregation \\
–policy cache_aware \\
–prefill http://x.x.x.x:8000 8995 \\
–decode http://x.x.x.x:8001 \\
–host 127.0.0.1 \\
–port 6688 \\
–mini-lb
​

3 ais_bench 配置

3.1 修改测试参数

调整Python测试代码配置：

vim ais_bench/benchmark/configs/models/vllm_api/vllm_api_stream_chat.py

# vllm_api_stream_chat.py
from ais_bench.benchmark.models import VLLMCustomAPIChatStream
from ais_bench.benchmark.utils.model_postprocessors import extract_non_reasoning_content

models = [
dict(
attr="service",
type=VLLMCustomAPIChatStream,
abbr='vllm-api-stream-chat',
path="/mnt/share/weights/Qwen3-235B-A22B-W8A8",
model="Qwen3",
request_rate = 10.5,
retry = 2,
host_ip = "127.0.0.1",
host_port = 6688,
max_out_len = 1500,
batch_size = 960, #880
trust_remote_code=False,
generation_kwargs = dict(
temperature = 0,
ignore_eos = True,
),
pred_postprocessor=dict(type=extract_non_reasoning_content)
)
]

3.2 执行测试命令

ais_bench –models vllm_api_stream_chat –datasets gsm8k_gen_0_shot_cot_str_perf –debug –summarizer default_perf –mode perf –num-prompts 4240