> \u90e8\u7f72\u76ee\u6807：\u5728\u4e00\u53f0 8 \u5361 RTX 5090（\u6bcf\u5361 32 GB）\u670d\u52a1\u5668\u4e0a，\u7528 vLLM \u8dd1\u8d77 GLM-5.3-Flash \u7684 NVFP4 \u91cf\u5316\u7248，\u5bf9\u5916\u63d0\u4f9b OpenAI \u517c\u5bb9 API，\u652f\u6301 `\/think` \u6df1\u5ea6\u601d\u8003\u6a21\u5f0f\u4e0e Function Call\u3002\u6700\u7ec8\u65b9\u6848\u91c7\u7528 **vLLM \u5b98\u65b9 nightly \u955c\u50cf + Docker**，\u670d\u52a1\u73b0\u7a33\u5b9a\u8fd0\u884c\u4e8e `http:\/\/10.168.2.103:8000\/v1`（\u5bb9\u5668\u5df2\u8fde\u7eed\u8fd0\u884c 9 \u5929+）\u3002<\/p>\n<\/p>\n
## \u4e00\u3001\u4e3a\u4ec0\u4e48\u662f\u8fd9\u5957\u7ec4\u5408<\/p>\n<\/p>\n
**\u6a21\u578b**：`GLM-5.3-Flash-NVFP4`，\u67b6\u6784\u4e3a `Glm5NextForConditionalGeneration`（`model_type: glm5_next`），\u539f\u59cb dtype bfloat16\u3002NVFP4 \u91cf\u5316\u540e\u6743\u91cd\u843d\u76d8 **185 GB**\u2014\u2014\u5168\u91cf bf16 \u663e\u7136\u653e\u4e0d\u8fdb\u4efb\u4f55\u5355\u673a，\u5373\u4fbf 4bit \u4e5f\u8fdc\u8d85\u5355\u5361，\u5fc5\u987b\u8d70\u5f20\u91cf\u5e76\u884c\u3002<\/p>\n<\/p>\n
**\u786c\u4ef6**：8 \u00d7 RTX 5090 32G，\u603b\u663e\u5b58\u7ea6 **255.6 GB**\u30025090 \u662f Blackwell \u67b6\u6784（SM120），\u7b97\u529b\u65b0\u3001\u663e\u5b58\u5927，\u4f46\u5bf9\u6846\u67b6\u7248\u672c\u8981\u6c42\u82db\u523b：\u7a33\u5b9a\u7248 vLLM \u5c1a\u672a\u5408\u5165 `glm5_next` \u67b6\u6784\u652f\u6301 + NVFP4 \u53cd\u91cf\u5316 kernel + SM120 \u540e\u7aef\u4e09\u4ef6\u5957，**\u5fc5\u987b\u7528 nightly \u6784\u5efa\u7684 vLLM + CUDA 13.0**\u3002<\/p>\n<\/p>\n
**\u91cf\u5316\u683c\u5f0f**（\u6765\u81ea `config.json` \u7684 `quantization_config`）：<\/p>\n<\/p>\n
```json<\/p>\n
"format": "nvfp4-pack-quantized",<\/p>\n
"input_activations": {<\/p>\n
\u00a0 "num_bits": 4,<\/p>\n
\u00a0 "group_size": 16,<\/p>\n
\u00a0 "scale_dtype": "torch.float8_e4m3fn",<\/p>\n
\u00a0 "strategy": "tensor_group",<\/p>\n
\u00a0 "dynamic": "local"<\/p>\n
}<\/p>\n
```<\/p>\n<\/p>\n
\u6ce8\u610f targets \u53ea\u8986\u76d6\u4e86 MoE \u7684 experts \u5c42（`.*\\\\.layers\\\\.(?:[3-9]|[1-3][0-9]|4[0-4])\\\\.mlp\\\\.experts\\\\..*(gate|up|down)`）\u2014\u2014**\u4e13\u5bb6\u77e9\u9635\u8d70 4bit，\u6ce8\u610f\u529b\u4e0e\u5171\u4eab\u5c42\u4fdd\u6301 bf16**，\u8fd9\u662f\u7cbe\u5ea6\u4e0e\u5e26\u5bbd\u7684\u7ecf\u5178\u6298\u4e2d\u3002vLLM \u901a\u8fc7 `compressed-tensors` \u53cd\u91cf\u5316\u5668\u539f\u751f\u8bc6\u522b，\u65e0\u9700\u624b\u52a8\u6307\u5b9a `–quantization`\u3002<\/p>\n<\/p>\n
## \u4e8c\u3001\u8f6f\u4ef6\u6808：\u5b98\u65b9 nightly \u955c\u50cf + Docker<\/p>\n<\/p>\n
\u5bbf\u4e3b\u673a\u4e0a\u5148\u8bd5\u8fc7 conda \u73af\u5883\u88f8\u8dd1（conda env `glm53`，torch 2.13.0+cu130），\u80fd\u8dd1\u901a\u4f46\u7248\u672c\u7ba1\u7406\u3001\u4f9d\u8d56\u9694\u79bb\u90fd\u4e0d\u5e72\u51c0，**\u6700\u7ec8\u751f\u4ea7\u65b9\u6848\u5207\u6362\u5230\u5b98\u65b9 vLLM \u955c\u50cf**：<\/p>\n<\/p>\n
| \u7ec4\u4ef6 | \u7248\u672c\/\u6765\u6e90 |<\/p>\n
|—|—|<\/p>\n
| \u955c\u50cf | `vllm\/vllm-openai:nightly-385dce36b`（\u5b98\u65b9 Buildkite \u6d41\u6c34\u7ebf\u6784\u5efa #6303） |<\/p>\n
| vLLM | 0.28.1rc1.dev580+g385dce36b（nightly，commit 385dce36b） |<\/p>\n
| \u57fa\u7840\u955c\u50cf CUDA | 13.0.2（\u8981\u6c42 NVIDIA \u9a71\u52a8 \u2265 535） |<\/p>\n
| \u8fd0\u884c\u65b9\u5f0f | Docker \u5bb9\u5668 `glm53-flash`，`–gpus all`，`–restart unless-stopped` |<\/p>\n
| \u6a21\u578b\u6302\u8f7d | `\/home\/x640\/models` \u2192 \u5bb9\u5668\u5185 `\/models` |<\/p>\n<\/p>\n
\u9009\u5b98\u65b9 nightly \u955c\u50cf\u800c\u4e0d\u662f\u81ea\u5efa\u955c\u50cf，\u6838\u5fc3\u539f\u56e0\u662f SM120 + NVFP4 \u7684 kernel \u7ec4\u5408\u5728 nightly \u91cc\u662f\u9f50\u7684，`ENTRYPOINT ["vllm" "serve"]`，\u6a21\u578b\u8def\u5f84\u901a\u8fc7\u6302\u8f7d\u4f20\u53c2\u5373\u53ef，\u96f6\u989d\u5916\u6253\u5305\u5de5\u4f5c\u3002<\/p>\n<\/p>\n
## \u4e09\u3001\u542f\u52a8\u547d\u4ee4（\u53ef\u76f4\u63a5\u6284）<\/p>\n<\/p>\n
\u4ee5\u4e0b `docker run` \u4f9d\u636e\u670d\u52a1\u5668\u4e0a\u5b9e\u9645\u5bb9\u5668\u7684 `docker inspect` \u7ed3\u679c\u8fd8\u539f（\u955c\u50cf\u5728\u670d\u52a1\u5668\u4e0a\u4ee5 sha \u5f15\u7528，label \u4e2d\u6807\u6ce8\u4e86 upstream tag）：<\/p>\n<\/p>\n
```bash<\/p>\n
docker run -d –gpus all \\\\<\/p>\n
\u00a0 –name glm53-flash \\\\<\/p>\n
\u00a0 –restart unless-stopped \\\\<\/p>\n
\u00a0 -p 8000:8000 \\\\<\/p>\n
\u00a0 -v \/home\/x640\/models:\/models \\\\<\/p>\n
\u00a0 -v \/home\/x640\/glm53_site:\/site \\\\<\/p>\n
\u00a0 -e VLLM_GLM53_CUDA_SPARSE_MLA=1 \\\\<\/p>\n
\u00a0 -e VLLM_GLM53_MOE_INPUT_SCALE=1.0 \\\\<\/p>\n
\u00a0 -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \\\\<\/p>\n
\u00a0 -e VLLM_USE_BREAKABLE_CUDAGRAPH=1 \\\\<\/p>\n
\u00a0 -e NCCL_MIN_NCHANNELS=32 \\\\<\/p>\n
\u00a0 -e NCCL_P2P_LEVEL=PXB \\\\<\/p>\n
\u00a0 vllm\/vllm-openai:nightly-385dce36b \\\\<\/p>\n
\u00a0 \/models\/GLM-5.3-Flash-NVFP4 \\\\<\/p>\n
\u00a0 –served-model-name glm-5.3-flash \\\\<\/p>\n
\u00a0 –tensor-parallel-size 8 \\\\<\/p>\n
\u00a0 –kv-cache-dtype fp8 \\\\<\/p>\n
\u00a0 –block-size 256 \\\\<\/p>\n
\u00a0 –max-model-len 131072 \\\\<\/p>\n
\u00a0 –max-num-seqs 4 \\\\<\/p>\n
\u00a0 –max-num-batched-tokens 512 \\\\<\/p>\n
\u00a0 –gpu-memory-utilization 0.82 \\\\<\/p>\n
\u00a0 –moe-backend flashinfer_cutlass \\\\<\/p>\n
\u00a0 –no-enable-flashinfer-autotune \\\\<\/p>\n
\u00a0 –tool-call-parser glm47 \\\\<\/p>\n
\u00a0 –enable-auto-tool-choice \\\\<\/p>\n
\u00a0 –reasoning-parser glm45 \\\\<\/p>\n
\u00a0 –trust-remote-code \\\\<\/p>\n
\u00a0 –enforce-eager<\/p>\n
```<\/p>\n<\/p>\n
### \u670d\u52a1\u53c2\u6570\u9010\u6761\u89e3\u91ca<\/p>\n<\/p>\n
– **`–tensor-parallel-size 8`**：8 \u5361\u4e00\u5f20\u5e76\u884c\u9762，NVFP4 \u6743\u91cd\u5206\u7247\u540e\u6bcf\u5361\u7ea6 23 GB\u3002<\/p>\n
– **`–gpu-memory-utilization 0.82`**：\u6bd4\u5e38\u89c1\u7684 0.9 \u4fdd\u5b88\u2014\u2014SM120 \u4e0a CUDA context\u3001NCCL buffer \u7684\u989d\u5916\u5f00\u9500\u66f4\u5927，0.82 \u662f\u5b9e\u6d4b\u7a33\u5b9a\u7684\u5206\u754c\u7ebf（\u5bb9\u5668\u89c6\u89d2\u7684\u6a21\u578b\/KV \u9884\u7b97；\u5bbf\u4e3b\u673a nvidia-smi \u5b9e\u9645\u6bcf\u5361 ~30.3\/32.6 GB，\u542b\u9a71\u52a8\u4fa7\u5360\u7528）\u3002<\/p>\n
– **`–max-model-len 131072`**：128K \u4e0a\u4e0b\u6587，\u53d6\u6a21\u578b config \u7684\u4e0a\u9650\u3002<\/p>\n
– **`–kv-cache-dtype fp8`**：KV cache \u4e5f\u538b\u6210 FP8，\u76f4\u63a5\u628a\u53ef\u670d\u52a1\u7684 KV token \u6570\u7ffb\u500d\u3002<\/p>\n
– **`–block-size 256`**：\u5927 block \u51cf\u5c11 KV \u7ba1\u7406\u5f00\u9500，\u914d\u5408 MLA \u7c7b\u67b6\u6784\u4f7f\u7528\u3002<\/p>\n
– **`–max-num-seqs 4` + `–max-num-batched-tokens 512`**：\u523b\u610f\u6536\u7a84\u7684\u8c03\u5ea6\u6c34\u4f4d\u3002256 GB \u663e\u5b58\u770b\u7740\u591a，128K \u5355\u8bf7\u6c42\u7684 KV \u5c31\u8981\u5403\u6389\u4e00\u5927\u5757，\u5c0f batch + \u5c0f prefill chunk \u4fdd\u8bc1\u957f\u8bf7\u6c42\u4e0b\u4e0d OOM\u3001\u65f6\u5ef6\u53ef\u9884\u671f\u3002<\/p>\n
– **`–moe-backend flashinfer_cutlass`**：MoE \u8d70 FlashInfer \u7684 CUTLASS \u8def\u5f84，\u4e0e NVFP4 \u53cd\u91cf\u5316\u914d\u5408\u6700\u4f18\u3002<\/p>\n
– **`–no-enable-flashinfer-autotune`**：\u5173\u6389 FlashInfer \u81ea\u52a8\u8c03\u4f18，\u7701\u4e0b\u6bcf\u6b21\u542f\u52a8\u5341\u51e0\u5206\u949f\u7684 tune \u65f6\u95f4，\u6536\u76ca\u5fae\u4e4e\u5176\u5fae\u3002<\/p>\n
– **`–tool-call-parser glm47` + `–enable-auto-tool-choice` + `–reasoning-parser glm45`**：GLM \u7cfb\u5217\u7684\u601d\u8003\u5185\u5bb9\u4e0e\u5de5\u5177\u8c03\u7528\u89e3\u6790\u5668\u3002\u6ca1\u8fd9\u4e24\u4e2a，\u524d\u7aef\u62ff\u5230\u7684\u5c06\u662f\u539f\u59cb `<think>` \u6807\u7b7e\u548c\u6587\u672c\u5f62\u5f0f\u7684\u5de5\u5177\u8c03\u7528，\u65e0\u6cd5\u7ed3\u6784\u5316\u62c6\u5206\u3002<\/p>\n
– **`–trust-remote-code`**：`glm5_next` \u67b6\u6784\u9700\u8981\u6a21\u578b\u4ed3\u5e93\u5185\u7684\u81ea\u5b9a\u4e49\u4ee3\u7801\u3002<\/p>\n
– **`–enforce-eager`**：\u5173\u95ed CUDA Graph \u6355\u83b7\u3002\u4ee3\u4ef7\u662f decode \u7a0d\u6162，\u6362\u6765\u7684\u662f\u663e\u5b58\u4f59\u91cf\u548c\u542f\u52a8\u7a33\u5b9a\u6027\u2014\u2014\u5728 0.82 \u6c34\u4f4d + fp8 KV \u7684\u7ec4\u5408\u4e0b\u8fd9\u662f\u5fc5\u8981\u7684\u53d6\u820d\u3002<\/p>\n<\/p>\n
### \u5bb9\u5668\u73af\u5883\u53d8\u91cf，\u6bcf\u4e2a\u90fd\u662f\u8e29\u5751\u6362\u6765\u7684<\/p>\n<\/p>\n
1. **`VLLM_ENGINE_READY_TIMEOUT_S=3600`**\u2014\u2014\u9ed8\u8ba4\u7684\u5f15\u64ce\u5c31\u7eea\u8d85\u65f6\u8fdc\u5c0f\u4e8e 185 GB \u6743\u91cd\u7684\u52a0\u8f7d\u65f6\u95f4（\u5b9e\u6d4b **632 \u79d2**），vLLM \u4f1a"\u8bef\u5224\u6b7b\u4ea1"\u63d0\u524d\u9000\u51fa\u3002\u653e\u5bbd\u5230 1 \u5c0f\u65f6\u3002<\/p>\n
2. **`VLLM_GLM53_CUDA_SPARSE_MLA=1`**\u2014\u2014\u6253\u5f00 GLM-5.3 \u7684\u7a00\u758f MLA \u6ce8\u610f\u529b\u8def\u5f84（SM120 \u4e13\u7528 kernel），\u8fd9\u662f\u957f\u4e0a\u4e0b\u6587\u541e\u5410\u7684\u5173\u952e\u5f00\u5173\u3002<\/p>\n
3. **`VLLM_GLM53_MOE_INPUT_SCALE=1.0`**\u2014\u2014MoE \u4e13\u5bb6\u5c42 NVFP4 \u8f93\u5165 scale \u7684\u5bf9\u9f50\u53c2\u6570\u3002<\/p>\n
4. **`VLLM_USE_BREAKABLE_CUDAGRAPH=1`**\u2014\u2014\u5141\u8bb8 vLLM \u5728\u663e\u5b58\u5403\u7d27\u65f6\u653e\u5f03\/\u5206\u6bb5 CUDA Graph \u800c\u4e0d\u662f\u5d29\u6e83\u3002<\/p>\n
5. **`NCCL_MIN_NCHANNELS=32` + `NCCL_P2P_LEVEL=PXB`**\u2014\u20148 \u5361 TP \u7684 all-reduce \u901a\u9053\u6570\u4e0e P2P \u62d3\u6251\u7ea7\u522b（\u540c PCIe switch \u5185\u4f18\u5148\u8d70 PXB），\u4e3a 5090 \u8fd9\u7c7b\u6d88\u8d39\u5361\u7684 P2P \u94fe\u8def\u5fae\u8c03\u3002<\/p>\n<\/p>\n
## \u56db\u3001\u542f\u52a8\u8fc7\u7a0b\u5b9e\u5f55<\/p>\n<\/p>\n
\u4ece\u5bb9\u5668\u65e5\u5fd7（`docker logs glm53-flash`，2026-09-21 12:47:16 \u8d77\u6b65）：<\/p>\n<\/p>\n
```<\/p>\n
12:47:16 \u00a0vLLM API server \u542f\u52a8\u6a2a\u5e45 (version 0.28.1rc1.dev580+g385dce36b)<\/p>\n
12:58:36 \u00a0Loading weights took 632.80 seconds<\/p>\n
12:59:37 \u00a0GPU KV cache size: 149,796 tokens<\/p>\n
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Maximum concurrency for 131,072 tokens per request: 1.14x<\/p>\n
```<\/p>\n<\/p>\n
\u51e0\u4e2a\u503c\u5f97\u6ce8\u610f\u7684\u70b9：<\/p>\n<\/p>\n
– **\u6743\u91cd\u52a0\u8f7d 10 \u5206\u534a**\u662f 185 GB \u4ece\u78c1\u76d8\u704c\u8fdb 8 \u5361\u7684\u7269\u7406\u6781\u9650，\u671f\u95f4\u65e5\u5fd7\u9759\u9ed8，\u522b\u614c，\u66f4\u522b\u63d0\u524d kill\u3002<\/p>\n
– **KV cache 149,796 tokens\u3001131K \u5355\u8bf7\u6c42\u5e76\u53d1 1.14x**：\u8fd9\u662f\u6700\u8bda\u5b9e\u7684\u5bb9\u91cf\u6570\u5b57\u2014\u2014fp8 KV + 0.82 \u6c34\u4f4d\u4e0b，\u7cfb\u7edf\u540c\u65f6\u585e\u5f97\u4e0b\u7ea6\u4e00\u4e2a\u534a\u6ee1\u957f 128K \u8bf7\u6c42\u3002\u65e5\u5e38\u77ed\u5bf9\u8bdd\u7684\u5e76\u53d1\u8fdc\u4e0d\u6b62\u8fd9\u4e2a\u6570（\u77ed\u8bf7\u6c42\u7684 KV \u5360\u7528\u6309\u6bd4\u4f8b\u7f29\u5c0f），\u4f46**\u957f\u6587\u6863\u573a\u666f\u5fc5\u987b\u6309 1~2 \u4e2a\u5e76\u53d1\u6765\u89c4\u5212\u4e1a\u52a1**\u3002<\/p>\n
– \u6a21\u578b\u5728\u5bb9\u5668\u5185\u4ee5 `\/models\/GLM-5.3-Flash-NVFP4` \u8def\u5f84\u52a0\u8f7d（\u5bbf\u4e3b\u673a\u6302\u8f7d\u8fdb\u6765\u7684）\u3002<\/p>\n<\/p>\n
## \u4e94\u3001\u9a8c\u6536：API \u5b9e\u6d4b<\/p>\n<\/p>\n
### 1. \u6a21\u578b\u6ce8\u518c<\/p>\n<\/p>\n
```bash<\/p>\n
$ curl http:\/\/10.168.2.103:8000\/v1\/models<\/p>\n
{"id":"glm-5.3-flash","root":"\/models\/GLM-5.3-Flash-NVFP4",<\/p>\n
\u00a0"max_model_len":131072, …}<\/p>\n
```<\/p>\n<\/p>\n
### 2. \u601d\u8003\u6a21\u5f0f\u62c6\u5206（reasoning parser \u751f\u6548）<\/p>\n<\/p>\n
```bash<\/p>\n
$ curl -s http:\/\/10.168.2.103:8000\/v1\/chat\/completions \\\\<\/p>\n
\u00a0 \u00a0 -H "Content-Type: application\/json" \\\\<\/p>\n
\u00a0 \u00a0 -d '{"model":"glm-5.3-flash",<\/p>\n
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0"messages":[{"role":"user","content":"What is 17*23?"}],<\/p>\n
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0"max_tokens":256}'<\/p>\n
```<\/p>\n<\/p>\n
\u8fd4\u56de：<\/p>\n<\/p>\n
– `message.content` \u2192 `"391"`（\u7b54\u6848\u6b63\u786e）<\/p>\n
– `message.reasoning_content` \u2192 \u601d\u8003\u8fc7\u7a0b\u72ec\u7acb\u6210\u5b57\u6bb5，\u4e0e\u6b63\u6587\u5e72\u51c0\u5206\u79bb<\/p>\n
– `usage` \u2192 prompt 25 tokens \/ completion 47 tokens，\u5176\u4e2d `reasoning_tokens: 43`<\/p>\n<\/p>\n
### 3. \u663e\u5b58\u6c34\u4f4d<\/p>\n<\/p>\n
```<\/p>\n
\u6bcf\u5361：~30.3 GiB \/ 32.6 GiB（\u7ea6 93%，\u542b CUDA context \u4e0e NCCL \u5f00\u9500）<\/p>\n
```<\/p>\n<\/p>\n
## \u516d\u3001\u5ba2\u6237\u7aef\u5bf9\u63a5\u6ce8\u610f\u4e8b\u9879<\/p>\n<\/p>\n
– **base URL**：`http:\/\/10.168.2.103:8000\/v1`，\u6a21\u578b\u540d\u56fa\u5b9a `glm-5.3-flash`（\u6ce8\u610f\u4e0d\u662f\u78c1\u76d8\u76ee\u5f55\u540d）\u3002<\/p>\n
– **\u601d\u8003\u5185\u5bb9**：\u9700\u8981\u6df1\u5ea6\u601d\u8003\u65f6\u5728 user \u6d88\u606f\u5f00\u5934\u52a0 `\/think`，\u7ed3\u679c\u8bfb `reasoning_content` \u5b57\u6bb5；\u4e0d\u8981\u81ea\u5df1\u53bb\u89e3\u6790 `<think>` \u6807\u7b7e\u3002<\/p>\n
– **\u5de5\u5177\u8c03\u7528**：OpenAI tools \u534f\u8bae\u76f4\u63a5\u53ef\u7528（`–enable-auto-tool-choice` \u5df2\u5f00）\u3002<\/p>\n
– **\u957f\u4e0a\u4e0b\u6587\u5e76\u53d1**：\u5982\u4e0a\u6587 KV cache \u6570\u636e，131K \u7ea7\u522b\u8bf7\u6c42\u7684\u7a33\u6001\u5e76\u53d1\u7ea6 1.14x，\u6279\u91cf\u957f\u6587\u6458\u8981\u52a1\u5fc5\u4e32\u884c\u6216\u5206\u6279\u3002<\/p>\n
– **Windows \u4e0b curl \u53d1\u4e2d\u6587 body \u6709\u7f16\u7801\u5751**：Git Bash \u4f1a\u628a UTF-8 \u8f6c GBK \u5bfc\u81f4 `There was an error parsing the body`\u3002\u8981\u4e48\u7528\u7eaf ASCII \u6d4b\u8bd5\u4f53，\u8981\u4e48\u8d70 Python\/Postman \u53d1\u8bf7\u6c42\u3002<\/p>\n<\/p>\n
## \u4e03\u3001\u603b\u7ed3<\/p>\n<\/p>\n
| \u9879\u76ee | \u6570\u503c |<\/p>\n
|—|—|<\/p>\n
| \u90e8\u7f72\u5f62\u6001 | Docker（vllm\/vllm-openai nightly-385dce36b，CUDA 13.0.2） |<\/p>\n
| \u603b\u663e\u5b58 \/ \u5b9e\u9645\u5360\u7528 | 255.6 GB \/ ~93% |<\/p>\n
| \u6743\u91cd\u843d\u76d8 | 185 GB（NVFP4，\u4e13\u5bb6\u5c42 4bit + \u5171\u4eab\u5c42 bf16） |<\/p>\n
| \u5355\u5361\u6a21\u578b\u5206\u7247 | ~23 GiB |<\/p>\n
| \u6743\u91cd\u52a0\u8f7d\u8017\u65f6 | 632.8 s（\u7ea6 10 \u5206\u534a） |<\/p>\n
| \u6700\u5927\u4e0a\u4e0b\u6587 | 131072 tokens |<\/p>\n
| KV cache | 149,796 tokens（fp8），131K \u8bf7\u6c42\u5e76\u53d1 1.14x |<\/p>\n
| \u542f\u52a8\u603b\u65f6\u957f | \u7ea6 11~12 \u5206\u949f |<\/p>\n<\/p>\n
\u6838\u5fc3\u7ecf\u9a8c\u4e00\u53e5\u8bdd：**\u65b0\u67b6\u6784 + \u65b0\u91cf\u5316 + \u65b0 GPU（SM120）\u610f\u5473\u7740\u5fc5\u987b\u4e0a nightly \u6846\u67b6，\u800c Docker \u5b98\u65b9\u955c\u50cf\u662f\u552f\u4e00\u7701\u5fc3\u7684\u8f7d\u4f53**；\u5269\u4e0b\u7684\u5751\u51e0\u4e4e\u5168\u90e8\u96c6\u4e2d\u5728"\u52a0\u8f7d\u8d85\u65f6\u3001KV \u5bb9\u91cf\u89c4\u5212\u3001\u663e\u5b58\u6c34\u4f4d"\u8fd9\u4e09\u4ef6\u4e8b\u4e0a，\u5404\u81ea\u90fd\u53ea\u9700\u8981\u4e00\u884c\u53c2\u6570\u5c31\u80fd\u89e3\u51b3\u3002<\/p>\n<\/p>\n
\u90e8\u7f72\u672c\u8eab\u6ca1\u6709\u4efb\u4f55\u65e0\u6cd5\u590d\u73b0\u7684\u9ed1\u9b54\u6cd5\u2014\u2014`docker run` \u6284\u8d70，\u6539\u6539\u6302\u8f7d\u8def\u5f84\u5c31\u80fd\u8dd1\u3002<\/p><\/p>\n","protected":false},"excerpt":{"rendered":"
> \u90e8\u7f72\u76ee\u6807：\u5728\u4e00\u53f0 8 \u5361 RTX 5090（\u6bcf\u5361 32 GB）\u670d\u52a1\u5668\u4e0a，\u7528 vLLM \u8dd1\u8d77 GLM-5.3-Flash \u7684 NVFP4 \u91cf\u5316\u7248，\u5bf9\u5916\u63d0\u4f9b OpenAI \u517c\u5bb9 API，\u652f\u6301 \/think \u6df1\u5ea6\u601d\u8003\u6a21\u5f0f\u4e0e Function Call\u3002\u6700\u7ec8\u65b9\u6848\u91c7\u7528 **vLLM \u5b98\u65b9 nightly \u955c\u50cf Docker**，\u670d\u52a1\u73b0\u7a33\u5b9a\u8fd0\u884c\u4e8e http:\/\/10.168.2.103<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[81,50],"topic":[],"class_list":["post-111869","post","type-post","status-publish","format-standard","hentry","category-server","tag-python","tag-50"],"yoast_head":"\n