{"id":82900,"date":"2026-07-25T14:33:36","date_gmt":"2026-07-25T06:33:36","guid":{"rendered":"https:\/\/www.wsisp.com\/helps\/82900.html"},"modified":"2026-07-25T14:33:36","modified_gmt":"2026-07-25T06:33:36","slug":"4090-8%e5%8d%a1%e6%9c%8d%e5%8a%a1%e5%99%a8-%e7%94%9f%e4%ba%a7%e5%8a%9b%ef%bc%88%e8%af%bb%e5%8f%96%ef%bc%9a813tokens-s%ef%bc%8c%e8%be%93%e5%87%ba%ef%bc%9a101tokens-s%ef%bc%89-qwen3-6-27b-awq%e9%83%a8","status":"publish","type":"post","link":"https:\/\/www.wsisp.com\/helps\/82900.html","title":{"rendered":"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a"},"content":{"rendered":"

\n

\n

\u9644：4\u670d\u52a1\u6a21\u578b API\u8bbe\u7f6e\u6d4b\u8bd5\u00a0\u6d41\u7a0b<\/p>\n

0. \u524d\u671f\u90e8\u7f72\u51c6\u5907\u6d41\u7a0b（\u4f9d\u8d56\u5b89\u88c5 + \u6a21\u578b\u4e0b\u8f7d + \u6821\u9a8c\u811a\u672c）<\/p>\n

0.1 \u6574\u4f53\u90e8\u7f72\u8bf4\u660e<\/p>\n

\u672c\u6b21\u63a8\u7406\u90e8\u7f72\u91c7\u7528\u79bb\u7ebf\u7eaf\u51c0\u90e8\u7f72\u65b9\u6848，\u57fa\u4e8eLinux\u670d\u52a1\u5668\u3001Conda\u865a\u62df\u73af\u5883\u9694\u79bb，\u5168\u7a0b\u9501\u5b9a\u4f9d\u8d56\u7248\u672c\u3001\u56fa\u5316\u73af\u5883\u53d8\u91cf\u3001\u5185\u7f51\u62f7\u8d1d\u6a21\u578b，\u65e0\u5916\u7f51\u4e0d\u7a33\u5b9a\u4e0b\u8f7d，\u53ef1:1\u590d\u523b\u751f\u4ea7\u73af\u5883\u3002\u672c\u7ae0\u5b8c\u6574\u8bb0\u5f55：\u73af\u5883\u642d\u5efa\u3001\u5b89\u88c5\u547d\u4ee4\u3001\u4f9d\u8d56\u7248\u672c\u3001\u73af\u5883\u53d8\u91cf\u3001\u6a21\u578b\u62f7\u8d1d\u3001\u6587\u4ef6\u6821\u9a8c\u3001\u4f9d\u8d56\u81ea\u68c0\u811a\u672c\u3002<\/p>\n

0.2 \u73af\u5883\u4f9d\u8d56\u5b89\u88c5\u6d41\u7a0b<\/p>\n

0.2.1 \u521b\u5efa\u9694\u79bb\u865a\u62df\u73af\u5883<\/p>\n\n\n\n
\n

bash<\/span>
\n# \u521b\u5efa\u4e13\u5c5e\u5927\u6a21\u578b\u63a8\u7406\u73af\u5883，python3.11\u9002\u914dvllm0.20.2
\nconda create -n vllm_env python=3.11 -y
\n# \u6fc0\u6d3b\u73af\u5883
\nconda activate vllm_env<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

0.2.2 \u57fa\u7840\u6df1\u5ea6\u5b66\u4e60\u4f9d\u8d56\u5b89\u88c5<\/p>\n\n\n\n
\n

bash<\/span>
\n# CUDA12.1 \u9002\u914d4090\u9a71\u52a8，\u56fa\u5b9aTorch\u7248\u672c\u675c\u7edd\u517c\u5bb9\u95ee\u9898
\npip install torch==2.5.1 torchvision torchaudio –index-url https:\/\/download.pytorch.org\/whl\/cu121
\n# \u6a21\u578b\u52a0\u8f7d\u57fa\u7840\u5e93
\npip install transformers==4.48.3 accelerate sentencepiece protobuf<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

0.2.3 \u63a8\u7406&\u91cf\u5316\u4e13\u9879\u4f9d\u8d56\u5b89\u88c5<\/p>\n\n\n\n
\n

bash<\/span>
\n# \u9ad8\u6027\u80fd\u63a8\u7406\u6846\u67b6（\u672c\u6b21\u56fa\u5316\u7248\u672c）
\npip install vllm==0.20.2
\n# AWQ\u91cf\u5316\u89e3\u6790 + Marlin4090\u9ad8\u901f\u5185\u6838
\npip install autoawq==0.2.7 marlin-kernels
\n# \u6ce8\u610f\u529b\u52a0\u901f\u5185\u6838
\npip install flashinfer xformers<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

0.2.4 \u751f\u4ea7\u56fa\u5316\u4f9d\u8d56\u7248\u672c\u8868（\u4e0d\u53ef\u53d8\u66f4）<\/p>\n\n\n\n\n\n\n\n\n
\n

\u4f9d\u8d56\u5e93<\/p>\n<\/td>\n

\n

\u56fa\u5316\u7248\u672c<\/p>\n<\/td>\n

\n

\u7528\u9014\u8bf4\u660e<\/p>\n<\/td>\n<\/tr>\n

\n

vllm<\/p>\n<\/td>\n

\n

0.20.2<\/p>\n<\/td>\n

\n

\u63a8\u7406\u5f15\u64ce\u3001\u538b\u6d4b\u5de5\u5177\u3001\u5206\u5e03\u5f0f\u5f20\u91cf\u5e76\u884c<\/p>\n<\/td>\n<\/tr>\n

\n

torch<\/p>\n<\/td>\n

\n

2.5.1<\/p>\n<\/td>\n

\n

\u5e95\u5c42CUDA\u5f20\u91cf\u8fd0\u7b97，\u9002\u914dCUDA12.1<\/p>\n<\/td>\n<\/tr>\n

\n

autoawq<\/p>\n<\/td>\n

\n

0.2.7<\/p>\n<\/td>\n

\n

\u89e3\u6790AWQ4bit\u91cf\u5316\u6743\u91cd<\/p>\n<\/td>\n<\/tr>\n

\n

transformers<\/p>\n<\/td>\n

\n

4.48.3<\/p>\n<\/td>\n

\n

\u6a21\u578b\u7ed3\u6784\u8bfb\u53d6\u3001\u5206\u8bcd\u5668\u52a0\u8f7d<\/p>\n<\/td>\n<\/tr>\n

\n

marlin-kernels<\/p>\n<\/td>\n

\n

\u6700\u65b0\u7a33\u5b9a\u7248<\/p>\n<\/td>\n

\n

4090\u663e\u5361\u4e13\u5c5e\u91cf\u5316\u9ad8\u901f\u5185\u6838<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

0.3 \u7cfb\u7edf\u73af\u5883\u53d8\u91cf\u914d\u7f6e（\u6c38\u4e45\u5199\u5165）<\/p>\n

\u9488\u5bf9\u53cc\u53614090\u786c\u4ef6\u9650\u5236（\u65e0P2P\u76f4\u901a），\u4e13\u95e8\u914d\u7f6e\u73af\u5883\u53d8\u91cf\u89c4\u907f\u8b66\u544a\u3001\u7a33\u5b9a\u901a\u4fe1\u3001\u9650\u5236\u663e\u5361\u5360\u7528\u3002<\/p>\n\n\n\n
\n

bash<\/span>
\n# \u5199\u5165\u7cfb\u7edf\u73af\u5883\u53d8\u91cf，\u6c38\u4e45\u751f\u6548
\nvim ~\/.bashrc
\n# \u5728\u672b\u5c3e\u6dfb\u52a0\u4ee5\u4e0b\u5185\u5bb9
\nexport VLLM_SLEEP_WHEN_IDLE=1
\nexport NCCL_P2P_DISABLE=1
\nexport CUDA_VISIBLE_DEVICES=0,1<\/p>\n

# \u751f\u6548\u914d\u7f6e
\nsource ~\/.bashrc<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

    \n
  • NCCL_P2P_DISABLE=1：\u5173\u95ed\u663e\u5361P2P\u68c0\u6d4b，\u89e3\u51b34090\u4e0d\u652f\u6301SymmMem\u62a5\u9519\u3002<\/li>\n<\/ul>\n
      \n
    • CUDA_VISIBLE_DEVICES=0,1：\u5f3a\u5236\u53ea\u4f7f\u7528\u4e24\u5f20\u663e\u5361，\u9632\u6b62\u4e32\u5361\u3002<\/li>\n<\/ul>\n
        \n
      • VLLM_SLEEP_WHEN_IDLE：\u7a7a\u95f2\u81ea\u52a8\u964d\u9891，\u964d\u4f4e\u670d\u52a1\u5668\u529f\u8017\u4e0e\u6e29\u5ea6\u3002<\/li>\n<\/ul>\n

        0.4 \u6a21\u578b\u4e0b\u8f7d\u3001\u62f7\u8d1d\u4e0e\u6821\u9a8c\u6d41\u7a0b<\/p>\n

        0.4.1 \u6a21\u578b\u57fa\u7840\u4fe1\u606f<\/p>\n

          \n
        • \u6a21\u578b\u540d\u79f0：Qwen3.6-27B-AWQ<\/li>\n<\/ul>\n
            \n
          • \u5b58\u653e\u8def\u5f84：\/data\/models\/Qwen3.6-27B-AWQ<\/li>\n<\/ul>\n
              \n
            • \u6a21\u578b\u683c\u5f0f：safetensors（8\u5206\u7247\u6743\u91cd）<\/li>\n<\/ul>\n
                \n
              • \u78c1\u76d8\u5360\u7528：20.35GB<\/li>\n<\/ul>\n
                  \n
                • \u91cf\u5316\u65b9\u5f0f：4bit-AWQ + Marlin\u786c\u4ef6\u52a0\u901f<\/li>\n<\/ul>\n

                  0.4.2 \u6a21\u578b\u62f7\u8d1d\u6d41\u7a0b（\u5185\u7f51\u79bb\u7ebf）<\/p>\n

                  \u672c\u670d\u52a1\u5668\u4e0d\u4f7f\u7528\u5916\u7f51Huggingface\u4e0b\u8f7d，\u91c7\u7528ModelScope\u8d44\u6e90\u62f7\u8d1d，\u4fdd\u8bc1\u6a21\u578b\u5b8c\u6574\u3001\u65e0\u635f\u574f\u3001\u65e0\u9650\u901f\u3002<\/p>\n\n\n\n
                  \n

                  bash<\/span>
                  \n# \u521b\u5efa\u6a21\u578b\u6839\u76ee\u5f55
                  \nmkdir -p \/data\/models\/
                  \n# \u5185\u7f51\u8d44\u6e90\u62f7\u8d1d\u81f3\u672c\u5730\u6a21\u578b\u76ee\u5f55
                  \ncp -r \/Qwen3.6-27B-AWQ \/data\/models\/<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                  0.4.3 \u6a21\u578b\u4eba\u5de5\u6821\u9a8c\u6b65\u9aa4<\/p>\n

                • \u6743\u91cd\u5206\u7247：\u51718\u4e2asafetensors\u6587\u4ef6，\u5206\u7247\u9f50\u5168\u65e0\u7f3a\u5931\u3002<\/li>\n
                • \u6587\u4ef6\u7cfb\u7edf：\u670d\u52a1\u5668\u78c1\u76d8\u683c\u5f0fXFS，\u8bfb\u5199\u7a33\u5b9a，\u65e0\u7f51\u7edc\u78c1\u76d8IO\u5ef6\u8fdf\u3002<\/li>\n
                • \u52a0\u8f7d\u6821\u9a8c：vLLM\u542f\u52a8\u65e0\u6743\u91cd\u62a5\u9519\u3001\u65e0\u54c8\u5e0c\u6821\u9a8c\u5931\u8d25\u3002<\/li>\n
                • \u5185\u6838\u8bc6\u522b：\u81ea\u52a8\u8bc6\u522bAWQ&Marlin，\u542f\u7528\u4e13\u7528\u91cf\u5316\u52a0\u901f\u5185\u6838\u3002<\/li>\n

                  0.5 \u751f\u4ea7\u7ea7\u4f9d\u8d56\u81ea\u68c0\u811a\u672c（\u53ef\u76f4\u63a5\u590d\u7528）<\/p>\n

                  \u4e3a\u65b9\u4fbf\u540e\u671f\u8fd0\u7ef4\u73af\u5883\u6821\u9a8c，\u7f16\u5199\u4e00\u952e\u4f9d\u8d56\u68c0\u67e5\u811a\u672c，\u68c0\u6d4bCUDA\u3001Torch\u3001vLLM\u3001AWQ\u4f9d\u8d56\u662f\u5426\u9f50\u5168\u3001\u7248\u672c\u662f\u5426\u5339\u914d\u3002<\/p>\n\n\n\n
                  \n

                  bash<\/span>
                  \n#!\/bin\/bash
                  \n# filename: env_check.sh
                  \n# \u5927\u6a21\u578b\u63a8\u7406\u73af\u5883\u81ea\u68c0\u811a\u672c<\/p>\n

                  echo "========== CUDA \u7248\u672c\u68c0\u6d4b =========="
                  \nnvidia-smi
                  \nnvcc –version<\/p>\n

                  echo "========== Python \u7248\u672c\u68c0\u6d4b =========="
                  \npython –version<\/p>\n

                  echo "========== \u5173\u952e\u4f9d\u8d56\u7248\u672c\u68c0\u6d4b =========="
                  \npip show torch
                  \npip show vllm
                  \npip show transformers
                  \npip show autoawq<\/p>\n

                  echo "========== GPU \u53ef\u7528\u6027\u68c0\u6d4b =========="
                  \npython -c "import torch;print('GPU\u53ef\u7528\u6570\u91cf:',torch.cuda.device_count())"<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                  \u4f7f\u7528\u65b9\u5f0f：\u8d4b\u4e88\u6743\u9650\u3001\u4e00\u952e\u6267\u884c，\u5feb\u901f\u6392\u67e5\u73af\u5883\u5f02\u5e38\u3002<\/p>\n\n\n\n
                  \n

                  bash<\/span>
                  \nchmod +x env_check.sh
                  \n.\/env_check.sh<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                  1. \u6d4b\u8bd5\u57fa\u672c\u4fe1\u606f<\/p>\n

                  1.1 \u786c\u4ef6\u73af\u5883<\/p>\n\n\n\n\n\n\n\n
                  \n

                  \u786c\u4ef6\u9879\u76ee<\/p>\n<\/td>\n

                  		\n

                  \u53c2\u6570\u8be6\u60c5<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  GPU\u578b\u53f7<\/p>\n<\/td>\n

                  		\n

                  NVIDIA RTX 4090 24GB<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  GPU\u6570\u91cf<\/p>\n<\/td>\n

                  		\n

                  2\u5f20（\u53cc\u5361\u5f20\u91cf\u5e76\u884c）<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  \u7cfb\u7edf\u5185\u5b58<\/p>\n<\/td>\n

                  		\n

                  512GB（\u5927\u5185\u5b58\u65e0\u52a0\u8f7d\u74f6\u9888）<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  \u786c\u76d8\u6587\u4ef6\u7cfb\u7edf<\/p>\n<\/td>\n

                  		\n

                  XFS<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                  1.2 \u8f6f\u4ef6\u73af\u5883<\/p>\n\n\n\n\n\n\n\n\n
                  \n

                  \u8f6f\u4ef6\u9879\u76ee<\/p>\n<\/td>\n

                  		\n

                  \u7248\u672c\/\u8bf4\u660e<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  \u63a8\u7406\u6846\u67b6<\/p>\n<\/td>\n

                  		\n

                  vLLM 0.20.2<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  \u91cf\u5316\u7b97\u6cd5<\/p>\n<\/td>\n

                  		\n

                  AWQ + Marlin \u9ad8\u901f\u5185\u6838<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  \u901a\u4fe1\u540e\u7aef<\/p>\n<\/td>\n

                  		\n

                  NCCL 2.28.9<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  KV\u7f13\u5b58\u683c\u5f0f<\/p>\n<\/td>\n

                  		\n

                  FP8（\u663e\u5b58\u538b\u7f29）<\/p>\n<\/td>\n<\/tr>\n

                  \n

                  Attention\u540e\u7aef<\/p>\n<\/td>\n

                  		\n

                  FlashInfer<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                  2. \u538b\u6d4b\u547d\u4ee4\u4e0e\u6838\u5fc3\u53c2\u6570\u8bf4\u660e<\/p>\n

                  2.1 \u5b8c\u6574\u538b\u6d4b\u547d\u4ee4<\/p>\n\n\n\n
                  \n

                  bash<\/span>
                  \nvllm bench latency \\\\
                  \n\u00a0\u00a0–model \/data\/models\/Qwen3.6-27B-AWQ \\\\
                  \n\u00a0\u00a0–trust-remote-code \\\\
                  \n\u00a0\u00a0–tensor-parallel-size 2 \\\\
                  \n\u00a0\u00a0–gpu-memory-utilization 0.75 \\\\
                  \n\u00a0\u00a0–max-model-len 4096 \\\\
                  \n\u00a0\u00a0–input-len 1024 \\\\
                  \n\u00a0\u00a0–output-len 1024 \\\\
                  \n\u00a0\u00a0–kv-cache-dtype fp8 \\\\
                  \n\u00a0\u00a0–enforce-eager<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                  2.2 \u5173\u952e\u957f\u5ea6\u53c2\u6570\u901a\u4fd7\u8be6\u89e3（\u91cd\u70b9）<\/p>\n

                  \u4e3a\u65b9\u4fbf\u8fd0\u7ef4\u4eba\u5458\u7406\u89e3，\u91c7\u7528\u7b14\u8bb0\u672c\u751f\u6d3b\u5316\u6bd4\u55bb，\u76f4\u767d\u533a\u5206\u4e09\u4e2a\u957f\u5ea6\u53c2\u6570：<\/p>\n

                  2.2.1 max-model-len = 4096（\u6700\u5927\u4e0a\u4e0b\u6587\u7a97\u53e3）<\/p>\n

                  \u6bd4\u55bb：\u4e00\u672c\u603b\u51714096\u9875\u7684\u7b14\u8bb0\u672c（\u786c\u4ef6\u5929\u82b1\u677f）<\/p>\n

                  \u6a21\u578b\u6700\u5927\u8bb0\u5fc6\u4e0a\u9650，\u5386\u53f2\u5bf9\u8bdd+\u63d0\u95ee+\u56de\u7b54\u5168\u90e8\u52a0\u8d77\u6765\u4e0d\u80fd\u8d85\u8fc74096token，\u8d85\u51fa\u81ea\u52a8\u622a\u65ad\u524d\u6587\u3002<\/p>\n

                  2.2.2 input-len = 1024（\u538b\u6d4b\u8f93\u5165\u957f\u5ea6）<\/p>\n

                  \u6bd4\u55bb：\u4f60\u5199\u7ed9AI\u7684\u9898\u76ee，\u5360\u75281024\u9875<\/p>\n

                  \u538b\u6d4b\u56fa\u5b9a\u8f93\u5165\u957f\u6587\u672c，\u6a21\u62df\u771f\u5b9e\u4e1a\u52a1\u957f\u6587\u6848\u3001\u957f\u6587\u6863\u3001\u957f\u95ee\u9898\u8f93\u5165\u3002<\/p>\n

                  2.2.3 output-len = 1024（\u538b\u6d4b\u751f\u6210\u957f\u5ea6）<\/p>\n

                  \u6bd4\u55bb：AI\u5199\u51fa\u7684\u7b54\u6848，\u5360\u75281024\u9875<\/p>\n

                  \u5f3a\u5236\u6a21\u578b\u5fc5\u987b\u751f\u62101024token，\u4e0d\u5141\u8bb8\u63d0\u524d\u7ed3\u675f，\u7528\u4e8e\u6d4b\u8bd5\u957f\u6587\u672c\u751f\u6210\u80fd\u529b\u3002<\/p>\n

                  2.3 \u4e09\u8005\u7ec4\u5408\u5173\u7cfb<\/p>\n

                  4096（\u603b\u5bb9\u91cf） = 1024\u8f93\u5165 + 1024\u8f93\u51fa + \u9884\u7559\u7f13\u5b58\u7a7a\u95f4<\/p>\n

                  \u4e2d\u6587\u901a\u7528\u6362\u7b97：1token \u2248 1.5~2\u4e2a\u6c49\u5b57<\/p>\n

                    \n
                  • \u8f93\u51651024token \u2248 2000\u6c49\u5b57<\/li>\n<\/ul>\n
                      \n
                    • \u8f93\u51fa1024token \u2248 2000\u6c49\u5b57<\/li>\n<\/ul>\n
                        \n
                      • \u6700\u5927\u4e0a\u4e0b\u6587 \u2248 8000\u6c49\u5b57<\/li>\n<\/ul>\n

                        2.4 \u5176\u4f59\u6838\u5fc3\u53c2\u6570\u8bf4\u660e<\/p>\n\n\n\n\n\n\n\n
                        \n

                        \u53c2\u6570<\/p>\n<\/td>\n

                        		\n

                        \u53d6\u503c<\/p>\n<\/td>\n

                        		\n

                        \u4f5c\u7528\u8bf4\u660e<\/p>\n<\/td>\n<\/tr>\n

                        \n

                        tensor-parallel-size<\/p>\n<\/td>\n

                        		\n

                        2<\/p>\n<\/td>\n

                        		\n

                        \u53cc\u5361\u5747\u5206\u6743\u91cd，\u964d\u4f4e\u5355\u5361\u663e\u5b58\u538b\u529b<\/p>\n<\/td>\n<\/tr>\n

                        \n

                        gpu-memory-utilization<\/p>\n<\/td>\n

                        		\n

                        0.75<\/p>\n<\/td>\n

                        		\n

                        \u663e\u5b58\u5b89\u5168\u4e0a\u9650，\u675c\u7eddOOM<\/p>\n<\/td>\n<\/tr>\n

                        \n

                        kv-cache-dtype<\/p>\n<\/td>\n

                        		\n

                        fp8<\/p>\n<\/td>\n

                        		\n

                        KV\u7f13\u5b58\u538b\u7f29，\u8282\u7701\u663e\u5b58\u3001\u63d0\u5347\u541e\u5410<\/p>\n<\/td>\n<\/tr>\n

                        \n

                        enforce-eager<\/p>\n<\/td>\n

                        		\n

                        \u542f\u7528<\/p>\n<\/td>\n

                        		\n

                        \u5173\u95edCUDAGraph，\u6362\u53d6\u5de5\u4e1a\u7ea7\u7a33\u5b9a\u6027<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                        3. \u90e8\u7f72\u6d41\u7a0b\u4e0e\u65e5\u5fd7\u5173\u952e\u8bb0\u5f55<\/p>\n

                        3.1 \u6a21\u578b\u52a0\u8f7d\u9636\u6bb5<\/p>\n

                      • \u5185\u6838\u81ea\u52a8\u9002\u914d：\u542f\u7528MarlinLinearKernel\u3001FlashInfer\u6ce8\u610f\u529b\u5185\u6838\u3002<\/li>\n
                      • \u6743\u91cd\u52a0\u8f7d\u8017\u65f6：4.80s，\u5b8c\u6574\u542f\u52a8\u8017\u65f66.25s\u3002<\/li>\n
                      • \u6a21\u578b\u663e\u5b58\u5360\u7528：9.98GiB\u3002<\/li>\n
                      • KV\u7f13\u5b58\u80fd\u529b：\u53ef\u7528\u7f13\u5b586.14GiB，\u6700\u5927\u7f13\u5b58Token174,762，\u7406\u8bba\u5e76\u53d142.67\u8def\u3002<\/li>\n

                        3.2 \u65e0\u5bb3\u544a\u8b66\u65e5\u5fd7\u6c47\u603b<\/p>\n

                        \u672c\u6b21\u6240\u6709\u8b66\u544a\u65e0\u62a5\u9519\u3001\u4e0d\u5f71\u54cd\u751f\u4ea7，\u5168\u90e8\u4e3a\u786c\u4ef6\/\u73af\u5883\u517c\u5bb9\u63d0\u793a：<\/p>\n

                          \n
                        • NIXL is not available：\u5206\u5e03\u5f0f\u9ad8\u901f\u4f20\u8f93\u7ec4\u4ef6\u672a\u5b89\u88c5，\u5355\u8282\u70b9\u65e0\u5f71\u54cd\u3002<\/li>\n<\/ul>\n
                            \n
                          • SymmMemCommunicator：4090\u7b97\u529b\u67b6\u67848.9\u4e0d\u652f\u6301\u5bf9\u79f0\u901a\u4fe1，\u81ea\u52a8\u964d\u7ea7NCCL\u3002<\/li>\n<\/ul>\n
                              \n
                            • Custom allreduce disabled：\u65e0\u663e\u5361P2P\u76f4\u8fde，\u81ea\u52a8\u5173\u95ed\u81ea\u5b9a\u4e49\u901a\u4fe1\u3002<\/li>\n<\/ul>\n

                              4. \u5b9e\u6d4b\u6027\u80fd\u7ed3\u679c\u4e0e\u6570\u636e\u89e3\u6790<\/p>\n

                              4.1 \u6838\u5fc3\u7a33\u5b9a\u6027\u80fd\u6307\u6807<\/p>\n\n\n\n\n\n\n\n
                              \n

                              \u6027\u80fd\u6307\u6807<\/p>\n<\/td>\n

                              		\n

                              \u5b9e\u6d4b\u6570\u503c<\/p>\n<\/td>\n

                              		\n

                              \u8bf4\u660e<\/p>\n<\/td>\n<\/tr>\n

                              \n

                              Prompt\u7f16\u7801\u541e\u5410（\u8bfb\u9898\u901f\u5ea6）<\/p>\n<\/td>\n

                              		\n

                              813.1 tokens\/s<\/p>\n<\/td>\n

                              		\n

                              \u6279\u91cf\u5e76\u884c\u7f16\u7801，\u901f\u5ea6\u6781\u9ad8，\u975e\u751f\u6210\u901f\u5ea6<\/p>\n<\/td>\n<\/tr>\n

                              \n

                              Generation\u751f\u6210\u541e\u5410（\u771f\u5b9e\u8f93\u51fa）<\/p>\n<\/td>\n

                              		\n

                              74~101 tokens\/s<\/p>\n<\/td>\n

                              		\n

                              \u7528\u6237\u771f\u5b9e\u611f\u77e5\u751f\u6210\u901f\u5ea6，\u4f18\u79c0\u6c34\u5e73<\/p>\n<\/td>\n<\/tr>\n

                              \n

                              \u7a33\u5b9a\u5e76\u53d1\u6570<\/p>\n<\/td>\n

                              		\n

                              8 reqs<\/p>\n<\/td>\n

                              		\n

                              \u65e0\u6392\u961f\u3001\u65e0\u5361\u987f<\/p>\n<\/td>\n<\/tr>\n

                              \n

                              KV\u7f13\u5b58\u4f7f\u7528\u7387<\/p>\n<\/td>\n

                              		\n

                              12.5%~15.7%<\/p>\n<\/td>\n

                              		\n

                              \u663e\u5b58\u4f59\u91cf\u6781\u5927<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                              4.2 \u5173\u952e\u6570\u636e\u901a\u4fd7\u89e3\u91ca<\/p>\n

                                \n
                              • 813 tokens\/s：\u8f93\u5165\u7f16\u7801\u901f\u5ea6，GPU\u4e00\u6b21\u6027\u6279\u91cf\u8bfb\u5b8c\u957f\u6587\u672c，\u6240\u4ee5\u901f\u5ea6\u6781\u9ad8\u3002<\/li>\n<\/ul>\n
                                  \n
                                • 74~101 tokens\/s：\u771f\u5b9e\u9010\u5b57\u751f\u6210\u901f\u5ea6，\u81ea\u56de\u5f52\u751f\u6210，\u4e3a\u7528\u6237\u5b9e\u9645\u4f53\u9a8c\u901f\u5ea6\u3002<\/li>\n<\/ul>\n

                                  5. \u4f18\u7f3a\u70b9\u603b\u7ed3<\/p>\n

                                  5.1 \u4f18\u70b9<\/p>\n

                                • \u90e8\u7f72\u6781\u5ea6\u7a33\u5b9a，\u65e0\u5d29\u6e83\u3001\u65e0OOM\u3002<\/li>\n
                                • FP8+AWQ\u663e\u5b58\u63a7\u5236\u4f18\u79c0，\u5b8c\u7f8e\u9002\u914d\u53cc4090\u3002<\/li>\n
                                • \u751f\u6210\u541e\u541075+ token\/s，\u6ee1\u8db3\u4f01\u4e1a\u751f\u4ea7\u529b\u573a\u666f\u3002<\/li>\n
                                • \u73af\u5883\u5168\u90e8\u56fa\u5316，\u53ef\u590d\u523b\u3001\u53ef\u8fc1\u79fb\u3002<\/li>\n

                                  5.2 \u73b0\u5b58\u77ed\u677f<\/p>\n

                                • \u7981\u7528CUDAGraph，\u727a\u7272\u5c11\u91cf\u6781\u81f4\u901f\u5ea6\u6362\u53d6\u7a33\u5b9a\u6027\u3002<\/li>\n
                                • 4090\u786c\u4ef6\u65e0P2P，\u901a\u4fe1\u6027\u80fd\u8f7b\u5fae\u53d7\u9650\u3002<\/li>\n

                                  6. \u751f\u4ea7\u4f18\u5316\u5efa\u8bae<\/p>\n

                                • \u5f00\u542f\u524d\u7f00\u7f13\u5b58：\u6dfb\u52a0 –enable-prefix-caching<\/span>，\u91cd\u590d\u5bf9\u8bdd\u63d0\u901f30%+\u3002<\/li>\n
                                • \u663e\u5b58\u5229\u7528\u7387\u53ef\u8c03\u81f30.80，\u63d0\u5347\u6700\u5927\u5e76\u53d1\u3002<\/li>\n
                                • \u6302\u673a\u751f\u4ea7\u957f\u671f\u8fd0\u884c\u4fdd\u6301 –enforce-eager\u00a0\u5f00\u542f，\u4fdd\u8bc1\u7a33\u8fd0\u884c\u3002<\/li>\n

                                  7. \u6700\u7ec8\u6d4b\u8bd5\u7ed3\u8bba<\/p>\n

                                  \u672c\u6b21\u5728\u53ccRTX4090\u670d\u52a1\u5668\u5b8c\u6210Qwen3.6-27B-AWQ\u5de5\u4e1a\u7ea7\u538b\u6d4b，\u91c7\u7528vLLM\u63a8\u7406\u3001AWQ\u91cf\u5316\u3001FP8 KV\u7f13\u5b58\u3001\u53cc\u5361\u5f20\u91cf\u5e76\u884c\u3002\u73af\u5883\u7eaf\u51c0\u3001\u4f9d\u8d56\u56fa\u5316\u3001\u6a21\u578b\u6821\u9a8c\u5b8c\u6574\u3001\u5168\u7a0b\u65e0\u62a5\u9519\u65e0\u6ea2\u51fa\u3002<\/p>\n

                                  \u5b9e\u6d4b\u8f93\u5165\u7f16\u7801\u541e\u5410813.1tokens\/s，\u771f\u5b9e\u751f\u6210\u541e\u5410\u7a33\u5b9a101tokens\/s，8\u8def\u5e76\u53d1\u5e73\u7a33\u8fd0\u884c，\u663e\u5b58\u4f59\u91cf\u5145\u8db3\u3002<\/p>\n

                                  \u7ed3\u8bba：\u6574\u5957\u90e8\u7f72\u65b9\u6848\u6210\u719f\u3001\u7a33\u5b9a\u3001\u53ef\u76f4\u63a5\u6295\u4ea7，\u6240\u6709\u53c2\u6570\u6c38\u4e45\u56fa\u5316，\u4f5c\u4e3a\u672c\u670d\u52a1\u5668\u5b98\u65b9\u751f\u4ea7\u529b\u90e8\u7f72\u65b9\u6848\u3002<\/p>\n

                                  \u6d4b\u8bd5\u4eba\u5458：\u6280\u672f\u6d4b\u8bd5\u7ec4<\/p>\n

                                  \u6d4b\u8bd5\u65f6\u95f4：2026\u5e7405\u670812\u65e5<\/p>\n

                                  \n

                                  \u9644：<\/p>\n

                                  8 \u5361 4090 \u670d\u52a1\u5668 4 \u670d\u52a1\u8bbe\u7f6e\u6d4b\u8bd5\u6d41\u7a0b<\/p>\n

                                  \u4e00\u3001\u73af\u5883\u57fa\u7840\u4fe1\u606f<\/p>\n

                                    \n
                                  • \u670d\u52a1\u5668：8 \u5361 RTX 4090<\/li>\n<\/ul>\n
                                      \n
                                    • \u6a21\u578b：Qwen3.6-27B-AWQ<\/li>\n<\/ul>\n
                                        \n
                                      • \u865a\u62df\u73af\u5883\u8def\u5f84：\/opt\/venv\/vllm<\/li>\n<\/ul>\n
                                          \n
                                        • \u5185\u7f51\u7f51\u5361：enp194s0f0<\/li>\n<\/ul>\n
                                            \n
                                          • \u5185\u7f51 IP：[10.255.254.44](10.255.254.44)<\/li>\n<\/ul>\n
                                              \n
                                            • \u516c\u7f51 IP：[218.31.112.74](218.31.112.74)<\/li>\n<\/ul>\n
                                                \n
                                              • \u5bf9\u5916\u7aef\u53e3：2196<\/li>\n<\/ul>\n
                                                  \n
                                                • \u540e\u7aef\u5185\u7f51\u7aef\u53e3：8000\u30018001\u30018002\u30018003<\/li>\n<\/ul>\n
                                                • \u7edf\u4e00 API Key：sk-qwen-27b-lb-2026<\/li>\n
                                                • \u67b6\u6784：4 \u4e2a vLLM \u540e\u7aef\u670d\u52a1 + Nginx \u8d1f\u8f7d\u5747\u8861 + \u53cd\u5411\u4ee3\u7406 + \u6807\u51c6 OpenAI \u63a5\u53e3\u5bf9\u5916<\/li>\n

                                                  \u4e8c\u3001\u524d\u671f\u95ee\u9898\u6392\u67e5\u4e0e\u6839\u56e0<\/p>\n

                                                • \u73b0\u8c61：vLLM \u670d\u52a1\u8fd0\u884c 2 \u5c0f\u65f6\u81ea\u52a8\u5168\u90e8\u9000\u51fa<\/li>\n
                                                • \u62a5\u9519\u65e5\u5fd7：Broken pipe<\/span>\u3001TCPStore server has shut down too early<\/span>\u3001NCCL \u901a\u4fe1\u5f02\u5e38<\/li>\n
                                                    \n
                                                  • \u6839\u56e0：<\/li>\n<\/ul>\n
                                                      \n
                                                    • vLLM \u591a\u5361\u5206\u5e03\u5f0f\u968f\u673a\u9009\u9519\u7f51\u5361 \/ IP<\/li>\n<\/ul>\n
                                                        \n
                                                      • NCCL\/TCPStore \u901a\u4fe1\u94fe\u8def\u4e0d\u7a33\u5b9a<\/li>\n<\/ul>\n
                                                      • \u672a\u56fa\u5b9a\u5185\u7f51\u7f51\u5361\u4e0e\u901a\u4fe1 IP，\u957f\u65f6\u95f4\u7a7a\u95f2\u8fde\u63a5\u65ad\u88c2<\/li>\n
                                                      • \u89e3\u51b3\u65b9\u6848：<\/li>\n
                                                      • \u5f3a\u5236\u7ed1\u5b9a\u56fa\u5b9a\u5185\u7f51\u7f51\u5361 enp194s0f0<\/li>\n
                                                      • \u56fa\u5b9a\u901a\u4fe1\u5185\u7f51 IP [10.255.254.44](10.255.254.44)<\/li>\n
                                                          \n
                                                        • \u66f4\u6362\u5206\u5e03\u5f0f\u540e\u7aef\u4e3a mp \u89c4\u907f NCCL \u4e0d\u7a33\u5b9a<\/li>\n<\/ul>\n
                                                            \n
                                                          • Nginx \u4ee3\u7406\u540e\u7aef\u6539\u4e3a\u5185\u7f51 IP \u7aef\u53e3<\/li>\n<\/ul>\n

                                                            \u4e09\u3001\u7cfb\u7edf\u73af\u5883\u53d8\u91cf\u6c38\u4e45\u914d\u7f6e<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \necho 'export NCCL_SOCKET_IFNAME=enp194s0f0' >> ~\/.bashrc
                                                            \necho 'export GLOO_SOCKET_IFNAME=enp194s0f0' >> ~\/.bashrc
                                                            \necho 'export VLLM_HOST_IP=10.255.254.44' >> ~\/.bashrc
                                                            \nsource ~\/.bashrc<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            \u56db\u3001\u8fdb\u5165\u865a\u62df\u73af\u5883\u5e76\u52a0\u8f7d\u8fd0\u884c\u53d8\u91cf<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \nsource \/opt\/venv\/vllm\/bin\/activate<\/p>\n

                                                            export VLLM_SLEEP_WHEN_IDLE=1
                                                            \nexport VLLM_USE_DEEP_GEMM=0
                                                            \nexport VLLM_USE_FLASHINFER_MOE_FP16=1
                                                            \nexport VLLM_USE_FLASHINFER_SAMPLER=0
                                                            \nexport OMP_NUM_THREADS=4
                                                            \nexport VLLM_USE_MODELSCOPE=true
                                                            \nexport VLLM_API_KEY=sk-qwen-27b-lb-2026<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            \u4e94\u3001\u5355\u7ec8\u7aef\u4e00\u952e\u542f\u52a8 4 \u4e2a vLLM \u540e\u7aef\u670d\u52a1<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \nCUDA_VISIBLE_DEVICES=0,1 vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\
                                                            \n–trust-remote-code \\\\
                                                            \n–tensor-parallel-size 2 \\\\
                                                            \n–gpu-memory-utilization 0.80 \\\\
                                                            \n–max-model-len 262144 \\\\
                                                            \n–kv-cache-dtype fp8 \\\\
                                                            \n–disable-custom-all-reduce \\\\
                                                            \n–enforce-eager \\\\
                                                            \n–port 8000 \\\\
                                                            \n–host 10.255.254.44 \\\\
                                                            \n–distributed-executor-backend mp \\\\
                                                            \n–api-key $VLLM_API_KEY > vllm_8000.log 2>&1 &<\/p>\n

                                                            CUDA_VISIBLE_DEVICES=2,3 vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\
                                                            \n–trust-remote-code \\\\
                                                            \n–tensor-parallel-size 2 \\\\
                                                            \n–gpu-memory-utilization 0.80 \\\\
                                                            \n–max-model-len 262144 \\\\
                                                            \n–kv-cache-dtype fp8 \\\\
                                                            \n–disable-custom-all-reduce \\\\
                                                            \n–enforce-eager \\\\
                                                            \n–port 8001 \\\\
                                                            \n–host 10.255.254.44 \\\\
                                                            \n–distributed-executor-backend mp \\\\
                                                            \n–api-key $VLLM_API_KEY > vllm_8001.log 2>&1 &<\/p>\n

                                                            CUDA_VISIBLE_DEVICES=4,5 vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\
                                                            \n–trust-remote-code \\\\
                                                            \n–tensor-parallel-size 2 \\\\
                                                            \n–gpu-memory-utilization 0.80 \\\\
                                                            \n–max-model-len 262144 \\\\
                                                            \n–kv-cache-dtype fp8 \\\\
                                                            \n–disable-custom-all-reduce \\\\
                                                            \n–enforce-eager \\\\
                                                            \n–port 8002 \\\\
                                                            \n–host 10.255.254.44 \\\\
                                                            \n–distributed-executor-backend mp \\\\
                                                            \n–api-key $VLLM_API_KEY > vllm_8002.log 2>&1 &<\/p>\n

                                                            CUDA_VISIBLE_DEVICES=6,7 vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\
                                                            \n–trust-remote-code \\\\
                                                            \n–tensor-parallel-size 2 \\\\
                                                            \n–gpu-memory-utilization 0.80 \\\\
                                                            \n–max-model-len 262144 \\\\
                                                            \n–kv-cache-dtype fp8 \\\\
                                                            \n–disable-custom-all-reduce \\\\
                                                            \n–enforce-eager \\\\
                                                            \n–port 8003 \\\\
                                                            \n–host 10.255.254.44 \\\\
                                                            \n–distributed-executor-backend mp \\\\
                                                            \n–api-key $VLLM_API_KEY > vllm_8003.log 2>&1 &<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            \u516d\u3001\u67e5\u770b vLLM \u670d\u52a1\u662f\u5426\u6b63\u5e38\u8fd0\u884c<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \nps -ef | grep vllm<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            \u4e03\u3001Nginx \u8d1f\u8f7d\u5747\u8861 + \u53cd\u5411\u4ee3\u7406\u914d\u7f6e<\/p>\n

                                                            1. \u5199\u5165\u914d\u7f6e\u6587\u4ef6<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \nsudo tee \/etc\/nginx\/conf.d\/vllm_lb.conf >\/dev\/null <<'EOF'
                                                            \nupstream vllm_backend {
                                                            \n\u00a0\u00a0\u00a0\u00a0server 10.255.254.44:8000;
                                                            \n\u00a0\u00a0\u00a0\u00a0server 10.255.254.44:8001;
                                                            \n\u00a0\u00a0\u00a0\u00a0server 10.255.254.44:8002;
                                                            \n\u00a0\u00a0\u00a0\u00a0server 10.255.254.44:8003;
                                                            \n\u00a0\u00a0\u00a0\u00a0ip_hash;
                                                            \n}<\/p>\n

                                                            server {
                                                            \n\u00a0\u00a0\u00a0\u00a0listen 0.0.0.0:2196;
                                                            \n\u00a0\u00a0\u00a0\u00a0server_name _;<\/p>\n

                                                            \u00a0\u00a0\u00a0\u00a0location \/ {
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_pass http:\/\/vllm_backend;
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_set_header Host $host;
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_set_header X-Real-IP $remote_addr;
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_set_header X-Forwarded-Proto $scheme;
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_connect_timeout 300s;
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_send_timeout 300s;
                                                            \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0proxy_read_timeout 300s;
                                                            \n\u00a0\u00a0\u00a0\u00a0}
                                                            \n}
                                                            \nEOF<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            2. \u6821\u9a8c\u914d\u7f6e\u5e76\u91cd\u542f Nginx<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \nsudo nginx -t
                                                            \nsudo systemctl restart nginx<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            3. \u9632\u706b\u5899\u653e\u884c\u5bf9\u5916\u7aef\u53e3<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \nsudo ufw allow 2196\/tcp
                                                            \nsudo ufw reload<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            \u516b\u3001\u670d\u52a1\u8fde\u901a\u6027\u6d4b\u8bd5\u547d\u4ee4<\/p>\n

                                                            1. \u67e5\u770b\u6a21\u578b\u5217\u8868\u6d4b\u8bd5<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \ncurl -H "Authorization: Bearer sk-qwen-27b-lb-2026" http:\/\/127.0.0.1:2196\/v1\/models<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            2. \u5bf9\u8bdd\u63a5\u53e3\u6d4b\u8bd5\u793a\u4f8b<\/p>\n\n\n\n
                                                            \n

                                                            bash<\/span>
                                                            \ncurl -H "Authorization: Bearer sk-qwen-27b-lb-2026" \\\\
                                                            \nhttp:\/\/218.31.112.74:2196\/v1\/chat\/completions \\\\
                                                            \n-H "Content-Type: application\/json" \\\\
                                                            \n-d '{"model":"\/data\/models\/Qwen3.6-27B-AWQ","messages":[{"role":"user","content":"\u7b80\u77ed\u4ecb\u7ecd\u4eba\u5de5\u667a\u80fd"}]}'<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                            \u4e5d\u3001\u5bf9\u5916\u6b63\u5f0f\u670d\u52a1\u63a5\u5165\u4fe1\u606f<\/p>\n

                                                              \n
                                                            • \u63a5\u53e3\u57fa\u7840\u5730\u5740：http:\/\/218.31.112.74:2196\/v1<\/span><\/li>\n<\/ul>\n
                                                                \n
                                                              • API Key：sk-qwen-27b-lb-2026<\/span><\/li>\n<\/ul>\n
                                                                  \n
                                                                • \u6a21\u578b\u540d\u79f0：\/data\/models\/Qwen3.6-27B-AWQ<\/span><\/li>\n<\/ul>\n
                                                                • \u63a5\u53e3\u6807\u51c6：\u517c\u5bb9 OpenAI \u683c\u5f0f，\u652f\u6301 ChatBox\u3001\u5404\u7c7b\u8c03\u7528\u5ba2\u6237\u7aef<\/li>\n

                                                                  \u5341\u3001\u90e8\u7f72\u7a33\u5b9a\u4fdd\u969c\u8bf4\u660e<\/p>\n

                                                                • \u56fa\u5b9a\u5185\u7f51\u7f51\u5361\u4e0e IP，\u675c\u7edd NCCL\/TCPStore \u901a\u4fe1\u65ad\u88c2<\/li>\n
                                                                • \u6539\u7528 mp \u5206\u5e03\u5f0f\u6267\u884c\u540e\u7aef，\u89c4\u907f\u591a\u5361\u968f\u673a\u5d29\u6e83<\/li>\n
                                                                • 4 \u4e2a\u540e\u7aef\u670d\u52a1\u8d1f\u8f7d\u5747\u8861，\u81ea\u52a8\u5206\u6d41<\/li>\n
                                                                • \u7edf\u4e00\u5bf9\u5916\u7aef\u53e3，\u5185\u7f51\u670d\u52a1\u9694\u79bb\u66f4\u5b89\u5168<\/li>\n
                                                                • \u89e3\u51b3\u957f\u65f6\u95f4\u8fd0\u884c\u81ea\u52a8\u9000\u51fa\u95ee\u9898，\u53ef 7\u00d724 \u5c0f\u65f6\u7a33\u5b9a\u63d0\u4f9b\u670d\u52a1<\/li>\n
                                                                    \n
                                                                  • \u670d\u52a1\u65e0\u4eba\u503c\u5b88\u90e8\u7f72<\/li>\n<\/ul>\n

                                                                    （\u5f00\u673a\u81ea\u542f + \u5d29\u6e83\u81ea\u52a8\u91cd\u542f）<\/p>\n

                                                                    1\u3001\u505c\u6b62\u539f\u6709\u624b\u52a8\u670d\u52a1<\/p>\n\n\n\n
                                                                    \n

                                                                    b<\/span>ash<\/span><\/p>\n

                                                                    \npkill -f vllm<\/p>\n

                                                                    sleep 3<\/p>\n

                                                                    ps -ef | grep vllm<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    2\u3001\u521b\u5efa 4 \u4e2a systemd \u6258\u7ba1\u670d\u52a1\u6587\u4ef6<\/p>\n

                                                                    1. \u521b\u5efa vllm-8000 \u670d\u52a1<\/p>\n\n\n\n
                                                                    \n

                                                                    Bash<\/span><\/p>\n

                                                                    \nsudo tee \/etc\/systemd\/system\/vllm-8000.service >\/dev\/null <<'EOF'<\/p>\n

                                                                    [Unit]<\/p>\n

                                                                    Description=vLLM Service 8000<\/p>\n

                                                                    After=network.target<\/p>\n

                                                                    \n

                                                                    [Service]<\/p>\n

                                                                    User=hmtsgpu<\/p>\n

                                                                    WorkingDirectory=\/home\/hmtsgpu<\/p>\n

                                                                    Environment="PATH=\/opt\/venv\/vllm\/bin:\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin"<\/p>\n

                                                                    Environment="VLLM_SLEEP_WHEN_IDLE=1"<\/p>\n

                                                                    Environment="VLLM_USE_DEEP_GEMM=0"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_MOE_FP16=1"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_SAMPLER=0"<\/p>\n

                                                                    Environment="OMP_NUM_THREADS=4"<\/p>\n

                                                                    Environment="VLLM_USE_MODELSCOPE=true"<\/p>\n

                                                                    Environment="VLLM_API_KEY=sk-qwen-27b-lb-2026"<\/p>\n

                                                                    Environment="NCCL_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="GLOO_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="VLLM_HOST_IP=10.255.254.44"<\/p>\n

                                                                    Environment="CUDA_VISIBLE_DEVICES=0,1"<\/p>\n

                                                                    \n

                                                                    ExecStart=\/opt\/venv\/vllm\/bin\/vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\<\/p>\n

                                                                    –trust-remote-code \\\\<\/p>\n

                                                                    –tensor-parallel-size 2 \\\\<\/p>\n

                                                                    –gpu-memory-utilization 0.80 \\\\<\/p>\n

                                                                    –max-model-len 262144 \\\\<\/p>\n

                                                                    –kv-cache-dtype fp8 \\\\<\/p>\n

                                                                    –disable-custom-all-reduce \\\\<\/p>\n

                                                                    –enforce-eager \\\\<\/p>\n

                                                                    –port 8000 \\\\<\/p>\n

                                                                    –host 10.255.254.44 \\\\<\/p>\n

                                                                    –distributed-executor-backend mp \\\\<\/p>\n

                                                                    –api-key sk-qwen-27b-lb-2026<\/p>\n

                                                                    \n

                                                                    Restart=always<\/p>\n

                                                                    RestartSec=5<\/p>\n

                                                                    LimitNOFILE=65535<\/p>\n

                                                                    \n

                                                                    [Install]<\/p>\n

                                                                    WantedBy=multi-user.target<\/p>\n

                                                                    EOF<\/p>\n

                                                                    \n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    \n

                                                                    2. \u521b\u5efa vllm-8001\u00a0\u670d\u52a1<\/p>\n\n\n\n
                                                                    \n

                                                                    Bash<\/span><\/p>\n

                                                                    \nsudo tee \/etc\/systemd\/system\/vllm-8001.service >\/dev\/null <<'EOF'<\/p>\n

                                                                    [Unit]<\/p>\n

                                                                    Description=vLLM Service 8001<\/p>\n

                                                                    After=network.target<\/p>\n

                                                                    \n

                                                                    [Service]<\/p>\n

                                                                    User=hmtsgpu<\/p>\n

                                                                    WorkingDirectory=\/home\/hmtsgpu<\/p>\n

                                                                    Environment="PATH=\/opt\/venv\/vllm\/bin:\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin"<\/p>\n

                                                                    Environment="VLLM_SLEEP_WHEN_IDLE=1"<\/p>\n

                                                                    Environment="VLLM_USE_DEEP_GEMM=0"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_MOE_FP16=1"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_SAMPLER=0"<\/p>\n

                                                                    Environment="OMP_NUM_THREADS=4"<\/p>\n

                                                                    Environment="VLLM_USE_MODELSCOPE=true"<\/p>\n

                                                                    Environment="VLLM_API_KEY=sk-qwen-27b-lb-2026"<\/p>\n

                                                                    Environment="NCCL_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="GLOO_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="VLLM_HOST_IP=10.255.254.44"<\/p>\n

                                                                    Environment="CUDA_VISIBLE_DEVICES=2,3"<\/p>\n

                                                                    \n

                                                                    ExecStart=\/opt\/venv\/vllm\/bin\/vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\<\/p>\n

                                                                    –trust-remote-code \\\\<\/p>\n

                                                                    –tensor-parallel-size 2 \\\\<\/p>\n

                                                                    –gpu-memory-utilization 0.80 \\\\<\/p>\n

                                                                    –max-model-len 262144 \\\\<\/p>\n

                                                                    –kv-cache-dtype fp8 \\\\<\/p>\n

                                                                    –disable-custom-all-reduce \\\\<\/p>\n

                                                                    –enforce-eager \\\\<\/p>\n

                                                                    –port 8001 \\\\<\/p>\n

                                                                    –host 10.255.254.44 \\\\<\/p>\n

                                                                    –distributed-executor-backend mp \\\\<\/p>\n

                                                                    –api-key sk-qwen-27b-lb-2026<\/p>\n

                                                                    \n

                                                                    Restart=always<\/p>\n

                                                                    RestartSec=5<\/p>\n

                                                                    LimitNOFILE=65535<\/p>\n

                                                                    \n

                                                                    [Install]<\/p>\n

                                                                    WantedBy=multi-user.target<\/p>\n

                                                                    EOF<\/p>\n

                                                                    \n

                                                                    \n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    <\/h4>\n

                                                                    3. \u521b\u5efa vllm-8002\u00a0\u670d\u52a1<\/p>\n\n\n\n
                                                                    \n

                                                                    Bash<\/span><\/p>\n

                                                                    \nsudo tee \/etc\/systemd\/system\/vllm-8002.service >\/dev\/null <<'EOF'<\/p>\n

                                                                    [Unit]<\/p>\n

                                                                    Description=vLLM Service 8002<\/p>\n

                                                                    After=network.target<\/p>\n

                                                                    \n

                                                                    [Service]<\/p>\n

                                                                    User=hmtsgpu<\/p>\n

                                                                    WorkingDirectory=\/home\/hmtsgpu<\/p>\n

                                                                    Environment="PATH=\/opt\/venv\/vllm\/bin:\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin"<\/p>\n

                                                                    Environment="VLLM_SLEEP_WHEN_IDLE=1"<\/p>\n

                                                                    Environment="VLLM_USE_DEEP_GEMM=0"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_MOE_FP16=1"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_SAMPLER=0"<\/p>\n

                                                                    Environment="OMP_NUM_THREADS=4"<\/p>\n

                                                                    Environment="VLLM_USE_MODELSCOPE=true"<\/p>\n

                                                                    Environment="VLLM_API_KEY=sk-qwen-27b-lb-2026"<\/p>\n

                                                                    Environment="NCCL_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="GLOO_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="VLLM_HOST_IP=10.255.254.44"<\/p>\n

                                                                    Environment="CUDA_VISIBLE_DEVICES=4,5"<\/p>\n

                                                                    \n

                                                                    ExecStart=\/opt\/venv\/vllm\/bin\/vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\<\/p>\n

                                                                    –trust-remote-code \\\\<\/p>\n

                                                                    –tensor-parallel-size 2 \\\\<\/p>\n

                                                                    –gpu-memory-utilization 0.80 \\\\<\/p>\n

                                                                    –max-model-len 262144 \\\\<\/p>\n

                                                                    –kv-cache-dtype fp8 \\\\<\/p>\n

                                                                    –disable-custom-all-reduce \\\\<\/p>\n

                                                                    –enforce-eager \\\\<\/p>\n

                                                                    –port 8002 \\\\<\/p>\n

                                                                    –host 10.255.254.44 \\\\<\/p>\n

                                                                    –distributed-executor-backend mp \\\\<\/p>\n

                                                                    –api-key sk-qwen-27b-lb-2026<\/p>\n

                                                                    \n

                                                                    Restart=always<\/p>\n

                                                                    RestartSec=5<\/p>\n

                                                                    LimitNOFILE=65535<\/p>\n

                                                                    \n

                                                                    [Install]<\/p>\n

                                                                    WantedBy=multi-user.target<\/p>\n

                                                                    EOF<\/p>\n

                                                                    \n

                                                                    \n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    \n

                                                                    \n

                                                                    4. \u521b\u5efa vllm-8003\u00a0\u670d\u52a1<\/p>\n\n\n\n
                                                                    \n

                                                                    Bash<\/span><\/p>\n

                                                                    \nsudo tee \/etc\/systemd\/system\/vllm-8003.service >\/dev\/null <<'EOF'<\/p>\n

                                                                    [Unit]<\/p>\n

                                                                    Description=vLLM Service 8003<\/p>\n

                                                                    After=network.target<\/p>\n

                                                                    \n

                                                                    [Service]<\/p>\n

                                                                    User=hmtsgpu<\/p>\n

                                                                    WorkingDirectory=\/home\/hmtsgpu<\/p>\n

                                                                    Environment="PATH=\/opt\/venv\/vllm\/bin:\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin"<\/p>\n

                                                                    Environment="VLLM_SLEEP_WHEN_IDLE=1"<\/p>\n

                                                                    Environment="VLLM_USE_DEEP_GEMM=0"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_MOE_FP16=1"<\/p>\n

                                                                    Environment="VLLM_USE_FLASHINFER_SAMPLER=0"<\/p>\n

                                                                    Environment="OMP_NUM_THREADS=4"<\/p>\n

                                                                    Environment="VLLM_USE_MODELSCOPE=true"<\/p>\n

                                                                    Environment="VLLM_API_KEY=sk-qwen-27b-lb-2026"<\/p>\n

                                                                    Environment="NCCL_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="GLOO_SOCKET_IFNAME=enp194s0f0"<\/p>\n

                                                                    Environment="VLLM_HOST_IP=10.255.254.44"<\/p>\n

                                                                    Environment="CUDA_VISIBLE_DEVICES=6,7"<\/p>\n

                                                                    \n

                                                                    ExecStart=\/opt\/venv\/vllm\/bin\/vllm serve \/data\/models\/Qwen3.6-27B-AWQ \\\\<\/p>\n

                                                                    –trust-remote-code \\\\<\/p>\n

                                                                    –tensor-parallel-size 2 \\\\<\/p>\n

                                                                    –gpu-memory-utilization 0.80 \\\\<\/p>\n

                                                                    –max-model-len 262144 \\\\<\/p>\n

                                                                    –kv-cache-dtype fp8 \\\\<\/p>\n

                                                                    –disable-custom-all-reduce \\\\<\/p>\n

                                                                    –enforce-eager \\\\<\/p>\n

                                                                    –port 8003 \\\\<\/p>\n

                                                                    –host 10.255.254.44 \\\\<\/p>\n

                                                                    –distributed-executor-backend mp \\\\<\/p>\n

                                                                    –api-key sk-qwen-27b-lb-2026<\/p>\n

                                                                    \n

                                                                    Restart=always<\/p>\n

                                                                    RestartSec=5<\/p>\n

                                                                    LimitNOFILE=65535<\/p>\n

                                                                    \n

                                                                    [Install]<\/p>\n

                                                                    WantedBy=multi-user.target<\/p>\n

                                                                    EOF<\/p>\n

                                                                    \n

                                                                    \n

                                                                    \n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    \n

                                                                    \u4e09\u3001\u52a0\u8f7d\u670d\u52a1\u5e76\u8bbe\u7f6e\u5f00\u673a\u81ea\u542f<\/p>\n\n\n\n
                                                                    \n

                                                                    bash<\/span><\/p>\n

                                                                    sudo systemctl daemon-reload<\/p>\n

                                                                    sudo systemctl enable vllm-8000<\/p>\n

                                                                    sudo systemctl enable vllm-8001<\/p>\n

                                                                    sudo systemctl enable vllm-8002<\/p>\n

                                                                    sudo systemctl enable vllm-8003<\/p>\n

                                                                    \n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    \u56db\u3001\u4e00\u952e\u542f\u52a8\u5168\u90e8 4 \u4e2a\u670d\u52a1<\/p>\n\n\n\n
                                                                    \n

                                                                    bas<\/span><\/p>\n

                                                                    sudo systemctl start vllm-8000<\/p>\n

                                                                    sudo systemctl start vllm-8001<\/p>\n

                                                                    sudo systemctl start vllm-8002<\/p>\n

                                                                    sudo systemctl start vllm-8003<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    \u4e94\u3001\u67e5\u770b\u670d\u52a1\u8fd0\u884c\u72b6\u6001<\/p>\n\n\n\n
                                                                    \n

                                                                    bash<\/span><\/p>\n

                                                                    sudo systemctl status vllm-8000<\/p>\n

                                                                    sudo systemctl status vllm-8001<\/p>\n

                                                                    sudo systemctl status vllm-8002<\/p>\n

                                                                    sudo systemctl status vllm-8003<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    \u516d\u3001\u65e0\u4eba\u503c\u5b88\u6838\u5fc3\u80fd\u529b\u8bf4\u660e<\/p>\n

                                                                  • \u670d\u52a1\u5668\u5f00\u673a \u2192 4 \u4e2a vLLM \u670d\u52a1\u81ea\u52a8\u542f\u52a8<\/li>\n
                                                                  • \u670d\u52a1\u5f02\u5e38\u5d29\u6e83 \u2192 5 \u79d2\u81ea\u52a8\u91cd\u542f<\/li>\n
                                                                  • \u5173\u95ed\u7ec8\u7aef \/ \u65ad\u5f00\u8fde\u63a5 \u2192 \u670d\u52a1\u4e0d\u53d7\u5f71\u54cd<\/li>\n
                                                                  • \u957f\u65f6\u95f4\u8fd0\u884c \u2192 \u7a33\u5b9a\u4e0d\u6389\u7ebf\u3001\u4e0d\u81ea\u52a8\u9000\u51fa<\/li>\n
                                                                  • Nginx + \u8d1f\u8f7d\u5747\u8861 + \u516c\u7f51\u8bbf\u95ee\u6c38\u4e45\u53ef\u7528<\/li>\n

                                                                    \u4e03\u3001\u65e5\u5e38\u8fd0\u7ef4\u547d\u4ee4<\/h3>\n\n\n\n
                                                                    \n

                                                                    bash<\/span><\/p>\n

                                                                    # \u67e5\u770b\u5355\u4e2a\u670d\u52a1\u72b6\u6001<\/p>\n

                                                                    sudo systemctl status vllm-8000<\/p>\n

                                                                    # \u91cd\u542f\u5355\u4e2a\u670d\u52a1<\/p>\n

                                                                    sudo systemctl restart vllm-8000<\/p>\n

                                                                    # \u91cd\u542f\u5168\u90e8 4 \u4e2a\u670d\u52a1<\/p>\n

                                                                    sudo systemctl restart vllm-8000 vllm-8001 vllm-8002 vllm-8003<\/p>\n

                                                                    # \u67e5\u770b\u5b9e\u65f6\u65e5\u5fd7sudo journalctl -u vllm-8000 -f<\/p>\n

                                                                    \n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

                                                                    \n","protected":false},"excerpt":{"rendered":"

                                                                    \u9644：4\u670d\u52a1\u6a21\u578b API\u8bbe\u7f6e\u6d4b\u8bd5\u00a0\u6d41\u7a0b0. \u524d\u671f\u90e8\u7f72\u51c6\u5907\u6d41\u7a0b（\u4f9d\u8d56\u5b89\u88c5 \u6a21\u578b\u4e0b\u8f7d \u6821\u9a8c\u811a\u672c）0.1 \u6574\u4f53\u90e8\u7f72\u8bf4\u660e\u672c\u6b21\u63a8\u7406\u90e8\u7f72\u91c7\u7528\u79bb\u7ebf\u7eaf\u51c0\u90e8\u7f72\u65b9\u6848，\u57fa\u4e8eLinux\u670d\u52a1\u5668\u3001Conda\u865a\u62df\u73af\u5883\u9694\u79bb，\u5168\u7a0b\u9501\u5b9a\u4f9d\u8d56\u7248\u672c\u3001\u56fa\u5316\u73af\u5883\u53d8\u91cf\u3001\u5185\u7f51\u62f7\u8d1d\u6a21\u578b，\u65e0\u5916\u7f51\u4e0d\u7a33\u5b9a\u4e0b\u8f7d，\u53ef1:1\u590d\u523b\u751f\u4ea7\u73af\u5883\u3002\u672c\u7ae0\u5b8c\u6574\u8bb0\u5f55：\u73af\u5883\u642d\u5efa\u3001\u5b89\u88c5\u547d\u4ee4\u3001\u4f9d\u8d56\u7248\u672c\u3001\u73af\u5883\u53d8\u91cf\u3001\u6a21\u578b\u62f7\u8d1d\u3001\u6587\u4ef6\u6821\u9a8c\u3001\u4f9d\u8d56\u81ea\u68c0\u811a\u672c\u3002<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[43,44],"topic":[],"class_list":["post-82900","post","type-post","status-publish","format-standard","hentry","category-server","tag-43","tag-44"],"yoast_head":"\n4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.wsisp.com\/helps\/82900.html\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"og:description\" content=\"\u9644：4\u670d\u52a1\u6a21\u578b API\u8bbe\u7f6e\u6d4b\u8bd5\u00a0\u6d41\u7a0b0. \u524d\u671f\u90e8\u7f72\u51c6\u5907\u6d41\u7a0b（\u4f9d\u8d56\u5b89\u88c5 \u6a21\u578b\u4e0b\u8f7d \u6821\u9a8c\u811a\u672c）0.1 \u6574\u4f53\u90e8\u7f72\u8bf4\u660e\u672c\u6b21\u63a8\u7406\u90e8\u7f72\u91c7\u7528\u79bb\u7ebf\u7eaf\u51c0\u90e8\u7f72\u65b9\u6848，\u57fa\u4e8eLinux\u670d\u52a1\u5668\u3001Conda\u865a\u62df\u73af\u5883\u9694\u79bb，\u5168\u7a0b\u9501\u5b9a\u4f9d\u8d56\u7248\u672c\u3001\u56fa\u5316\u73af\u5883\u53d8\u91cf\u3001\u5185\u7f51\u62f7\u8d1d\u6a21\u578b，\u65e0\u5916\u7f51\u4e0d\u7a33\u5b9a\u4e0b\u8f7d，\u53ef1:1\u590d\u523b\u751f\u4ea7\u73af\u5883\u3002\u672c\u7ae0\u5b8c\u6574\u8bb0\u5f55：\u73af\u5883\u642d\u5efa\u3001\u5b89\u88c5\u547d\u4ee4\u3001\u4f9d\u8d56\u7248\u672c\u3001\u73af\u5883\u53d8\u91cf\u3001\u6a21\u578b\u62f7\u8d1d\u3001\u6587\u4ef6\u6821\u9a8c\u3001\u4f9d\u8d56\u81ea\u68c0\u811a\u672c\u3002\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.wsisp.com\/helps\/82900.html\" \/>\n<meta property=\"og:site_name\" content=\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"article:published_time\" content=\"2026-07-25T06:33:36+00:00\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/82900.html\",\"url\":\"https:\/\/www.wsisp.com\/helps\/82900.html\",\"name\":\"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"isPartOf\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\"},\"datePublished\":\"2026-07-25T06:33:36+00:00\",\"dateModified\":\"2026-07-25T06:33:36+00:00\",\"author\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/82900.html#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.wsisp.com\/helps\/82900.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/82900.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.wsisp.com\/helps\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\",\"url\":\"https:\/\/www.wsisp.com\/helps\/\",\"name\":\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"description\":\"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"contentUrl\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"caption\":\"admin\"},\"sameAs\":[\"http:\/\/wp.wsisp.com\"],\"url\":\"https:\/\/www.wsisp.com\/helps\/author\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.wsisp.com\/helps\/82900.html","og_locale":"zh_CN","og_type":"article","og_title":"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","og_description":"\u9644：4\u670d\u52a1\u6a21\u578b API\u8bbe\u7f6e\u6d4b\u8bd5\u00a0\u6d41\u7a0b0. \u524d\u671f\u90e8\u7f72\u51c6\u5907\u6d41\u7a0b（\u4f9d\u8d56\u5b89\u88c5 \u6a21\u578b\u4e0b\u8f7d \u6821\u9a8c\u811a\u672c）0.1 \u6574\u4f53\u90e8\u7f72\u8bf4\u660e\u672c\u6b21\u63a8\u7406\u90e8\u7f72\u91c7\u7528\u79bb\u7ebf\u7eaf\u51c0\u90e8\u7f72\u65b9\u6848，\u57fa\u4e8eLinux\u670d\u52a1\u5668\u3001Conda\u865a\u62df\u73af\u5883\u9694\u79bb，\u5168\u7a0b\u9501\u5b9a\u4f9d\u8d56\u7248\u672c\u3001\u56fa\u5316\u73af\u5883\u53d8\u91cf\u3001\u5185\u7f51\u62f7\u8d1d\u6a21\u578b，\u65e0\u5916\u7f51\u4e0d\u7a33\u5b9a\u4e0b\u8f7d，\u53ef1:1\u590d\u523b\u751f\u4ea7\u73af\u5883\u3002\u672c\u7ae0\u5b8c\u6574\u8bb0\u5f55：\u73af\u5883\u642d\u5efa\u3001\u5b89\u88c5\u547d\u4ee4\u3001\u4f9d\u8d56\u7248\u672c\u3001\u73af\u5883\u53d8\u91cf\u3001\u6a21\u578b\u62f7\u8d1d\u3001\u6587\u4ef6\u6821\u9a8c\u3001\u4f9d\u8d56\u81ea\u68c0\u811a\u672c\u3002","og_url":"https:\/\/www.wsisp.com\/helps\/82900.html","og_site_name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","article_published_time":"2026-07-25T06:33:36+00:00","author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"8 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.wsisp.com\/helps\/82900.html","url":"https:\/\/www.wsisp.com\/helps\/82900.html","name":"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","isPartOf":{"@id":"https:\/\/www.wsisp.com\/helps\/#website"},"datePublished":"2026-07-25T06:33:36+00:00","dateModified":"2026-07-25T06:33:36+00:00","author":{"@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41"},"breadcrumb":{"@id":"https:\/\/www.wsisp.com\/helps\/82900.html#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.wsisp.com\/helps\/82900.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.wsisp.com\/helps\/82900.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.wsisp.com\/helps"},{"@type":"ListItem","position":2,"name":"4090-8\u5361\u670d\u52a1\u5668-\u751f\u4ea7\u529b\uff08\u8bfb\u53d6\uff1a813tokens\/s\uff0c\u8f93\u51fa\uff1a101tokens\/s\uff09-Qwen3.6-27B-AWQ\u90e8\u7f72\u548c\u6d4b\u8bd5\u62a5\u544a"}]},{"@type":"WebSite","@id":"https:\/\/www.wsisp.com\/helps\/#website","url":"https:\/\/www.wsisp.com\/helps\/","name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","description":"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41","name":"admin","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/","url":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","contentUrl":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","caption":"admin"},"sameAs":["http:\/\/wp.wsisp.com"],"url":"https:\/\/www.wsisp.com\/helps\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/82900","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/comments?post=82900"}],"version-history":[{"count":0,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/82900\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media?parent=82900"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/categories?post=82900"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/tags?post=82900"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/topic?post=82900"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}