{"id":91928,"date":"2026-08-08T19:32:46","date_gmt":"2026-08-08T11:32:46","guid":{"rendered":"https:\/\/www.wsisp.com\/helps\/91928.html"},"modified":"2026-08-08T19:32:46","modified_gmt":"2026-08-08T11:32:46","slug":"%e9%87%8f%e5%8c%96%e5%9f%ba%e7%a1%80%ef%bc%9a%e4%bb%8e-fp32-%e5%88%b0-int4-%e7%9a%84%e5%8e%8b%e7%bc%a9%e8%89%ba%e6%9c%af","status":"publish","type":"post","link":"https:\/\/www.wsisp.com\/helps\/91928.html","title":{"rendered":"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f"},"content":{"rendered":"<\/p>\n

\u672c\u7ae0\u7b80\u4ecb<\/h3>\n

\u5927\u6a21\u578b\u63a8\u7406\u7684\u74f6\u9888\u5f80\u5f80\u4e0d\u662f\u7b97\u529b\u800c\u662f\u8bbf\u5b58：decode \u9636\u6bb5\u6bcf\u751f\u6210\u4e00\u4e2a token \u90fd\u8981\u628a\u5168\u90e8\u6743\u91cd\u4ece\u663e\u5b58\u8bfb\u4e00\u904d，\u628a\u6bcf\u4e2a\u53c2\u6570\u7684\u5b57\u8282\u6570\u780d\u534a，\u7406\u8bba\u8017\u65f6\u5c31\u80fd\u780d\u534a\u3002\u91cf\u5316（Quantization）\u6b63\u662f\u76f4\u63a5\u653b\u51fb"\u5b57\u8282\u6570"\u8fd9\u4e2a\u5206\u6bcd\u7684\u7b2c\u4e00\u624b\u6bb5\u3002\u4f46"INT8 \u662f\u4ec0\u4e48\u3001\u4e3a\u4ec0\u4e48\u80fd\u538b\u7f29\u3001\u4ee3\u4ef7\u662f\u4ec0\u4e48"\u8fd9\u7c7b\u95ee\u9898，\u5f88\u591a\u5de5\u7a0b\u5e08\u5e76\u6ca1\u6709\u4ece\u6570\u5b66\u4e0a\u8bb2\u6e05\u695a\u8fc7\u3002<\/p>\n

\u672c\u7bc7\u8981\u89e3\u51b3\u56db\u4e2a\u95ee\u9898：<\/p>\n

  • FP32\/FP16\/BF16\/FP8\/INT8\/INT4 \u8fd9\u4e9b\u683c\u5f0f，\u5728\u4f4d\u5bbd\u3001\u52a8\u6001\u8303\u56f4\u3001\u7cbe\u5ea6\u4e0a\u5230\u5e95\u5dee\u5728\u54ea？<\/li>\n
  • \u91cf\u5316\u6620\u5c04\u7684\u6570\u5b66\u672c\u8d28\u662f\u4ec0\u4e48？scale\u3001zero-point\u3001clamp \u4e09\u4e2a\u8981\u7d20\u5982\u4f55\u51b3\u5b9a\u8bef\u5dee\u4e0a\u754c？<\/li>\n
  • Per-tensor \/ Per-channel \/ Per-group \/ Per-token \u56db\u79cd\u7c92\u5ea6\u600e\u4e48\u9009？PTQ \u548c QAT \u4e24\u6761\u8def\u7ebf\u600e\u4e48\u9009？<\/li>\n
  • \u80fd\u4e0d\u80fd\u624b\u5199\u4e00\u4e2a\u91cf\u5316\u5668，\u4eb2\u773c\u770b\u4e00\u770b\u8bef\u5dee\u957f\u4ec0\u4e48\u6837？<\/li>\n

    \u8bfb\u5b8c\u540e，\u4f60\u5e94\u8be5\u80fd\u770b\u61c2\u4efb\u4f55\u4e00\u4efd\u91cf\u5316\u65b9\u6848\u6587\u6863\u91cc\u7684\u672f\u8bed\u4e0e\u53c2\u6570，\u5e76\u4e3a\u540e\u7eed （\u6fc0\u6d3b outlier \u4e0e SmoothQuant\u3001GPTQ\u3001AWQ\u3001KV Cache \u91cf\u5316\u3001FP8 \u5b9e\u8df5）\u6253\u597d\u6570\u5b66\u57fa\u7840\u3002<\/p>\n

    1. \u6570\u503c\u8868\u793a\u57fa\u7840：\u4f4d\u5bbd\u3001\u8303\u56f4\u4e0e\u7cbe\u5ea6\u7684\u4e09\u89d2\u7ea6\u675f<\/h3>\n

    1.1 \u6d6e\u70b9\u5bb6\u65cf：\u6307\u6570\u4f4d\u6362\u8303\u56f4，\u5c3e\u6570\u4f4d\u6362\u7cbe\u5ea6<\/h4>\n

    \u6d6e\u70b9\u6570\u7684\u901a\u7528\u7ed3\u6784\u662f"\u7b26\u53f7\u4f4d + \u6307\u6570\u4f4d + \u5c3e\u6570\u4f4d"：<\/p>\n

    \u503c = (-1)^sign \u00d7 2^(exponent – bias) \u00d7 (1 + mantissa \/ 2^m)<\/p>\n

      \n
    • \u6307\u6570\u4f4d\u8d8a\u591a，\u52a8\u6001\u8303\u56f4\u8d8a\u5927：\u80fd\u8868\u793a\u7684\u6700\u5927\/\u6700\u5c0f\u503c\u8de8\u5ea6\u8d8a\u5927，\u4e0d\u5bb9\u6613\u6ea2\u51fa；<\/li>\n
    • \u5c3e\u6570\u4f4d\u8d8a\u591a，\u76f8\u5bf9\u7cbe\u5ea6\u8d8a\u9ad8：\u76f8\u90bb\u4e24\u4e2a\u53ef\u8868\u793a\u6570\u4e4b\u95f4\u7684\u95f4\u9694\u8d8a\u5c0f\u3002<\/li>\n<\/ul>\n

      FP32（1+8+23）\u662f\u6df1\u5ea6\u5b66\u4e60\u7684\u57fa\u51c6\u683c\u5f0f\u3002\u540e\u6765\u7684\u4f4e\u7cbe\u5ea6\u6d6e\u70b9\u683c\u5f0f，\u672c\u8d28\u4e0a\u90fd\u662f\u5728"\u6307\u6570\u4f4d \u00d7 \u5c3e\u6570\u4f4d"\u7684\u9884\u7b97\u91cc\u505a\u4e0d\u540c\u6743\u8861：<\/p>\n

        \n
      • FP16（1+5+10）：\u6307\u6570\u53ea\u5269 5 \u4f4d，\u6700\u5927\u53ea\u80fd\u8868\u793a\u5230 65504，\u8bad\u7ec3\u65f6\u68af\u5ea6\u5bb9\u6613\u6ea2\u51fa，\u56e0\u6b64\u9700\u8981 loss scaling；<\/li>\n
      • BF16（Brain Floating Point 16，1+8+7）：\u4fdd\u7559\u4e0e FP32 \u76f8\u540c\u7684 8 \u4f4d\u6307\u6570，\u52a8\u6001\u8303\u56f4\u4e0d\u53d7\u5f71\u54cd，\u4ee3\u4ef7\u662f\u5c3e\u6570\u53ea\u6709 7 \u4f4d\u3001\u7cbe\u5ea6\u53d8\u7c97\u3002Google \u4e3a TPU \u8bbe\u8ba1，\u73b0\u5df2\u6210\u4e3a LLM \u8bad\u7ec3\u4e0e\u63a8\u7406\u7684\u4e3b\u6d41\u683c\u5f0f；<\/li>\n
      • TF32（TensorFloat-32，1+8+10）：NVIDIA \u5728 Ampere \u67b6\u6784（A100，2020）\u5f15\u5165\u7684\u5185\u90e8\u683c\u5f0f\u2014\u2014\u5bf9\u5916\u4ecd\u662f FP32 \u5b58\u50a8，tensor core \u5185\u90e8\u622a\u65ad\u6210 19 \u4f4d\u8ba1\u7b97，\u7528\u7cbe\u5ea6\u6362\u901f\u5ea6；<\/li>\n
      • FP8（1+4+3 \u6216 1+5+2）：Hopper \u67b6\u6784（H100，2022）\u5f15\u5165，E4M3 \u6700\u5927\u7ea6 448，\u901a\u5e38\u914d\u5408 Transformer Engine \u7684\u9010\u5c42\u7f29\u653e\u4f7f\u7528\u3002<\/li>\n<\/ul>\n

        1.2 \u6574\u6570\u5bb6\u65cf：\u5747\u5300\u7f51\u683c\u4e0a\u7684\u5b9a\u70b9\u6570<\/h4>\n

        INT8\/INT4 \u662f\u53e6\u4e00\u5957\u903b\u8f91：\u6ca1\u6709\u6307\u6570，\u6240\u6709\u53ef\u8868\u793a\u503c\u6784\u6210\u4e00\u4e2a\u7edd\u5bf9\u5747\u5300\u7684\u7f51\u683c\u3002\u8303\u56f4\u5185\u5206\u8fa8\u7387\u6052\u5b9a，\u8303\u56f4\u5916\u7684\u503c\u76f4\u63a5\u88ab clamp（\u622a\u65ad）\u3002\u8fd9\u610f\u5473\u7740\u6574\u6570\u683c\u5f0f\u7684"\u52a8\u6001\u8303\u56f4"\u4e0d\u662f\u6307\u6570\u7ed9\u7684，\u800c\u662f\u9760\u4e00\u4e2a\u5916\u90e8\u6d6e\u70b9\u7f29\u653e\u7cfb\u6570（scale）\u6491\u51fa\u6765\u7684\u2014\u2014\u8fd9\u6b63\u662f\u7b2c 2 \u8282\u91cf\u5316\u6620\u5c04\u8981\u89e3\u51b3\u7684\u95ee\u9898\u3002<\/p>\n

        1.3 \u4e03\u79cd\u683c\u5f0f\u6a2a\u5411\u5bf9\u6bd4<\/h4>\n\n\u6570\u636e\u7c7b\u578b\u4f4d\u5bbd\u6307\u6570\u4f4d\/\u5c3e\u6570\u4f4d\u52a8\u6001\u8303\u56f4\u5178\u578b\u5e94\u7528\u573a\u666f<\/tr>\n\n\n\n\n\n\n\n\n
        FP32<\/td>\n32<\/td>\n8 \/ 23<\/td>\n\u00b13.4\u00d710\u00b3\u2078<\/td>\n\u8bad\u7ec3\u57fa\u51c6\u3001master weights\u3001\u9ad8\u7cbe\u5ea6\u7d2f\u52a0<\/td>\n<\/tr>\n
        TF32<\/td>\n19（\u5b58 32）<\/td>\n8 \/ 10<\/td>\n\u00b13.4\u00d710\u00b3\u2078<\/td>\nAmpere+ \u8bad\u7ec3\u77e9\u9635\u4e58\u5185\u90e8\u52a0\u901f<\/td>\n<\/tr>\n
        FP16<\/td>\n16<\/td>\n5 \/ 10<\/td>\n\u00b165504<\/td>\n\u6df7\u5408\u7cbe\u5ea6\u8bad\u7ec3\u3001\u63a8\u7406\u5b58\u50a8\u4e0e\u8ba1\u7b97<\/td>\n<\/tr>\n
        BF16<\/td>\n16<\/td>\n8 \/ 7<\/td>\n\u00b13.4\u00d710\u00b3\u2078<\/td>\nLLM \u8bad\u7ec3\u4e0e\u63a8\u7406\u4e3b\u6d41\u683c\u5f0f<\/td>\n<\/tr>\n
        FP8-E4M3<\/td>\n8<\/td>\n4 \/ 3<\/td>\n\u00b1448<\/td>\nHopper+ \u8bad\u7ec3\u524d\u5411\u3001\u63a8\u7406 GEMM\u3001KV Cache<\/td>\n<\/tr>\n
        INT8<\/td>\n8<\/td>\n\u2014（\u5b9a\u70b9）<\/td>\n[-128, 127] \u5747\u5300\u7f51\u683c<\/td>\n\u63a8\u7406\u6743\u91cd\/\u6fc0\u6d3b\/KV \u91cf\u5316，\u5de5\u4e1a\u754c\u6700\u6210\u719f<\/td>\n<\/tr>\n
        INT4<\/td>\n4<\/td>\n\u2014（\u5b9a\u70b9）<\/td>\n[-8, 7] \u5747\u5300\u7f51\u683c<\/td>\n\u6781\u81f4\u6743\u91cd\u91cf\u5316（W4A16，GPTQ\/AWQ）<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        \u786c\u4ef6\u4fa7\u7684\u5bf9\u7167（NVIDIA \u5b98\u65b9\u516c\u5f00\u89c4\u683c）：H100 SXM \u7684 FP16\/BF16 tensor core \u5cf0\u503c\u7ea6 989.5 TFLOPS（\u7a20\u5bc6），FP8 \u7ffb\u500d\u5230\u7ea6 1979 TFLOPS；\u663e\u5b58\u5e26\u5bbd 3.35 TB\/s（80GB HBM3）\u3002\u5230 Blackwell \u4e00\u4ee3，B200 \u8fdb\u4e00\u6b65\u652f\u6301 FP4，\u663e\u5b58\u5e26\u5bbd\u63d0\u5347\u5230 8 TB\/s（192GB HBM3e）\u3002\u683c\u5f0f\u4f4d\u5bbd\u6bcf\u780d\u534a，\u540c\u6837\u5e26\u5bbd\u4e0b\u5355\u4f4d\u65f6\u95f4\u80fd\u642c\u8fd0\u7684\u5143\u7d20\u6570\u5c31\u7ffb\u500d\u2014\u2014\u8fd9\u5c31\u662f\u91cf\u5316\u52a0\u901f\u7684\u5168\u90e8\u7269\u7406\u57fa\u7840\u3002<\/p>\n

        \u4e00\u53e5\u8bdd\u603b\u7ed3：\u6d6e\u70b9\u683c\u5f0f\u7528\u6307\u6570\u4f4d\u6362\u52a8\u6001\u8303\u56f4\u3001\u7528\u5c3e\u6570\u4f4d\u6362\u7cbe\u5ea6；\u6574\u6570\u683c\u5f0f\u6ca1\u6709\u6307\u6570，\u9760\u5916\u90e8 scale \u5b9a\u8303\u56f4\u3002\u91cf\u5316\u9009\u578b\u7684\u7b2c\u4e00\u6b65，\u662f\u770b\u6e05\u4f60\u7684\u6570\u636e\u5206\u5e03\u843d\u5728"\u8303\u56f4-\u7cbe\u5ea6"\u5730\u56fe\u7684\u54ea\u4e2a\u4f4d\u7f6e\u3002<\/p>\n

        2. \u91cf\u5316\u6620\u5c04\u539f\u7406：scale\u3001zero-point \u4e0e clamp<\/h3>\n

        2.1 \u4ece\u8fde\u7eed\u5230\u79bb\u6563：\u4eff\u5c04\u6620\u5c04<\/h4>\n

        \u91cf\u5316\u7684\u6570\u5b66\u672c\u8d28，\u662f\u628a\u8fde\u7eed\u6d6e\u70b9\u503c\u6620\u5c04\u5230\u6709\u9650\u7684\u6574\u6570\u7f51\u683c\u4e0a\u3002\u6700\u5e38\u7528\u7684\u662f\u5747\u5300\u4eff\u5c04\u91cf\u5316（uniform affine quantization），\u5bf9\u6d6e\u70b9\u5f20\u91cf x \u7684\u6bcf\u4e2a\u5143\u7d20：<\/p>\n

        \u91cf\u5316（float \u2192 int）： q = clamp(round(x \/ s) + z, q_min, q_max)
        \n\u53cd\u91cf\u5316（int \u2192 float）：x\u0302 = (q – z) \u00b7 s<\/p>\n

        \u4e09\u4e2a\u8981\u7d20：<\/p>\n

          \n
        • scale s：\u6b65\u957f，\u4e00\u4e2a\u91cf\u5316\u7801\u5bf9\u5e94\u591a\u5c11\u6d6e\u70b9\u5355\u4f4d\u3002s \u662f\u552f\u4e00\u540c\u65f6\u51b3\u5b9a\u7cbe\u5ea6\u4e0e\u7801\u70b9\u5229\u7528\u7387\u7684\u65cb\u94ae；<\/li>\n
        • zero-point z：\u6574\u6570\u96f6\u70b9，\u4fdd\u8bc1\u6d6e\u70b9 0 \u80fd\u88ab\u7cbe\u786e\u8868\u793a（\u5bf9 padding\u3001ReLU \u540e\u7684\u96f6\u503c\u5f88\u91cd\u8981）；<\/li>\n
        • clamp：\u628a\u8d85\u51fa\u7801\u70b9\u8303\u56f4\u7684\u503c\u622a\u65ad\u5230 [q_min, q_max]，\u7531\u6b64\u5f15\u5165\u88c1\u526a\u8bef\u5dee（\u7b2c 5 \u8282\u8be6\u8ff0）\u3002<\/li>\n<\/ul>\n

          2.2 \u5bf9\u79f0\u91cf\u5316：\u96f6\u70b9\u56fa\u5b9a\u5728 0<\/h4>\n

          \u5bf9\u79f0\u91cf\u5316（Symmetric Quantization，\u5bf9\u79f0\u91cf\u5316）\u4ee4 z = 0，\u7801\u70b9\u5173\u4e8e 0 \u5bf9\u79f0：<\/p>\n

          s = max|x| \/ q_max （INT8 \u65f6 q_max = 127）
          \nq = clamp(round(x \/ s), -127, 127)<\/p>\n

          \u6ce8\u610f INT8 \u523b\u610f\u53ea\u7528 [-127, 127] \u800c\u4e0d\u7528 -128：\u4fdd\u6301\u6b63\u8d1f\u5bf9\u79f0，\u907f\u514d -(-128) \u8fd9\u7c7b\u7b26\u53f7\u7ffb\u8f6c\u5728\u5b9a\u70b9\u8fd0\u7b97\u4e2d\u6ea2\u51fa\u3002\u6743\u91cd\u5206\u5e03\u5929\u7136\u96f6\u4e2d\u5fc3，\u5bf9\u79f0\u91cf\u5316\u4e0d\u9700\u8981\u5b58 zero-point，kernel \u91cc\u5c11\u4e00\u6b21\u51cf\u6cd5，\u662f\u6743\u91cd\u91cf\u5316\u7684\u9ed8\u8ba4\u9009\u62e9\u3002<\/p>\n

          2.3 \u975e\u5bf9\u79f0\u91cf\u5316：\u8ba9\u7801\u70b9\u94fa\u6ee1\u5b9e\u9645\u533a\u95f4<\/h4>\n

          \u975e\u5bf9\u79f0\u91cf\u5316（Asymmetric Quantization，\u975e\u5bf9\u79f0\u91cf\u5316）\u5141\u8bb8 z \u2260 0，\u7528\u5b9e\u9645\u7684\u6700\u5c0f\/\u6700\u5927\u503c\u5b9a\u6807：<\/p>\n

          s = (x_max – x_min) \/ (q_max – q_min)
          \nz = clamp(round(q_min – x_min \/ s), q_min, q_max)<\/p>\n

          \u5f53\u6570\u636e\u660e\u663e\u504f\u79bb\u96f6\u4e2d\u5fc3\u65f6（\u6bd4\u5982 GELU \u4e4b\u540e\u7684\u6fc0\u6d3b，\u975e\u8d1f\u4e14\u504f\u659c），\u5bf9\u79f0\u91cf\u5316\u4f1a\u628a\u4e00\u534a\u7801\u70b9\u6d6a\u8d39\u5728\u6c38\u8fdc\u4e0d\u51fa\u73b0\u7684\u8d1f\u503c\u4e0a；\u975e\u5bf9\u79f0\u91cf\u5316\u901a\u8fc7\u5e73\u79fb zero-point，\u628a 256 \u4e2a\u7801\u70b9\u5168\u90e8\u94fa\u5230\u6570\u636e\u5b9e\u9645\u8986\u76d6\u7684\u533a\u95f4，\u6709\u6548\u5206\u8fa8\u7387\u63a5\u8fd1\u7ffb\u500d\u3002\u4ee3\u4ef7\u662f\u8981\u591a\u5b58\u4e00\u4e2a z，kernel \u91cc\u591a\u4e00\u6b65\u504f\u79fb\u5904\u7406\u3002<\/p>\n

          2.4 \u8bef\u5dee\u754c\u63a8\u5bfc：|x\u0302 \u2212 x| \u2264 s\/2<\/h4>\n

          \u4ee5\u5bf9\u79f0\u91cf\u5316\u4e3a\u4f8b\u3002\u5f53\u5143\u7d20\u6ca1\u6709\u89e6\u53d1 clamp \u65f6，round \u7684\u820d\u5165\u8bef\u5dee\u81f3\u591a\u4e3a\u534a\u4e2a\u6b65\u957f：<\/p>\n

          |round(x\/s) \u2212 x\/s| \u2264 1\/2
          \n\u27f9 |x\u0302 \u2212 x| = s \u00b7 |round(x\/s) \u2212 x\/s| \u2264 s\/2<\/p>\n

          \u8bb0\u4f4f\u8fd9\u4e2a\u4e0d\u7b49\u5f0f：\u5355\u70b9\u7edd\u5bf9\u8bef\u5dee |x\u0302 \u2212 x| \u2264 s\/2\u3002\u5b83\u662f\u7edd\u5bf9\u8bef\u5dee\u800c\u975e\u76f8\u5bf9\u8bef\u5dee\u2014\u2014\u5bf9\u5927\u5e45\u503c\u5143\u7d20，\u91cf\u5316\u51e0\u4e4e\u65e0\u611f；\u5bf9\u5c0f\u5e45\u503c\u5143\u7d20，\u5f53 |x| < s\/2 \u65f6\u4f1a\u88ab\u76f4\u63a5\u5f52\u96f6\u3002\u800c s = max|x| \/ 127 \u7531\u8986\u76d6\u8303\u56f4\u5185\u7684\u6700\u5927\u503c\u51b3\u5b9a：\u4e00\u4e2a outlier \u843d\u8fdb\u6765，\u8303\u56f4\u5185\u6240\u6709\u5143\u7d20\u7684\u5206\u8fa8\u7387\u90fd\u88ab\u62c9\u4f4e\u3002\u8fd9\u4e00\u4e2a\u4e0d\u7b49\u5f0f，\u5c31\u662f\u7b2c 3 \u8282"\u7c92\u5ea6\u4e4b\u4e89"\u548c\u7b2c 4.2 \u7bc7 SmoothQuant \u7684\u5168\u90e8\u6839\u6e90\u3002<\/p>\n

          3. \u91cf\u5316\u7c92\u5ea6：\u4e00\u4e2a scale \u7ba1\u591a\u5c11\u5143\u7d20<\/h3>\n

          \u540c\u4e00\u4e2a INT8，"\u4e00\u4e2a scale \u8986\u76d6\u591a\u5927\u8303\u56f4"\u6709\u56db\u79cd\u5178\u578b\u9009\u62e9\u3002\u4ee5\u5f62\u72b6\u4e3a [M, K] \u7684\u6743\u91cd\u77e9\u9635（M \u8f93\u5165\u7ef4\u3001K \u8f93\u51fa\u901a\u9053）\u4e3a\u4f8b：<\/p>\n\n\u7c92\u5ea6scale \u6570\u91cf\u7cbe\u5ea6\u5143\u6570\u636e\u5f00\u9500\u5178\u578b\u573a\u666f<\/tr>\n\n\n\n\n\n
          Per-tensor<\/td>\n1<\/td>\n\u6700\u4f4e<\/td>\n\u6700\u5c0f<\/td>\n\u9759\u6001\u6fc0\u6d3b\u91cf\u5316\u3001\u65e9\u671f\u90e8\u7f72\u65b9\u6848<\/td>\n<\/tr>\n
          Per-channel<\/td>\nK（\u6bcf\u8f93\u51fa\u901a\u9053 1 \u4e2a）<\/td>\n\u9ad8<\/td>\n\u5c0f<\/td>\n\u6743\u91cd\u91cf\u5316\u9ed8\u8ba4\u914d\u7f6e（W8\/W4）<\/td>\n<\/tr>\n
          Per-group<\/td>\nK \u00d7 M\/g（g \u5e38\u53d6 64\/128）<\/td>\n\u66f4\u9ad8<\/td>\n\u4e2d（\u5b58\u50a8\u7565\u6da8）<\/td>\nINT4 \u6743\u91cd，GPTQ\/AWQ \u6807\u914d<\/td>\n<\/tr>\n
          Per-token<\/td>\n\u6bcf token 1 \u4e2a<\/td>\n\u9ad8<\/td>\n\u5c0f<\/td>\n\u6fc0\u6d3b \/ KV Cache \u52a8\u6001\u91cf\u5316<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

          \u7c92\u5ea6\u9009\u62e9\u7684\u672c\u8d28，\u662f\u8ba9 scale \u7684\u8986\u76d6\u8303\u56f4\u5339\u914d\u6570\u636e\u7684 outlier \u7ed3\u6784：<\/p>\n

            \n
          • \u6743\u91cd\u662f\u9759\u6001\u7684，\u53ef\u4ee5\u79bb\u7ebf\u6cbf\u4efb\u610f\u7ef4\u5ea6\u5f52\u7ea6，per-channel \u51e0\u4e4e\u96f6\u6210\u672c；INT4 \u518d\u4e0a group-wise \u8fdb\u4e00\u6b65\u538b\u8bef\u5dee；<\/li>\n
          • \u6fc0\u6d3b\u9010 token \u5230\u8fbe，\u53ea\u80fd\u6cbf\u7279\u5f81\u7ef4\u5728\u7ebf\u5f52\u7ea6，per-token \u52a8\u6001\u91cf\u5316\u662f\u552f\u4e00\u5728\u7ebf\u53ef\u884c\u7684\u7ec6\u7c92\u5ea6\u65b9\u6848；<\/li>\n
          • LLM \u6fc0\u6d3b\u7684 outlier \u6309 channel \u805a\u96c6，per-tensor \/ per-token \u90fd\u4f1a\u88ab\u5b83\u6491\u5927 s\u2014\u2014\u8fd9\u6b63\u662f SmoothQuant \u8981\u628a\u91cf\u5316\u96be\u5ea6\u4ece\u6fc0\u6d3b\u4fa7\u8fc1\u79fb\u5230\u6743\u91cd\u4fa7\u7684\u539f\u56e0（\u672c\u7cfb\u5217\u7b2c 4.2 \u7bc7\u5c55\u5f00）\u3002<\/li>\n<\/ul>\n

            \u7c92\u5ea6\u4e5f\u4e0d\u662f\u8d8a\u7ec6\u8d8a\u597d：group=64 \u65f6，\u6bcf 64 \u4e2a INT8 \u8981\u989d\u5916\u5b58\u4e00\u4e2a FP16 scale，\u5b58\u50a8\u4ece 1 B\/\u5143\u7d20\u6da8\u5230\u7ea6 1.03 B\/\u5143\u7d20，kernel \u91cc\u8fd8\u8981\u591a\u4e00\u6b21\u5f52\u7ea6\u4e0e\u5e7f\u64ad\u3002\u7cbe\u5ea6\u6536\u76ca\u9012\u51cf\u3001\u5f00\u9500\u7ebf\u6027\u9012\u589e，\u5de5\u7a0b\u4e0a\u53d6\u4e24\u8005\u7684\u4ea4\u70b9\u3002<\/p>\n

            4. \u91cf\u5316\u8303\u5f0f：PTQ vs QAT<\/h3>\n

            \u6309\u91cf\u5316\u53c2\u6570\u5728\u4f55\u65f6\u786e\u5b9a，\u5206\u4e3a\u4e24\u6761\u8def\u7ebf：<\/p>\n

              \n
            • PTQ（Post-Training Quantization，\u8bad\u7ec3\u540e\u91cf\u5316）：\u6a21\u578b\u8bad\u7ec3\u5b8c\u6210\u540e，\u7528\u5c11\u91cf\u6821\u51c6\u6570\u636e（\u901a\u5e38\u51e0\u767e\u5230\u51e0\u5343\u6761）\u7edf\u8ba1\u5404\u5f20\u91cf\u5206\u5e03，\u79bb\u7ebf\u7b97\u51fa scale\/zero-point \u540e\u76f4\u63a5\u90e8\u7f72\u3002\u6210\u672c\u4f4e\u3001\u65e0\u9700\u91cd\u8bad\u7ec3，\u662f INT8\/INT4 \u6743\u91cd\u91cf\u5316\u7684\u4e3b\u6d41；<\/li>\n
            • QAT（Quantization-Aware Training，\u91cf\u5316\u611f\u77e5\u8bad\u7ec3）：\u5728\u8bad\u7ec3\u56fe\u4e2d\u63d2\u5165\u4f2a\u91cf\u5316（fake quant）\u8282\u70b9，\u524d\u5411\u6a21\u62df\u91cf\u5316\u8bef\u5dee，\u53cd\u5411\u7528 STE（Straight-Through Estimator，\u76f4\u901a\u4f30\u8ba1\u5668）\u628a\u68af\u5ea6\u7ed5\u8fc7\u4e0d\u53ef\u5bfc\u7684 round \u4f20\u56de\u53bb，\u8ba9\u6a21\u578b\u5728\u5fae\u8c03\u4e2d"\u5b66\u4f1a\u9002\u5e94"\u91cf\u5316\u3002\u6210\u672c\u63a5\u8fd1\u4e00\u6b21\u5fae\u8c03，\u4f46\u80fd\u6362\u56de\u66f4\u4f4e\u6bd4\u7279\u4e0b\u7684\u7cbe\u5ea6\u3002<\/li>\n<\/ul>\n

              flowchart LR
              \n subgraph PTQ["PTQ：\u8bad\u7ec3\u540e\u91cf\u5316"]
              \n A1["FP \u6a21\u578b"] –> A2["\u6821\u51c6\u6570\u636e\u7edf\u8ba1\u5206\u5e03"] –> A3["\u8ba1\u7b97 scale \/ zero-point"] –> A4["\u91cf\u5316\u90e8\u7f72"]
              \n end
              \n subgraph QAT["QAT：\u91cf\u5316\u611f\u77e5\u8bad\u7ec3"]
              \n B1["FP \u6a21\u578b"] –> B2["\u63d2\u5165\u4f2a\u91cf\u5316\u8282\u70b9"] –> B3["\u5fae\u8c03（STE \u53cd\u4f20\u68af\u5ea6）"] –> B4["\u56fa\u5316\u53c2\u6570\u540e\u90e8\u7f72"]
              \n end<\/p>\n\n\u7ef4\u5ea6PTQQAT<\/tr>\n\n\n\n\n\n
              \u989d\u5916\u6210\u672c<\/td>\n\u51e0\u767e\u6761\u6821\u51c6\u6570\u636e\u3001\u5206\u949f\u7ea7<\/td>\n\u5b8c\u6574\u5fae\u8c03\u6d41\u6c34\u7ebf\u3001\u5c0f\u65f6\u5230\u5929\u7ea7<\/td>\n<\/tr>\n
              \u7cbe\u5ea6\u8868\u73b0<\/td>\nINT8 \u57fa\u672c\u65e0\u635f，INT4 \u9700 GPTQ\/AWQ \u8865\u507f<\/td>\n\u66f4\u4f4e\u6bd4\u7279\u4e0b\u4ecd\u8f83\u7a33<\/td>\n<\/tr>\n
              \u5de5\u7a0b\u95e8\u69db<\/td>\n\u4f4e，\u5de5\u5177\u94fe\u6210\u719f<\/td>\n\u9ad8，\u9700\u8bad\u7ec3\u6846\u67b6\u652f\u6301<\/td>\n<\/tr>\n
              \u4f55\u65f6\u9009<\/td>\n\u9ed8\u8ba4\u9996\u9009<\/td>\nPTQ \u7cbe\u5ea6\u4e0d\u8fbe\u6807\u65f6\u7684\u540e\u624b<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

              \u5bf9\u63a8\u7406\u5de5\u7a0b\u5e08\u6765\u8bf4，\u65e5\u5e38\u7edd\u5927\u591a\u6570\u573a\u666f\u662f PTQ；QAT \u66f4\u591a\u51fa\u73b0\u5728\u6a21\u578b\u53d1\u5e03\u65b9\u7684\u751f\u4ea7\u6d41\u6c34\u7ebf\u4e0a（\u4f8b\u5982\u5382\u5546\u76f4\u63a5\u653e\u51fa\u5b98\u65b9\u91cf\u5316 checkpoint）\u3002<\/p>\n

              5. \u91cf\u5316\u8bef\u5dee\u7684\u4e09\u4e2a\u6765\u6e90<\/h3>\n

              \u91cf\u5316\u4e0d\u662f\u514d\u8d39\u7684\u3002\u8bef\u5dee\u6309\u6765\u6e90\u5206\u4e09\u7c7b，\u5de5\u7a0b\u5bf9\u7b56\u5404\u4e0d\u76f8\u540c：<\/p>\n

              \u2460 \u820d\u5165\u8bef\u5dee（Rounding Error）\u3002round \u64cd\u4f5c\u5f15\u5165，\u843d\u5728 \u00b1s\/2 \u4ee5\u5185，\u8fd1\u4f3c\u5747\u5300\u5206\u5e03\u3001\u5747\u503c\u7ea6\u4e3a 0\u3002\u53ea\u8981 round \u5b9e\u73b0\u6b63\u786e（\u8b66\u60d5\u90e8\u5206\u786c\u4ef6\u7684\u9690\u5f0f float\u2192int cast \u662f\u5411\u96f6\u622a\u65ad\u800c\u975e\u56db\u820d\u4e94\u5165，\u4f1a\u5f15\u5165\u5747\u503c \u2248 \u2212s\/2 \u7684\u7cfb\u7edf\u6027\u504f\u5dee），\u820d\u5165\u8bef\u5dee\u5728\u70b9\u79ef\u7d2f\u52a0\u65f6\u4f1a\u90e8\u5206\u62b5\u6d88，\u662f\u4e09\u7c7b\u8bef\u5dee\u91cc\u6700"\u6e29\u548c"\u7684\u3002<\/p>\n

              \u2461 \u88c1\u526a\u8bef\u5dee（Clipping Error）\u3002\u5f53 |x| \u8d85\u51fa\u91cf\u5316\u8303\u56f4\u65f6\u88ab clamp \u5230\u8fb9\u754c，\u8bef\u5dee\u53ef\u4ee5\u8fdc\u5927\u4e8e s\/2，\u4e14\u96c6\u4e2d\u5728 outlier \u4e0a\u2014\u2014\u800c outlier \u5f80\u5f80\u6070\u662f\u6a21\u578b\u91cc\u6700\u91cd\u8981\u7684\u503c\u3002\u51cf\u5c0f s（\u4f8b\u5982\u6309 99.9 \u5206\u4f4d\u800c\u975e\u6700\u5927\u503c\u5b9a\u6807）\u80fd\u7f29\u5c0f\u6574\u4f53\u820d\u5165\u8bef\u5dee，\u4ee3\u4ef7\u662f\u653e\u5927\u88c1\u526a\u8bef\u5dee，\u4e24\u8005\u5b58\u5728\u6700\u4f18\u6743\u8861；\u6821\u51c6\u7b97\u6cd5（percentile\u3001MSE \u6700\u4f18\u622a\u65ad）\u627e\u7684\u5c31\u662f\u8fd9\u4e2a\u70b9\u3002<\/p>\n

              \u2462 \u7d2f\u79ef\u8bef\u5dee（Accumulation Error）\u3002\u5355\u5143\u7d20\u8bef\u5dee\u5f88\u5c0f，\u4f46 GEMM（General Matrix Multiply，\u901a\u7528\u77e9\u9635\u4e58）\u662f\u6210\u5343\u4e0a\u4e07\u6b21\u4e58\u52a0：\u65e0\u504f\u8bef\u5dee\u90e8\u5206\u62b5\u6d88，\u6709\u504f\u8bef\u5dee\u7ebf\u6027\u7d2f\u79ef\u3002\u4e24\u4e2a\u5de5\u7a0b\u542b\u4e49：\u4e00\u662f round \u6a21\u5f0f\u5fc5\u987b\u6838\u5b9e（\u89c1\u2460）；\u4e8c\u662f\u7d2f\u52a0\u5668\u7cbe\u5ea6\u5fc5\u987b\u591f\u2014\u2014INT8\u00d7INT8 \u5fc5\u987b\u7528 INT32 \u7d2f\u52a0\u5668，\u53cd\u91cf\u5316\u653e\u5728\u7d2f\u52a0\u5b8c\u6210\u540e\u4e00\u6b21\u6027\u505a，\u800c\u4e0d\u662f\u9010\u5143\u7d20\u5148\u8f6c\u6d6e\u70b9\u518d\u4e58\u3002<\/p>\n

              6. \u5b9e\u6218：\u624b\u5199\u4e00\u4e2a FP16\u2192INT8 \u91cf\u5316\u5668<\/h3>\n

              \u7406\u8bba\u8bb2\u5b8c，\u52a8\u624b\u9a8c\u8bc1\u3002\u76ee\u6807：\u624b\u5199\u5bf9\u79f0 INT8 \u91cf\u5316\u7684 quantize() \/ dequantize()，\u5728\u4e00\u4efd"\u50cf LLM \u6743\u91cd"\u7684\u5408\u6210\u6570\u636e（\u6b63\u6001\u4e3b\u4f53 + \u5c11\u91cf\u91cd\u5c3e）\u4e0a\u505a\u91cf\u5316\u2014\u53cd\u91cf\u5316，\u7edf\u8ba1\u8bef\u5dee\u5e76\u4e0e\u7b2c 2.4 \u8282\u7684\u7406\u8bba\u754c\u5bf9\u7167\u3002<\/p>\n

              6.1 numpy \u7248<\/h4>\n

              # \u6d4b\u8bd5\u73af\u5883：Windows 11, Python 3.12, numpy 1.26.4
              \nimport numpy as np<\/p>\n

              rng = np.random.default_rng(42)<\/p>\n

              def quantize(x, qmax=127):
              \n """\u5bf9\u79f0\u5747\u5300\u91cf\u5316：float -> int8 + scale"""
              \n s = np.abs(x).max() \/ qmax # scale：\u4e00\u4e2a\u91cf\u5316\u7801\u5bf9\u5e94\u591a\u5c11\u6d6e\u70b9\u5355\u4f4d
              \n q = np.clip(np.round(x \/ s), -qmax, qmax)
              \n return q.astype(np.int8), s<\/p>\n

              def dequantize(q, s):
              \n """\u53cd\u91cf\u5316：int8 -> float"""
              \n return q.astype(np.float64) * s<\/p>\n

              # \u5408\u6210\u4e00\u4efd"\u50cf LLM \u6743\u91cd"\u7684\u6570\u636e：\u6b63\u6001\u4e3b\u4f53 + \u5c11\u91cf\u91cd\u5c3e
              \nw = rng.standard_normal(200_000) * 0.05
              \nw[:500] *= 6.0
              \nw16 = w.astype(np.float16).astype(np.float64) # \u6a21\u62df FP16 \u5b58\u50a8<\/p>\n

              q, s = quantize(w16)
              \nw_hat = dequantize(q, s)
              \nerr = w_hat – w16<\/p>\n

              print(f"scale s = {s:.6f}")
              \nprint(f"\u7406\u8bba\u5355\u70b9\u8bef\u5dee\u4e0a\u754c s\/2 = {s\/2:.6f}")
              \nprint(f"\u5b9e\u9645\u6700\u5927\u7edd\u5bf9\u8bef\u5dee = {np.abs(err).max():.6f}")
              \nprint(f"\u5e73\u5747\u7edd\u5bf9\u8bef\u5dee (MAE) = {np.abs(err).mean():.6f}")
              \nprint(f"\u76f8\u5bf9\u8bef\u5dee (L2 \u8303\u6570\u6bd4) = {np.linalg.norm(err) \/ np.linalg.norm(w16):.6%}")
              \nprint(f"INT8 \u7801\u70b9\u5229\u7528\u7387 = {len(np.unique(q))} \/ 255")<\/p>\n

              \u771f\u5b9e\u8fd0\u884c\u8f93\u51fa：<\/p>\n

              scale s = 0.006882
              \n\u7406\u8bba\u5355\u70b9\u8bef\u5dee\u4e0a\u754c s\/2 = 0.003441
              \n\u5b9e\u9645\u6700\u5927\u7edd\u5bf9\u8bef\u5dee = 0.003441
              \n\u5e73\u5747\u7edd\u5bf9\u8bef\u5dee (MAE) = 0.001720
              \n\u76f8\u5bf9\u8bef\u5dee (L2 \u8303\u6570\u6bd4) = 3.813329%
              \nINT8 \u7801\u70b9\u5229\u7528\u7387 = 160 \/ 255<\/p>\n

              \u4e09\u4e2a\u503c\u5f97\u6ce8\u610f\u7684\u70b9：<\/p>\n

            • \u5b9e\u9645\u6700\u5927\u8bef\u5dee 0.003441 \u6070\u597d\u7b49\u4e8e\u7406\u8bba\u754c s\/2\u2014\u2014\u7b2c 2.4 \u8282\u7684\u4e0d\u7b49\u5f0f\u88ab\u4e25\u683c\u9a8c\u8bc1，\u8fb9\u754c\u4e0a\u786e\u5b9e\u6709\u5143\u7d20"\u8e29\u7ebf"；<\/li>\n
            • MAE \u2248 0.00172 \u2248 s\/4，\u4e0e"\u820d\u5165\u8bef\u5dee\u5728 \u00b1s\/2 \u4e0a\u5747\u5300\u5206\u5e03"\u7684\u7406\u8bba\u671f\u671b\u4e00\u81f4；<\/li>\n
            • \u7801\u70b9\u53ea\u7528\u4e86 160\/255\u2014\u2014\u91cd\u5c3e outlier \u628a max|x| \u6491\u5230\u7ea6 0.87，s \u968f\u4e4b\u53d8\u5927，\u5927\u91cf\u4e2d\u95f4\u7801\u70b9\u7a7a\u7f6e\u3002\u8fd9\u6b63\u662f per-tensor \u7c92\u5ea6\u5728\u771f\u5b9e LLM \u6570\u636e\u4e0a\u7684\u5178\u578b\u75c5\u75c7，\u4e5f\u662f\u7b2c 3 \u8282\u7c92\u5ea6\u7ec6\u5316\u4e0e\u7b2c 4.2 \u7bc7 SmoothQuant \u7684\u52a8\u673a\u3002<\/li>\n

              6.2 PyTorch \u7248：\u8d34\u8fdb\u771f\u5b9e\u63a8\u7406\u6d41\u7a0b<\/h4>\n

              \u628a\u540c\u6837\u7684\u903b\u8f91\u7ffb\u8bd1\u6210 torch，\u5e76\u52a0\u4e0a per-channel \u7c92\u5ea6\u2014\u2014\u8fd9\u624d\u662f\u5de5\u7a0b\u91cc\u771f\u6b63\u4f7f\u7528\u7684\u5f62\u6001：<\/p>\n

              # \u6d4b\u8bd5\u73af\u5883：torch 2.5.1, CUDA 12.4（CPU \u4ea6\u53ef\u8fd0\u884c）
              \nimport torch<\/p>\n

              def quantize_int8(x: torch.Tensor, per_channel: bool = False):
              \n """\u5bf9\u79f0 INT8 \u91cf\u5316：FP16 -> INT8 + scale\u3002<\/p>\n

              per_channel=True \u65f6\u6cbf\u8f93\u51fa\u901a\u9053（dim 0）\u9010\u901a\u9053\u5b9a\u6807，\u662f\u6743\u91cd\u91cf\u5316\u7684\u5de5\u7a0b\u9ed8\u8ba4\u3002
              \n """
              \n q_max = 127
              \n if per_channel:
              \n s = x.abs().amax(dim=1, keepdim=True) \/ q_max # \u5f62\u72b6 [out, 1]，\u53ef\u5e7f\u64ad
              \n else:
              \n s = x.abs().max() \/ q_max # \u6807\u91cf
              \n q = torch.clamp(torch.round(x \/ s), -q_max, q_max).to(torch.int8)
              \n return q, s<\/p>\n

              def dequantize_int8(q: torch.Tensor, s: torch.Tensor) -> torch.Tensor:
              \n """\u53cd\u91cf\u5316：INT8 -> FP16，\u9010\u5143\u7d20\u4e58\u56de scale\u3002"""
              \n return q.to(torch.float16) * s.to(torch.float16)<\/p>\n

              if __name__ == "__main__":
              \n torch.manual_seed(42)
              \n w = (torch.randn(4096, 512) * 0.05).to(torch.float16) # \u6a21\u62df\u4e00\u5c42\u6743\u91cd [out, in]
              \n w[:8, :] *= 6.0 # \u5236\u9020\u51e0\u4e2a outlier \u8f93\u51fa\u901a\u9053<\/p>\n

              q_t, s_t = quantize_int8(w, per_channel=False)
              \n q_c, s_c = quantize_int8(w, per_channel=True)<\/p>\n

              err_t = (dequantize_int8(q_t, s_t) – w).abs().max().item()
              \n err_c = (dequantize_int8(q_c, s_c) – w).abs().max().item()
              \n print(f"per-tensor max|err| = {err_t:.6f} (s\/2 = {(s_t \/ 2).item():.6f})")
              \n print(f"per-channel max|err| = {err_c:.6f}")<\/p>\n

              \u81ea\u67e5\u8981\u70b9：amax(dim=1, keepdim=True) \u4fdd\u6301\u5e7f\u64ad\u5f62\u72b6；torch.round \u4e3a\u56db\u820d\u4e94\u5165（half-to-even）；INT8 \u5f20\u91cf\u53c2\u4e0e\u8fd0\u7b97\u524d\u5148\u663e\u5f0f\u8f6c\u56de FP16，\u907f\u514d\u6574\u578b\u6ea2\u51fa\u8bed\u4e49\u3002\u6ce8\u610f\u771f\u5b9e kernel \u91cc\u5e76\u4e0d\u4f1a\u771f\u7684\u5148\u53cd\u91cf\u5316\u518d\u505a GEMM（\u90a3\u7b49\u4e8e\u767d\u91cf\u5316），\u800c\u662f int8\u00d7int8 \u7d2f\u52a0\u6210 int32\u3001\u6700\u540e\u4e00\u6b21\u6027\u4e58 scale\u2014\u2014kernel \u7ea7\u4f18\u5316\u5c5e\u4e8e\u540e\u7eed\u7bc7\u7ae0\u7684\u8bdd\u9898\u3002<\/p>\n

              6.3 \u53ef\u89c6\u5316：\u91cf\u5316\u524d\u540e\u5206\u5e03\u5bf9\u6bd4<\/h4>\n

              \"\"<\/p>\n

              \u56fe 1：\u5de6\u56fe\u4e3a\u539f\u59cb FP16 \u6743\u91cd\u5206\u5e03（\u6b63\u6001\u4e3b\u4f53 + \u91cd\u5c3e）；\u53f3\u56fe\u53e0\u52a0 INT8 \u91cf\u5316\u2014\u53cd\u91cf\u5316\u540e\u7684\u5206\u5e03\u3002\u91cf\u5316\u540e\u6574\u4f53\u8f6e\u5ed3\u4fdd\u6301，\u4f46\u76f4\u65b9\u56fe\u5448\u79bb\u6563\u9636\u68af\u72b6（\u7801\u70b9\u6709\u9650），outlier \u9644\u8fd1\u7684\u7801\u70b9\u5bc6\u5ea6\u660e\u663e\u53d8\u7a00\u2014\u2014s \u88ab\u91cd\u5c3e\u6491\u5927\u7684\u4ee3\u4ef7\u6e05\u6670\u53ef\u89c1\u3002\u5b8c\u6574\u7ed8\u56fe\u4ee3\u7801\u5982\u4e0b，\u53ef\u76f4\u63a5\u590d\u73b0：<\/p>\n

              # \u6d4b\u8bd5\u73af\u5883：Windows 11, Python 3.12, numpy 1.26.4, matplotlib 3.9.2
              \n# \u8fd0\u884c：python scripts\/quant-ch4\/fig_4_1.py
              \nimport numpy as np
              \nimport matplotlib<\/p>\n

              matplotlib.use("Agg")
              \nimport matplotlib.pyplot as plt<\/p>\n

              rng = np.random.default_rng(42)<\/p>\n

              # ———- 1. \u624b\u5199\u5bf9\u79f0\u91cf\u5316 \/ \u53cd\u91cf\u5316 ———-
              \ndef quantize(x, qmax=127):
              \n """\u5bf9\u79f0\u5747\u5300\u91cf\u5316：float -> int8 + scale"""
              \n s = np.abs(x).max() \/ qmax # scale：\u4e00\u4e2a\u91cf\u5316\u7801\u5bf9\u5e94\u591a\u5c11\u6d6e\u70b9\u5355\u4f4d
              \n q = np.clip(np.round(x \/ s), -qmax, qmax)
              \n return q.astype(np.int8), s<\/p>\n

              def dequantize(q, s):
              \n """\u53cd\u91cf\u5316：int8 -> float"""
              \n return q.astype(np.float64) * s<\/p>\n

              # \u5408\u6210\u4e00\u4efd"\u50cf LLM \u6743\u91cd"\u7684\u6570\u636e：\u6b63\u6001\u4e3b\u4f53 + \u5c11\u91cf\u91cd\u5c3e
              \nw = rng.standard_normal(200_000) * 0.05
              \nw[:500] *= 6.0
              \nw16 = w.astype(np.float16).astype(np.float64) # \u6a21\u62df FP16 \u5b58\u50a8<\/p>\n

              q, s = quantize(w16)
              \nw_hat = dequantize(q, s)
              \nerr = w_hat – w16<\/p>\n

              print(f"scale s = {s:.6f}")
              \nprint(f"\u7406\u8bba\u5355\u70b9\u8bef\u5dee\u4e0a\u754c s\/2 = {s\/2:.6f}")
              \nprint(f"\u5b9e\u9645\u6700\u5927\u7edd\u5bf9\u8bef\u5dee = {np.abs(err).max():.6f}")
              \nprint(f"\u5e73\u5747\u7edd\u5bf9\u8bef\u5dee (MAE) = {np.abs(err).mean():.6f}")
              \nprint(f"\u76f8\u5bf9\u8bef\u5dee (L2 \u8303\u6570\u6bd4) = {np.linalg.norm(err) \/ np.linalg.norm(w16):.6%}")
              \nprint(f"INT8 \u7801\u70b9\u5229\u7528\u7387 = {len(np.unique(q))} \/ 255")<\/p>\n

              # ———- 2. \u7ed8\u56fe ———-
              \nfig, axes = plt.subplots(1, 2, figsize=(11, 4))<\/p>\n

              axes[0].hist(w16, bins=200, color="#4C72B0", alpha=0.85)
              \naxes[0].set_title("Original FP16 Weights")
              \naxes[0].set_xlabel("value")
              \naxes[0].set_ylabel("count")<\/p>\n

              axes[1].hist(w16, bins=200, color="#4C72B0", alpha=0.6, label="FP16")
              \naxes[1].hist(w_hat, bins=200, color="#DD8452", alpha=0.6, label="INT8 dequantized")
              \naxes[1].set_title("After INT8 Symmetric Quantization")
              \naxes[1].set_xlabel("value")
              \naxes[1].legend()<\/p>\n

              fig.suptitle(f"FP16 vs INT8 (scale={s:.4f}, rel L2 err={np.linalg.norm(err)\/np.linalg.norm(w16):.2%})")
              \nfig.tight_layout()
              \nfig.savefig("4.1-fp16-vs-int8-dist.png", dpi=150) # \u4fdd\u5b58\u5230\u5f53\u524d\u76ee\u5f55，\u4e0e\u6587\u7ae0\u914d\u56fe\u4e00\u81f4<\/p>\n

              \u60f3\u5feb\u901f\u4f53\u9a8c\u5de5\u4e1a\u7ea7\u91cf\u5316\u63a8\u7406，vLLM \u4e00\u6761\u547d\u4ee4\u5373\u53ef（FP8 \u52a8\u6001\u91cf\u5316 + FP8 KV Cache，KV Cache \u5373 Key-Value Cache，\u952e\u503c\u7f13\u5b58）：<\/p>\n

              # \u6d4b\u8bd5\u73af\u5883：H100 80GB, CUDA 12.4, vLLM 0.10.0, torch 2.5.1
              \nvllm serve meta-llama\/Llama-3.1-8B-Instruct \\\\
              \n –quantization fp8 –kv-cache-dtype fp8<\/p>\n

              7. \u5e38\u89c1\u95ee\u9898（FAQ）<\/h3>\n

              Q1：\u4e3a\u4ec0\u4e48\u5bf9\u79f0 INT8 \u91cf\u5316\u7528 [-127, 127] \u800c\u4e0d\u7528\u5b8c\u6574\u7684 [-128, 127]？ \u4fdd\u6301\u6b63\u8d1f\u5bf9\u79f0：-128 \u6ca1\u6709\u5bf9\u5e94\u7684\u6b63\u503c，\u5b9a\u70b9\u4e58\u52a0\u91cc\u7684\u7b26\u53f7\u7ffb\u8f6c -(-128) \u4f1a\u6ea2\u51fa；\u4e14 z=0 \u65f6\u6620\u5c04\u5173\u7cfb\u6700\u7b80\u5355\u3002TensorRT \u7b49\u4e3b\u6d41\u63a8\u7406\u6846\u67b6\u9ed8\u8ba4\u90fd\u7528 \u00b1127，\u635f\u5931\u4e00\u4e2a\u7801\u70b9\u7684\u4ee3\u4ef7\u53ef\u4ee5\u5ffd\u7565\u3002<\/p>\n

              Q2：\u91cf\u5316\u540e\u7cbe\u5ea6\u660e\u663e\u4e0b\u964d，\u7b2c\u4e00\u6b65\u67e5\u4ec0\u4e48？ \u753b\u51fa\u6bcf\u4e2a channel \u7684 max|x|，\u5148\u627e outlier channel；\u7136\u540e\u628a\u7c92\u5ea6\u4ece per-tensor \u7ec6\u5316\u5230 per-channel \u6216 group-wise；\u518d\u68c0\u67e5\u6821\u51c6\u96c6\u662f\u5426\u8986\u76d6\u771f\u5b9e\u6570\u636e\u5206\u5e03；\u6700\u540e\u6838\u5b9e round\/clamp \u7684\u5b9e\u73b0\u884c\u4e3a（\u4e0d\u5c11 DSL \u7684\u9690\u5f0f float\u2192int \u8f6c\u6362\u662f\u5411\u96f6\u622a\u65ad\u800c\u975e\u56db\u820d\u4e94\u5165，\u4f1a\u5f15\u5165\u7cfb\u7edf\u6027\u504f\u5dee）\u3002\u6309\u8fd9\u4e2a\u987a\u5e8f\u6392\u67e5，\u7edd\u5927\u591a\u6570\u7cbe\u5ea6\u95ee\u9898\u90fd\u80fd\u5b9a\u4f4d\u3002<\/p>\n

              Q3：\u63a8\u7406\u90e8\u7f72\u9009 FP16 \u8fd8\u662f BF16？ \u7b97\u529b\u4e0a\u4e24\u8005\u4e00\u81f4（H100 \u4e0a FP16\/BF16 tensor core \u5cf0\u503c\u76f8\u540c）\u3002BF16 \u52a8\u6001\u8303\u56f4\u4e0e FP32 \u76f8\u540c，\u6fc0\u6d3b\u4e0e\u68af\u5ea6\u4e0d\u6613\u6ea2\u51fa，\u662f LLM \u4e3b\u6d41；FP16 \u5c3e\u6570\u591a 3 \u4f4d\u3001\u76f8\u5bf9\u7cbe\u5ea6\u7565\u9ad8，\u4f46\u6700\u5927\u503c\u53ea\u6709 65504，\u957f\u5e8f\u5217 logits \u5b58\u5728\u6ea2\u51fa\u98ce\u9669\u3002\u65b0\u90e8\u7f72\u7684\u6a21\u578b\u9ed8\u8ba4\u9009 BF16，\u9664\u975e\u6709\u660e\u786e\u7406\u7531\u3002<\/p>\n

              Q4：INT4 \u80fd\u540c\u65f6\u91cf\u5316\u6fc0\u6d3b\u5417（W4A4）？ \u76ee\u524d\u57fa\u672c\u4e0d\u80fd\u30024 bit \u53ea\u6709 16 \u4e2a\u7801\u70b9，\u6fc0\u6d3b\u7684 outlier channel \u4f1a\u628a per-token \u7684 s \u6491\u5927\u5230\u8ba9\u7edd\u5927\u591a\u6570\u5c0f\u503c\u76f4\u63a5\u5f52\u96f6\u3002\u5de5\u4e1a\u754c INT4 \u51e0\u4e4e\u53ea\u7528\u4e8e\u7eaf\u6743\u91cd\u91cf\u5316 W4A16（\u6743\u91cd 4 bit\u3001\u6fc0\u6d3b 16 bit），\u914d\u5408 group-wise scale \u4e0e GPTQ\/AWQ \u8865\u507f\u7b97\u6cd5；\u6fc0\u6d3b\u4fa7 INT4 \u4ecd\u505c\u7559\u5728\u7814\u7a76\u9636\u6bb5\u3002<\/p>\n

              Q5：\u4e3a\u4ec0\u4e48 INT4 \u91cf\u5316\u540e\u6a21\u578b\u6587\u4ef6\u4e0d\u662f\u6b63\u597d\u7f29\u5230 FP16 \u7684 1\/4？ \u4e09\u90e8\u5206"\u989d\u5916"\u4f53\u79ef：group-wise \u7684 scale\/zero-point \u5143\u6570\u636e（group=64 \u65f6\u7ea6\u589e\u52a0 6%）；embedding \u4e0e lm_head \u901a\u5e38\u4fdd\u6301 FP16 \u4e0d\u91cf\u5316；\u90e8\u5206\u654f\u611f\u5c42（\u9996\u672b\u5c42\u3001MoE \u8def\u7531）\u5e38\u88ab\u8df3\u8fc7\u3002\u5b9e\u9645\u4f53\u79ef\u7ea6\u4e3a FP16 \u7248\u7684 30%，\u5c5e\u6b63\u5e38\u73b0\u8c61\u3002<\/p>\n

              \u672c\u7ae0\u5c0f\u7ed3<\/h3>\n
                \n
              • \u6570\u503c\u683c\u5f0f\u7684\u672c\u8d28\u662f"\u8303\u56f4-\u7cbe\u5ea6"\u6743\u8861：\u6d6e\u70b9\u7528\u6307\u6570\u4f4d\u6362\u8303\u56f4\u3001\u5c3e\u6570\u4f4d\u6362\u7cbe\u5ea6；\u6574\u6570\u662f\u5747\u5300\u7f51\u683c，\u8303\u56f4\u9760\u5916\u90e8 scale \u6491\u51fa\u3002<\/li>\n
              • \u91cf\u5316\u6620\u5c04\u4e09\u8981\u7d20：scale \u5b9a\u6b65\u957f\u3001zero-point \u5b9a\u96f6\u70b9\u3001clamp \u622a\u8fb9\u754c；\u6838\u5fc3\u4e0d\u7b49\u5f0f |x\u0302 \u2212 x| \u2264 s\/2，\u4e14 s \u7531\u8986\u76d6\u8303\u56f4\u5185\u7684\u6700\u5927\u503c\u51b3\u5b9a\u3002<\/li>\n
              • \u7c92\u5ea6\u662f\u91cf\u5316\u6700\u91cd\u8981\u7684\u5de5\u7a0b\u51b3\u7b56：\u6743\u91cd per-channel \u8d77\u6b65\u3001INT4 \u4e0a group-wise；\u6fc0\u6d3b\u4e0e KV Cache \u7528 per-token \u52a8\u6001\u91cf\u5316；\u7c92\u5ea6\u65b9\u5411\u8981\u5bf9\u7740 outlier \u7684\u65b9\u5411\u3002<\/li>\n
              • PTQ \u662f\u9ed8\u8ba4\u8def\u7ebf（\u6821\u51c6\u6570\u636e + \u79bb\u7ebf\u5b9a\u6807），QAT \u662f PTQ \u7cbe\u5ea6\u4e0d\u8fbe\u6807\u65f6\u7684\u540e\u624b（\u4f2a\u91cf\u5316 + STE \u5fae\u8c03）\u3002<\/li>\n
              • \u8bef\u5dee\u4e09\u6765\u6e90：\u820d\u5165（\u00b1s\/2 \u5185\u3001\u8fd1\u4f3c\u65e0\u504f）\u3001\u88c1\u526a（outlier \u4e0a\u3001\u53ef\u8d85\u754c）\u3001\u7d2f\u79ef（\u6709\u504f\u8bef\u5dee\u968f\u70b9\u79ef\u7ebf\u6027\u589e\u957f，\u7d2f\u52a0\u5668\u8981\u7528 INT32）\u3002<\/li>\n
              • \u52a8\u624b\u5b9e\u9a8c\u9a8c\u8bc1\u4e86\u7406\u8bba：\u6700\u5927\u8bef\u5dee\u6070\u597d\u8e29\u5230 s\/2，MAE \u2248 s\/4，\u91cd\u5c3e outlier \u5bfc\u81f4\u7801\u70b9\u5229\u7528\u7387\u53ea\u6709 160\/255\u2014\u2014\u4e0b\u4e00\u7bc7 SmoothQuant \u5c31\u4ece\u8fd9\u4e2a\u95ee\u9898\u8bb2\u8d77\u3002<\/li>\n<\/ul>\n

                \u5ef6\u4f38\u9605\u8bfb<\/h3>\n
                  \n
                • Mixed Precision Training（Micikevicius et al., 2017）：FP16 \u6df7\u5408\u7cbe\u5ea6\u8bad\u7ec3\u7684\u5960\u57fa\u4e4b\u4f5c，\u7406\u89e3 FP16\/BF16 \u53d6\u820d\u7684\u8d77\u70b9；<\/li>\n
                • A White Paper on Neural Network Quantization（Nagel et al., 2021）：\u91cf\u5316\u9886\u57df\u6700\u7cfb\u7edf\u7684\u7efc\u8ff0，PTQ\/QAT\u3001\u7c92\u5ea6\u3001\u6821\u51c6\u7b56\u7565\u5747\u6709\u8986\u76d6；<\/li>\n
                • LLM.int8()（Dettmers et al., 2022）：\u9996\u6b21\u7cfb\u7edf\u63ed\u793a LLM \u6fc0\u6d3b\u7684 outlier feature \u73b0\u8c61\u2014\u2014\u672c\u7cfb\u5217\u7b2c 4.2 \u7bc7 SmoothQuant \u7684\u95ee\u9898\u6e90\u5934；<\/li>\n
                • GPTQ（Frantar et al., 2022）\u4e0e AWQ（Lin et al., 2023）：INT4 \u6743\u91cd\u91cf\u5316\u7684\u4e24\u4e2a\u4e8b\u5b9e\u6807\u51c6，\u5206\u522b\u5bf9\u5e94\u672c\u7cfb\u5217\u7b2c 4.3\u30014.4 \u7bc7；<\/li>\n
                • NVIDIA Transformer Engine \u6587\u6863：FP8 \u5728\u771f\u5b9e\u8bad\u7ec3\/\u63a8\u7406\u4e2d\u7684\u5de5\u7a0b\u5f62\u6001，\u5bf9\u5e94\u672c\u7cfb\u5217\u7b2c 4.6 \u7bc7\u3002<\/li>\n<\/ul>\n

                  \u53c2\u8003\u6587\u732e<\/h3>\n
                    \n
                  • Mixed Precision Training：[1710.03740] Mixed Precision Training<\/li>\n
                  • A White Paper on Neural Network Quantization：[2106.08295] A White Paper on Neural Network Quantization<\/li>\n
                  • LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale：[2208.07339] LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale<\/li>\n
                  • GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers：[2210.17323] GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers<\/li>\n
                  • SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models：[2211.10438] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models<\/li>\n
                  • AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration：https:\/\/arxiv.org\/abs\/2306.00978<\/li>\n
                  • PyTorch Quantization \u5b98\u65b9\u6587\u6863：https:\/\/pytorch.org\/docs\/stable\/quantization.html<\/li>\n
                  • NVIDIA H100 Tensor Core GPU：https:\/\/www.nvidia.com\/en-us\/data-center\/h100\/<\/li>\n
                  • NVIDIA Blackwell Architecture：The Engine Behind AI Factories | NVIDIA Blackwell Architecture<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"

                    \u672c\u7ae0\u7b80\u4ecb
                    \n\u5927\u6a21\u578b\u63a8\u7406\u7684\u74f6\u9888\u5f80\u5f80\u4e0d\u662f\u7b97\u529b\u800c\u662f\u8bbf\u5b58：decode \u9636\u6bb5\u6bcf\u751f\u6210\u4e00\u4e2a token \u90fd\u8981\u628a\u5168\u90e8\u6743\u91cd\u4ece\u663e\u5b58\u8bfb\u4e00\u904d，\u628a\u6bcf\u4e2a\u53c2\u6570\u7684\u5b57\u8282\u6570\u780d\u534a，\u7406\u8bba\u8017\u65f6\u5c31\u80fd\u780d\u534a\u3002\u91cf\u5316（Quantization）\u6b63\u662f\u76f4\u63a5\u653b\u51fb\\”\u5b57\u8282\u6570\\”\u8fd9\u4e2a\u5206\u6bcd\u7684\u7b2c\u4e00\u624b\u6bb5\u3002\u4f46\\”INT8 \u662f\u4ec0\u4e48\u3001\u4e3a\u4ec0\u4e48\u80fd\u538b\u7f29\u3001\u4ee3\u4ef7\u662f\u4ec0\u4e48\\”\u8fd9\u7c7b\u95ee\u9898，\u5f88\u591a\u5de5\u7a0b\u5e08\u5e76\u6ca1\u6709\u4ece\u6570\u5b66\u4e0a\u8bb2\u6e05\u695a\u8fc7\u3002
                    \n\u672c\u7bc7\u8981\u89e3\u51b3\u56db\u4e2a\u95ee\u9898：
                    \nFP32\/FP16\/BF1<\/p>\n","protected":false},"author":2,"featured_media":91927,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[152,50],"topic":[],"class_list":["post-91928","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-server","tag-pytorch","tag-50"],"yoast_head":"\n\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.wsisp.com\/helps\/91928.html\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"og:description\" content=\"\u672c\u7ae0\u7b80\u4ecb \u5927\u6a21\u578b\u63a8\u7406\u7684\u74f6\u9888\u5f80\u5f80\u4e0d\u662f\u7b97\u529b\u800c\u662f\u8bbf\u5b58：decode \u9636\u6bb5\u6bcf\u751f\u6210\u4e00\u4e2a token \u90fd\u8981\u628a\u5168\u90e8\u6743\u91cd\u4ece\u663e\u5b58\u8bfb\u4e00\u904d，\u628a\u6bcf\u4e2a\u53c2\u6570\u7684\u5b57\u8282\u6570\u780d\u534a，\u7406\u8bba\u8017\u65f6\u5c31\u80fd\u780d\u534a\u3002\u91cf\u5316（Quantization）\u6b63\u662f\u76f4\u63a5\u653b\u51fb"\u5b57\u8282\u6570"\u8fd9\u4e2a\u5206\u6bcd\u7684\u7b2c\u4e00\u624b\u6bb5\u3002\u4f46"INT8 \u662f\u4ec0\u4e48\u3001\u4e3a\u4ec0\u4e48\u80fd\u538b\u7f29\u3001\u4ee3\u4ef7\u662f\u4ec0\u4e48"\u8fd9\u7c7b\u95ee\u9898，\u5f88\u591a\u5de5\u7a0b\u5e08\u5e76\u6ca1\u6709\u4ece\u6570\u5b66\u4e0a\u8bb2\u6e05\u695a\u8fc7\u3002 \u672c\u7bc7\u8981\u89e3\u51b3\u56db\u4e2a\u95ee\u9898： FP32\/FP16\/BF1\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.wsisp.com\/helps\/91928.html\" \/>\n<meta property=\"og:site_name\" content=\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"article:published_time\" content=\"2026-08-08T11:32:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.wsisp.com\/helps\/wp-content\/uploads\/2026\/08\/20260808113245-6a7713dd65a2a.png\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/91928.html\",\"url\":\"https:\/\/www.wsisp.com\/helps\/91928.html\",\"name\":\"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"isPartOf\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\"},\"datePublished\":\"2026-08-08T11:32:46+00:00\",\"dateModified\":\"2026-08-08T11:32:46+00:00\",\"author\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/91928.html#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.wsisp.com\/helps\/91928.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/91928.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.wsisp.com\/helps\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\",\"url\":\"https:\/\/www.wsisp.com\/helps\/\",\"name\":\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"description\":\"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"contentUrl\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"caption\":\"admin\"},\"sameAs\":[\"http:\/\/wp.wsisp.com\"],\"url\":\"https:\/\/www.wsisp.com\/helps\/author\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.wsisp.com\/helps\/91928.html","og_locale":"zh_CN","og_type":"article","og_title":"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","og_description":"\u672c\u7ae0\u7b80\u4ecb \u5927\u6a21\u578b\u63a8\u7406\u7684\u74f6\u9888\u5f80\u5f80\u4e0d\u662f\u7b97\u529b\u800c\u662f\u8bbf\u5b58：decode \u9636\u6bb5\u6bcf\u751f\u6210\u4e00\u4e2a token \u90fd\u8981\u628a\u5168\u90e8\u6743\u91cd\u4ece\u663e\u5b58\u8bfb\u4e00\u904d，\u628a\u6bcf\u4e2a\u53c2\u6570\u7684\u5b57\u8282\u6570\u780d\u534a，\u7406\u8bba\u8017\u65f6\u5c31\u80fd\u780d\u534a\u3002\u91cf\u5316（Quantization）\u6b63\u662f\u76f4\u63a5\u653b\u51fb\"\u5b57\u8282\u6570\"\u8fd9\u4e2a\u5206\u6bcd\u7684\u7b2c\u4e00\u624b\u6bb5\u3002\u4f46\"INT8 \u662f\u4ec0\u4e48\u3001\u4e3a\u4ec0\u4e48\u80fd\u538b\u7f29\u3001\u4ee3\u4ef7\u662f\u4ec0\u4e48\"\u8fd9\u7c7b\u95ee\u9898，\u5f88\u591a\u5de5\u7a0b\u5e08\u5e76\u6ca1\u6709\u4ece\u6570\u5b66\u4e0a\u8bb2\u6e05\u695a\u8fc7\u3002 \u672c\u7bc7\u8981\u89e3\u51b3\u56db\u4e2a\u95ee\u9898： FP32\/FP16\/BF1","og_url":"https:\/\/www.wsisp.com\/helps\/91928.html","og_site_name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","article_published_time":"2026-08-08T11:32:46+00:00","og_image":[{"url":"https:\/\/www.wsisp.com\/helps\/wp-content\/uploads\/2026\/08\/20260808113245-6a7713dd65a2a.png"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"10 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.wsisp.com\/helps\/91928.html","url":"https:\/\/www.wsisp.com\/helps\/91928.html","name":"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","isPartOf":{"@id":"https:\/\/www.wsisp.com\/helps\/#website"},"datePublished":"2026-08-08T11:32:46+00:00","dateModified":"2026-08-08T11:32:46+00:00","author":{"@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41"},"breadcrumb":{"@id":"https:\/\/www.wsisp.com\/helps\/91928.html#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.wsisp.com\/helps\/91928.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.wsisp.com\/helps\/91928.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.wsisp.com\/helps"},{"@type":"ListItem","position":2,"name":"\u91cf\u5316\u57fa\u7840\uff1a\u4ece FP32 \u5230 INT4 \u7684\u538b\u7f29\u827a\u672f"}]},{"@type":"WebSite","@id":"https:\/\/www.wsisp.com\/helps\/#website","url":"https:\/\/www.wsisp.com\/helps\/","name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","description":"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41","name":"admin","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/","url":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","contentUrl":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","caption":"admin"},"sameAs":["http:\/\/wp.wsisp.com"],"url":"https:\/\/www.wsisp.com\/helps\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/91928","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/comments?post=91928"}],"version-history":[{"count":0,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/91928\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media\/91927"}],"wp:attachment":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media?parent=91928"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/categories?post=91928"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/tags?post=91928"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/topic?post=91928"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}