{"id":123675,"date":"2026-11-08T01:24:52","date_gmt":"2026-11-07T17:24:52","guid":{"rendered":"https:\/\/www.wsisp.com\/helps\/123675.html"},"modified":"2026-11-08T01:24:52","modified_gmt":"2026-11-07T17:24:52","slug":"%e6%a8%a1%e5%9e%8b%e4%b8%8a%e4%b8%8b%e6%96%87%e7%aa%97%e5%8f%a3%e7%ae%a1%e7%90%86%ef%bc%8c%e9%81%bf%e5%85%8d%e8%b6%85%e9%95%bf%e5%af%b9%e8%af%9d%e8%b6%85%e9%99%90","status":"publish","type":"post","link":"https:\/\/www.wsisp.com\/helps\/123675.html","title":{"rendered":"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650"},"content":{"rendered":"

\/* MD \/ \u5bcc\u6587\u672c\u4e2d\u7684 .toc\uff08\u542b\u535a\u5ba2\u56ed\u642c\u5bb6\u7b49\u5d4c\u5957\u7ed3\u6784\uff09\uff1b.toc-box \u5728\u4fa7\u680f\uff0c\u4e0d\u53d7\u5f71\u54cd *\/
\n #content_views .toc,
\n \/* \u7f16\u8f91\u5668\u5e38\u5728\u76ee\u5f55\u524d\u540e\u63d2\u5165\u7a7a p\uff08:empty \u4ecd\u5360 20px\uff09\uff0c\u4e00\u5e76\u53bb\u6389\u907f\u514d\u9876\u7a7a\u9699 *\/
\n #content_views.markdown_views > p:empty:has(+ .toc),
\n #content_views.markdown_views > .toc + p:empty,
\n \/* \u5bcc\u6587\u672c\u65e7\u7248\u76ee\u5f55\u6807\u8bb0 *\/
\n #content_views.htmledit_views #main-toc,
\n #content_views.htmledit_views #hr-toc,
\n #content_views.htmledit_views p[id*=\\”-toc\\”] {
\n display: none !important;
\n }
\n \/* \u76ee\u5f55\u53bb\u6389\u540e\uff0c\u7d27\u8ddf\u7684\u9996\u4e2a\u6807\u9898\u4e0d\u518d\u591a\u51fa\u4e00\u5757\u4e0a\u8fb9\u8ddd *\/
\n #content_views.markdown_views > .toc + h1,
\n #content_views.markdown_views > .toc + h2,
\n #content_views.markdown_views > .toc + h3,
\n #content_views.markdown_views > .toc + h4,
\n #content_views.markdown_views > .toc + p:empty + h1,
\n #content_views.markdown_views > .toc + p:empty + h2,
\n #content_views.markdown_views > .toc + p:empty + h3,
\n #content_views.markdown_views > .toc + p:empty + h4 {
\n margin-top: 0 !important;
\n }<\/p>\n

\u6458\u8981
\n\u89e3\u51b3LLM\u5e94\u7528\u5728\u5904\u7406\u957f\u5bf9\u8bdd\u6216\u6d77\u91cf\u6587\u6863\u65f6\u5f15\u53d1\u7684\u4e0a\u4e0b\u6587\u8d85\u9650\u3001API\u62a5\u9519\u3001\u5ef6\u8fdf\u6fc0\u589e\u53ca\u5e7b\u89c9\u9891\u53d1\u95ee\u9898\u3002\u9002\u7528\u4e8e\u9700\u6784\u5efa\u9ad8\u53ef\u7528\u957f\u4e0a\u4e0b\u6587AI\u5e94\u7528\u7684\u540e\u7aef\u4e0e\u7b97\u6cd5\u5de5\u7a0b\u5e08\u3002\u8bfb\u5b8c\u53ef\u83b7\u5f97\u57fa\u4e8e\u6ed1\u52a8\u7a97\u53e3\u3001\u6458\u8981\u538b\u7f29\u4e0eRAG\u68c0\u7d22\u56de\u586b\u7684\u7cfb\u7edf\u7ea7\u7ba1\u7406\u7b56\u7565，\u4ee5\u53caToken\u7cbe\u51c6\u8ba1\u6570\u4e0e\u4e0a\u4e0b\u6587\u88c1\u526a\u7684\u53ef\u8fd0\u884c\u4ee3\u7801\u3002<\/p>\n

\u5173\u952e\u8bcd
\n\u4e0a\u4e0b\u6587\u7a97\u53e3\u3001Token\u8d85\u9650\u3001\u6ed1\u52a8\u7a97\u53e3\u3001\u6458\u8981\u538b\u7f29\u3001RAG\u3001tiktoken\u3001\u4e0a\u4e0b\u6587\u88c1\u526a<\/p>\n

  • \n

    \u80cc\u666f\u4e0e\u95ee\u9898
    \nLLM\u7684\u4e0a\u4e0b\u6587\u7a97\u53e3\u7269\u7406\u53d7\u9650（\u59824K\u30018K\u3001128K），\u8d85\u957f\u5bf9\u8bdd\u76f4\u63a5\u5bfc\u81f4API\u62a5\u9519（\u5982OpenAI\u7684context_length_exceeded）\u6216\u6a21\u578b\u5bf9\u4e2d\u6bb5\u4fe1\u606f\u7684\u6ce8\u610f\u529b\u8870\u51cf（Lost in the Middle）\u3002\u76f4\u63a5\u65e0\u8111\u622a\u65ad\u4f1a\u4e22\u5931\u7cfb\u7edf\u6307\u4ee4\u6216\u8fd1\u671f\u5173\u952e\u5b9e\u4f53，\u5bfc\u81f4\u6a21\u578b\u8f93\u51fa\u201c\u5931\u5fc6\u201d\u6216\u7b54\u975e\u6240\u95ee\u3002\u6838\u5fc3\u77db\u76fe\u5728\u4e8e：\u4fdd\u7559\u5168\u90e8\u5386\u53f2\u4ee5\u7ef4\u6301\u8fde\u8d2f\u6027，\u4e0e\u63a7\u5236Token\u4ee5\u538b\u4f4e\u6210\u672c\/\u5ef6\u8fdf\u4e4b\u95f4\u5b58\u5728\u4e0d\u53ef\u8c03\u548c\u7684\u51b2\u7a81\u3002\u4e0a\u4e0b\u6587\u7ba1\u7406\u7684\u6838\u5fc3\u76ee\u6807\u662f\u5728\u6709\u9650\u7684Token\u9884\u7b97\u5185，\u52a8\u6001\u88c5\u8f7d\u6700\u9ad8\u4fe1\u606f\u5bc6\u5ea6\u7684\u6587\u672c\u3002<\/p>\n<\/li>\n

  • \n

    \u6838\u5fc3\u6982\u5ff5\u4e0e\u73af\u5883
    \n2.1 \u4f9d\u8d56\u4e0e\u7248\u672c
    \n\u7ec4\u4ef6\u7248\u672c\u8981\u6c42\u4f5c\u7528
    \ntiktoken>= 0.7OpenAI\u5b98\u65b9Token\u8ba1\u7b97\u5de5\u5177
    \nlangchain>= 0.3\u63d0\u4f9b\u957f\u6587\u672c\u5206\u5272\u4e0e\u6458\u8981\u94fe\u7ec4\u4ef6
    \ntransformers>= 4.44\u672c\u5730LLM Tokenizer\u52a0\u8f7d
    \n\u4ee5\u4e0b\u4ee3\u7801\u4e3a\u6700\u5c0f\u5316\u6f14\u793a\u5b9e\u73b0，\u751f\u4ea7\u73af\u5883\u9700\u63a5\u5165\u771f\u5b9e\u7684\u4f1a\u8bdd\u6570\u636e\u5e93\u4e0e\u5f02\u6b65\u4efb\u52a1\u961f\u5217\u3002<\/p>\n<\/li>\n

    2.2 \u5173\u952e\u672f\u8bed
    \nLost in the Middle：\u6a21\u578b\u5bf9\u4e0a\u4e0b\u6587\u5f00\u5934（\u7cfb\u7edf\u6307\u4ee4）\u548c\u7ed3\u5c3e（\u6700\u65b0\u95ee\u9898）\u7684\u6ce8\u610f\u529b\u6700\u9ad8，\u4e2d\u95f4\u90e8\u5206\u6613\u88ab\u5ffd\u7565\u7684\u73b0\u8c61\u3002<\/p>\n

    Token\u9884\u7b97（Token Budget）：\u5355\u6b21\u8bf7\u6c42\u5141\u8bb8\u7684\u6700\u5927Token\u6570\u3002\u5fc5\u987b\u4e3a\u6a21\u578b\u8f93\u51fa（Completion）\u9884\u7559\u81f3\u5c11 20%-30% \u7684\u7a7a\u95f4\u3002<\/p>\n

    \u6ed1\u52a8\u7a97\u53e3（Sliding Window）：\u4fdd\u7559\u6700\u8fd1N\u8f6e\u5bf9\u8bdd\u6216\u6700\u8fd1M\u4e2aToken，\u4e22\u5f03\u6700\u8001\u7684\u5386\u53f2，\u4fdd\u8bc1O(1)\u7684\u4e0a\u4e0b\u6587\u66f4\u65b0\u3002<\/p>\n

    \u6458\u8981\u538b\u7f29（Summarization）：\u5c06\u5386\u53f2\u6d88\u606f\u4ea4\u7ed9\u5c0f\u6a21\u578b\u751f\u6210\u6458\u8981，\u4ee5\u6781\u5c11\u7684Token\u4fdd\u7559\u6838\u5fc3\u8bed\u4e49，\u4ee3\u4ef7\u662f\u4fe1\u606f\u6709\u635f\u3002<\/p>\n

  • \u6838\u5fc3\u5185\u5bb9
    \n3.1 \u53ef\u8fd0\u884c\u4ee3\u7801：\u57fa\u4e8eToken\u9884\u7b97\u7684\u6ed1\u52a8\u7a97\u53e3\u88c1\u526a
    \npython<\/li>\n

    file: context_management.py<\/h2>\n

    pip install tiktoken langchain<\/h2>\n

    import tiktoken<\/p>\n

    def manage_context(
    \nmessages: list[dict],
    \nmodel_name: str = \u201cgpt-3.5-turbo\u201d,
    \nmax_tokens: int = 4096,
    \ncompletion_reserve: int = 1024
    \n) -> list[dict]:
    \n\u201c\u201d"
    \n\u6839\u636eToken\u9884\u7b97\u88c1\u526a\u5bf9\u8bdd\u5386\u53f2，\u4f18\u5148\u4fdd\u7559System\u548c\u6700\u8fd1\u6d88\u606f\u3002
    \n\u201c\u201d"
    \nencoding = tiktoken.encoding_for_model(model_name)<\/p>\n

    # 1. \u8ba1\u7b97\u7cfb\u7edf\u6307\u4ee4\u4e0e\u6700\u65b0\u95ee\u9898\u7684Token
    \nsystem_msg = [m for m in messages if m["role"] == "system"]
    \nlatest_msg = messages[-1:] if messages[-1]["role"] != "system" else []<\/p>\n

    fixed_tokens = sum(len(encoding.encode(m["content"])) for m in system_msg + latest_msg)<\/p>\n

    # \u5269\u4f59\u53ef\u7528Token\u9884\u7b97
    \navailable_budget = max_tokens – completion_reserve – fixed_tokens
    \nif available_budget <= 0:
    \n raise ValueError("System\u6307\u4ee4\u4e0e\u6700\u65b0\u6d88\u606f\u8fc7\u957f，\u8d85\u51fa\u57fa\u7840\u9884\u7b97")<\/p>\n

    # 2. \u4ece\u540e\u5f80\u524d\u904d\u5386\u5386\u53f2\u6d88\u606f，\u586b\u6ee1\u5269\u4f59\u9884\u7b97
    \nhistory = [m for m in messages if m not in system_msg and m not in latest_msg]
    \ntrimmed_history = []
    \ncurrent_tokens = 0<\/p>\n

    for msg in reversed(history):
    \n msg_tokens = len(encoding.encode(msg["content"]))
    \n if current_tokens + msg_tokens <= available_budget:
    \n trimmed_history.insert(0, msg)
    \n current_tokens += msg_tokens
    \n else:
    \n break # \u9884\u7b97\u8017\u5c3d，\u4e22\u5f03\u66f4\u65e9\u7684\u5386\u53f2<\/p>\n

    # 3. \u91cd\u65b0\u62fc\u88c5：System + \u88c1\u526a\u540e\u7684\u5386\u53f2 + \u6700\u65b0\u6d88\u606f
    \nreturn system_msg + trimmed_history + latest_msg<\/p>\n

    \u6d4b\u8bd5\u7528\u4f8b<\/h2>\n

    if name == \u201cmain\u201d:
    \nlong_messages = [
    \n{\u201crole\u201d: \u201csystem\u201d, \u201ccontent\u201d: \u201c\u4f60\u662f\u4e00\u4e2a\u6280\u672f\u6587\u6863\u52a9\u624b\u3002\u201d},
    \n{\u201crole\u201d: \u201cuser\u201d, \u201ccontent\u201d: \u201c\u8bf7\u8be6\u7ec6\u89e3\u91ca\u5fae\u670d\u52a1\u67b6\u6784\u3002\u201d * 10},
    \n{\u201crole\u201d: \u201cassistant\u201d, \u201ccontent\u201d: \u201c\u5fae\u670d\u52a1\u662f\u4e00\u79cd\u67b6\u6784\u6a21\u5f0f\u2026\u201d * 10},
    \n{\u201crole\u201d: \u201cuser\u201d, \u201ccontent\u201d: \u201c\u90a3\u4e48\u670d\u52a1\u7f51\u683c（Service Mesh）\u5462？\u201d * 10},
    \n{\u201crole\u201d: \u201cuser\u201d, \u201ccontent\u201d: \u201c\u603b\u7ed3\u4e00\u4e0b\u521a\u624d\u8ba8\u8bba\u7684\u5fae\u670d\u52a1\u4f18\u7f3a\u70b9\u3002\u201d} # \u6700\u65b0\u95ee\u9898
    \n]<\/p>\n

    trimmed = manage_context(long_messages, max_tokens=1000, completion_reserve=200)
    \nprint(f"\u88c1\u526a\u540e\u6d88\u606f\u6570: {len(trimmed)}")
    \nfor m in trimmed:
    \n print(f"[{m['role']}] {m['content'][:30]}…")<\/p>\n

    \u5165\u53e3\u547d\u4ee4：<\/p>\n

    bash
    \npython context_management.py
    \n\u9884\u671f\u8f93\u51fa：<\/p>\n

    text
    \n\u88c1\u526a\u540e\u6d88\u606f\u6570: 4
    \n[system] \u4f60\u662f\u4e00\u4e2a\u6280\u672f\u6587\u6863\u52a9\u624b\u3002\u2026
    \n[user] \u90a3\u4e48\u670d\u52a1\u7f51\u683c（Service Mesh）\u5462？\u2026
    \n[user] \u603b\u7ed3\u4e00\u4e0b\u521a\u624d\u8ba8\u8bba\u7684\u5fae\u670d\u52a1\u4f18\u7f3a\u70b9\u3002\u2026
    \n(\u6ce8：\u6700\u65e9\u7684\u4e00\u6761\u5fae\u670d\u52a1\u5b9a\u4e49\u56e0\u8d85\u51faToken\u9884\u7b97\u88ab\u81ea\u52a8\u6ed1\u7a97\u4e22\u5f03，\u4fdd\u8bc1\u4e86\u6700\u65b0\u95ee\u9898\u80fd\u9001\u5165\u6a21\u578b\u3002)<\/p>\n

    3.2 \u56db\u79cd\u4e0a\u4e0b\u6587\u7ba1\u7406\u7b56\u7565\u5bf9\u6bd4
    \n\u7b56\u7565\u5b9e\u73b0\u590d\u6742\u5ea6Token\u524a\u51cf\u7387\u4fe1\u606f\u5b8c\u6574\u5ea6\u9002\u7528\u573a\u666f
    \n\u6ed1\u52a8\u7a97\u53e3\u4f4e（O(N)\u904d\u5386）\u89c6\u7a97\u53e3\u5927\u5c0f\u4ec5\u4fdd\u7559\u8fd1\u671f，\u8fdc\u671f\u4e22\u5931\u95f2\u804a\u3001\u9ad8\u9891\u5373\u65f6\u95ee\u7b54
    \n\u622a\u65ad（Truncation）\u6700\u4f4e（\u76f4\u63a5\u5207）\u6781\u9ad8\u6700\u5dee，\u6613\u4e22\u7cfb\u7edf\u6307\u4ee4\u6781\u4f4e\u9891\u515c\u5e95，\u4e0d\u63a8\u8350
    \n\u6458\u8981\u538b\u7f29\u4e2d（\u9700\u8c03\u7528\u5c0f\u6a21\u578b）70%-90%\u53d9\u4e8b\u8fde\u8d2f，\u7ec6\u8282\u4e22\u5931\u957f\u5267\u60c5\u751f\u6210\u3001\u4e2a\u4eba\u52a9\u7406
    \nRAG\u68c0\u7d22\u56de\u586b\u9ad8（\u9700\u5411\u91cf\u5e93）90%\u4ee5\u4e0a\u4ec5\u4fdd\u7559\u4e0e\u5f53\u524d\u95ee\u9898\u76f8\u5173\u7247\u6bb5\u4f01\u4e1a\u77e5\u8bc6\u5e93\u3001\u5ba2\u670d\u5de5\u5355
    \n3.3 \u907f\u5751\u6e05\u5355
    \n\u98ce\u9669\u70b9\u89e6\u53d1\u6761\u4ef6\u5904\u7f6e\u65b9\u5f0f
    \n\u88c1\u526a\u5bfc\u81f4\u201c\u5931\u5fc6\u201d\u7cfb\u7edf\u63d0\u793a\u8bcd\u6216\u5173\u952e\u5b9e\u4f53\u88ab\u622a\u65ad\u5f3a\u5236\u5c06 System \u6d88\u606f\u4e0e\u6700\u65b0 User \u6d88\u606f\u8bbe\u4e3a\u4e0d\u53ef\u88c1\u526a\u7684\u201c\u951a\u70b9\u201d
    \n\u6458\u8981\u53d1\u751f\u5e7b\u89c9\u7528\u5c0f\u6a21\u578b\u538b\u7f29\u4e8b\u5b9e\u578b\u6570\u636e\u6458\u8981\u4ec5\u7528\u4e8e\u538b\u7f29\u53d9\u4e8b\/\u95f2\u804a；UUID\u3001\u91d1\u989d\u3001\u4ee3\u7801\u8def\u5f84\u7b49\u5b58\u5165\u7ed3\u6784\u5316\u6570\u636e\u5e93\u6309\u9700\u6ce8\u5165
    \n\u8f93\u51fa\u622a\u65adAPI\u62a5\u9519\u672a\u9884\u7559 max_tokens \u7a7a\u95f4\u6bcf\u6b21\u8bf7\u6c42\u7684\u8ba1\u7b97\u516c\u5f0f：\u8f93\u5165Token + max_tokens <= \u6a21\u578b\u4e0a\u4e0b\u6587\u4e0a\u9650
    \n\u591a\u6a21\u6001Token\u8ba1\u7b97\u9519\u8bef\u56fe\u7247\/\u97f3\u9891\u76f4\u63a5\u8ba1\u5165\u6587\u672cToken\u9700\u4f7f\u7528\u7279\u5b9a\u6a21\u578b\u7684 tokenizer \u4e14\u6309 image_tokens \u89c4\u5219\u5355\u72ec\u7d2f\u52a0
    \n4. \u5e38\u89c1\u95ee\u9898\u4e0e\u6392\u9519
    \n\u73b0\u8c61：\u6309\u7167\u5b57\u7b26\u6570\u622a\u65ad，\u4f46API\u4ecd\u62a5 context_length_exceeded\u3002
    \n\u539f\u56e0：\u4e2d\u82f1\u6587Token\u5bc6\u5ea6\u4e0d\u540c（1\u4e2a\u6c49\u5b57\u7ea6\u53601.5-2\u4e2aToken），\u5b57\u7b26\u6570\u4e0eToken\u6570\u4e0d\u6210\u6b63\u6bd4\u3002
    \n\u89e3\u51b3：\u5fc5\u987b\u4f7f\u7528\u5b98\u65b9\u63d0\u4f9b\u7684 tiktoken \u6216\u5bf9\u5e94\u6a21\u578b\u7684 tokenizer \u8fdb\u884c\u7cbe\u51c6\u8ba1\u7b97，\u7981\u6b62\u7528 len(text) \u4f30\u7b97\u3002<\/p>\n

    \u73b0\u8c61：\u957f\u5bf9\u8bdd\u8fdb\u884c\u5230\u7b2c10\u8f6e\u540e，\u6a21\u578b\u5f00\u59cb\u91cd\u590d\u524d\u6587\u6216\u903b\u8f91\u6df7\u4e71\u3002
    \n\u539f\u56e0：\u5178\u578b\u7684\u201cLost in the Middle\u201d\u73b0\u8c61，\u4e0a\u4e0b\u6587\u8fc7\u957f\u5bfc\u81f4\u6ce8\u610f\u529b\u673a\u5236\u5931\u6548\u3002
    \n\u89e3\u51b3：\u4e0d\u8981\u585e\u6ee1\u6574\u4e2a\u4e0a\u4e0b\u6587\u7a97\u53e3\u3002\u5c06\u5b9e\u9645\u4f7f\u7528\u7684Token\u91cf\u63a7\u5236\u5728\u6a21\u578b\u4e0a\u9650\u768450%-70%\u5de6\u53f3，\u7559\u51fa\u6ce8\u610f\u529b\u4f59\u91cf\u3002<\/p>\n

    \u73b0\u8c61：RAG\u68c0\u7d22\u540e，\u9996\u5c3e\u5e26\u6709\u5927\u91cf\u65e0\u5173\u7684\u6bb5\u843d\u3002
    \n\u539f\u56e0：\u68c0\u7d22\u51fa\u7684\u7247\u6bb5\u672a\u505a\u4e8c\u6b21\u88c1\u526a\u6216\u6392\u5e8f，\u76f4\u63a5\u62fc\u63a5\u5360\u6ee1\u7a97\u53e3\u3002
    \n\u89e3\u51b3：\u5bf9\u53ec\u56de\u7ed3\u679c\u8fdb\u884c Rerank \u91cd\u6392，\u4ec5\u4fdd\u7559 Top-3 \u7247\u6bb5；\u5e76\u5bf9\u7247\u6bb5\u8fdb\u884c\u8bed\u4e49\u53bb\u91cd\u3002<\/p>\n

    \u73b0\u8c61：\u6d41\u5f0f\u8f93\u51fa\u521a\u5f00\u59cb\u5c31\u7a81\u7136\u7ec8\u6b62\u3002
    \n\u539f\u56e0：\u8f93\u5165Token\u6070\u597d\u7b49\u4e8e max_tokens，\u6a21\u578b\u6ca1\u6709\u5269\u4f59\u7a7a\u95f4\u751f\u6210\u56de\u7b54\u3002
    \n\u89e3\u51b3：\u52a8\u6001\u8c03\u6574 max_tokens \u53c2\u6570，\u786e\u4fdd \u6a21\u578b\u4e0a\u9650 – \u8f93\u5165Token \u59cb\u7ec8\u5927\u4e8e200\u3002<\/p>\n

  • \u4f18\u5316\u5efa\u8bae\u4e0e\u8fb9\u754c
    \n\u5206\u5c42\u6cbb\u7406：\u91c7\u7528 \u5b9e\u65f6\u6ed1\u52a8\u7a97\u53e3 + \u540e\u53f0\u5f02\u6b65\u6458\u8981\u3002\u524d\u53f0\u8bf7\u6c42\u53ea\u5e26\u6700\u8fd1K\u8f6e\u5bf9\u8bdd\u4fdd\u8bc1\u4f4e\u5ef6\u8fdf；\u540e\u53f0\u7ef4\u62a4\u4e00\u4efd\u957f\u671f\u8bb0\u5fc6\u6458\u8981，\u6309\u9700\u4ee5System Prompt\u5f62\u5f0f\u6ce8\u5165\u3002<\/li>\n

    \u4e0a\u4e0b\u6587\u7f13\u5b58（Prefix Caching）：\u5728\u591a\u8f6e\u5bf9\u8bdd\u4e2d，\u7cfb\u7edf\u63d0\u793a\u8bcd\u548c\u5386\u53f2\u524d\u7f00\u662f\u4e0d\u53d8\u7684\u3002\u5229\u7528 vLLM \u7b49\u5f15\u64ce\u7684\u524d\u7f00\u7f13\u5b58\u7279\u6027，\u53ef\u4ee5\u5927\u5e45\u964d\u4f4e\u957f\u4e0a\u4e0b\u6587\u7684TTFT\u3002<\/p>\n

    \u63d0\u793a\u8bcd\u538b\u7f29（Prompt Compression）：\u4f7f\u7528 LLMLingua \u7b49\u5de5\u5177，\u5728\u4fdd\u6301\u8bed\u4e49\u7684\u524d\u63d0\u4e0b，\u76f4\u63a5\u5254\u9664\u63d0\u793a\u8bcd\u4e2d\u7684\u505c\u7528\u8bcd\u3001\u5197\u4f59\u4fee\u9970\u8bed，\u5b9e\u73b0\u65e0\u635f\/\u8fd1\u65e0\u635f\u538b\u7f29\u3002<\/p>\n

    \u8fb9\u754c：\u4ee3\u7801\u5206\u6790\u3001\u5408\u540c\u5ba1\u67e5\u7b49\u4efb\u52a1\u4e25\u7981\u4f7f\u7528\u6709\u635f\u538b\u7f29（\u5982\u6458\u8981），\u5fc5\u987b\u4f9d\u8d56\u5927\u7a97\u53e3\u6a21\u578b（\u5982128K）\u6216 RAG \u7cbe\u51c6\u5b9a\u4f4d，\u786e\u4fdd\u6bcf\u4e00\u884c\u4ee3\u7801\u6216\u6761\u6b3e\u7684\u7edd\u5bf9\u51c6\u786e\u6027\u3002<\/p>\n

  • \u6c42\u804c\u4e0e\u9762\u8bd5\u5173\u8054
    \n\u4ec0\u4e48\u662f\u201cLost in the Middle\u201d\u73b0\u8c61？\u5982\u4f55\u7f13\u89e3？
    \n\u601d\u8def：\u6a21\u578b\u5bf9\u8d85\u957f\u4e0a\u4e0b\u6587\u4e2d\u95f4\u90e8\u5206\u7684\u6ce8\u610f\u529b\u6700\u5f31\u3002\u7f13\u89e3\u65b9\u5f0f：\u5c06\u6700\u5173\u952e\u6307\u4ee4\u7f6e\u4e8e\u9996\u5c3e；\u51cf\u5c11\u65e0\u6548Context，\u514b\u5236\u585e\u6ee1\u7a97\u53e3；\u4f7f\u7528RAG\u6309\u9700\u63d0\u4f9b\u4fe1\u606f\u3002<\/li>\n

    \u5982\u4f55\u7cbe\u786e\u8ba1\u7b97Prompt\u7684Token\u6d88\u8017？
    \n\u601d\u8def：\u7981\u6b62\u6309\u5b57\u6570\u4f30\u7b97\u3002\u4f7f\u7528\u5bf9\u5e94\u6a21\u578b\u7684 Tokenizer（\u5982 tiktoken, sentencepiece）\u8fdb\u884c\u7f16\u7801\u8ba1\u7b97\u3002\u4e0d\u540c\u8bed\u8a00\u7684Token\u5bc6\u5ea6\u5dee\u5f02\u5de8\u5927\u3002<\/p>\n

    \u6ed1\u52a8\u7a97\u53e3\u4e0e\u6458\u8981\u538b\u7f29\u5404\u81ea\u7684\u4f18\u7f3a\u70b9\u662f\u4ec0\u4e48？
    \n\u601d\u8def：\u6ed1\u52a8\u7a97\u53e3\u5b9e\u73b0\u7b80\u5355\u3001\u5ef6\u8fdf\u4f4e，\u4f46\u4e22\u5931\u8fdc\u671f\u4fe1\u606f；\u6458\u8981\u538b\u7f29\u4fdd\u7559\u8bed\u4e49\u8fde\u8d2f\u6027，\u4f46\u9700\u989d\u5916\u63a8\u7406\u5f00\u9500\u4e14\u53ef\u80fd\u5f15\u5165\u5e7b\u89c9\u3002<\/p>\n

    \u5728RAG\u67b6\u6784\u4e2d，\u5982\u4f55\u63a7\u5236\u586b\u5165\u4e0a\u4e0b\u6587\u7684Token\u6570？
    \n\u601d\u8def：\u4f7f\u7528\u8bed\u4e49\u5206\u5272（Chunking）\u63a7\u5236\u5355\u5757\u5927\u5c0f；\u4f7f\u7528Rerank\u6a21\u578b\u53ea\u53d6Top-K；\u9650\u5236\u6700\u7ec8\u62fc\u63a5\u8fdb\u5165Prompt\u7684\u603bToken\u9884\u7b97\u3002<\/p>\n

    \u7cfb\u7edf\u63d0\u793a\u8bcd（System Prompt）\u5728\u88c1\u526a\u65f6\u5e94\u9075\u5faa\u4ec0\u4e48\u539f\u5219？
    \n\u601d\u8def：\u7edd\u5bf9\u4fdd\u7559\u539f\u5219\u3002System Prompt \u5b9a\u4e49\u4e86\u6a21\u578b\u7684\u89d2\u8272\u548c\u7ea2\u7ebf，\u5fc5\u987b\u4f5c\u4e3a\u201c\u951a\u70b9\u201d\u9501\u5b9a，\u4e0d\u53c2\u4e0e\u6ed1\u7a97\u88c1\u526a\u3002<\/p>\n

    \u5982\u4f55\u5728\u540e\u53f0\u5b9e\u73b0\u4e0d\u963b\u585e\u4e3b\u7ebf\u7a0b\u7684\u957f\u5bf9\u8bdd\u6458\u8981\u538b\u7f29？
    \n\u601d\u8def：\u5c06\u65e7\u5bf9\u8bdd\u6254\u8fdb\u6d88\u606f\u961f\u5217（\u5982Kafka\/RabbitMQ），\u7531\u540e\u53f0Worker（\u5982Celery）\u8c03\u7528\u8f7b\u91cfLLM\u751f\u6210\u6458\u8981，\u5b8c\u6210\u540e\u66f4\u65b0\u6570\u636e\u5e93，\u4e0b\u6b21\u8bf7\u6c42\u65f6\u76f4\u63a5\u8bfb\u53d6\u6700\u65b0\u6458\u8981\u3002<\/p>\n

    \u603b\u7ed3
    \n\u7cbe\u51c6\u8ba1\u7b97Token\u662f\u7ba1\u7406\u4e0a\u4e0b\u6587\u7684\u5148\u51b3\u6761\u4ef6，\u5fc5\u987b\u4f7f\u7528 tiktoken \u7b49\u5de5\u5177\u675c\u7edd\u5b57\u7b26\u4f30\u7b97\u3002<\/p>\n

    \u6ed1\u52a8\u7a97\u53e3\u662f\u6210\u672c\u6700\u4f4e\u7684\u4fdd\u5e95\u7b56\u7565，\u4f46\u5fc5\u987b\u5c06 System Prompt \u4e0e \u6700\u65b0User\u8f93\u5165 \u8bbe\u4e3a\u4e0d\u53ef\u88c1\u526a\u951a\u70b9\u3002<\/p>\n

    \u6458\u8981\u538b\u7f29\u9002\u5408\u53d9\u4e8b\u573a\u666f，\u4e8b\u5b9e\u578b\u6570\u636e（ID\u3001\u6570\u503c\u3001\u4ee3\u7801）\u5fc5\u987b\u8d70\u7ed3\u6784\u5316\u5b58\u50a8，\u4e25\u9632\u6458\u8981\u5e7b\u89c9\u3002<\/p>\n

    RAG\u662f\u89e3\u51b3\u8d85\u957f\u5916\u6302\u77e5\u8bc6\u7684\u6700\u4f73\u8def\u5f84，\u4f46\u9700\u914d\u5408 Rerank \u907f\u514d\u65e0\u6548\u7247\u6bb5\u585e\u6ee1\u7a97\u53e3\u3002<\/p>\n

    \u5207\u5fcc\u585e\u6ee1\u4e0a\u4e0b\u6587\u7a97\u53e3，\u4fdd\u7559 30%-50% \u7684\u4f59\u91cf\u80fd\u6709\u6548\u7f13\u89e3\u201cLost in the Middle\u201d\u5bfc\u81f4\u7684\u6ce8\u610f\u529b\u8870\u51cf\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"

    \/* MD \/ \u5bcc\u6587\u672c\u4e2d\u7684 .toc\uff08\u542b\u535a\u5ba2\u56ed\u642c\u5bb6\u7b49\u5d4c\u5957\u7ed3\u6784\uff09\uff1b.toc-box \u5728\u4fa7\u680f\uff0c\u4e0d\u53d7\u5f71\u54cd *\/#content_views .toc,\/* \u7f16\u8f91\u5668\u5e38\u5728\u76ee\u5f55\u524d\u540e\u63d2\u5165\u7a7a p\uff08:empty \u4ecd\u5360 20px\uff09\uff0c\u4e00\u5e76\u53bb\u6389\u907f\u514d\u9876\u7a7a\u9699 *\/#content_views.markdown_views > p:empty:has(+ .toc),#content_views.markdown_views > .toc + p:empty,\/* \u5bcc\u6587\u672c\u65e7\u7248\u76ee\u5f55\u6807\u8bb0 *\/#content<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[14183,58,4119,100,44],"topic":[],"class_list":["post-123675","post","type-post","status-publish","format-standard","hentry","category-server","tag-tiktoken","tag-linux","tag-rag","tag-100","tag-44"],"yoast_head":"\n\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.wsisp.com\/helps\/123675.html\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"og:description\" content=\"\/* MD \/ \u5bcc\u6587\u672c\u4e2d\u7684 .toc\uff08\u542b\u535a\u5ba2\u56ed\u642c\u5bb6\u7b49\u5d4c\u5957\u7ed3\u6784\uff09\uff1b.toc-box \u5728\u4fa7\u680f\uff0c\u4e0d\u53d7\u5f71\u54cd *\/#content_views .toc,\/* \u7f16\u8f91\u5668\u5e38\u5728\u76ee\u5f55\u524d\u540e\u63d2\u5165\u7a7a p\uff08:empty \u4ecd\u5360 20px\uff09\uff0c\u4e00\u5e76\u53bb\u6389\u907f\u514d\u9876\u7a7a\u9699 *\/#content_views.markdown_views > p:empty:has(+ .toc),#content_views.markdown_views > .toc + p:empty,\/* \u5bcc\u6587\u672c\u65e7\u7248\u76ee\u5f55\u6807\u8bb0 *\/#content\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.wsisp.com\/helps\/123675.html\" \/>\n<meta property=\"og:site_name\" content=\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-07T17:24:52+00:00\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/123675.html\",\"url\":\"https:\/\/www.wsisp.com\/helps\/123675.html\",\"name\":\"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"isPartOf\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\"},\"datePublished\":\"2026-11-07T17:24:52+00:00\",\"dateModified\":\"2026-11-07T17:24:52+00:00\",\"author\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/123675.html#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.wsisp.com\/helps\/123675.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/123675.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.wsisp.com\/helps\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\",\"url\":\"https:\/\/www.wsisp.com\/helps\/\",\"name\":\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"description\":\"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"contentUrl\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"caption\":\"admin\"},\"sameAs\":[\"http:\/\/wp.wsisp.com\"],\"url\":\"https:\/\/www.wsisp.com\/helps\/author\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.wsisp.com\/helps\/123675.html","og_locale":"zh_CN","og_type":"article","og_title":"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","og_description":"\/* MD \/ \u5bcc\u6587\u672c\u4e2d\u7684 .toc\uff08\u542b\u535a\u5ba2\u56ed\u642c\u5bb6\u7b49\u5d4c\u5957\u7ed3\u6784\uff09\uff1b.toc-box \u5728\u4fa7\u680f\uff0c\u4e0d\u53d7\u5f71\u54cd *\/#content_views .toc,\/* \u7f16\u8f91\u5668\u5e38\u5728\u76ee\u5f55\u524d\u540e\u63d2\u5165\u7a7a p\uff08:empty \u4ecd\u5360 20px\uff09\uff0c\u4e00\u5e76\u53bb\u6389\u907f\u514d\u9876\u7a7a\u9699 *\/#content_views.markdown_views > p:empty:has(+ .toc),#content_views.markdown_views > .toc + p:empty,\/* \u5bcc\u6587\u672c\u65e7\u7248\u76ee\u5f55\u6807\u8bb0 *\/#content","og_url":"https:\/\/www.wsisp.com\/helps\/123675.html","og_site_name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","article_published_time":"2026-11-07T17:24:52+00:00","author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"4 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.wsisp.com\/helps\/123675.html","url":"https:\/\/www.wsisp.com\/helps\/123675.html","name":"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","isPartOf":{"@id":"https:\/\/www.wsisp.com\/helps\/#website"},"datePublished":"2026-11-07T17:24:52+00:00","dateModified":"2026-11-07T17:24:52+00:00","author":{"@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41"},"breadcrumb":{"@id":"https:\/\/www.wsisp.com\/helps\/123675.html#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.wsisp.com\/helps\/123675.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.wsisp.com\/helps\/123675.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.wsisp.com\/helps"},{"@type":"ListItem","position":2,"name":"\u6a21\u578b\u4e0a\u4e0b\u6587\u7a97\u53e3\u7ba1\u7406\uff0c\u907f\u514d\u8d85\u957f\u5bf9\u8bdd\u8d85\u9650"}]},{"@type":"WebSite","@id":"https:\/\/www.wsisp.com\/helps\/#website","url":"https:\/\/www.wsisp.com\/helps\/","name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","description":"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41","name":"admin","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/","url":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","contentUrl":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","caption":"admin"},"sameAs":["http:\/\/wp.wsisp.com"],"url":"https:\/\/www.wsisp.com\/helps\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/123675","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/comments?post=123675"}],"version-history":[{"count":0,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/123675\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media?parent=123675"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/categories?post=123675"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/tags?post=123675"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/topic?post=123675"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}