{"id":35989,"date":"2025-05-07T09:10:34","date_gmt":"2025-05-07T01:10:34","guid":{"rendered":"https:\/\/www.wsisp.com\/helps\/35989.html"},"modified":"2025-05-07T09:10:34","modified_gmt":"2025-05-07T01:10:34","slug":"%e5%bc%80%e6%ba%90%e6%a8%a1%e5%9e%8b%e5%ba%94%e7%94%a8%e8%90%bd%e5%9c%b0-deepseek-r1-distill-qwen-7b-lora%e5%be%ae%e8%b0%83-llama-factory-%e5%8d%95%e6%9c%ba%e5%8d%95%e5%8d%a1-v100%ef%bc%88%e4%b8%80","status":"publish","type":"post","link":"https:\/\/www.wsisp.com\/helps\/35989.html","title":{"rendered":"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09"},"content":{"rendered":"

\u4e00\u3001\u524d\u8a00<\/h2>\n

\u00a0\u00a0\u00a0 \u5982\u4eca，\u5927\u8bed\u8a00\u6a21\u578b\u9886\u57df\u70ed\u95f9\u975e\u51e1，\u5404\u79cd\u6a21\u578b\u4e0d\u65ad\u6d8c\u73b0\u3002DeepSeek-R1-Distill-Qwen-7B \u6a21\u578b\u51ed\u501f\u5176\u51fa\u8272\u7684\u6548\u679c\u548c\u6027\u80fd，\u5438\u5f15\u4e86\u4f17\u591a\u5f00\u53d1\u8005\u7684\u76ee\u5149\u3002\u800c LLaMa-Factory \u4f5c\u4e3a\u5f3a\u5927\u7684\u5fae\u8c03\u5de5\u5177，\u80fd\u8ba9\u6a21\u578b\u66f4\u597d\u5730\u6ee1\u8db3\u4e2a\u6027\u5316\u9700\u6c42\u3002<\/p>\n

\u00a0\u00a0\u00a0 \u5728\u672c\u7bc7\u4e2d，\u5c06\u6df1\u5165\u63a2\u8ba8\u5982\u4f55\u8fd0\u7528 LLaMa-Factory \u5bf9 DeepSeek-R1-Distill-Qwen-7B \u6a21\u578b\u8fdb\u884c\u5fae\u8c03，\u63a2\u7d22\u5982\u4f55\u901a\u8fc7\u5fae\u8c03，\u8ba9\u6a21\u578b\u66f4\u597d\u5730\u4e3a\u6211\u4eec\u6240\u7528\u3002<\/p>\n

\"\"<\/p>\n

\n

\u4e8c\u3001\u672f\u8bed\u4ecb\u7ecd<\/h2>\n

2.1. LoRA\u5fae\u8c03<\/h3>\n

\u00a0 \u00a0 LoRA (Low-Rank Adaptation) \u7528\u4e8e\u5fae\u8c03\u5927\u578b\u8bed\u8a00\u6a21\u578b (LLM)\u3002 \u00a0\u662f\u4e00\u79cd\u6709\u6548\u7684\u81ea\u9002\u5e94\u7b56\u7565，\u5b83\u4e0d\u4f1a\u5f15\u5165\u989d\u5916\u7684\u63a8\u7406\u5ef6\u8fdf，\u5e76\u5728\u4fdd\u6301\u6a21\u578b\u8d28\u91cf\u7684\u540c\u65f6\u663e\u7740\u51cf\u5c11\u4e0b\u6e38\u4efb\u52a1\u7684\u53ef\u8bad\u7ec3\u53c2\u6570\u6570\u91cf\u3002<\/p>\n

2.2. \u53c2\u6570\u9ad8\u6548\u5fae\u8c03(PEFT)\u00a0<\/h3>\n

\u00a0 \u00a0 \u4ec5\u5fae\u8c03\u5c11\u91cf (\u989d\u5916) \u6a21\u578b\u53c2\u6570，\u540c\u65f6\u51bb\u7ed3\u9884\u8bad\u7ec3 LLM \u7684\u5927\u90e8\u5206\u53c2\u6570，\u4ece\u800c\u5927\u5927\u964d\u4f4e\u4e86\u8ba1\u7b97\u548c\u5b58\u50a8\u6210\u672c\u3002<\/p>\n

2.3. LLaMA-Factory<\/h3>\n

\u00a0 \u00a0 \u662f\u4e00\u4e2a\u4e0e LLaMA（Large Language Model Meta AI）\u76f8\u5173\u7684\u9879\u76ee，\u65e8\u5728\u4e3a\u7528\u6237\u63d0\u4f9b\u4e00\u79cd\u7b80\u5316\u548c\u4f18\u5316\u7684\u65b9\u5f0f\u6765\u8bad\u7ec3\u3001\u5fae\u8c03\u548c\u90e8\u7f72\u5927\u578b\u8bed\u8a00\u6a21\u578b\u3002\u8be5\u5de5\u5177\u901a\u5e38\u5305\u62ec\u4e00\u7cfb\u5217\u529f\u80fd，\u5982\u6570\u636e\u5904\u7406\u3001\u6a21\u578b\u914d\u7f6e\u3001\u8bad\u7ec3\u76d1\u63a7\u7b49，\u4ee5\u5e2e\u52a9\u7814\u7a76\u4eba\u5458\u548c\u5f00\u53d1\u8005\u66f4\u9ad8\u6548\u5730\u4f7f\u7528 LLaMA \u6a21\u578b\u3002<\/p>\n

\u00a0 \u00a0 LLaMA-Factory\u652f\u6301\u7684\u6a21\u578b\u5217\u8868：<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

2.4. DeepSeek-R1-Distill-Qwen-7B<\/h3>\n

\u00a0 \u00a0 \u662f\u4e00\u4e2a\u7531DeepSeek\u5f00\u53d1\u7684\u6a21\u578b，\u5b83\u662f\u901a\u8fc7\u84b8\u998f\u6280\u672f\u5c06Qwen-7B\u5927\u578b\u6a21\u578b\u7684\u4e00\u90e8\u5206\u77e5\u8bc6\u7cbe\u534e\u63d0\u53d6\u51fa\u6765，\u4ee5\u9002\u5e94\u66f4\u5c0f\u578b\u7684\u6a21\u578b\u9700\u6c42\u3002<\/p>\n

\n

\u4e09\u3001\u524d\u7f6e\u6761\u4ef6<\/h2>\n

\u00a03.1. \u57fa\u7840\u73af\u5883\u53ca\u524d\u7f6e\u6761\u4ef6<\/h3>\n

\u00a0\u00a0 \u00a0 1. \u64cd\u4f5c\u7cfb\u7edf：centos7<\/p>\n

\u00a0 \u00a0 \u00a02. NVIDIA Tesla V100 32GB \u00a0 CUDA Version: 12.2\u00a0<\/p>\n

\"\"<\/p>\n

\u00a0 \u00a0 \u00a03. \u63d0\u524d\u4e0b\u8f7d\u597dDeepSeek-R1-Distill-Qwen-7B\u6a21\u578b\u00a0 \u00a0 \u00a0 \u00a0 \u00a0<\/p>\n

\u00a0\u901a\u8fc7\u4ee5\u4e0b\u4e24\u4e2a\u5730\u5740\u8fdb\u884c\u4e0b\u8f7d，\u4f18\u5148\u63a8\u8350\u9b54\u642d\u00a0 \u00a0 \u00a0 \u00a0\u00a0<\/p>\n

huggingface：<\/p>\n

https:\/\/huggingface.co\/deepseek-ai\/DeepSeek-R1-Distill-Qwen-7B\/tree\/main<\/p>\n

\"\"<\/p>\n

ModelScope：<\/p>\n

\u9b54\u642d\u793e\u533a<\/p>\n

\"\"<\/p>\n

\u00a0\u6309\u9700\u9009\u62e9SDK\u6216\u8005Git\u65b9\u5f0f\u4e0b\u8f7d<\/p>\n

\"\"<\/p>\n<\/p>\n

\u00a0 \u4f7f\u7528git-lfs\u65b9\u5f0f\u4e0b\u8f7d\u793a\u4f8b：<\/p>\n

\"\"<\/p>\n<\/p>\n

3.2.\u00a0Anaconda\u5b89\u88c5<\/h3>\n

1\u3001Update System
\nsudo yum update -y
\nsudo yum upgrade -y<\/p>\n

2\u3001Download Anaconda
\nwget https:\/\/repo.anaconda.com\/archive\/Anaconda3-2022.10-Linux-x86_64.sh<\/p>\n

3\u3001Verify Data Integrity
\nsha256sum Anaconda3-2022.10-Linux-x86_64.sh<\/p>\n

4\u3001Run Anaconda Installation Script
\nbash Anaconda3-2022.10-Linux-x86_64.sh<\/p>\n

\u5b89\u88c5\u76ee\u5f55：\/opt\/anaconda3<\/p>\n

\u6ce8：\u5b89\u88c5\u4f4d\u7f6e\u53ef\u4ee5\u5728\u6267\u884c\u5b89\u88c5\u811a\u672c\u7684\u65f6\u5019\u76f4\u63a5\u6307\u5b9a，\u53ef\u4ee5\u8fd9\u6837\u4fee\u6539\u6267\u884c\u5185\u5bb9
\nbash Anaconda3-2022.10-Linux-x86_64.sh -p \/opt\/anaconda3<\/p>\n

Do you wish the installer to initialize Anaconda3 by running conda init?
\nyes<\/p>\n

\u5982\u679c\u6ca1\u6709\u6267\u884c\u521d\u59cb\u5316，\u53ef\u4ee5\u6267\u884c：\/opt\/anaconda3\/bin\/conda init<\/p>\n

\u6ce8：\u521d\u59cb\u5316\u65f6，anaconda\u5c06\u914d\u7f6e\u5199\u5165\u4e86~\/.bashrc \u6587\u4ef6，\u76f4\u63a5\u6267\u884c
\nsource ~\/.bashrc<\/p>\n

5\u3001Verify Installation
\nconda –version<\/p>\n

6\u3001\u914d\u7f6e\u955c\u50cf\u6e90
\nconda config –add channels https:\/\/mirrors.tuna.tsinghua.edu.cn\/anaconda\/pkgs\/free\/
\nconda config –add channels https:\/\/mirrors.tuna.tsinghua.edu.cn\/anaconda\/pkgs\/main\/
\nconda config –set show_channel_urls yes <\/p>\n

3.3.\u4e0b\u8f7dLLaMA-Factory<\/h3>\n

\u65b9\u5f0f\u4e00：\u76f4\u63a5\u4e0b\u8f7d<\/p>\n

\u5730\u5740：GitHub – hiyouga\/LLaMA-Factory: Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)<\/p>\n

\"\"<\/p>\n

\u65b9\u5f0f\u4e8c：\u4f7f\u7528git\u514b\u9686\u9879\u76ee<\/p>\n

git clone –depth 1 https:\/\/github.com\/hiyouga\/LLaMA-Factory.git <\/p>\n

\"\"<\/p>\n

\u4e0b\u8f7d\u597d\u7684\u9879\u76ee\u653e\u7f6e\u5728\/data\/service\u76ee\u5f55\u4e0b<\/p>\n

3.4. \u5b89\u88c5\u4f9d\u8d56<\/h3>\n

conda create –name llama_factory python=3.10
\nconda activate llama_factory
\ncd \/data\/service\/LLaMA-Factory
\npip install -e ".[torch,metrics]" -i https:\/\/pypi.tuna.tsinghua.edu.cn\/simple <\/p>\n

\u00a0 PS：\u8f6f\u786c\u4ef6\u8981\u6c42<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\n

\u56db\u3001\u6280\u672f\u5b9e\u73b0<\/h2>\n

4.1.\u6570\u636e\u51c6\u5907<\/h3>\n

\u6709\u4e24\u79cd\u683c\u5f0f\u9009\u62e9，\u5305\u62ecalpaca\u548csharegpt<\/p>\n

alpaca\u793a\u4f8b\u683c\u5f0f：<\/p>\n

[
\n {
\n "instruction": "\u4eba\u7c7b\u6307\u4ee4（\u5fc5\u586b）",
\n "input": "\u4eba\u7c7b\u8f93\u5165（\u9009\u586b）",
\n "output": "\u6a21\u578b\u56de\u7b54（\u5fc5\u586b）",
\n "system": "\u7cfb\u7edf\u63d0\u793a\u8bcd（\u9009\u586b）",
\n "history": [
\n ["\u7b2c\u4e00\u8f6e\u6307\u4ee4（\u9009\u586b）", "\u7b2c\u4e00\u8f6e\u56de\u7b54（\u9009\u586b）"],
\n ["\u7b2c\u4e8c\u8f6e\u6307\u4ee4（\u9009\u586b）", "\u7b2c\u4e8c\u8f6e\u56de\u7b54（\u9009\u586b）"]
\n ]
\n }
\n] <\/p>\n

\u5bf9\u4e8e\u4e0a\u8ff0\u683c\u5f0f\u7684\u6570\u636e，dataset_info.json\u00a0\u4e2d\u7684\u6570\u636e\u96c6\u63cf\u8ff0\u5e94\u4e3a：<\/p>\n

"\u6570\u636e\u96c6\u540d\u79f0": {
\n "file_name": "data.json",
\n "columns": {
\n "prompt": "instruction",
\n "query": "input",
\n "response": "output",
\n "system": "system",
\n "history": "history"
\n }
\n} <\/p>\n

sharegpt\u793a\u4f8b\u683c\u5f0f：<\/p>\n

  • \u76f8\u6bd4 alpaca \u683c\u5f0f\u7684\u6570\u636e\u96c6，sharegpt \u683c\u5f0f\u652f\u6301\u66f4\u591a\u7684\u89d2\u8272\u79cd\u7c7b，\u4f8b\u5982 human\u3001gpt\u3001observation\u3001function \u7b49\u7b49\u3002\u5b83\u4eec\u6784\u6210\u4e00\u4e2a\u5bf9\u8c61\u5217\u8868\u5448\u73b0\u5728\u00a0conversations\u00a0\u5217\u4e2d\u3002<\/li>\n
  • \u6ce8\u610f\u5176\u4e2d human \u548c observation \u5fc5\u987b\u51fa\u73b0\u5728\u5947\u6570\u4f4d\u7f6e，gpt \u548c function \u5fc5\u987b\u51fa\u73b0\u5728\u5076\u6570\u4f4d\u7f6e\u3002<\/li>\n

    [
    \n {
    \n "conversations": [
    \n {
    \n "from": "human",
    \n "value": "\u4eba\u7c7b\u6307\u4ee4"
    \n },
    \n {
    \n "from": "function_call",
    \n "value": "\u5de5\u5177\u53c2\u6570"
    \n },
    \n {
    \n "from": "observation",
    \n "value": "\u5de5\u5177\u7ed3\u679c"
    \n },
    \n {
    \n "from": "gpt",
    \n "value": "\u6a21\u578b\u56de\u7b54"
    \n }
    \n ],
    \n "system": "\u7cfb\u7edf\u63d0\u793a\u8bcd（\u9009\u586b）",
    \n "tools": "\u5de5\u5177\u63cf\u8ff0（\u9009\u586b）"
    \n }
    \n] <\/p>\n

    \u5bf9\u4e8e\u4e0a\u8ff0\u683c\u5f0f\u7684\u6570\u636e，dataset_info.json\u00a0\u4e2d\u7684\u6570\u636e\u96c6\u63cf\u8ff0\u5e94\u4e3a：<\/p>\n

    "\u6570\u636e\u96c6\u540d\u79f0": {
    \n "file_name": "data.json",
    \n "formatting": "sharegpt",
    \n "columns": {
    \n "messages": "conversations",
    \n "system": "system",
    \n "tools": "tools"
    \n }
    \n} <\/p>\n

    \u4ee5\u4e0b\u91c7\u7528sharegpt\u683c\u5f0f，\u5728\/data\/service\/LLaMA-Factory\/data\/\u76ee\u5f55\u4e0a\u4f20\u6d4b\u8bd5\u6570\u636e，\u6587\u4ef6\u540d\u4e3ads_zh_demo.json<\/span>，\u6d4b\u8bd5\u6570\u636e\u793a\u4f8b\u5982\u4e0b：<\/p>\n

    [
    \n{
    \n"conversations": [
    \n{
    \n"from": "user",
    \n"value": "\u4ec0\u4e48\u662f\u6df1\u5ea6\u5b66\u4e60，\u5b83\u4e0e\u4f20\u7edf\u673a\u5668\u5b66\u4e60\u6709\u4ec0\u4e48\u533a\u522b？"
    \n},
    \n{
    \n"from": "assistant",
    \n"value": "\u6df1\u5ea6\u5b66\u4e60\u662f\u673a\u5668\u5b66\u4e60\u7684\u4e00\u4e2a\u5b50\u9886\u57df，\u4f7f\u7528\u4eba\u5de5\u795e\u7ecf\u7f51\u7edc，\u5c24\u5176\u662f\u6df1\u5ea6\u795e\u7ecf\u7f51\u7edc，\u6765\u8fdb\u884c\u6570\u636e\u5efa\u6a21\u3002\u4e0e\u4f20\u7edf\u673a\u5668\u5b66\u4e60\u65b9\u6cd5\u76f8\u6bd4，\u6df1\u5ea6\u5b66\u4e60\u80fd\u591f\u81ea\u52a8\u63d0\u53d6\u7279\u5f81，\u65e0\u9700\u624b\u52a8\u7279\u5f81\u5de5\u7a0b，\u9002\u5408\u5904\u7406\u5927\u89c4\u6a21\u6570\u636e\u96c6\u548c\u590d\u6742\u6a21\u5f0f\u3002"
    \n}
    \n]
    \n},
    \n{
    \n"conversations": [
    \n{
    \n"from": "user",
    \n"value": "\u5982\u4f55\u8bc4\u4f30\u4e00\u4e2a\u673a\u5668\u5b66\u4e60\u6a21\u578b\u7684\u6027\u80fd？"
    \n},
    \n{
    \n"from": "assistant",
    \n"value": "\u8bc4\u4f30\u673a\u5668\u5b66\u4e60\u6a21\u578b\u7684\u6027\u80fd\u901a\u5e38\u4f7f\u7528\u591a\u79cd\u6307\u6807，\u5305\u62ec\u51c6\u786e\u7387\u3001\u7cbe\u786e\u7387\u3001\u53ec\u56de\u7387\u3001F1-score\u3001ROC\u66f2\u7ebf\u548cAUC\u503c\u3002\u9009\u62e9\u5408\u9002\u7684\u6307\u6807\u53d6\u51b3\u4e8e\u5177\u4f53\u4efb\u52a1\u7684\u6027\u8d28\u548c\u76ee\u6807\u3002"
    \n}
    \n]
    \n}
    \n]<\/p>\n

    \u4fee\u6539\u6570\u636e\u96c6\u63cf\u8ff0\u6587\u4ef6dataset_info.json<\/p>\n

    vi \/data\/service\/LLaMA-Factory\/data\/dataset_info.json <\/p>\n

    \u589e\u52a0\u4ee5\u4e0b\u5185\u5bb9：<\/p>\n

    "ds_zh_demo": {
    \n"file_name": "ds_zh_demo.json",
    \n"formatting": "sharegpt",
    \n"columns": {
    \n "messages": "conversations"
    \n},
    \n"tags": {
    \n "role_tag": "from",
    \n "content_tag": "value",
    \n "user_tag": "user",
    \n "assistant_tag": "assistant"
    \n}
    \n}<\/p>\n

    4.2.\u914d\u7f6e\u6587\u4ef6\u51c6\u5907<\/h3>\n

    1) \u5907\u4efd\u539f\u6709\u7684\u914d\u7f6e\u6587\u4ef6<\/p>\n

    cp \/data\/service\/LLaMA-Factory\/examples\/train_lora\/llama3_lora_sft.yaml \/data\/service\/LLaMA-Factory\/examples\/train_lora\/llama3_lora_sft.yaml.bak <\/p>\n

    2) \u521b\u5efa\u65b0\u7684\u914d\u7f6e\u6587\u4ef6<\/p>\n

    mv \/data\/service\/LLaMA-Factory\/examples\/train_lora\/llama3_lora_sft.yaml \/data\/service\/LLaMA-Factory\/examples\/train_lora\/ds_qwen7b_lora_sft.yaml <\/p>\n

    3) \u4fee\u6539\u914d\u7f6e\u6587\u4ef6\u5185\u5bb9<\/p>\n

    vi \/data\/service\/LLaMA-Factory\/examples\/train_lora\/ds_qwen7b_lora_sft.yaml <\/p>\n

    \u00a0 \u5185\u5bb9\u5982\u4e0b：<\/p>\n

    ### model
    \nmodel_name_or_path: \/data\/model\/DeepSeek-R1-Distill-Qwen-7B
    \ntrust_remote_code: true<\/p>\n

    ### method
    \nstage: sft
    \ndo_train: true
    \nfinetuning_type: lora
    \nlora_rank: 8
    \nlora_target: all<\/p>\n

    ### dataset
    \ndataset: ds_zh_demo
    \ntemplate: deepseek3
    \ncutoff_len: 4096
    \nmax_samples: 4019
    \noverwrite_cache: true
    \npreprocessing_num_workers: 16<\/p>\n

    ### output
    \noutput_dir: \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B
    \nlogging_steps: 10
    \nsave_steps: 500
    \nplot_loss: true
    \noverwrite_output_dir: true<\/p>\n

    ### train
    \nper_device_train_batch_size: 1
    \ngradient_accumulation_steps: 8
    \nlearning_rate: 1.0e-4
    \nnum_train_epochs: 1.0
    \nlr_scheduler_type: cosine
    \nwarmup_ratio: 0.1
    \nbf16: true
    \nddp_timeout: 180000000<\/p>\n

    ### eval
    \nval_size: 0.1
    \nper_device_eval_batch_size: 1
    \neval_strategy: steps
    \neval_steps: 500 <\/p>\n

    \u00a0 \u9700\u8981\u5173\u6ce8\u4ee5\u4e0b\u53c2\u6570<\/p>\n

  • model_name_or_path：\u6a21\u578b\u8def\u5f84<\/li>\n
  • dataset：\u6570\u636e\u96c6\u540d\u79f0，\u5bf9\u5e94\u4e0a\u9762\u58f0\u660e\u7684qwen_zh_demo<\/li>\n
  • template：\u6a21\u7248<\/li>\n
  • cutoff_len：\u63a7\u5236\u8f93\u5165\u5e8f\u5217\u7684\u6700\u5927\u957f\u5ea6<\/li>\n
  • output_dir：\u5fae\u8c03\u540e\u6743\u91cd\u4fdd\u5b58\u8def\u5f84<\/li>\n
  • gradient_accumulation_steps：\u68af\u5ea6\u7d2f\u79ef\u7684\u6b65\u6570，GPU\u8d44\u6e90\u4e0d\u8db3\u65f6\u9700\u8981\u51cf\u5c11\u8be5\u503c<\/li>\n
  • num_train_epochs：\u8bad\u7ec3\u7684\u8f6e\u6570<\/li>\n

    4.3.\u542f\u52a8\u5fae\u8c03<\/h3>\n

    conda activate llama_factory
    \ncd \/data\/service\/LLaMA-Factory
    \nllamafactory-cli train \/data\/service\/LLaMA-Factory\/examples\/train_lora\/ds_qwen7b_lora_sft.yaml<\/p>\n

    # \u540e\u53f0\u8fd0\u884c
    \nnohup llamafactory-cli train \/data\/service\/LLaMA-Factory\/examples\/train_lora\/ds_qwen7b_lora_sft.yaml > output.log 2>&1 & <\/p>\n

    \"\"<\/h3>\n

    4.4.\u5fae\u8c03\u7ed3\u679c<\/h3>\n

    [INFO|configuration_utils.py:1052] 2025-02-18 16:39:55,400 >> loading configuration file \/data\/model\/DeepSeek-R1-Distill-Qwen-7B\/generation_config.json
    \n[INFO|configuration_utils.py:1099] 2025-02-18 16:39:55,400 >> Generate config GenerationConfig {
    \n "bos_token_id": 151646,
    \n "do_sample": true,
    \n "eos_token_id": 151643,
    \n "temperature": 0.6,
    \n "top_p": 0.95
    \n}<\/p>\n

    [INFO|2025-02-18 16:39:55] llamafactory.model.model_utils.checkpointing:157 >> Gradient checkpointing enabled.
    \n[INFO|2025-02-18 16:39:55] llamafactory.model.model_utils.attention:157 >> Using torch SDPA for faster training and inference.
    \n[INFO|2025-02-18 16:39:55] llamafactory.model.adapter:157 >> Upcasting trainable params to float32.
    \n[INFO|2025-02-18 16:39:55] llamafactory.model.adapter:157 >> Fine-tuning method: LoRA
    \n[INFO|2025-02-18 16:39:55] llamafactory.model.model_utils.misc:157 >> Found linear modules: down_proj,o_proj,up_proj,k_proj,v_proj,q_proj,gate_proj
    \n[INFO|2025-02-18 16:39:55] llamafactory.model.loader:157 >> trainable params: 20,185,088 || all params: 7,635,801,600 || trainable%: 0.2643
    \nDetected kernel version 4.18.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
    \n[INFO|trainer.py:667] 2025-02-18 16:39:55,807 >> Using auto half precision backend
    \n[INFO|trainer.py:2243] 2025-02-18 16:39:56,634 >> ***** Running training *****
    \n[INFO|trainer.py:2244] 2025-02-18 16:39:56,634 >> Num examples = 3,617
    \n[INFO|trainer.py:2245] 2025-02-18 16:39:56,634 >> Num Epochs = 1
    \n[INFO|trainer.py:2246] 2025-02-18 16:39:56,634 >> Instantaneous batch size per device = 1
    \n[INFO|trainer.py:2249] 2025-02-18 16:39:56,634 >> Total train batch size (w. parallel, distributed & accumulation) = 8
    \n[INFO|trainer.py:2250] 2025-02-18 16:39:56,634 >> Gradient Accumulation steps = 8
    \n[INFO|trainer.py:2251] 2025-02-18 16:39:56,634 >> Total optimization steps = 452
    \n[INFO|trainer.py:2252] 2025-02-18 16:39:56,638 >> Number of trainable parameters = 20,185,088
    \n 0%| | 0\/452 [00:00<?, ?it\/s]\/usr\/local\/miniconda3\/envs\/llama_factory\/lib\/python3.10\/site-packages\/torch\/utils\/checkpoint.py:295: FutureWarning: `torch.cpu.amp.autocast(args…)` is deprecated. Please use `torch.amp.autocast('cpu', args…)` instead.
    \n with torch.enable_grad(), device_autocast_ctx, torch.cpu.amp.autocast(**ctx.cpu_autocast_kwargs): # type: ignore[attr-defined]
    \n100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 452\/452 [4:06:28<00:00, 31.87s\/it][INFO|trainer.py:3705] 2025-02-18 20:46:24,795 >> Saving model checkpoint to \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B\/checkpoint-452
    \n[INFO|configuration_utils.py:670] 2025-02-18 20:46:24,819 >> loading configuration file \/data\/model\/DeepSeek-R1-Distill-Qwen-7B\/config.json
    \n[INFO|configuration_utils.py:739] 2025-02-18 20:46:24,820 >> Model config Qwen2Config {
    \n "architectures": [
    \n "Qwen2ForCausalLM"
    \n ],
    \n "attention_dropout": 0.0,
    \n "bos_token_id": 151643,
    \n "eos_token_id": 151643,
    \n "hidden_act": "silu",
    \n "hidden_size": 3584,
    \n "initializer_range": 0.02,
    \n "intermediate_size": 18944,
    \n "max_position_embeddings": 131072,
    \n "max_window_layers": 28,
    \n "model_type": "qwen2",
    \n "num_attention_heads": 28,
    \n "num_hidden_layers": 28,
    \n "num_key_value_heads": 4,
    \n "rms_norm_eps": 1e-06,
    \n "rope_scaling": null,
    \n "rope_theta": 10000,
    \n "sliding_window": null,
    \n "tie_word_embeddings": false,
    \n "torch_dtype": "bfloat16",
    \n "transformers_version": "4.45.0",
    \n "use_cache": true,
    \n "use_mrope": false,
    \n "use_sliding_window": false,
    \n "vocab_size": 152064
    \n}<\/p>\n

    [INFO|tokenization_utils_base.py:2649] 2025-02-18 20:46:25,042 >> tokenizer config file saved in \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B\/checkpoint-452\/tokenizer_config.json
    \n[INFO|tokenization_utils_base.py:2658] 2025-02-18 20:46:25,043 >> Special tokens file saved in \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B\/checkpoint-452\/special_tokens_map.json
    \n[INFO|trainer.py:2505] 2025-02-18 20:46:25,377 >> <\/p>\n

    Training completed. Do not forget to share your model on huggingface.co\/models =)<\/p>\n

    100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 452\/452 [4:06:28<00:00, 32.72s\/it]
    \n[INFO|trainer.py:3705] 2025-02-18 20:46:25,379 >> Saving model checkpoint to \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B
    \n[INFO|configuration_utils.py:670] 2025-02-18 20:46:25,401 >> loading configuration file \/data\/model\/DeepSeek-R1-Distill-Qwen-7B\/config.json
    \n[INFO|configuration_utils.py:739] 2025-02-18 20:46:25,401 >> Model config Qwen2Config {
    \n "architectures": [
    \n "Qwen2ForCausalLM"
    \n ],
    \n "attention_dropout": 0.0,
    \n "bos_token_id": 151643,
    \n "eos_token_id": 151643,
    \n "hidden_act": "silu",
    \n "hidden_size": 3584,
    \n "initializer_range": 0.02,
    \n "intermediate_size": 18944,
    \n "max_position_embeddings": 131072,
    \n "max_window_layers": 28,
    \n "model_type": "qwen2",
    \n "num_attention_heads": 28,
    \n "num_hidden_layers": 28,
    \n "num_key_value_heads": 4,
    \n "rms_norm_eps": 1e-06,
    \n "rope_scaling": null,
    \n "rope_theta": 10000,
    \n "sliding_window": null,
    \n "tie_word_embeddings": false,
    \n "torch_dtype": "bfloat16",
    \n "transformers_version": "4.45.0",
    \n "use_cache": true,
    \n "use_mrope": false,
    \n "use_sliding_window": false,
    \n "vocab_size": 152064
    \n}<\/p>\n

    [INFO|tokenization_utils_base.py:2649] 2025-02-18 20:46:25,556 >> tokenizer config file saved in \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B\/tokenizer_config.json
    \n[INFO|tokenization_utils_base.py:2658] 2025-02-18 20:46:25,556 >> Special tokens file saved in \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B\/special_tokens_map.json
    \n{'loss': 3.6592, 'grad_norm': 0.38773563504219055, 'learning_rate': 2.173913043478261e-05, 'epoch': 0.02}
    \n{'loss': 3.667, 'grad_norm': 0.698821485042572, 'learning_rate': 4.347826086956522e-05, 'epoch': 0.04}
    \n{'loss': 3.4784, 'grad_norm': 0.41371676325798035, 'learning_rate': 6.521739130434783e-05, 'epoch': 0.07}
    \n{'loss': 3.2962, 'grad_norm': 0.4966348111629486, 'learning_rate': 8.695652173913044e-05, 'epoch': 0.09}
    \n{'loss': 3.0158, 'grad_norm': 0.333425909280777, 'learning_rate': 9.997605179330019e-05, 'epoch': 0.11}
    \n{'loss': 3.2221, 'grad_norm': 0.3786776065826416, 'learning_rate': 9.970689785771798e-05, 'epoch': 0.13}
    \n{'loss': 2.8439, 'grad_norm': 0.3683229386806488, 'learning_rate': 9.914027086842322e-05, 'epoch': 0.15}
    \n{'loss': 3.0528, 'grad_norm': 0.42745739221572876, 'learning_rate': 9.82795618288397e-05, 'epoch': 0.18}
    \n{'loss': 2.9092, 'grad_norm': 0.45462721586227417, 'learning_rate': 9.712992168898436e-05, 'epoch': 0.2}
    \n{'loss': 3.1055, 'grad_norm': 0.5547119379043579, 'learning_rate': 9.56982305193869e-05, 'epoch': 0.22}
    \n{'loss': 2.9412, 'grad_norm': 0.5830215811729431, 'learning_rate': 9.399305633701373e-05, 'epoch': 0.24}
    \n{'loss': 2.7873, 'grad_norm': 0.5862609148025513, 'learning_rate': 9.202460382960448e-05, 'epoch': 0.27}
    \n{'loss': 2.8255, 'grad_norm': 0.5828853845596313, 'learning_rate': 8.980465328528219e-05, 'epoch': 0.29}
    \n{'loss': 2.6266, 'grad_norm': 0.6733331084251404, 'learning_rate': 8.734649009291585e-05, 'epoch': 0.31}
    \n{'loss': 2.8745, 'grad_norm': 0.6904928684234619, 'learning_rate': 8.46648252351431e-05, 'epoch': 0.33}
    \n{'loss': 2.8139, 'grad_norm': 0.7874809503555298, 'learning_rate': 8.177570724986628e-05, 'epoch': 0.35}
    \n{'loss': 2.7818, 'grad_norm': 0.8345168232917786, 'learning_rate': 7.86964261870916e-05, 'epoch': 0.38}
    \n{'loss': 2.7198, 'grad_norm': 0.8806198239326477, 'learning_rate': 7.544541013588645e-05, 'epoch': 0.4}
    \n{'loss': 2.7231, 'grad_norm': 0.9481658935546875, 'learning_rate': 7.204211494069292e-05, 'epoch': 0.42}
    \n{'loss': 2.7371, 'grad_norm': 0.9718573093414307, 'learning_rate': 6.850690776699573e-05, 'epoch': 0.44}
    \n{'loss': 2.6862, 'grad_norm': 1.2056019306182861, 'learning_rate': 6.486094521315022e-05, 'epoch': 0.46}
    \n{'loss': 2.4661, 'grad_norm': 1.200085163116455, 'learning_rate': 6.112604669781572e-05, 'epoch': 0.49}
    \n{'loss': 2.4841, 'grad_norm': 1.1310691833496094, 'learning_rate': 5.732456388071247e-05, 'epoch': 0.51}
    \n{'loss': 2.3755, 'grad_norm': 1.1279083490371704, 'learning_rate': 5.3479246898159063e-05, 'epoch': 0.53}
    \n{'loss': 2.5552, 'grad_norm': 1.2654848098754883, 'learning_rate': 4.96131082139099e-05, 'epoch': 0.55}
    \n{'loss': 2.6197, 'grad_norm': 1.3887016773223877, 'learning_rate': 4.574928490008264e-05, 'epoch': 0.58}
    \n{'loss': 2.3773, 'grad_norm': 1.3009178638458252, 'learning_rate': 4.1910900172361764e-05, 'epoch': 0.6}
    \n{'loss': 2.3881, 'grad_norm': 1.346793532371521, 'learning_rate': 3.812092500812646e-05, 'epoch': 0.62}
    \n{'loss': 2.4821, 'grad_norm': 1.7273674011230469, 'learning_rate': 3.440204067565511e-05, 'epoch': 0.64}
    \n{'loss': 2.3563, 'grad_norm': 1.529177188873291, 'learning_rate': 3.077650299710653e-05, 'epoch': 0.66}
    \n{'loss': 2.1308, 'grad_norm': 1.5957469940185547, 'learning_rate': 2.7266009157601224e-05, 'epoch': 0.69}
    \n{'loss': 2.1709, 'grad_norm': 1.4444897174835205, 'learning_rate': 2.3891567857490372e-05, 'epoch': 0.71}
    \n{'loss': 2.275, 'grad_norm': 1.5686719417572021, 'learning_rate': 2.067337358489085e-05, 'epoch': 0.73}
    \n{'loss': 2.2075, 'grad_norm': 1.5931408405303955, 'learning_rate': 1.7630685760908622e-05, 'epoch': 0.75}
    \n{'loss': 2.1727, 'grad_norm': 1.7681787014007568, 'learning_rate': 1.4781713480810184e-05, 'epoch': 0.77}
    \n{'loss': 2.3562, 'grad_norm': 1.742925763130188, 'learning_rate': 1.2143506540914128e-05, 'epoch': 0.8}
    \n{'loss': 2.1187, 'grad_norm': 1.6716198921203613, 'learning_rate': 9.731853403356705e-06, 'epoch': 0.82}
    \n{'loss': 2.2564, 'grad_norm': 1.915489912033081, 'learning_rate': 7.561186709365653e-06, 'epoch': 0.84}
    \n{'loss': 2.261, 'grad_norm': 2.132519245147705, 'learning_rate': 5.644496906502233e-06, 'epoch': 0.86}
    \n{'loss': 2.1632, 'grad_norm': 1.591231107711792, 'learning_rate': 3.9932545067728366e-06, 'epoch': 0.88}
    \n{'loss': 2.1266, 'grad_norm': 1.584917664527893, 'learning_rate': 2.6173414408598827e-06, 'epoch': 0.91}
    \n{'loss': 2.2944, 'grad_norm': 1.5982666015625, 'learning_rate': 1.524991919285429e-06, 'epoch': 0.93}
    \n{'loss': 2.3799, 'grad_norm': 2.1475727558135986, 'learning_rate': 7.227431544266194e-07, 'epoch': 0.95}
    \n{'loss': 2.1196, 'grad_norm': 1.6714484691619873, 'learning_rate': 2.153962382888841e-07, 'epoch': 0.97}
    \n{'loss': 2.1427, 'grad_norm': 1.7334465980529785, 'learning_rate': 5.987410165758656e-09, 'epoch': 1.0}
    \n{'train_runtime': 14788.7396, 'train_samples_per_second': 0.245, 'train_steps_per_second': 0.031, 'train_loss': 2.6206856934370193, 'epoch': 1.0}
    \n***** train metrics *****
    \n epoch = 0.9997
    \n total_flos = 100517734GF
    \n train_loss = 2.6207
    \n train_runtime = 4:06:28.73
    \n train_samples_per_second = 0.245
    \n train_steps_per_second = 0.031
    \nFigure saved at: \/data\/model\/sft\/DeepSeek-R1-Distill-Qwen-7B\/training_loss.png
    \n[WARNING|2025-02-18 20:46:25] llamafactory.extras.ploting:162 >> No metric eval_loss to plot.
    \n[WARNING|2025-02-18 20:46:25] llamafactory.extras.ploting:162 >> No metric eval_accuracy to plot.
    \n[INFO|trainer.py:4021] 2025-02-18 20:46:25,781 >>
    \n***** Running Evaluation *****
    \n[INFO|trainer.py:4023] 2025-02-18 20:46:25,781 >> Num examples = 402
    \n[INFO|trainer.py:4026] 2025-02-18 20:46:25,781 >> Batch size = 1
    \n100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 402\/402 [09:03<00:00, 1.35s\/it]t]
    \n[INFO|modelcard.py:449] 2025-02-18 20:55:30,409 >> Dropping the following result as it does not have all the necessary fields:
    \n{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
    \n***** eval metrics *****
    \n epoch = 0.9997
    \n eval_loss = 2.2648
    \n eval_runtime = 0:09:04.62
    \n eval_samples_per_second = 0.738
    \n eval_steps_per_second = 0.738 <\/p>\n

    \u751f\u6210\u7684\u6743\u91cd\u6587\u4ef6：<\/p>\n

    \"\"<\/p>\n

    \n

    \u4e94\u3001\u9644\u5e26\u8bf4\u660e<\/h2>\n

    5.1. dataset_info.json<\/h3>\n

    \u5305\u542b\u4e86\u6240\u6709\u53ef\u7528\u7684\u6570\u636e\u96c6\u3002\u5982\u679c\u60a8\u5e0c\u671b\u4f7f\u7528\u81ea\u5b9a\u4e49\u6570\u636e\u96c6，\u8bf7\u52a1\u5fc5\u5728\u00a0dataset_info.json\u00a0\u6587\u4ef6\u4e2d\u6dfb\u52a0\u6570\u636e\u96c6\u63cf\u8ff0，\u5e76\u901a\u8fc7\u4fee\u6539\u00a0dataset: \u6570\u636e\u96c6\u540d\u79f0\u00a0\u914d\u7f6e\u6765\u4f7f\u7528\u6570\u636e\u96c6\u3002<\/p>\n

    "\u6570\u636e\u96c6\u540d\u79f0": {
    \n "hf_hub_url": "Hugging Face \u7684\u6570\u636e\u96c6\u4ed3\u5e93\u5730\u5740（\u82e5\u6307\u5b9a，\u5219\u5ffd\u7565 script_url \u548c file_name）",
    \n "ms_hub_url": "ModelScope \u7684\u6570\u636e\u96c6\u4ed3\u5e93\u5730\u5740（\u82e5\u6307\u5b9a，\u5219\u5ffd\u7565 script_url \u548c file_name）",
    \n "script_url": "\u5305\u542b\u6570\u636e\u52a0\u8f7d\u811a\u672c\u7684\u672c\u5730\u6587\u4ef6\u5939\u540d\u79f0（\u82e5\u6307\u5b9a，\u5219\u5ffd\u7565 file_name）",
    \n "file_name": "\u8be5\u76ee\u5f55\u4e0b\u6570\u636e\u96c6\u6587\u4ef6\u5939\u6216\u6587\u4ef6\u7684\u540d\u79f0（\u82e5\u4e0a\u8ff0\u53c2\u6570\u672a\u6307\u5b9a，\u5219\u6b64\u9879\u5fc5\u9700）",
    \n "formatting": "\u6570\u636e\u96c6\u683c\u5f0f（\u53ef\u9009，\u9ed8\u8ba4：alpaca，\u53ef\u4ee5\u4e3a alpaca \u6216 sharegpt）",
    \n "ranking": "\u662f\u5426\u4e3a\u504f\u597d\u6570\u636e\u96c6（\u53ef\u9009，\u9ed8\u8ba4：False）",
    \n "subset": "\u6570\u636e\u96c6\u5b50\u96c6\u7684\u540d\u79f0（\u53ef\u9009，\u9ed8\u8ba4：None）",
    \n "split": "\u6240\u4f7f\u7528\u7684\u6570\u636e\u96c6\u5207\u5206（\u53ef\u9009，\u9ed8\u8ba4：train）",
    \n "folder": "Hugging Face \u4ed3\u5e93\u7684\u6587\u4ef6\u5939\u540d\u79f0（\u53ef\u9009，\u9ed8\u8ba4：None）",
    \n "num_samples": "\u8be5\u6570\u636e\u96c6\u6240\u4f7f\u7528\u7684\u6837\u672c\u6570\u91cf\u3002（\u53ef\u9009，\u9ed8\u8ba4：None）",
    \n "columns（\u53ef\u9009）": {
    \n "prompt": "\u6570\u636e\u96c6\u4ee3\u8868\u63d0\u793a\u8bcd\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：instruction）",
    \n "query": "\u6570\u636e\u96c6\u4ee3\u8868\u8bf7\u6c42\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：input）",
    \n "response": "\u6570\u636e\u96c6\u4ee3\u8868\u56de\u7b54\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：output）",
    \n "history": "\u6570\u636e\u96c6\u4ee3\u8868\u5386\u53f2\u5bf9\u8bdd\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "messages": "\u6570\u636e\u96c6\u4ee3\u8868\u6d88\u606f\u5217\u8868\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：conversations）",
    \n "system": "\u6570\u636e\u96c6\u4ee3\u8868\u7cfb\u7edf\u63d0\u793a\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "tools": "\u6570\u636e\u96c6\u4ee3\u8868\u5de5\u5177\u63cf\u8ff0\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "images": "\u6570\u636e\u96c6\u4ee3\u8868\u56fe\u50cf\u8f93\u5165\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "videos": "\u6570\u636e\u96c6\u4ee3\u8868\u89c6\u9891\u8f93\u5165\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "audios": "\u6570\u636e\u96c6\u4ee3\u8868\u97f3\u9891\u8f93\u5165\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "chosen": "\u6570\u636e\u96c6\u4ee3\u8868\u66f4\u4f18\u56de\u7b54\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "rejected": "\u6570\u636e\u96c6\u4ee3\u8868\u66f4\u5dee\u56de\u7b54\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）",
    \n "kto_tag": "\u6570\u636e\u96c6\u4ee3\u8868 KTO \u6807\u7b7e\u7684\u8868\u5934\u540d\u79f0（\u9ed8\u8ba4：None）"
    \n },
    \n "tags（\u53ef\u9009，\u7528\u4e8e sharegpt \u683c\u5f0f）": {
    \n "role_tag": "\u6d88\u606f\u4e2d\u4ee3\u8868\u53d1\u9001\u8005\u8eab\u4efd\u7684\u952e\u540d（\u9ed8\u8ba4：from）",
    \n "content_tag": "\u6d88\u606f\u4e2d\u4ee3\u8868\u6587\u672c\u5185\u5bb9\u7684\u952e\u540d（\u9ed8\u8ba4：value）",
    \n "user_tag": "\u6d88\u606f\u4e2d\u4ee3\u8868\u7528\u6237\u7684 role_tag（\u9ed8\u8ba4：human）",
    \n "assistant_tag": "\u6d88\u606f\u4e2d\u4ee3\u8868\u52a9\u624b\u7684 role_tag（\u9ed8\u8ba4：gpt）",
    \n "observation_tag": "\u6d88\u606f\u4e2d\u4ee3\u8868\u5de5\u5177\u8fd4\u56de\u7ed3\u679c\u7684 role_tag（\u9ed8\u8ba4：observation）",
    \n "function_tag": "\u6d88\u606f\u4e2d\u4ee3\u8868\u5de5\u5177\u8c03\u7528\u7684 role_tag（\u9ed8\u8ba4：function_call）",
    \n "system_tag": "\u6d88\u606f\u4e2d\u4ee3\u8868\u7cfb\u7edf\u63d0\u793a\u7684 role_tag（\u9ed8\u8ba4：system，\u4f1a\u8986\u76d6 system column）"
    \n }
    \n} <\/p>\n

    5.2. \u81ea\u5b9a\u4e49\u5bf9\u8bdd\u6a21\u7248<\/h3>\n

    \u5728 template.py \u4e2d\u6dfb\u52a0\u81ea\u5df1\u7684\u5bf9\u8bdd\u6a21\u677f\u3002<\/p>\n

    https:\/\/github.com\/hiyouga\/LLaMA-Factory\/blob\/main\/src\/llamafactory\/data\/template.py<\/p>\n

    # Copyright 2025 the LlamaFactory team.
    \n#
    \n# Licensed under the Apache License, Version 2.0 (the "License");
    \n# you may not use this file except in compliance with the License.
    \n# You may obtain a copy of the License at
    \n#
    \n# http:\/\/www.apache.org\/licenses\/LICENSE-2.0
    \n#
    \n# Unless required by applicable law or agreed to in writing, software
    \n# distributed under the License is distributed on an "AS IS" BASIS,
    \n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    \n# See the License for the specific language governing permissions and
    \n# limitations under the License.<\/p>\n

    from dataclasses import dataclass
    \nfrom typing import TYPE_CHECKING, Dict, List, Optional, Sequence, Tuple, Type, Union<\/p>\n

    from typing_extensions import override<\/p>\n

    from ..extras import logging
    \nfrom ..extras.misc import check_version
    \nfrom .data_utils import Role
    \nfrom .formatter import EmptyFormatter, FunctionFormatter, StringFormatter, ToolFormatter
    \nfrom .mm_plugin import get_mm_plugin<\/p>\n

    if TYPE_CHECKING:
    \n from transformers import PreTrainedTokenizer<\/p>\n

    from ..hparams import DataArguments
    \n from .formatter import SLOTS, Formatter
    \n from .mm_plugin import BasePlugin
    \n from .tool_utils import FunctionCall<\/p>\n

    logger = logging.get_logger(__name__)<\/p>\n

    @dataclass
    \nclass Template:
    \n format_user: "Formatter"
    \n format_assistant: "Formatter"
    \n format_system: "Formatter"
    \n format_function: "Formatter"
    \n format_observation: "Formatter"
    \n format_tools: "Formatter"
    \n format_prefix: "Formatter"
    \n default_system: str
    \n stop_words: List[str]
    \n thought_words: Tuple[str, str]
    \n efficient_eos: bool
    \n replace_eos: bool
    \n replace_jinja_template: bool
    \n mm_plugin: "BasePlugin"<\/p>\n

    def encode_oneturn(
    \n self,
    \n tokenizer: "PreTrainedTokenizer",
    \n messages: Sequence[Dict[str, str]],
    \n system: Optional[str] = None,
    \n tools: Optional[str] = None,
    \n ) -> Tuple[List[int], List[int]]:
    \n r"""
    \n Returns a single pair of token ids representing prompt and response respectively.
    \n """
    \n encoded_messages = self._encode(tokenizer, messages, system, tools)
    \n prompt_ids = []
    \n for encoded_ids in encoded_messages[:-1]:
    \n prompt_ids += encoded_ids<\/p>\n

    response_ids = encoded_messages[-1]
    \n return prompt_ids, response_ids<\/p>\n

    def encode_multiturn(
    \n self,
    \n tokenizer: "PreTrainedTokenizer",
    \n messages: Sequence[Dict[str, str]],
    \n system: Optional[str] = None,
    \n tools: Optional[str] = None,
    \n ) -> List[Tuple[List[int], List[int]]]:
    \n r"""
    \n Returns multiple pairs of token ids representing prompts and responses respectively.
    \n """
    \n encoded_messages = self._encode(tokenizer, messages, system, tools)
    \n return [(encoded_messages[i], encoded_messages[i + 1]) for i in range(0, len(encoded_messages), 2)]<\/p>\n

    def extract_tool(self, content: str) -> Union[str, List["FunctionCall"]]:
    \n r"""
    \n Extracts tool message.
    \n """
    \n return self.format_tools.extract(content)<\/p>\n

    def get_stop_token_ids(self, tokenizer: "PreTrainedTokenizer") -> List[int]:
    \n r"""
    \n Returns stop token ids.
    \n """
    \n stop_token_ids = {tokenizer.eos_token_id}
    \n for token in self.stop_words:
    \n stop_token_ids.add(tokenizer.convert_tokens_to_ids(token))<\/p>\n

    return list(stop_token_ids)<\/p>\n

    def _convert_elements_to_ids(self, tokenizer: "PreTrainedTokenizer", elements: "SLOTS") -> List[int]:
    \n r"""
    \n Converts elements to token ids.
    \n """
    \n token_ids = []
    \n for elem in elements:
    \n if isinstance(elem, str):
    \n if len(elem) != 0:
    \n token_ids += tokenizer.encode(elem, add_special_tokens=False)
    \n elif isinstance(elem, dict):
    \n token_ids += [tokenizer.convert_tokens_to_ids(elem.get("token"))]
    \n elif isinstance(elem, set):
    \n if "bos_token" in elem and tokenizer.bos_token_id is not None:
    \n token_ids += [tokenizer.bos_token_id]
    \n elif "eos_token" in elem and tokenizer.eos_token_id is not None:
    \n token_ids += [tokenizer.eos_token_id]
    \n else:
    \n raise ValueError(f"Input must be string, set[str] or dict[str, str], got {type(elem)}")<\/p>\n

    return token_ids<\/p>\n

    def _encode(
    \n self,
    \n tokenizer: "PreTrainedTokenizer",
    \n messages: Sequence[Dict[str, str]],
    \n system: Optional[str],
    \n tools: Optional[str],
    \n ) -> List[List[int]]:
    \n r"""
    \n Encodes formatted inputs to pairs of token ids.
    \n Turn 0: prefix + system + query resp
    \n Turn t: query resp
    \n """
    \n system = system or self.default_system
    \n encoded_messages = []
    \n for i, message in enumerate(messages):
    \n elements = []<\/p>\n

    if i == 0:
    \n elements += self.format_prefix.apply()
    \n if system or tools:
    \n tool_text = self.format_tools.apply(content=tools)[0] if tools else ""
    \n elements += self.format_system.apply(content=(system + tool_text))<\/p>\n

    if message["role"] == Role.USER.value:
    \n elements += self.format_user.apply(content=message["content"], idx=str(i \/\/ 2))
    \n elif message["role"] == Role.ASSISTANT.value:
    \n elements += self.format_assistant.apply(content=message["content"])
    \n elif message["role"] == Role.OBSERVATION.value:
    \n elements += self.format_observation.apply(content=message["content"])
    \n elif message["role"] == Role.FUNCTION.value:
    \n elements += self.format_function.apply(content=message["content"])
    \n else:
    \n raise NotImplementedError("Unexpected role: {}".format(message["role"]))<\/p>\n

    encoded_messages.append(self._convert_elements_to_ids(tokenizer, elements))<\/p>\n

    return encoded_messages<\/p>\n

    @staticmethod
    \n def _add_or_replace_eos_token(tokenizer: "PreTrainedTokenizer", eos_token: str) -> None:
    \n r"""
    \n Adds or replaces eos token to the tokenizer.
    \n """
    \n is_added = tokenizer.eos_token_id is None
    \n num_added_tokens = tokenizer.add_special_tokens({"eos_token": eos_token})<\/p>\n

    if is_added:
    \n logger.info_rank0(f"Add eos token: {tokenizer.eos_token}.")
    \n else:
    \n logger.info_rank0(f"Replace eos token: {tokenizer.eos_token}.")<\/p>\n

    if num_added_tokens > 0:
    \n logger.warning_rank0("New tokens have been added, make sure `resize_vocab` is True.")<\/p>\n

    def fix_special_tokens(self, tokenizer: "PreTrainedTokenizer") -> None:
    \n r"""
    \n Adds eos token and pad token to the tokenizer.
    \n """
    \n stop_words = self.stop_words
    \n if self.replace_eos:
    \n if not stop_words:
    \n raise ValueError("Stop words are required to replace the EOS token.")<\/p>\n

    self._add_or_replace_eos_token(tokenizer, eos_token=stop_words[0])
    \n stop_words = stop_words[1:]<\/p>\n

    if tokenizer.eos_token_id is None:
    \n self._add_or_replace_eos_token(tokenizer, eos_token="<|endoftext|>")<\/p>\n

    if tokenizer.pad_token_id is None:
    \n tokenizer.pad_token = tokenizer.eos_token
    \n logger.info_rank0(f"Add pad token: {tokenizer.pad_token}")<\/p>\n

    if stop_words:
    \n num_added_tokens = tokenizer.add_special_tokens(
    \n dict(additional_special_tokens=stop_words), replace_additional_special_tokens=False
    \n )
    \n logger.info_rank0("Add {} to stop words.".format(",".join(stop_words)))
    \n if num_added_tokens > 0:
    \n logger.warning_rank0("New tokens have been added, make sure `resize_vocab` is True.")<\/p>\n

    @staticmethod
    \n def _jinja_escape(content: str) -> str:
    \n r"""
    \n Escape single quotes in content.
    \n """
    \n return content.replace("'", r"\\\\'")<\/p>\n

    @staticmethod
    \n def _convert_slots_to_jinja(slots: "SLOTS", tokenizer: "PreTrainedTokenizer", placeholder: str = "content") -> str:
    \n r"""
    \n Converts slots to jinja template.
    \n """
    \n slot_items = []
    \n for slot in slots:
    \n if isinstance(slot, str):
    \n slot_pieces = slot.split("{{content}}")
    \n if slot_pieces[0]:
    \n slot_items.append("'" + Template._jinja_escape(slot_pieces[0]) + "'")
    \n if len(slot_pieces) > 1:
    \n slot_items.append(placeholder)
    \n if slot_pieces[1]:
    \n slot_items.append("'" + Template._jinja_escape(slot_pieces[1]) + "'")
    \n elif isinstance(slot, set): # do not use {{ eos_token }} since it may be replaced
    \n if "bos_token" in slot and tokenizer.bos_token_id is not None:
    \n slot_items.append("'" + tokenizer.bos_token + "'")
    \n elif "eos_token" in slot and tokenizer.eos_token_id is not None:
    \n slot_items.append("'" + tokenizer.eos_token + "'")
    \n elif isinstance(slot, dict):
    \n raise ValueError("Dict is not supported.")<\/p>\n

    return " + ".join(slot_items)<\/p>\n

    def _get_jinja_template(self, tokenizer: "PreTrainedTokenizer") -> str:
    \n r"""
    \n Returns the jinja template.
    \n """
    \n prefix = self._convert_slots_to_jinja(self.format_prefix.apply(), tokenizer)
    \n system = self._convert_slots_to_jinja(self.format_system.apply(), tokenizer, placeholder="system_message")
    \n user = self._convert_slots_to_jinja(self.format_user.apply(), tokenizer)
    \n assistant = self._convert_slots_to_jinja(self.format_assistant.apply(), tokenizer)
    \n jinja_template = ""
    \n if prefix:
    \n jinja_template += "{{ " + prefix + " }}"<\/p>\n

    if self.default_system:
    \n jinja_template += "{% set system_message = '" + self._jinja_escape(self.default_system) + "' %}"<\/p>\n

    jinja_template += (
    \n "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}"
    \n "{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}"
    \n "{% if system_message is defined %}{{ " + system + " }}{% endif %}"
    \n "{% for message in loop_messages %}"
    \n "{% set content = message['content'] %}"
    \n "{% if message['role'] == 'user' %}"
    \n "{{ " + user + " }}"
    \n "{% elif message['role'] == 'assistant' %}"
    \n "{{ " + assistant + " }}"
    \n "{% endif %}"
    \n "{% endfor %}"
    \n )
    \n return jinja_template<\/p>\n

    def fix_jinja_template(self, tokenizer: "PreTrainedTokenizer") -> None:
    \n r"""
    \n Replaces the jinja template in the tokenizer.
    \n """
    \n if tokenizer.chat_template is None or self.replace_jinja_template:
    \n try:
    \n tokenizer.chat_template = self._get_jinja_template(tokenizer)
    \n except ValueError as e:
    \n logger.info_rank0(f"Cannot add this chat template to tokenizer: {e}.")<\/p>\n

    @staticmethod
    \n def _convert_slots_to_ollama(
    \n slots: "SLOTS", tokenizer: "PreTrainedTokenizer", placeholder: str = "content"
    \n ) -> str:
    \n r"""
    \n Converts slots to ollama template.
    \n """
    \n slot_items = []
    \n for slot in slots:
    \n if isinstance(slot, str):
    \n slot_pieces = slot.split("{{content}}")
    \n if slot_pieces[0]:
    \n slot_items.append(slot_pieces[0])
    \n if len(slot_pieces) > 1:
    \n slot_items.append("{{ " + placeholder + " }}")
    \n if slot_pieces[1]:
    \n slot_items.append(slot_pieces[1])
    \n elif isinstance(slot, set): # do not use {{ eos_token }} since it may be replaced
    \n if "bos_token" in slot and tokenizer.bos_token_id is not None:
    \n slot_items.append(tokenizer.bos_token)
    \n elif "eos_token" in slot and tokenizer.eos_token_id is not None:
    \n slot_items.append(tokenizer.eos_token)
    \n elif isinstance(slot, dict):
    \n raise ValueError("Dict is not supported.")<\/p>\n

    return "".join(slot_items)<\/p>\n

    def _get_ollama_template(self, tokenizer: "PreTrainedTokenizer") -> str:
    \n r"""
    \n Returns the ollama template.
    \n """
    \n prefix = self._convert_slots_to_ollama(self.format_prefix.apply(), tokenizer)
    \n system = self._convert_slots_to_ollama(self.format_system.apply(), tokenizer, placeholder=".System")
    \n user = self._convert_slots_to_ollama(self.format_user.apply(), tokenizer, placeholder=".Content")
    \n assistant = self._convert_slots_to_ollama(self.format_assistant.apply(), tokenizer, placeholder=".Content")
    \n return (
    \n f"{prefix}{{{{ if .System }}}}{system}{{{{ end }}}}"
    \n f"""{{{{ range .Messages }}}}{{{{ if eq .Role "user" }}}}{user}"""
    \n f"""{{{{ else if eq .Role "assistant" }}}}{assistant}{{{{ end }}}}{{{{ end }}}}"""
    \n )<\/p>\n

    def get_ollama_modelfile(self, tokenizer: "PreTrainedTokenizer") -> str:
    \n r"""
    \n Returns the ollama modelfile.<\/p>\n

    TODO: support function calling.
    \n """
    \n modelfile = "# ollama modelfile auto-generated by llamafactory\\\\n\\\\n"
    \n modelfile += f'FROM .\\\\n\\\\nTEMPLATE """{self._get_ollama_template(tokenizer)}"""\\\\n\\\\n'<\/p>\n

    if self.default_system:
    \n modelfile += f'SYSTEM """{self.default_system}"""\\\\n\\\\n'<\/p>\n

    for stop_token_id in self.get_stop_token_ids(tokenizer):
    \n modelfile += f'PARAMETER stop "{tokenizer.convert_ids_to_tokens(stop_token_id)}"\\\\n'<\/p>\n

    modelfile += "PARAMETER num_ctx 4096\\\\n"
    \n return modelfile<\/p>\n

    @dataclass
    \nclass Llama2Template(Template):
    \n @override
    \n def _encode(
    \n self,
    \n tokenizer: "PreTrainedTokenizer",
    \n messages: Sequence[Dict[str, str]],
    \n system: str,
    \n tools: str,
    \n ) -> List[List[int]]:
    \n system = system or self.default_system
    \n encoded_messages = []
    \n for i, message in enumerate(messages):
    \n elements = []<\/p>\n

    system_text = ""
    \n if i == 0:
    \n elements += self.format_prefix.apply()
    \n if system or tools:
    \n tool_text = self.format_tools.apply(content=tools)[0] if tools else ""
    \n system_text = self.format_system.apply(content=(system + tool_text))[0]<\/p>\n

    if message["role"] == Role.USER.value:
    \n elements += self.format_user.apply(content=system_text + message["content"])
    \n elif message["role"] == Role.ASSISTANT.value:
    \n elements += self.format_assistant.apply(content=message["content"])
    \n elif message["role"] == Role.OBSERVATION.value:
    \n elements += self.format_observation.apply(content=message["content"])
    \n elif message["role"] == Role.FUNCTION.value:
    \n elements += self.format_function.apply(content=message["content"])
    \n else:
    \n raise NotImplementedError("Unexpected role: {}".format(message["role"]))<\/p>\n

    encoded_messages.append(self._convert_elements_to_ids(tokenizer, elements))<\/p>\n

    return encoded_messages<\/p>\n

    def _get_jinja_template(self, tokenizer: "PreTrainedTokenizer") -> str:
    \n prefix = self._convert_slots_to_jinja(self.format_prefix.apply(), tokenizer)
    \n system_message = self._convert_slots_to_jinja(
    \n self.format_system.apply(), tokenizer, placeholder="system_message"
    \n )
    \n user_message = self._convert_slots_to_jinja(self.format_user.apply(), tokenizer)
    \n assistant_message = self._convert_slots_to_jinja(self.format_assistant.apply(), tokenizer)
    \n jinja_template = ""
    \n if prefix:
    \n jinja_template += "{{ " + prefix + " }}"<\/p>\n

    if self.default_system:
    \n jinja_template += "{% set system_message = '" + self._jinja_escape(self.default_system) + "' %}"<\/p>\n

    jinja_template += (
    \n "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}"
    \n "{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}"
    \n "{% for message in loop_messages %}"
    \n "{% if loop.index0 == 0 and system_message is defined %}"
    \n "{% set content = " + system_message + " + message['content'] %}"
    \n "{% else %}{% set content = message['content'] %}{% endif %}"
    \n "{% if message['role'] == 'user' %}"
    \n "{{ " + user_message + " }}"
    \n "{% elif message['role'] == 'assistant' %}"
    \n "{{ " + assistant_message + " }}"
    \n "{% endif %}"
    \n "{% endfor %}"
    \n )
    \n return jinja_template<\/p>\n

    TEMPLATES: Dict[str, "Template"] = {}<\/p>\n

    def register_template(
    \n name: str,
    \n format_user: Optional["Formatter"] = None,
    \n format_assistant: Optional["Formatter"] = None,
    \n format_system: Optional["Formatter"] = None,
    \n format_function: Optional["Formatter"] = None,
    \n format_observation: Optional["Formatter"] = None,
    \n format_tools: Optional["Formatter"] = None,
    \n format_prefix: Optional["Formatter"] = None,
    \n default_system: str = "",
    \n stop_words: Optional[Sequence[str]] = None,
    \n thought_words: Optional[Tuple[str, str]] = None,
    \n efficient_eos: bool = False,
    \n replace_eos: bool = False,
    \n replace_jinja_template: bool = False,
    \n mm_plugin: "BasePlugin" = get_mm_plugin(name="base"),
    \n template_class: Type["Template"] = Template,
    \n) -> None:
    \n r"""
    \n Registers a chat template.<\/p>\n

    To add the following chat template:
    \n ```
    \n <s><user>user prompt here
    \n <model>model response here<\/s>
    \n <user>user prompt here
    \n <model>model response here<\/s>
    \n ```<\/p>\n

    The corresponding code should be:
    \n ```
    \n register_template(
    \n name="custom",
    \n format_user=StringFormatter(slots=["<user>{{content}}\\\\n<model>"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<\/s>\\\\n"]),
    \n format_prefix=EmptyFormatter("<s>"),
    \n )
    \n ```
    \n """
    \n if name in TEMPLATES:
    \n raise ValueError(f"Template {name} already exists.")<\/p>\n

    default_slots = ["{{content}}"] if efficient_eos else ["{{content}}", {"eos_token"}]
    \n default_user_formatter = StringFormatter(slots=["{{content}}"])
    \n default_assistant_formatter = StringFormatter(slots=default_slots)
    \n default_function_formatter = FunctionFormatter(slots=default_slots, tool_format="default")
    \n default_tool_formatter = ToolFormatter(tool_format="default")
    \n default_prefix_formatter = EmptyFormatter()
    \n TEMPLATES[name] = template_class(
    \n format_user=format_user or default_user_formatter,
    \n format_assistant=format_assistant or default_assistant_formatter,
    \n format_system=format_system or default_user_formatter,
    \n format_function=format_function or default_function_formatter,
    \n format_observation=format_observation or format_user or default_user_formatter,
    \n format_tools=format_tools or default_tool_formatter,
    \n format_prefix=format_prefix or default_prefix_formatter,
    \n default_system=default_system,
    \n stop_words=stop_words or [],
    \n thought_words=thought_words or ("<think>", "<\/think>"),
    \n efficient_eos=efficient_eos,
    \n replace_eos=replace_eos,
    \n replace_jinja_template=replace_jinja_template,
    \n mm_plugin=mm_plugin,
    \n )<\/p>\n

    def parse_template(tokenizer: "PreTrainedTokenizer") -> "Template":
    \n r"""
    \n Extracts a chat template from the tokenizer.
    \n """<\/p>\n

    def find_diff(short_str: str, long_str: str) -> str:
    \n i, j = 0, 0
    \n diff = ""
    \n while i < len(short_str) and j < len(long_str):
    \n if short_str[i] == long_str[j]:
    \n i += 1
    \n j += 1
    \n else:
    \n diff += long_str[j]
    \n j += 1<\/p>\n

    return diff<\/p>\n

    prefix = tokenizer.decode(tokenizer.encode(""))<\/p>\n

    messages = [{"role": "system", "content": "{{content}}"}]
    \n system_slot = tokenizer.apply_chat_template(messages, add_generation_prompt=False, tokenize=False)[len(prefix) :]<\/p>\n

    messages = [{"role": "system", "content": ""}, {"role": "user", "content": "{{content}}"}]
    \n user_slot_empty_system = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
    \n user_slot_empty_system = user_slot_empty_system[len(prefix) :]<\/p>\n

    messages = [{"role": "user", "content": "{{content}}"}]
    \n user_slot = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
    \n user_slot = user_slot[len(prefix) :]<\/p>\n

    messages = [{"role": "user", "content": "{{content}}"}, {"role": "assistant", "content": "{{content}}"}]
    \n assistant_slot = tokenizer.apply_chat_template(messages, add_generation_prompt=False, tokenize=False)
    \n assistant_slot = assistant_slot[len(prefix) + len(user_slot) :]<\/p>\n

    if len(user_slot) > len(user_slot_empty_system):
    \n default_system = find_diff(user_slot_empty_system, user_slot)
    \n sole_system = system_slot.replace("{{content}}", default_system, 1)
    \n user_slot = user_slot[len(sole_system) :]
    \n else: # if defaut_system is empty, user_slot_empty_system will be longer than user_slot
    \n default_system = ""<\/p>\n

    return Template(
    \n format_user=StringFormatter(slots=[user_slot]),
    \n format_assistant=StringFormatter(slots=[assistant_slot]),
    \n format_system=StringFormatter(slots=[system_slot]),
    \n format_function=FunctionFormatter(slots=[assistant_slot], tool_format="default"),
    \n format_observation=StringFormatter(slots=[user_slot]),
    \n format_tools=ToolFormatter(tool_format="default"),
    \n format_prefix=EmptyFormatter(slots=[prefix]) if prefix else EmptyFormatter(),
    \n default_system=default_system,
    \n stop_words=[],
    \n thought_words=("<think>", "<\/think>"),
    \n efficient_eos=False,
    \n replace_eos=False,
    \n replace_jinja_template=False,
    \n mm_plugin=get_mm_plugin(name="base"),
    \n )<\/p>\n

    def get_template_and_fix_tokenizer(tokenizer: "PreTrainedTokenizer", data_args: "DataArguments") -> "Template":
    \n r"""
    \n Gets chat template and fixes the tokenizer.
    \n """
    \n if data_args.template is None:
    \n if isinstance(tokenizer.chat_template, str):
    \n logger.warning_rank0("`template` was not specified, try parsing the chat template from the tokenizer.")
    \n template = parse_template(tokenizer)
    \n else:
    \n logger.warning_rank0("`template` was not specified, use `empty` template.")
    \n template = TEMPLATES["empty"] # placeholder
    \n else:
    \n if data_args.template not in TEMPLATES:
    \n raise ValueError(f"Template {data_args.template} does not exist.")<\/p>\n

    template = TEMPLATES[data_args.template]<\/p>\n

    if template.mm_plugin.__class__.__name__ != "BasePlugin":
    \n check_version("transformers>=4.45.0")<\/p>\n

    if data_args.train_on_prompt and template.efficient_eos:
    \n raise ValueError("Current template does not support `train_on_prompt`.")<\/p>\n

    if data_args.tool_format is not None:
    \n logger.info_rank0(f"Using tool format: {data_args.tool_format}.")
    \n default_slots = ["{{content}}"] if template.efficient_eos else ["{{content}}", {"eos_token"}]
    \n template.format_function = FunctionFormatter(slots=default_slots, tool_format=data_args.tool_format)
    \n template.format_tools = ToolFormatter(tool_format=data_args.tool_format)<\/p>\n

    template.fix_special_tokens(tokenizer)
    \n template.fix_jinja_template(tokenizer)
    \n return template<\/p>\n

    register_template(
    \n name="alpaca",
    \n format_user=StringFormatter(slots=["### Instruction:\\\\n{{content}}\\\\n\\\\n### Response:\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}", {"eos_token"}, "\\\\n\\\\n"]),
    \n default_system=(
    \n "Below is an instruction that describes a task. Write a response that appropriately completes the request.\\\\n\\\\n"
    \n ),
    \n replace_jinja_template=True,
    \n)<\/p>\n

    register_template(
    \n name="aquila",
    \n format_user=StringFormatter(slots=["Human: {{content}}###Assistant:"]),
    \n format_assistant=StringFormatter(slots=["{{content}}###"]),
    \n format_system=StringFormatter(slots=["System: {{content}}###"]),
    \n default_system=(
    \n "A chat between a curious human and an artificial intelligence assistant. "
    \n "The assistant gives helpful, detailed, and polite answers to the human's questions."
    \n ),
    \n stop_words=["<\/s>"],
    \n)<\/p>\n

    register_template(
    \n name="atom",
    \n format_user=StringFormatter(
    \n slots=[{"bos_token"}, "Human: {{content}}\\\\n", {"eos_token"}, {"bos_token"}, "Assistant:"]
    \n ),
    \n format_assistant=StringFormatter(slots=["{{content}}\\\\n", {"eos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="baichuan",
    \n format_user=StringFormatter(slots=[{"token": "<reserved_102>"}, "{{content}}", {"token": "<reserved_103>"}]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="baichuan2",
    \n format_user=StringFormatter(slots=["<reserved_106>{{content}}<reserved_107>"]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="belle",
    \n format_user=StringFormatter(slots=["Human: {{content}}\\\\n\\\\nBelle: "]),
    \n format_assistant=StringFormatter(slots=["{{content}}", {"eos_token"}, "\\\\n\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="bluelm",
    \n format_user=StringFormatter(slots=[{"token": "[|Human|]:"}, "{{content}}", {"token": "[|AI|]:"}]),
    \n)<\/p>\n

    register_template(
    \n name="breeze",
    \n format_user=StringFormatter(slots=["[INST] {{content}} [\/INST] "]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="chatglm2",
    \n format_user=StringFormatter(slots=["[Round {{idx}}]\\\\n\\\\n\u95ee：{{content}}\\\\n\\\\n\u7b54："]),
    \n format_prefix=EmptyFormatter(slots=[{"token": "[gMASK]"}, {"token": "sop"}]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="chatglm3",
    \n format_user=StringFormatter(slots=[{"token": "<|user|>"}, "\\\\n", "{{content}}", {"token": "<|assistant|>"}]),
    \n format_assistant=StringFormatter(slots=["\\\\n", "{{content}}"]),
    \n format_system=StringFormatter(slots=[{"token": "<|system|>"}, "\\\\n", "{{content}}"]),
    \n format_function=FunctionFormatter(slots=["{{content}}"], tool_format="glm4"),
    \n format_observation=StringFormatter(
    \n slots=[{"token": "<|observation|>"}, "\\\\n", "{{content}}", {"token": "<|assistant|>"}]
    \n ),
    \n format_tools=ToolFormatter(tool_format="glm4"),
    \n format_prefix=EmptyFormatter(slots=[{"token": "[gMASK]"}, {"token": "sop"}]),
    \n stop_words=["<|user|>", "<|observation|>"],
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="chatml",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_observation=StringFormatter(slots=["<|im_start|>tool\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n stop_words=["<|im_end|>", "<|im_start|>"],
    \n replace_eos=True,
    \n replace_jinja_template=True,
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="chatml_de",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_observation=StringFormatter(slots=["<|im_start|>tool\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n default_system="Du bist ein freundlicher und hilfsbereiter KI-Assistent.",
    \n stop_words=["<|im_end|>", "<|im_start|>"],
    \n replace_eos=True,
    \n replace_jinja_template=True,
    \n)<\/p>\n

    register_template(
    \n name="codegeex2",
    \n format_prefix=EmptyFormatter(slots=[{"token": "[gMASK]"}, {"token": "sop"}]),
    \n)<\/p>\n

    register_template(
    \n name="codegeex4",
    \n format_user=StringFormatter(slots=["<|user|>\\\\n{{content}}<|assistant|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|system|>\\\\n{{content}}"]),
    \n format_function=FunctionFormatter(slots=["{{content}}"], tool_format="glm4"),
    \n format_observation=StringFormatter(slots=["<|observation|>\\\\n{{content}}<|assistant|>\\\\n"]),
    \n format_tools=ToolFormatter(tool_format="glm4"),
    \n format_prefix=EmptyFormatter(slots=["[gMASK]<sop>"]),
    \n default_system=(
    \n "\u4f60\u662f\u4e00\u4f4d\u667a\u80fd\u7f16\u7a0b\u52a9\u624b，\u4f60\u53ebCodeGeeX\u3002\u4f60\u4f1a\u4e3a\u7528\u6237\u56de\u7b54\u5173\u4e8e\u7f16\u7a0b\u3001\u4ee3\u7801\u3001\u8ba1\u7b97\u673a\u65b9\u9762\u7684\u4efb\u4f55\u95ee\u9898，"
    \n "\u5e76\u63d0\u4f9b\u683c\u5f0f\u89c4\u8303\u3001\u53ef\u4ee5\u6267\u884c\u3001\u51c6\u786e\u5b89\u5168\u7684\u4ee3\u7801，\u5e76\u5728\u5fc5\u8981\u65f6\u63d0\u4f9b\u8be6\u7ec6\u7684\u89e3\u91ca\u3002"
    \n ),
    \n stop_words=["<|user|>", "<|observation|>"],
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="cohere",
    \n format_user=StringFormatter(
    \n slots=[
    \n (
    \n "<|START_OF_TURN_TOKEN|><|USER_TOKEN|>{{content}}<|END_OF_TURN_TOKEN|>"
    \n "<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>"
    \n )
    \n ]
    \n ),
    \n format_system=StringFormatter(slots=["<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>{{content}}<|END_OF_TURN_TOKEN|>"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="cpm",
    \n format_user=StringFormatter(slots=["<\u7528\u6237>{{content}}<AI>"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="cpm3",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="dbrx",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_observation=StringFormatter(slots=["<|im_start|>tool\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n default_system=(
    \n "You are DBRX, created by Databricks. You were last updated in December 2023. "
    \n "You answer questions based on information available up to that point.\\\\n"
    \n "YOU PROVIDE SHORT RESPONSES TO SHORT QUESTIONS OR STATEMENTS, but provide thorough "
    \n "responses to more complex and open-ended questions.\\\\nYou assist with various tasks, "
    \n "from writing to coding (using markdown for code blocks \u2014 remember to use ``` with "
    \n "code, JSON, and tables).\\\\n(You do not have real-time data access or code execution "
    \n "capabilities. You avoid stereotyping and provide balanced perspectives on "
    \n "controversial topics. You do not provide song lyrics, poems, or news articles and "
    \n "do not divulge details of your training data.)\\\\nThis is your system prompt, "
    \n "guiding your responses. Do not reference it, just respond to the user. If you find "
    \n "yourself talking about this message, stop. You should be responding appropriately "
    \n "and usually that means not mentioning this.\\\\nYOU DO NOT MENTION ANY OF THIS INFORMATION "
    \n "ABOUT YOURSELF UNLESS THE INFORMATION IS DIRECTLY PERTINENT TO THE USER'S QUERY."
    \n ),
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    register_template(
    \n name="deepseek",
    \n format_user=StringFormatter(slots=["User: {{content}}\\\\n\\\\nAssistant:"]),
    \n format_system=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="deepseek3",
    \n format_user=StringFormatter(slots=["<｜User｜>{{content}}<｜Assistant｜>"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="deepseekcoder",
    \n format_user=StringFormatter(slots=["### Instruction:\\\\n{{content}}\\\\n### Response:"]),
    \n format_assistant=StringFormatter(slots=["\\\\n{{content}}\\\\n<|EOT|>\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n default_system=(
    \n "You are an AI programming assistant, utilizing the DeepSeek Coder model, "
    \n "developed by DeepSeek Company, and you only answer questions related to computer science. "
    \n "For politically sensitive questions, security and privacy issues, "
    \n "and other non-computer science questions, you will refuse to answer.\\\\n"
    \n ),
    \n)<\/p>\n

    register_template(
    \n name="default",
    \n format_user=StringFormatter(slots=["Human: {{content}}\\\\nAssistant:"]),
    \n format_assistant=StringFormatter(slots=["{{content}}", {"eos_token"}, "\\\\n"]),
    \n format_system=StringFormatter(slots=["System: {{content}}\\\\n"]),
    \n)<\/p>\n

    register_template(
    \n name="empty",
    \n format_assistant=StringFormatter(slots=["{{content}}"]),
    \n)<\/p>\n

    register_template(
    \n name="exaone",
    \n format_user=StringFormatter(slots=["[|user|]{{content}}\\\\n[|assistant|]"]),
    \n format_assistant=StringFormatter(slots=["{{content}}", {"eos_token"}, "\\\\n"]),
    \n format_system=StringFormatter(slots=["[|system|]{{content}}[|endofturn|]\\\\n"]),
    \n)<\/p>\n

    register_template(
    \n name="falcon",
    \n format_user=StringFormatter(slots=["User: {{content}}\\\\nFalcon:"]),
    \n format_assistant=StringFormatter(slots=["{{content}}\\\\n"]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="fewshot",
    \n format_assistant=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="gemma",
    \n format_user=StringFormatter(slots=["<start_of_turn>user\\\\n{{content}}<end_of_turn>\\\\n<start_of_turn>model\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<end_of_turn>\\\\n"]),
    \n format_observation=StringFormatter(
    \n slots=["<start_of_turn>tool\\\\n{{content}}<end_of_turn>\\\\n<start_of_turn>model\\\\n"]
    \n ),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="glm4",
    \n format_user=StringFormatter(slots=["<|user|>\\\\n{{content}}<|assistant|>"]),
    \n format_assistant=StringFormatter(slots=["\\\\n{{content}}"]),
    \n format_system=StringFormatter(slots=["<|system|>\\\\n{{content}}"]),
    \n format_function=FunctionFormatter(slots=["{{content}}"], tool_format="glm4"),
    \n format_observation=StringFormatter(slots=["<|observation|>\\\\n{{content}}<|assistant|>"]),
    \n format_tools=ToolFormatter(tool_format="glm4"),
    \n format_prefix=EmptyFormatter(slots=["[gMASK]<sop>"]),
    \n stop_words=["<|user|>", "<|observation|>"],
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="granite3",
    \n format_user=StringFormatter(
    \n slots=[
    \n "<|start_of_role|>user<|end_of_role|>{{content}}<|end_of_text|>\\\\n<|start_of_role|>assistant<|end_of_role|>"
    \n ]
    \n ),
    \n format_assistant=StringFormatter(slots=["{{content}}<|end_of_text|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|start_of_role|>system<|end_of_role|>{{content}}<|end_of_text|>\\\\n"]),
    \n)<\/p>\n

    register_template(
    \n name="index",
    \n format_user=StringFormatter(slots=["reserved_0{{content}}reserved_1"]),
    \n format_system=StringFormatter(slots=["<unk>{{content}}"]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="intern",
    \n format_user=StringFormatter(slots=["<|User|>:{{content}}\\\\n<|Bot|>:"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<eoa>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|System|>:{{content}}\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n default_system=(
    \n "You are an AI assistant whose name is InternLM (\u4e66\u751f\u00b7\u6d66\u8bed).\\\\n"
    \n "- InternLM (\u4e66\u751f\u00b7\u6d66\u8bed) is a conversational language model that is developed by Shanghai AI Laboratory "
    \n "(\u4e0a\u6d77\u4eba\u5de5\u667a\u80fd\u5b9e\u9a8c\u5ba4). It is designed to be helpful, honest, and harmless.\\\\n"
    \n "- InternLM (\u4e66\u751f\u00b7\u6d66\u8bed) can understand and communicate fluently in the language "
    \n "chosen by the user such as English and \u4e2d\u6587."
    \n ),
    \n stop_words=["<eoa>"],
    \n)<\/p>\n

    register_template(
    \n name="intern2",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n default_system=(
    \n "You are an AI assistant whose name is InternLM (\u4e66\u751f\u00b7\u6d66\u8bed).\\\\n"
    \n "- InternLM (\u4e66\u751f\u00b7\u6d66\u8bed) is a conversational language model that is developed by Shanghai AI Laboratory "
    \n "(\u4e0a\u6d77\u4eba\u5de5\u667a\u80fd\u5b9e\u9a8c\u5ba4). It is designed to be helpful, honest, and harmless.\\\\n"
    \n "- InternLM (\u4e66\u751f\u00b7\u6d66\u8bed) can understand and communicate fluently in the language "
    \n "chosen by the user such as English and \u4e2d\u6587."
    \n ),
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    register_template(
    \n name="llama2",
    \n format_user=StringFormatter(slots=[{"bos_token"}, "[INST] {{content}} [\/INST]"]),
    \n format_system=StringFormatter(slots=["<<SYS>>\\\\n{{content}}\\\\n<<\/SYS>>\\\\n\\\\n"]),
    \n template_class=Llama2Template,
    \n)<\/p>\n

    # copied from llama2 template
    \nregister_template(
    \n name="llama2_zh",
    \n format_user=StringFormatter(slots=[{"bos_token"}, "[INST] {{content}} [\/INST]"]),
    \n format_system=StringFormatter(slots=["<<SYS>>\\\\n{{content}}\\\\n<<\/SYS>>\\\\n\\\\n"]),
    \n default_system="You are a helpful assistant. \u4f60\u662f\u4e00\u4e2a\u4e50\u4e8e\u52a9\u4eba\u7684\u52a9\u624b\u3002",
    \n template_class=Llama2Template,
    \n)<\/p>\n

    register_template(
    \n name="llama3",
    \n format_user=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>user<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_assistant=StringFormatter(slots=["{{content}}<|eot_id|>"]),
    \n format_system=StringFormatter(slots=["<|start_header_id|>system<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"]),
    \n format_function=FunctionFormatter(slots=["{{content}}<|eot_id|>"], tool_format="llama3"),
    \n format_observation=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>ipython<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_tools=ToolFormatter(tool_format="llama3"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n stop_words=["<|eot_id|>", "<|eom_id|>"],
    \n)<\/p>\n

    # copied from llama3 template
    \nregister_template(
    \n name="mllama",
    \n format_user=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>user<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_assistant=StringFormatter(slots=["{{content}}<|eot_id|>"]),
    \n format_system=StringFormatter(slots=["<|start_header_id|>system<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"]),
    \n format_function=FunctionFormatter(slots=["{{content}}<|eot_id|>"], tool_format="llama3"),
    \n format_observation=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>ipython<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_tools=ToolFormatter(tool_format="llama3"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n stop_words=["<|eot_id|>", "<|eom_id|>"],
    \n mm_plugin=get_mm_plugin(name="mllama", image_token="<|image|>"),
    \n)<\/p>\n

    # copied from vicuna template
    \nregister_template(
    \n name="llava",
    \n format_user=StringFormatter(slots=["USER: {{content}} ASSISTANT:"]),
    \n default_system=(
    \n "A chat between a curious user and an artificial intelligence assistant. "
    \n "The assistant gives helpful, detailed, and polite answers to the user's questions."
    \n ),
    \n mm_plugin=get_mm_plugin(name="llava", image_token="<image>"),
    \n)<\/p>\n

    # copied from vicuna template
    \nregister_template(
    \n name="llava_next",
    \n format_user=StringFormatter(slots=["USER: {{content}} ASSISTANT:"]),
    \n default_system=(
    \n "A chat between a curious user and an artificial intelligence assistant. "
    \n "The assistant gives helpful, detailed, and polite answers to the user's questions."
    \n ),
    \n mm_plugin=get_mm_plugin(name="llava_next", image_token="<image>"),
    \n)<\/p>\n

    # copied from llama3 template
    \nregister_template(
    \n name="llava_next_llama3",
    \n format_user=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>user<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_assistant=StringFormatter(slots=["{{content}}<|eot_id|>"]),
    \n format_system=StringFormatter(slots=["<|start_header_id|>system<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"]),
    \n format_function=FunctionFormatter(slots=["{{content}}<|eot_id|>"], tool_format="llama3"),
    \n format_observation=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>ipython<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_tools=ToolFormatter(tool_format="llama3"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n stop_words=["<|eot_id|>", "<|eom_id|>"],
    \n mm_plugin=get_mm_plugin(name="llava_next", image_token="<image>"),
    \n)<\/p>\n

    # copied from mistral template
    \nregister_template(
    \n name="llava_next_mistral",
    \n format_user=StringFormatter(slots=["[INST] {{content}}[\/INST]"]),
    \n format_assistant=StringFormatter(slots=[" {{content}}", {"eos_token"}]),
    \n format_system=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n format_function=FunctionFormatter(slots=["[TOOL_CALLS] {{content}}", {"eos_token"}], tool_format="mistral"),
    \n format_observation=StringFormatter(slots=["""[TOOL_RESULTS] {"content": {{content}}}[\/TOOL_RESULTS]"""]),
    \n format_tools=ToolFormatter(tool_format="mistral"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n mm_plugin=get_mm_plugin(name="llava_next", image_token="<image>"),
    \n template_class=Llama2Template,
    \n)<\/p>\n

    # copied from qwen template
    \nregister_template(
    \n name="llava_next_qwen",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_function=FunctionFormatter(slots=["{{content}}<|im_end|>\\\\n"], tool_format="qwen"),
    \n format_observation=StringFormatter(
    \n slots=["<|im_start|>user\\\\n<tool_response>\\\\n{{content}}\\\\n<\/tool_response><|im_end|>\\\\n<|im_start|>assistant\\\\n"]
    \n ),
    \n format_tools=ToolFormatter(tool_format="qwen"),
    \n default_system="You are a helpful assistant.",
    \n stop_words=["<|im_end|>"],
    \n mm_plugin=get_mm_plugin(name="llava_next", image_token="<image>"),
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="llava_next_yi",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n stop_words=["<|im_end|>"],
    \n mm_plugin=get_mm_plugin(name="llava_next", image_token="<image>"),
    \n)<\/p>\n

    # copied from vicuna template
    \nregister_template(
    \n name="llava_next_video",
    \n format_user=StringFormatter(slots=["USER: {{content}} ASSISTANT:"]),
    \n default_system=(
    \n "A chat between a curious user and an artificial intelligence assistant. "
    \n "The assistant gives helpful, detailed, and polite answers to the user's questions."
    \n ),
    \n mm_plugin=get_mm_plugin(name="llava_next_video", image_token="<image>", video_token="<video>"),
    \n)<\/p>\n

    # copied from mistral template
    \nregister_template(
    \n name="llava_next_video_mistral",
    \n format_user=StringFormatter(slots=["[INST] {{content}}[\/INST]"]),
    \n format_assistant=StringFormatter(slots=[" {{content}}", {"eos_token"}]),
    \n format_system=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n format_function=FunctionFormatter(slots=["[TOOL_CALLS] {{content}}", {"eos_token"}], tool_format="mistral"),
    \n format_observation=StringFormatter(slots=["""[TOOL_RESULTS] {"content": {{content}}}[\/TOOL_RESULTS]"""]),
    \n format_tools=ToolFormatter(tool_format="mistral"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n mm_plugin=get_mm_plugin(name="llava_next_video", image_token="<image>", video_token="<video>"),
    \n template_class=Llama2Template,
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="llava_next_video_yi",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n stop_words=["<|im_end|>"],
    \n mm_plugin=get_mm_plugin(name="llava_next_video", image_token="<image>", video_token="<video>"),
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="marco",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_observation=StringFormatter(slots=["<|im_start|>tool\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n default_system=(
    \n "\u4f60\u662f\u4e00\u4e2a\u7ecf\u8fc7\u826f\u597d\u8bad\u7ec3\u7684AI\u52a9\u624b，\u4f60\u7684\u540d\u5b57\u662fMarco-o1.\u7531\u963f\u91cc\u56fd\u9645\u6570\u5b57\u5546\u4e1a\u96c6\u56e2\u7684AI Business\u521b\u9020.\\\\n## \u91cd\u8981！！！！！\\\\n"
    \n "\u5f53\u4f60\u56de\u7b54\u95ee\u9898\u65f6，\u4f60\u7684\u601d\u8003\u5e94\u8be5\u5728<Thought>\u5185\u5b8c\u6210，<Output>\u5185\u8f93\u51fa\u4f60\u7684\u7ed3\u679c\u3002\\\\n"
    \n "<Thought>\u5e94\u8be5\u5c3d\u53ef\u80fd\u662f\u82f1\u6587，\u4f46\u662f\u67092\u4e2a\u7279\u4f8b，\u4e00\u4e2a\u662f\u5bf9\u539f\u6587\u4e2d\u7684\u5f15\u7528，\u53e6\u4e00\u4e2a\u662f\u662f\u6570\u5b66\u5e94\u8be5\u4f7f\u7528markdown\u683c\u5f0f，<Output>\u5185\u7684\u8f93\u51fa\u9700\u8981\u9075\u5faa\u7528\u6237\u8f93\u5165\u7684\u8bed\u8a00\u3002\\\\n"
    \n ),
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="minicpm_v",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n stop_words=["<|im_end|>"],
    \n default_system="You are a helpful assistant.",
    \n mm_plugin=get_mm_plugin(name="minicpm_v", image_token="<image>", video_token="<video>"),
    \n)<\/p>\n

    # copied from minicpm_v template
    \nregister_template(
    \n name="minicpm_o",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n stop_words=["<|im_end|>"],
    \n default_system="You are Qwen, created by Alibaba Cloud. You are a helpful assistant.",
    \n mm_plugin=get_mm_plugin(name="minicpm_v", image_token="<image>", video_token="<video>", audio_token="<audio>"),
    \n)<\/p>\n

    # mistral tokenizer v3 tekken
    \nregister_template(
    \n name="ministral",
    \n format_user=StringFormatter(slots=["[INST]{{content}}[\/INST]"]),
    \n format_system=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n format_function=FunctionFormatter(slots=["[TOOL_CALLS]{{content}}", {"eos_token"}], tool_format="mistral"),
    \n format_observation=StringFormatter(slots=["""[TOOL_RESULTS]{"content": {{content}}}[\/TOOL_RESULTS]"""]),
    \n format_tools=ToolFormatter(tool_format="mistral"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n template_class=Llama2Template,
    \n)<\/p>\n

    # mistral tokenizer v3
    \nregister_template(
    \n name="mistral",
    \n format_user=StringFormatter(slots=["[INST] {{content}}[\/INST]"]),
    \n format_assistant=StringFormatter(slots=[" {{content}}", {"eos_token"}]),
    \n format_system=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n format_function=FunctionFormatter(slots=["[TOOL_CALLS] {{content}}", {"eos_token"}], tool_format="mistral"),
    \n format_observation=StringFormatter(slots=["""[TOOL_RESULTS] {"content": {{content}}}[\/TOOL_RESULTS]"""]),
    \n format_tools=ToolFormatter(tool_format="mistral"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n template_class=Llama2Template,
    \n)<\/p>\n

    # mistral tokenizer v7 tekken (copied from ministral)
    \nregister_template(
    \n name="mistral_small",
    \n format_user=StringFormatter(slots=["[INST]{{content}}[\/INST]"]),
    \n format_system=StringFormatter(slots=["[SYSTEM_PROMPT]{{content}}[\/SYSTEM_PROMPT]"]),
    \n format_function=FunctionFormatter(slots=["[TOOL_CALLS]{{content}}", {"eos_token"}], tool_format="mistral"),
    \n format_observation=StringFormatter(slots=["""[TOOL_RESULTS]{"content": {{content}}}[\/TOOL_RESULTS]"""]),
    \n format_tools=ToolFormatter(tool_format="mistral"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="olmo",
    \n format_user=StringFormatter(slots=["<|user|>\\\\n{{content}}<|assistant|>\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"eos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="openchat",
    \n format_user=StringFormatter(slots=["GPT4 Correct User: {{content}}", {"eos_token"}, "GPT4 Correct Assistant:"]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    register_template(
    \n name="openchat-3.6",
    \n format_user=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>GPT4 Correct User<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>GPT4 Correct Assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n stop_words=["<|eot_id|>"],
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="opencoder",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_observation=StringFormatter(slots=["<|im_start|>tool\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n default_system="You are OpenCoder, created by OpenCoder Team.",
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    register_template(
    \n name="orion",
    \n format_user=StringFormatter(slots=["Human: {{content}}\\\\n\\\\nAssistant: ", {"eos_token"}]),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n)<\/p>\n

    # copied from gemma template
    \nregister_template(
    \n name="paligemma",
    \n format_user=StringFormatter(slots=["<start_of_turn>user\\\\n{{content}}<end_of_turn>\\\\n<start_of_turn>model\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<end_of_turn>\\\\n"]),
    \n format_observation=StringFormatter(
    \n slots=["<start_of_turn>tool\\\\n{{content}}<end_of_turn>\\\\n<start_of_turn>model\\\\n"]
    \n ),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n mm_plugin=get_mm_plugin(name="paligemma", image_token="<image>"),
    \n)<\/p>\n

    register_template(
    \n name="phi",
    \n format_user=StringFormatter(slots=["<|user|>\\\\n{{content}}<|end|>\\\\n<|assistant|>\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|system|>\\\\n{{content}}<|end|>\\\\n"]),
    \n stop_words=["<|end|>"],
    \n)<\/p>\n

    register_template(
    \n name="phi_small",
    \n format_user=StringFormatter(slots=["<|user|>\\\\n{{content}}<|end|>\\\\n<|assistant|>\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|system|>\\\\n{{content}}<|end|>\\\\n"]),
    \n format_prefix=EmptyFormatter(slots=[{"<|endoftext|>"}]),
    \n stop_words=["<|end|>"],
    \n)<\/p>\n

    register_template(
    \n name="phi4",
    \n format_user=StringFormatter(
    \n slots=["<|im_start|>user<|im_sep|>{{content}}<|im_end|><|im_start|>assistant<|im_sep|>"]
    \n ),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system<|im_sep|>{{content}}<|im_end|>"]),
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    # copied from ministral template
    \nregister_template(
    \n name="pixtral",
    \n format_user=StringFormatter(slots=["[INST]{{content}}[\/INST]"]),
    \n format_system=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n format_function=FunctionFormatter(slots=["[TOOL_CALLS]{{content}}", {"eos_token"}], tool_format="mistral"),
    \n format_observation=StringFormatter(slots=["""[TOOL_RESULTS]{"content": {{content}}}[\/TOOL_RESULTS]"""]),
    \n format_tools=ToolFormatter(tool_format="mistral"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n mm_plugin=get_mm_plugin(name="pixtral", image_token="[IMG]"),
    \n template_class=Llama2Template,
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="qwen",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_function=FunctionFormatter(slots=["{{content}}<|im_end|>\\\\n"], tool_format="qwen"),
    \n format_observation=StringFormatter(
    \n slots=["<|im_start|>user\\\\n<tool_response>\\\\n{{content}}\\\\n<\/tool_response><|im_end|>\\\\n<|im_start|>assistant\\\\n"]
    \n ),
    \n format_tools=ToolFormatter(tool_format="qwen"),
    \n default_system="You are a helpful assistant.",
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="qwen2_audio",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n default_system="You are a helpful assistant.",
    \n stop_words=["<|im_end|>"],
    \n mm_plugin=get_mm_plugin(name="qwen2_audio", audio_token="<|AUDIO|>"),
    \n)<\/p>\n

    # copied from qwen template
    \nregister_template(
    \n name="qwen2_vl",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n format_function=FunctionFormatter(slots=["{{content}}<|im_end|>\\\\n"], tool_format="qwen"),
    \n format_observation=StringFormatter(
    \n slots=["<|im_start|>user\\\\n<tool_response>\\\\n{{content}}\\\\n<\/tool_response><|im_end|>\\\\n<|im_start|>assistant\\\\n"]
    \n ),
    \n format_tools=ToolFormatter(tool_format="qwen"),
    \n default_system="You are a helpful assistant.",
    \n stop_words=["<|im_end|>"],
    \n mm_plugin=get_mm_plugin(name="qwen2_vl", image_token="<|image_pad|>", video_token="<|video_pad|>"),
    \n)<\/p>\n

    register_template(
    \n name="sailor",
    \n format_user=StringFormatter(slots=["<|im_start|>question\\\\n{{content}}<|im_end|>\\\\n<|im_start|>answer\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n default_system=(
    \n "You are an AI assistant named Sailor created by Sea AI Lab. "
    \n "Your answer should be friendly, unbiased, faithful, informative and detailed."
    \n ),
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    # copied from llama3 template
    \nregister_template(
    \n name="skywork_o1",
    \n format_user=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>user<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_assistant=StringFormatter(slots=["{{content}}<|eot_id|>"]),
    \n format_system=StringFormatter(slots=["<|start_header_id|>system<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"]),
    \n format_function=FunctionFormatter(slots=["{{content}}<|eot_id|>"], tool_format="llama3"),
    \n format_observation=StringFormatter(
    \n slots=[
    \n (
    \n "<|start_header_id|>ipython<|end_header_id|>\\\\n\\\\n{{content}}<|eot_id|>"
    \n "<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n"
    \n )
    \n ]
    \n ),
    \n format_tools=ToolFormatter(tool_format="llama3"),
    \n format_prefix=EmptyFormatter(slots=[{"bos_token"}]),
    \n default_system=(
    \n "You are Skywork-o1, a thinking model developed by Skywork AI, specializing in solving complex problems "
    \n "involving mathematics, coding, and logical reasoning through deep thought. When faced with a user's request, "
    \n "you first engage in a lengthy and in-depth thinking process to explore possible solutions to the problem. "
    \n "After completing your thoughts, you then provide a detailed explanation of the solution process "
    \n "in your response."
    \n ),
    \n stop_words=["<|eot_id|>", "<|eom_id|>"],
    \n)<\/p>\n

    register_template(
    \n name="solar",
    \n format_user=StringFormatter(slots=["### User:\\\\n{{content}}\\\\n\\\\n### Assistant:\\\\n"]),
    \n format_system=StringFormatter(slots=["### System:\\\\n{{content}}\\\\n\\\\n"]),
    \n efficient_eos=True,
    \n)<\/p>\n

    register_template(
    \n name="starchat",
    \n format_user=StringFormatter(slots=["<|user|>\\\\n{{content}}<|end|>\\\\n<|assistant|>"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|system|>\\\\n{{content}}<|end|>\\\\n"]),
    \n stop_words=["<|end|>"],
    \n)<\/p>\n

    register_template(
    \n name="telechat",
    \n format_user=StringFormatter(slots=["<_user>{{content}}<_bot>"]),
    \n format_system=StringFormatter(slots=["<_system>{{content}}<_end>"]),
    \n)<\/p>\n

    register_template(
    \n name="telechat2",
    \n format_user=StringFormatter(slots=["<_user>{{content}}<_bot>"]),
    \n format_system=StringFormatter(slots=["<_system>{{content}}"]),
    \n default_system=(
    \n "\u4f60\u662f\u4e2d\u56fd\u7535\u4fe1\u661f\u8fb0\u8bed\u4e49\u5927\u6a21\u578b，\u82f1\u6587\u540d\u662fTeleChat，\u4f60\u662f\u7531\u4e2d\u7535\u4fe1\u4eba\u5de5\u667a\u80fd\u79d1\u6280\u6709\u9650\u516c\u53f8\u548c\u4e2d\u56fd\u7535\u4fe1\u4eba\u5de5\u667a\u80fd\u7814\u7a76\u9662（TeleAI）\u7814\u53d1\u7684\u4eba\u5de5\u667a\u80fd\u52a9\u624b\u3002"
    \n ),
    \n)<\/p>\n

    register_template(
    \n name="vicuna",
    \n format_user=StringFormatter(slots=["USER: {{content}} ASSISTANT:"]),
    \n default_system=(
    \n "A chat between a curious user and an artificial intelligence assistant. "
    \n "The assistant gives helpful, detailed, and polite answers to the user's questions."
    \n ),
    \n replace_jinja_template=True,
    \n)<\/p>\n

    register_template(
    \n name="video_llava",
    \n format_user=StringFormatter(slots=["USER: {{content}} ASSISTANT:"]),
    \n default_system=(
    \n "A chat between a curious user and an artificial intelligence assistant. "
    \n "The assistant gives helpful, detailed, and polite answers to the user's questions."
    \n ),
    \n mm_plugin=get_mm_plugin(name="video_llava", image_token="<image>", video_token="<video>"),
    \n)<\/p>\n

    register_template(
    \n name="xuanyuan",
    \n format_user=StringFormatter(slots=["Human: {{content}} Assistant:"]),
    \n default_system=(
    \n "\u4ee5\u4e0b\u662f\u7528\u6237\u548c\u4eba\u5de5\u667a\u80fd\u52a9\u624b\u4e4b\u95f4\u7684\u5bf9\u8bdd\u3002\u7528\u6237\u4ee5Human\u5f00\u5934，\u4eba\u5de5\u667a\u80fd\u52a9\u624b\u4ee5Assistant\u5f00\u5934，"
    \n "\u4f1a\u5bf9\u4eba\u7c7b\u63d0\u51fa\u7684\u95ee\u9898\u7ed9\u51fa\u6709\u5e2e\u52a9\u3001\u9ad8\u8d28\u91cf\u3001\u8be6\u7ec6\u548c\u793c\u8c8c\u7684\u56de\u7b54，\u5e76\u4e14\u603b\u662f\u62d2\u7edd\u53c2\u4e0e\u4e0e\u4e0d\u9053\u5fb7\u3001"
    \n "\u4e0d\u5b89\u5168\u3001\u6709\u4e89\u8bae\u3001\u653f\u6cbb\u654f\u611f\u7b49\u76f8\u5173\u7684\u8bdd\u9898\u3001\u95ee\u9898\u548c\u6307\u793a\u3002\\\\n"
    \n ),
    \n)<\/p>\n

    register_template(
    \n name="xverse",
    \n format_user=StringFormatter(slots=["Human: {{content}}\\\\n\\\\nAssistant: "]),
    \n)<\/p>\n

    register_template(
    \n name="yayi",
    \n format_user=StringFormatter(slots=[{"token": "<|Human|>"}, ":\\\\n{{content}}\\\\n\\\\n", {"token": "<|YaYi|>"}, ":"]),
    \n format_assistant=StringFormatter(slots=["{{content}}\\\\n\\\\n"]),
    \n format_system=StringFormatter(slots=[{"token": "<|System|>"}, ":\\\\n{{content}}\\\\n\\\\n"]),
    \n default_system=(
    \n "You are a helpful, respectful and honest assistant named YaYi "
    \n "developed by Beijing Wenge Technology Co.,Ltd. "
    \n "Always answer as helpfully as possible, while being safe. "
    \n "Your answers should not include any harmful, unethical, "
    \n "racist, sexist, toxic, dangerous, or illegal content. "
    \n "Please ensure that your responses are socially unbiased and positive in nature.\\\\n\\\\n"
    \n "If a question does not make any sense, or is not factually coherent, "
    \n "explain why instead of answering something not correct. "
    \n "If you don't know the answer to a question, please don't share false information."
    \n ),
    \n stop_words=["<|End|>"],
    \n)<\/p>\n

    # copied from chatml template
    \nregister_template(
    \n name="yi",
    \n format_user=StringFormatter(slots=["<|im_start|>user\\\\n{{content}}<|im_end|>\\\\n<|im_start|>assistant\\\\n"]),
    \n format_assistant=StringFormatter(slots=["{{content}}<|im_end|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|im_start|>system\\\\n{{content}}<|im_end|>\\\\n"]),
    \n stop_words=["<|im_end|>"],
    \n)<\/p>\n

    register_template(
    \n name="yi_vl",
    \n format_user=StringFormatter(slots=["### Human: {{content}}\\\\n### Assistant:"]),
    \n format_assistant=StringFormatter(slots=["{{content}}\\\\n"]),
    \n default_system=(
    \n "This is a chat between an inquisitive human and an AI assistant. "
    \n "Assume the role of the AI assistant. Read all the images carefully, "
    \n "and respond to the human's questions with informative, helpful, detailed and polite answers. "
    \n "\u8fd9\u662f\u4e00\u4e2a\u597d\u5947\u7684\u4eba\u7c7b\u548c\u4e00\u4e2a\u4eba\u5de5\u667a\u80fd\u52a9\u624b\u4e4b\u95f4\u7684\u5bf9\u8bdd\u3002\u5047\u8bbe\u4f60\u626e\u6f14\u8fd9\u4e2aAI\u52a9\u624b\u7684\u89d2\u8272\u3002"
    \n "\u4ed4\u7ec6\u9605\u8bfb\u6240\u6709\u7684\u56fe\u50cf，\u5e76\u5bf9\u4eba\u7c7b\u7684\u95ee\u9898\u505a\u51fa\u4fe1\u606f\u4e30\u5bcc\u3001\u6709\u5e2e\u52a9\u3001\u8be6\u7ec6\u7684\u548c\u793c\u8c8c\u7684\u56de\u7b54\u3002\\\\n\\\\n"
    \n ),
    \n stop_words=["###"],
    \n efficient_eos=True,
    \n mm_plugin=get_mm_plugin(name="llava", image_token="<image>"),
    \n)<\/p>\n

    register_template(
    \n name="yuan",
    \n format_user=StringFormatter(slots=["{{content}}", {"token": "<sep>"}]),
    \n format_assistant=StringFormatter(slots=["{{content}}<eod>\\\\n"]),
    \n stop_words=["<eod>"],
    \n)<\/p>\n

    register_template(
    \n name="zephyr",
    \n format_user=StringFormatter(slots=["<|user|>\\\\n{{content}}", {"eos_token"}, "<|assistant|>\\\\n"]),
    \n format_system=StringFormatter(slots=["<|system|>\\\\n{{content}}", {"eos_token"}]),
    \n default_system="You are Zephyr, a helpful assistant.",
    \n)<\/p>\n

    register_template(
    \n name="ziya",
    \n format_user=StringFormatter(slots=["<human>:{{content}}\\\\n<bot>:"]),
    \n format_assistant=StringFormatter(slots=["{{content}}\\\\n"]),
    \n) <\/p>\n","protected":false},"excerpt":{"rendered":"

    \u6587\u7ae0\u6d4f\u89c8\u9605\u8bfb1.4w\u6b21\uff0c\u70b9\u8d5e91\u6b21\uff0c\u6536\u85cf118\u6b21\u3002\u4f7f\u7528LLaMA-Factory\u9ad8\u6548\u5fae\u8c03qwen2.5-7b-instruct_llama factory\u5fae\u8c03 deepseek-r1-7b-distill\u6a21\u578b<\/p>\n","protected":false},"author":2,"featured_media":35975,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[3078,86,224,51],"topic":[],"class_list":["post-35989","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-server","tag-3078","tag-86","tag-224","tag-51"],"yoast_head":"\n\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.wsisp.com\/helps\/35989.html\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"og:description\" content=\"\u6587\u7ae0\u6d4f\u89c8\u9605\u8bfb1.4w\u6b21\uff0c\u70b9\u8d5e91\u6b21\uff0c\u6536\u85cf118\u6b21\u3002\u4f7f\u7528LLaMA-Factory\u9ad8\u6548\u5fae\u8c03qwen2.5-7b-instruct_llama factory\u5fae\u8c03 deepseek-r1-7b-distill\u6a21\u578b\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.wsisp.com\/helps\/35989.html\" \/>\n<meta property=\"og:site_name\" content=\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"article:published_time\" content=\"2025-05-07T01:10:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.wsisp.com\/helps\/wp-content\/uploads\/2025\/05\/20250507011031-681ab3070b8f7.png\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"50 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/35989.html\",\"url\":\"https:\/\/www.wsisp.com\/helps\/35989.html\",\"name\":\"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"isPartOf\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\"},\"datePublished\":\"2025-05-07T01:10:34+00:00\",\"dateModified\":\"2025-05-07T01:10:34+00:00\",\"author\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/35989.html#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.wsisp.com\/helps\/35989.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/35989.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.wsisp.com\/helps\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\",\"url\":\"https:\/\/www.wsisp.com\/helps\/\",\"name\":\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"description\":\"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"contentUrl\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"caption\":\"admin\"},\"sameAs\":[\"http:\/\/wp.wsisp.com\"],\"url\":\"https:\/\/www.wsisp.com\/helps\/author\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.wsisp.com\/helps\/35989.html","og_locale":"zh_CN","og_type":"article","og_title":"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","og_description":"\u6587\u7ae0\u6d4f\u89c8\u9605\u8bfb1.4w\u6b21\uff0c\u70b9\u8d5e91\u6b21\uff0c\u6536\u85cf118\u6b21\u3002\u4f7f\u7528LLaMA-Factory\u9ad8\u6548\u5fae\u8c03qwen2.5-7b-instruct_llama factory\u5fae\u8c03 deepseek-r1-7b-distill\u6a21\u578b","og_url":"https:\/\/www.wsisp.com\/helps\/35989.html","og_site_name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","article_published_time":"2025-05-07T01:10:34+00:00","og_image":[{"url":"https:\/\/www.wsisp.com\/helps\/wp-content\/uploads\/2025\/05\/20250507011031-681ab3070b8f7.png"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"50 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.wsisp.com\/helps\/35989.html","url":"https:\/\/www.wsisp.com\/helps\/35989.html","name":"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09 - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","isPartOf":{"@id":"https:\/\/www.wsisp.com\/helps\/#website"},"datePublished":"2025-05-07T01:10:34+00:00","dateModified":"2025-05-07T01:10:34+00:00","author":{"@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41"},"breadcrumb":{"@id":"https:\/\/www.wsisp.com\/helps\/35989.html#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.wsisp.com\/helps\/35989.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.wsisp.com\/helps\/35989.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.wsisp.com\/helps"},{"@type":"ListItem","position":2,"name":"\u5f00\u6e90\u6a21\u578b\u5e94\u7528\u843d\u5730-DeepSeek-R1-Distill-Qwen-7B-LoRA\u5fae\u8c03-LLaMA-Factory-\u5355\u673a\u5355\u5361-V100\uff08\u4e00\uff09"}]},{"@type":"WebSite","@id":"https:\/\/www.wsisp.com\/helps\/#website","url":"https:\/\/www.wsisp.com\/helps\/","name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","description":"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41","name":"admin","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/","url":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","contentUrl":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","caption":"admin"},"sameAs":["http:\/\/wp.wsisp.com"],"url":"https:\/\/www.wsisp.com\/helps\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/35989","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/comments?post=35989"}],"version-history":[{"count":0,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/35989\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media\/35975"}],"wp:attachment":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media?parent=35989"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/categories?post=35989"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/tags?post=35989"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/topic?post=35989"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}