实战横评】主流 AI 编程工具实测对比，看完不再盲目选型

AI 编程工具横评实测报告（2026-08-04）

7 款 AI 编程工具，同一模型（deepseek-v4-flash）、同一测试题、同一网关并行实测。 附带 token 消耗对比 + 代码质量 + 速度 + 易用性 + 已知 bug 全记录。

目录

• 快速结论（30 秒版）
• 测试方法（完整版）
  • 测试题原文 / 统一环境 / 执行命令 / 判定标准 / 局限与偏差 / 选型决策
• 结果总表
• 分工具详解（7 款逐一分析）
• 模型对比：glm_for_coding vs deepseek-v4-flash
• 最终推荐
• 部署与配置坑（详版）
• Token 数据证据链
• 测试产物全集（7 份完整代码 + pytest 输出）

快速结论（30 秒版）

阅读导航：想复现测试看 测试方法；想挑工具看 最终推荐；想看各工具实际写的代码看 测试产物全集；想避坑看 部署与配置坑；想核对数字看 Token 证据链。

测试方法（完整版）

1. 测试题（逐字原文，所有工具收到的 prompt 完全相同）

Create todo.py (Python CLI todo app with add/list/done/delete commands and JSON persistence) and test_todo.py (pytest unit tests). Run the tests and make sure they pass.

中文翻译：创建 todo.py（带 add/list/done/delete 命令和 JSON 持久化的 Python CLI 待办应用）和 test_todo.py（pytest 单元测试）。运行测试并确保全部通过。

这个测试题考察的能力：

• 理解需求并拆解（CLI 命令设计、JSON 存储设计）
• 写出可测试的代码（函数/类结构可被 pytest 引用）
• 自己写测试用例覆盖功能
• 执行测试、发现失败、修复到全过（自我验证闭环）
• 工具链使用（argparse、json、pytest、文件读写）

2. 统一环境

3. 每个工具的具体执行命令（实测用的真实命令）

4. 判定标准

• 通过 = 该工具自己生成的 test_todo.py 用 pytest 跑通（/tmp/bench-venv/bin/python -m pytest test_todo.py），测试数量由工具自己决定（22/12/13/24/10/13/17 各不相同）
• 耗时 = 从启动命令到进程退出（含模型首 token 延迟 + 工具自我迭代）
• token 数据 = 各工具本地记录（来源见"Token 数据证据链"章节）
• 代码质量 = 人工检查产物（测试覆盖、边界处理、可读性）

5. 测试的局限与已知偏差（诚实声明）

6. 为什么选 deepseek-v4-flash（决策过程）

• 首轮用 glm_for_coding（网关 GLM 渠道）：首 token 40-70 秒，每工具 7-9 分钟，且 Codex heredoc 转义错乱、Crush 有 DEFAULT_FILE bug
• 用户建议测试 deepseek 模型 → 实测 1.7s 首 token，且 Codex/Crush 的问题自动消失
• 最终统一用 deepseek-v4-flash 重测全部 7 工具（第二轮），保证公平
• 首轮 glm 数据仅作对照参考，排名以 deepseek 轮为准

结果总表

* Codex 本地不记录 usage（OpenAI 兼容接口），网关无分 key 明细；按成功 session 的 6 轮 API 请求 × 累积上下文估算（input 12K→32K 递增），单任务成本约 $0.027 † Crush 的 crush.db sessions 表无 cache_read 字段，故无数据 ‡ Crush 记录值（completion_tokens 疑似只记末次请求，实际生成代码见产物）

分工具详解

🥇 1. OpenCode — 综合最优

最快（32s）+ 最省 token（11.7K input，其他工具的 1/5-1/7）+ 代码质量过关。

• 测试：12/12 通过
• 自动发现项目 .venv 里的 pytest 9.1.1 并直接使用
• 安装难点：GitHub 下载慢需镜像（ghfast.top），但配置极简
• Token 数据：本地 SQLite（~/.local/share/opencode/opencode.db session 表）可精确查询

配置要点：

// 项目级 opencode.json
{
"provider": { "myagentlab": {
"npm": "@ai-sdk/openai-compatible",
"options": { "baseURL": "https://api.myagentlab.homes/v1", "apiKey": "见 credentials" },
"models": { "deepseek-v4-flash": { "name": "deepseek-v4-flash" } }
}},
"model": "myagentlab/deepseek-v4-flash"
}

opencode run "任务描述"

🥈 2. Claude Code — 质量天花板

代码质量最高（22 个测试最多最全），但 token 消耗最大。

• 测试：22/22 通过
• 亮点：自己发现并修复了 mktemp 空文件导致 json.load 崩溃的 bug（加了空文件容错 + 2 个对应测试）；用 next_id 计数器保证删除后 id 不复用
• 系统提示词最完善，工具定义最全——这也是它 token 大的原因
• 生态最大（134K stars）、文档全、最成熟

Token 分析： input 227K 中 190K 是 cache read（价格是正常 input 的 1/10），实际新计算 ~36K，并非全价消耗。

配置要点：

export ANTHROPIC_API_KEY=<key> ANTHROPIC_BASE_URL=https://api.myagentlab.homes
claude -p "任务" –model deepseek-v4-flash –allowedTools 'Bash,Write,Read,Edit' –output-format stream-json –verbose

⚠️ root 用户不能用 –dangerously-skip-permissions（被拒），必须用 –allowedTools

🥉 3. Codex — 模型对了就起飞

换 deepseek 后从"失败×4"变"13/13 通过、78s"，但 token 数据缺失 + 有推理模型兼容隐患。

• 测试：13/13 通过，还做了完整 smoke test（add→done→list→delete 端到端）
• 重要历史：glm_for_coding 下失败 4 次（heredoc 被 chr(39) 转义搞坏、代码测试不一致、连接卡死）——是模型问题不是 Codex 问题
• 隐患：deepseek 是推理模型，Codex 多轮对话时触发 reasoning_content 必须回传 400 报错——复杂多轮任务会随机挂，只适合单轮/简单任务
• Terminal-Bench 2.1 排名第一（83.4%）的工具，能力本身最强
• Token 说明：本地 session 不记录 usage；曾尝试"账单前后差值法"但网关为共享 key（后台有 MRS refactor 等任务并发消耗），差值 $2.09 不可信；最终按 6 轮请求 × 累积上下文估算 input ~132K / output ~12K / 成本 ~$0.027

配置要点：

# ~/.codex/config.toml
model = "deepseek-v4-flash"
model_provider = "myagentlab"
[model_providers.myagentlab]
name = "myagentlab"
base_url = "https://api.myagentlab.homes/v1"
env_key = "OPENAI_API_KEY"
wire_api = "chat"

codex exec –dangerously-bypass-approvals-and-sandbox –skip-git-repo-check –json '任务'

4. Goose — 大器晚成

配置坑最多但修好后很稳，17/17 通过，77s。

• 测试：17/17 通过，代码质量好（126 行 todo.py 结构清晰，含损坏文件容错、时间戳、自动递增 ID）
• 自带任务管理（todo_write 工具），agent 会自己记账
• Linux Foundation 项目，48K stars，潜力大

⚠️ 配置坑（都是实测踩出来的）：

配置要点：

// ~/.config/goose/custom_providers/myagentlab.json
{
"name": "myagentlab",
"engine": "openai",
"api_key_env": "MYAGENTLAB_API_KEY",
"base_url": "https://api.myagentlab.homes/v1",
"models": [ { "name": "deepseek-v4-flash", "context_limit": 128000 } ],
"supports_streaming": false,
"requires_auth": true
}

MYAGENTLAB_API_KEY=<key> goose run -t '任务' –provider myagentlab –model deepseek-v4-flash

5. Crush — 焕然一新

从 glm 下 13/14（有 bug）变成 deepseek 下 13/13 全过、66s，但配置极坑 + token 记录不全。

• 测试：13/13 通过（glm_for_coding 下有个 DEFAULT_FILE 绑定 bug，换 deepseek 后自动消失）
• 速度快（66s），token 省（14.5K input）
• Charm 出品，UI 好看，但实用性一般

⚠️ 配置坑：

配置要点：

// ~/.config/crush/crush.json
{
"providers": {
"myagentlab": {
"type": "openai-compat",
"base_url": "https://api.myagentlab.homes/v1",
"api_key": "$MYAGENTLAB_API_KEY",
"models": [ { "id": "deepseek-v4-flash", "name": "DeepSeek V4 Flash", "context_window": 128000, "default_max_tokens": 8192 } ]
}
},
"defaultModel": "myagentlab/deepseek-v4-flash"
}

MYAGENTLAB_API_KEY=<key> crush run -m myagentlab/deepseek-v4-flash "任务"

6. Cline — 修好代码后全过，但 CLI 有真 bug

原始 22/24（2 个代码 bug），手动修复后 24/24；但 CLI 二进制有 team_status 崩溃 bug。

• 原始测试：22/24。两个 bug：
• _next_id() 用现存任务 max+1，删除后 id 复用（测试要求永不复用）
• except KeyError: print(f"Error: {exc}") 中 KeyError 的 str 自带引号
• 修复方法（已验证 24/24）：① 加 self._max_id 实例属性持续递增 ② exc.args[0] 取干净消息
• ⚠️ CLI 真 bug：team_status 工具与 deepseek tool schema 校验随机崩溃（约 1/3 概率报 Invalid schema for function 'team_status': null is not of type "array"），是 Cline v3.0.49 二进制缺陷
  • 缓解：~/.cline/data/settings/settings.json 设 "enableTeams": false（不完全稳定）
  • glm_for_coding 下无此问题（21/21 全过）
• CLI 安装配置最简单：cline auth openai -b <url> -k <key> -m <model> 一行搞定
• 真实场景是 VS Code 扩展，CLI 是轻量版

配置要点：

cline auth openai -b https://api.myagentlab.homes/v1 -k <key> -m deepseek-v4-flash
cline –json "任务"

7. Kilo Code — OpenCode 换皮但更费

与 OpenCode 同源（fork），功能几乎一样，但 token 消耗高（cache read 102K）且更慢。

• 测试：10/10 通过，还自己修了个 bug（self.done 属性遮蔽 done() 方法，改名 self.is_done）
• 配置和 OpenCode 几乎相同（kilo.jsonc）
• Token 分析：cache read 102K 是 7 个工具最高——它每次请求重发大量工具定义/上下文，但 cache 价格便宜（1/10）
• 结论：没理由选它，直接用 OpenCode 更好

配置要点：

// ~/.config/kilo/kilo.jsonc
{ "provider": { "myagentlab": {
"npm": "@ai-sdk/openai-compatible",
"options": { "baseURL": "https://api.myagentlab.homes/v1", "apiKey": "见 credentials" },
"models": { "deepseek-v4-flash": { "name": "deepseek-v4-flash" } }
}}, "model": "myagentlab/deepseek-v4-flash" }

kilo run -m myagentlab/deepseek-v4-flash "任务"

模型对比：glm_for_coding vs deepseek-v4-flash

结论：deepseek-v4-flash 是当前网关下最佳通用编码模型（快 + 干净 + 便宜）。

最终推荐

• 日常开发首选：OpenCode + deepseek-v4-flash（最快最省，质量过关）
• 要最高质量：Claude Code + deepseek-v4-flash（最稳最严谨，但 token 大）
• 单轮/简单任务：Codex（能力强，但推理模型多轮有兼容隐患）
• 避免：Kilo（OpenCode 的劣化版）、Cline CLI（team_status 崩溃 bug 未修，等新版本）

安装记录（踩坑汇总）

网络环境备注：GitHub 直连慢/超时 → 用 https://ghfast.top/https://github.com/… 镜像；npm 用 –registry=https://registry.npmmirror.com；Go 用 GOPROXY=https://goproxy.cn,direct。

测试产物存放

所有测试代码、日志、token 数据保留在 /home/agent/coding-benchmark/：

• test-<工具名>/todo.py + test_todo.py — 各工具产物（可复验 pytest）
• REPORT-deepseek.md — 精简版报告
• /tmp/<工具>-ds.log — 运行日志
• 各工具 token 数据来源：OpenCode/Kilo 本地 SQLite、Goose sessions.db usage_ledger、Cline session JSON metrics、Crush .crush/crush.db、Claude Code stream-json usage

文档维护：CEO Agent，2026-08-04

部署与配置坑（详版）

以下均为本次实测真实踩坑记录（含报错原文）。配置代码见上文各工具详解；完整生成代码见文末 测试产物全集。 侧重点：这里是"出了什么错、怎么解决"，上文详解是"怎么配"。

Claude Code

• root 用户限制：–dangerously-skip-permissions 直接拒绝（“cannot be used with root/sudo privileges”），必须用 –allowedTools 'Bash,Write,Read,Edit'
• 环境变量优先级：.bashrc 里的 ANTHROPIC_API_KEY/ANTHROPIC_BASE_URL export 覆盖 ~/.claude/settings.json——要换测试端点必须临时 export，不能只改 settings.json
• 生产配置隔离：测试时用 export 覆盖，不动 ~/.claude/settings.json 和 .bashrc

OpenCode

• GitHub 直连下载超时 → 用 https://ghfast.top/https://github.com/… 镜像下载 opencode-linux-x64.tar.gz（v1.18.12 glibc）
• npm opencode 包 404；@opencode-ai/cli 平台包是空壳——必须下二进制
• 曾遇 EEXIST bug：/tmp/opencode 被 musl 二进制留成文件，glibc 版 mkdir 失败 → rm /tmp/opencode 解决
• opencode version 不是子命令，TUI 是默认命令

Codex

• 配置 ~/.codex/config.toml：wire_api = "chat"（OpenAI 兼容），env_key = "OPENAI_API_KEY"
• 运行：codex exec –dangerously-bypass-approvals-and-sandbox –skip-git-repo-check –json
• 推理模型隐患：deepseek 是推理模型，Codex 多轮对话触发 reasoning_content 必须回传 400——复杂任务会挂，单轮任务 OK
• token 不落本地：session jsonl 无 usage 字段

Goose（坑最多）

• 变量名坑：必须用 OPENAI_API_KEY/OPENAI_BASE_URL（不是 GOOSE_API_KEY/GOOSE_BASE_URL！）——用错连默认 api.openai.com 必然超时
• 流式坑：默认流式报 Stream decode error → declarative provider 必须设 "supports_streaming": false
• 自定义 provider 路径：~/.config/goose/custom_providers/<name>.json
• 诊断方法：本地 SQLite ~/.local/share/goose/sessions/sessions.db 的 messages 表看请求是否真的发出去（之前只存 user 消息 = 请求没到模型）

Crush

• 配置文件坑：正确文件名是 ~/.config/crush/crush.json（不是 config.json！改 config.json 完全不生效，排查半天）
• models 必须是数组不是对象（对象结构报 “large model not found”）
• crush update-providers 会覆盖 providers.json 手动修改（远程拉 40 个 provider）
• -m provider/model 格式（如 -m myagentlab/deepseek-v4-flash）
• token 记录不全：crush.db sessions 表只有 prompt/completion/cost，无 cache_read

Cline

• 配置最简单：cline auth openai -b <url> -k <key> -m <model> 一行
• ⚠️ CLI 二进制 bug：team_status 工具与 deepseek tool schema 校验随机崩溃（~1/3 概率 Invalid schema for function 'team_status': null is not of type "array"）
  • 缓解：~/.cline/data/settings/settings.json 设 "enableTeams": false（不完全稳定）
  • glm_for_coding 下无此问题
• session 数据在 ~/.cline/data/sessions/<id>/，token 在 messages.json 的 assistant 消息 metrics 字段

Kilo Code

• 配置 ~/.config/kilo/kilo.jsonc（opencode fork，格式几乎相同）
• kilo run -m myagentlab/deepseek-v4-flash "任务"
• token 在 ~/.local/share/kilo/kilo.db session 表

Token 数据证据链（每个数字从哪来）

* Crush completion_tokens=40 疑似只记末次请求（实际生成 124 行代码），已按实际产物修正理解

复核命令示例（OpenCode）：

SELECT tokens_input, tokens_output, tokens_cache_read FROM session
WHERE model LIKE '%deepseek%' ORDER BY time_created DESC LIMIT 1;

复核命令示例（Goose）：

SELECT session_id, SUM(input_tokens), SUM(output_tokens), SUM(cache_read_tokens)
FROM usage_ledger GROUP BY session_id ORDER BY MIN(id) DESC LIMIT 2;

复核命令示例（Claude Code）：

grep -oE '"usage":\\{[^}]*\\}' /tmp/cc-ds.log | tail -1

为什么 Codex 没有精确值： 本地 session jsonl 无 usage 字段；网关 billing 接口只给总额（共享 key，后台 MRS 任务并发消耗，差值法不可信）；唯一精确途径是 new-api 后台按渠道查明细。

附：测试产物全集（7 份完整代码 + pytest 输出）

以下为 7 款工具在统一测试题下实际生成的全部代码 + pytest 复验输出（机器实跑，非手写）。 复验命令：/tmp/bench-venv/bin/python -m pytest test_todo.py -v（产物目录 /home/agent/coding-benchmark/test-<工具名>/）

本页存放 7 款工具在统一测试题下实际生成的全部代码 + pytest 复验输出。

复验命令：/tmp/bench-venv/bin/python -m pytest test_todo.py -v（产物目录 /home/agent/coding-benchmark/test-<工具名>/）

Claude Code v2.1.220 — 22/22

Claude Code 生成的 todo.py (4619 bytes)

#!/usr/bin/env python3
"""A simple command-line todo app with JSON persistence.

Usage:
todo.py add <task> Add a new task
todo.py list List all tasks
todo.py done <id> Mark a task as done
todo.py delete <id> Delete a task
todo.py –file <path> Use a custom JSON file (default: ~/.todo.json)

Tasks are stored in a JSON file of the form:
{"next_id": 2, "todos": [{"id": 1, "task": "…", "done": false}]}
The ``next_id`` counter is persisted so task IDs are never reused, even
after a delete. This prevents a stale reference from completing or
deleting the wrong task later.
"""

import argparse
import json
import os
import sys

DEFAULT_FILE = os.path.join(os.path.expanduser("~"), ".todo.json")

class TodoApp:
"""Todo list backed by a JSON file."""

def __init__(self, filepath=DEFAULT_FILE):
self.filepath = filepath

def load(self):
"""Return the store (``next_id`` counter + ``todos`` list)."""
if not os.path.exists(self.filepath):
return {"next_id": 1, "todos": []}
with open(self.filepath, "r") as f:
content = f.read()
if not content.strip():
# Empty file (e.g. created by touch/mktemp) means a fresh store.
return {"next_id": 1, "todos": []}
data = json.loads(content)
# Backwards compatibility: a bare list of todos.
if isinstance(data, list):
data = {
"next_id": max((t["id"] for t in data), default=0) + 1,
"todos": data,
}
return data

def save(self, data):
"""Write the store to the JSON file."""
with open(self.filepath, "w") as f:
json.dump(data, f, indent=2)

def _allocate_id(self, data):
next_id = data["next_id"]
data["next_id"] = next_id + 1
return next_id

def add(self, task):
"""Add a new task and return the created todo."""
data = self.load()
todo = {"id": self._allocate_id(data), "task": task, "done": False}
data["todos"].append(todo)
self.save(data)
return todo

def list_todos(self):
"""Return all todos in insertion order."""
return self.load()["todos"]

def done(self, todo_id):
"""Mark a todo as done and return it, or None if not found."""
data = self.load()
for todo in data["todos"]:
if todo["id"] == todo_id:
todo["done"] = True
self.save(data)
return todo
return None

def delete(self, todo_id):
"""Delete a todo and return it, or None if not found."""
data = self.load()
for i, todo in enumerate(data["todos"]):
if todo["id"] == todo_id:
removed = data["todos"].pop(i)
self.save(data)
return removed
return None

def main(argv=None):
parser = argparse.ArgumentParser(description="A simple todo app")
parser.add_argument(
"–file", default=DEFAULT_FILE, help="JSON file to store todos"
)
subparsers = parser.add_subparsers(dest="command", required=True)

add_parser = subparsers.add_parser("add", help="Add a task")
add_parser.add_argument("task", help="Task description")

subparsers.add_parser("list", help="List all tasks")

done_parser = subparsers.add_parser("done", help="Mark a task as done")
done_parser.add_argument("id", type=int, help="Task ID")

delete_parser = subparsers.add_parser("delete", help="Delete a task")
delete_parser.add_argument("id", type=int, help="Task ID")

args = parser.parse_args(argv)
app = TodoApp(args.file)

if args.command == "add":
todo = app.add(args.task)
print(f"Added todo #{todo['id']}: {todo['task']}")
elif args.command == "list":
todos = app.list_todos()
if not todos:
print("No todos.")
return 0
for todo in todos:
status = "[x]" if todo["done"] else "[ ]"
print(f"{todo['id']}. {status} {todo['task']}")
elif args.command == "done":
todo = app.done(args.id)
if todo is None:
print(f"Todo #{args.id} not found.", file=sys.stderr)
return 1
print(f"Done: {todo['task']}")
elif args.command == "delete":
todo = app.delete(args.id)
if todo is None:
print(f"Todo #{args.id} not found.", file=sys.stderr)
return 1
print(f"Deleted: {todo['task']}")
return 0

if __name__ == "__main__":
sys.exit(main())

Claude Code 生成的 test_todo.py (5048 bytes)

"""Unit tests for the todo app."""

import json

import pytest

from todo import TodoApp, main

@pytest.fixture
def app(tmp_path):
return TodoApp(filepath=tmp_path / "todos.json")

# — add ——————————————————————

def test_add_returns_created_todo(app):
todo = app.add("Buy milk")
assert todo == {"id": 1, "task": "Buy milk", "done": False}

def test_add_persists_to_json(app, tmp_path):
app.add("Write tests")
data = json.loads((tmp_path / "todos.json").read_text())
assert len(data["todos"]) == 1
assert data["todos"][0]["task"] == "Write tests"
assert data["todos"][0]["done"] is False

def test_add_persists_next_id_counter(app, tmp_path):
app.add("first")
app.add("second")
data = json.loads((tmp_path / "todos.json").read_text())
assert data["next_id"] == 3

def test_add_increments_ids(app):
app.add("first")
second = app.add("second")
assert second["id"] == 2
assert [t["id"] for t in app.list_todos()] == [1, 2]

def test_add_after_delete_does_not_reuse_id(app):
first = app.add("first")
app.delete(first["id"])
third = app.add("third")
assert third["id"] == 2
assert [t["id"] for t in app.list_todos()] == [2]

def test_ids_are_monotonic_across_restarts(tmp_path):
filepath = tmp_path / "todos.json"
app1 = TodoApp(filepath=filepath)
first = app1.add("first")
app2 = TodoApp(filepath=filepath)
app2.delete(first["id"])
third = app2.add("third")
assert third["id"] == 2

# — list —————————————————————–

def test_list_empty_when_no_file(tmp_path):
app = TodoApp(filepath=tmp_path / "nonexistent.json")
assert app.list_todos() == []

def test_list_empty_when_file_is_empty(tmp_path):
filepath = tmp_path / "todos.json"
filepath.write_text("")
app = TodoApp(filepath=filepath)
assert app.list_todos() == []

def test_add_over_empty_file(tmp_path):
filepath = tmp_path / "todos.json"
filepath.write_text("")
app = TodoApp(filepath=filepath)
todo = app.add("Task")
assert todo["id"] == 1

def test_list_empty_when_file_is_empty_store(tmp_path):
filepath = tmp_path / "todos.json"
filepath.write_text(json.dumps({"next_id": 1, "todos": []}))
app = TodoApp(filepath=filepath)
assert app.list_todos() == []

def test_list_returns_all_todos_in_order(app):
app.add("a")
app.add("b")
todos = app.list_todos()
assert [t["task"] for t in todos] == ["a", "b"]

# — done —————————————————————–

def test_done_marks_todo(app):
todo = app.add("Task")
result = app.done(todo["id"])
assert result["done"] is True
assert app.list_todos()[0]["done"] is True

def test_done_persists_to_json(app, tmp_path):
todo = app.add("Task")
app.done(todo["id"])
data = json.loads((tmp_path / "todos.json").read_text())
assert data["todos"][0]["done"] is True

def test_done_missing_returns_none(app):
assert app.done(999) is None

# — delete —————————————————————

def test_delete_removes_todo(app):
todo = app.add("Task")
removed = app.delete(todo["id"])
assert removed["task"] == "Task"
assert app.list_todos() == []

def test_delete_persists_to_json(app, tmp_path):
todo = app.add("Task")
app.delete(todo["id"])
data = json.loads((tmp_path / "todos.json").read_text())
assert data["todos"] == []

def test_delete_missing_returns_none(app):
assert app.delete(999) is None

# — CLI ——————————————————————

def test_cli_add_and_list(tmp_path, capsys):
filepath = tmp_path / "todos.json"
assert main(["–file", str(filepath), "add", "Buy milk"]) == 0
assert "Added todo #1: Buy milk" in capsys.readouterr().out

assert main(["–file", str(filepath), "list"]) == 0
out = capsys.readouterr().out
assert "1." in out
assert "Buy milk" in out

def test_cli_list_empty(tmp_path, capsys):
filepath = tmp_path / "todos.json"
assert main(["–file", str(filepath), "list"]) == 0
assert "No todos." in capsys.readouterr().out

def test_cli_done_and_delete(tmp_path, capsys):
filepath = tmp_path / "todos.json"
main(["–file", str(filepath), "add", "Task"])

assert main(["–file", str(filepath), "done", "1"]) == 0
assert "Done: Task" in capsys.readouterr().out

assert main(["–file", str(filepath), "delete", "1"]) == 0
assert "Deleted: Task" in capsys.readouterr().out

def test_cli_done_missing_id_returns_nonzero(tmp_path, capsys):
filepath = tmp_path / "todos.json"
assert main(["–file", str(filepath), "done", "1"]) == 1
assert "not found" in capsys.readouterr().err

def test_cli_delete_missing_id_returns_nonzero(tmp_path, capsys):
filepath = tmp_path / "todos.json"
assert main(["–file", str(filepath), "delete", "1"]) == 1
assert "not found" in capsys.readouterr().err

Claude Code pytest 复验输出

_after_delete_does_not_reuse_id PASSED [ 22%]
test_todo.py::test_ids_are_monotonic_across_restarts PASSED [ 27%]
test_todo.py::test_list_empty_when_no_file PASSED [ 31%]
test_todo.py::test_list_empty_when_file_is_empty PASSED [ 36%]
test_todo.py::test_add_over_empty_file PASSED [ 40%]
test_todo.py::test_list_empty_when_file_is_empty_store PASSED [ 45%]
test_todo.py::test_list_returns_all_todos_in_order PASSED [ 50%]
test_todo.py::test_done_marks_todo PASSED [ 54%]
test_todo.py::test_done_persists_to_json PASSED [ 59%]
test_todo.py::test_done_missing_returns_none PASSED [ 63%]
test_todo.py::test_delete_removes_todo PASSED [ 68%]
test_todo.py::test_delete_persists_to_json PASSED [ 72%]
test_todo.py::test_delete_missing_returns_none PASSED [ 77%]
test_todo.py::test_cli_add_and_list PASSED [ 81%]
test_todo.py::test_cli_list_empty PASSED [ 86%]
test_todo.py::test_cli_done_and_delete PASSED [ 90%]
test_todo.py::test_cli_done_missing_id_returns_nonzero PASSED [ 95%]
test_todo.py::test_cli_delete_missing_id_returns_nonzero PASSED [100%]

============================== 22 passed in 0.16s ==============================

OpenCode v1.18.12 — 12/12

OpenCode 生成的 todo.py (3444 bytes)

#!/usr/bin/env python3
"""A simple CLI todo app with JSON persistence."""

import argparse
import json
import os
import sys

DEFAULT_STORE = "todos.json"

class TodoApp:
def __init__(self, store_path=DEFAULT_STORE):
self.store_path = store_path
self.todos = self._load()

def _load(self):
if not os.path.exists(self.store_path):
return []
try:
with open(self.store_path, "r") as fh:
data = json.load(fh)
except (json.JSONDecodeError, OSError):
return []
if not isinstance(data, list):
return []
return data

def _save(self):
with open(self.store_path, "w") as fh:
json.dump(self.todos, fh, indent=2)

def add(self, title):
todo = {"id": self._next_id(), "title": title, "done": False}
self.todos.append(todo)
self._save()
return todo

def _next_id(self):
return max((t["id"] for t in self.todos), default=0) + 1

def list(self, show_done=True):
todos = self.todos
if not show_done:
todos = [t for t in todos if not t["done"]]
return todos

def done(self, todo_id):
for todo in self.todos:
if todo["id"] == todo_id:
todo["done"] = True
self._save()
return todo
raise KeyError(f"No todo with id {todo_id}")

def delete(self, todo_id):
for todo in self.todos:
if todo["id"] == todo_id:
self.todos.remove(todo)
self._save()
return todo
raise KeyError(f"No todo with id {todo_id}")

def main(argv=None):
parser = argparse.ArgumentParser(prog="todo", description="A simple todo app.")
parser.add_argument(
"–store", default=DEFAULT_STORE, help="path to the JSON store file"
)
sub = parser.add_subparsers(dest="command", required=True)

p_add = sub.add_parser("add", help="add a todo")
p_add.add_argument("title", help="title of the todo")

p_list = sub.add_parser("list", help="list todos")
p_list.add_argument(
"–all", action="store_true", help="show completed todos too"
)

p_done = sub.add_parser("done", help="mark a todo as done")
p_done.add_argument("id", type=int, help="todo id")

p_delete = sub.add_parser("delete", help="delete a todo")
p_delete.add_argument("id", type=int, help="todo id")

args = parser.parse_args(argv)

app = TodoApp(args.store)

try:
if args.command == "add":
todo = app.add(args.title)
print(f"Added todo {todo['id']}: {todo['title']}")
elif args.command == "list":
todos = app.list(show_done=args.all)
if not todos:
print("No todos.")
return 0
for todo in todos:
status = "[x]" if todo["done"] else "[ ]"
print(f"{status} {todo['id']}: {todo['title']}")
elif args.command == "done":
todo = app.done(args.id)
print(f"Marked todo {todo['id']} as done: {todo['title']}")
elif args.command == "delete":
todo = app.delete(args.id)
print(f"Deleted todo {todo['id']}: {todo['title']}")
except KeyError as exc:
print(str(exc), file=sys.stderr)
return 1
return 0

if __name__ == "__main__":
sys.exit(main())

OpenCode 生成的 test_todo.py (2052 bytes)

import json
import os

import pytest

from todo import TodoApp

@pytest.fixture
def app(tmp_path):
store = tmp_path / "todos.json"
return TodoApp(str(store))

def test_add_creates_todo(app):
todo = app.add("Buy milk")
assert todo["title"] == "Buy milk"
assert todo["done"] is False
assert todo["id"] == 1

def test_add_persists_to_disk(app):
app.add("Walk the dog")
assert os.path.exists(app.store_path)
with open(app.store_path) as fh:
data = json.load(fh)
assert data[0]["title"] == "Walk the dog"

def test_list_empty(app):
assert app.list() == []

def test_list_returns_all_todos(app):
app.add("A")
app.add("B")
assert len(app.list()) == 2

def test_list_hides_done_when_requested(app):
app.add("A")
app.add("B")
app.done(1)
remaining = app.list(show_done=False)
assert [t["id"] for t in remaining] == [2]

def test_done_marks_todo_completed(app):
app.add("A")
todo = app.done(1)
assert todo["done"] is True
assert app.todos[0]["done"] is True

def test_done_unknown_id_raises(app):
with pytest.raises(KeyError):
app.done(999)

def test_delete_removes_todo(app):
app.add("A")
app.add("B")
todo = app.delete(1)
assert todo["id"] == 1
assert len(app.list()) == 1
assert app.list()[0]["id"] == 2

def test_delete_unknown_id_raises(app):
with pytest.raises(KeyError):
app.delete(999)

def test_next_id_increments_after_delete(app):
app.add("A")
app.add("B")
app.delete(1)
todo = app.add("C")
assert todo["id"] == 3

def test_load_existing_store(tmp_path):
store = tmp_path / "todos.json"
store.write_text(json.dumps([{"id": 1, "title": "existing", "done": True}]))
app = TodoApp(str(store))
assert len(app.list()) == 1
assert app.list()[0]["title"] == "existing"

def test_load_corrupt_store_returns_empty(tmp_path):
store = tmp_path / "todos.json"
store.write_text("{not valid json")
app = TodoApp(str(store))
assert app.list() == []

OpenCode pytest 复验输出

============================= test session starts ==============================
platform linux — Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 — /tmp/bench-venv/bin/python
cachedir: .pytest_cache
rootdir: /home/agent/coding-benchmark/test-opencode
collecting … collected 12 items

test_todo.py::test_add_creates_todo PASSED [ 8%]
test_todo.py::test_add_persists_to_disk PASSED [ 16%]
test_todo.py::test_list_empty PASSED [ 25%]
test_todo.py::test_list_returns_all_todos PASSED [ 33%]
test_todo.py::test_list_hides_done_when_requested PASSED [ 41%]
test_todo.py::test_done_marks_todo_completed PASSED [ 50%]
test_todo.py::test_done_unknown_id_raises PASSED [ 58%]
test_todo.py::test_delete_removes_todo PASSED [ 66%]
test_todo.py::test_delete_unknown_id_raises PASSED [ 75%]
test_todo.py::test_next_id_increments_after_delete PASSED [ 83%]
test_todo.py::test_load_existing_store PASSED [ 91%]
test_todo.py::test_load_corrupt_store_returns_empty PASSED [100%]

============================== 12 passed in 0.07s ==============================

Codex v0.50.0 — 13/13

Codex 生成的 todo.py (4025 bytes)

#!/usr/bin/env python3
"""A simple command-line todo application with JSON persistence."""

import argparse
import json
import sys
from pathlib import Path

DEFAULT_FILE = Path(__file__).parent / "todos.json"

def load_todos(path=DEFAULT_FILE):
"""Load todos from a JSON file, returning an empty list if missing or corrupt."""
if not Path(path).exists():
return []
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
except (json.JSONDecodeError, OSError):
return []
return data if isinstance(data, list) else []

def save_todos(todos, path=DEFAULT_FILE):
"""Save a list of todos to a JSON file."""
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
json.dump(todos, f, indent=2)

def add_todo(description, path=DEFAULT_FILE):
"""Add a new todo and return its id."""
todos = load_todos(path)
todo_id = max((t["id"] for t in todos), default=0) + 1
todos.append({"id": todo_id, "description": description, "done": False})
save_todos(todos, path)
return todo_id

def list_todos(path=DEFAULT_FILE):
"""Return the list of todos."""
return load_todos(path)

def mark_done(todo_id, path=DEFAULT_FILE):
"""Mark a todo as done; return True if it was found."""
todos = load_todos(path)
for todo in todos:
if todo["id"] == todo_id:
todo["done"] = True
save_todos(todos, path)
return True
return False

def delete_todo(todo_id, path=DEFAULT_FILE):
"""Delete a todo; return True if it was found."""
todos = load_todos(path)
remaining = [t for t in todos if t["id"] != todo_id]
if len(remaining) == len(todos):
return False
save_todos(remaining, path)
return True

def main(argv=None):
"""Parse command-line arguments and run the requested command."""
parser = argparse.ArgumentParser(prog="todo", description="A simple todo app.")
subparsers = parser.add_subparsers(dest="command", required=True)

add_parser = subparsers.add_parser("add", help="Add a new todo")
add_parser.add_argument("description", nargs="+", help="Todo description")
add_parser.add_argument("-f", "–file", default=DEFAULT_FILE, help="JSON file to use")

list_parser = subparsers.add_parser("list", help="List all todos")
list_parser.add_argument("-f", "–file", default=DEFAULT_FILE, help="JSON file to use")

done_parser = subparsers.add_parser("done", help="Mark a todo as done")
done_parser.add_argument("todo_id", type=int, help="Todo ID")
done_parser.add_argument("-f", "–file", default=DEFAULT_FILE, help="JSON file to use")

delete_parser = subparsers.add_parser("delete", help="Delete a todo")
delete_parser.add_argument("todo_id", type=int, help="Todo ID")
delete_parser.add_argument("-f", "–file", default=DEFAULT_FILE, help="JSON file to use")

args = parser.parse_args(argv)

if args.command == "add":
description = " ".join(args.description)
todo_id = add_todo(description, args.file)
print(f"Added todo #{todo_id}: {description}")
elif args.command == "list":
todos = list_todos(args.file)
if not todos:
print("No todos.")
return 0
for todo in todos:
status = "[x]" if todo["done"] else "[ ]"
print(f"{status} {todo['id']}: {todo['description']}")
elif args.command == "done":
if mark_done(args.todo_id, args.file):
print(f"Marked todo #{args.todo_id} as done.")
else:
print(f"No todo with id {args.todo_id}.", file=sys.stderr)
return 1
elif args.command == "delete":
if delete_todo(args.todo_id, args.file):
print(f"Deleted todo #{args.todo_id}.")
else:
print(f"No todo with id {args.todo_id}.", file=sys.stderr)
return 1
return 0

if __name__ == "__main__":
sys.exit(main())

Codex 生成的 test_todo.py (3197 bytes)

import pytest

import todo

def test_load_missing_file_returns_empty_list(tmp_path):
assert todo.load_todos(tmp_path / "nope.json") == []

def test_save_and_load_round_trip(tmp_path):
path = tmp_path / "todos.json"
todos = [{"id": 1, "description": "buy milk", "done": False}]
todo.save_todos(todos, path)
assert todo.load_todos(path) == todos

def test_load_corrupt_json_returns_empty_list(tmp_path):
path = tmp_path / "todos.json"
path.write_text("{not json", encoding="utf-8")
assert todo.load_todos(path) == []

def test_load_non_list_json_returns_empty_list(tmp_path):
path = tmp_path / "todos.json"
path.write_text('{"key": "value"}', encoding="utf-8")
assert todo.load_todos(path) == []

def test_add_todo_assigns_incrementing_ids(tmp_path):
path = tmp_path / "todos.json"
first = todo.add_todo("first task", path)
second = todo.add_todo("second task", path)
assert first == 1
assert second == 2
todos = todo.load_todos(path)
assert todos == [
{"id": 1, "description": "first task", "done": False},
{"id": 2, "description": "second task", "done": False},
]

def test_mark_done(tmp_path):
path = tmp_path / "todos.json"
todo.add_todo("task", path)
assert todo.mark_done(1, path) is True
assert todo.load_todos(path)[0]["done"] is True

def test_mark_done_missing_id_returns_false(tmp_path):
path = tmp_path / "todos.json"
assert todo.mark_done(99, path) is False

def test_delete_todo(tmp_path):
path = tmp_path / "todos.json"
todo.add_todo("keep", path)
todo.add_todo("remove", path)
assert todo.delete_todo(2, path) is True
assert [t["description"] for t in todo.load_todos(path)] == ["keep"]

def test_delete_missing_id_returns_false(tmp_path):
path = tmp_path / "todos.json"
assert todo.delete_todo(99, path) is False

def test_main_add_and_list(tmp_path, capsys):
path = tmp_path / "todos.json"
assert todo.main(["add", "buy", "milk", "-f", str(path)]) == 0
assert todo.main(["list", "-f", str(path)]) == 0
out = capsys.readouterr().out
assert "buy milk" in out
assert "Added todo" in out

def test_list_shows_done_status(tmp_path, capsys):
path = tmp_path / "todos.json"
todo.add_todo("pending", path)
todo.add_todo("finished", path)
todo.mark_done(2, path)
todo.main(["list", "-f", str(path)])
out = capsys.readouterr().out
assert "[ ] 1: pending" in out
assert "[x] 2: finished" in out

def test_main_done_and_delete(tmp_path, capsys):
path = tmp_path / "todos.json"
todo.main(["add", "task", "-f", str(path)])
assert todo.main(["done", "1", "-f", str(path)]) == 0
assert todo.main(["delete", "1", "-f", str(path)]) == 0
assert todo.main(["list", "-f", str(path)]) == 0
assert "No todos." in capsys.readouterr().out

def test_main_missing_todo_returns_error(tmp_path, capsys):
path = tmp_path / "todos.json"
assert todo.main(["done", "1", "-f", str(path)]) == 1
assert "No todo with id 1" in capsys.readouterr().err
assert todo.main(["delete", "1", "-f", str(path)]) == 1
assert "No todo with id 1" in capsys.readouterr().err

Codex pytest 复验输出

============================= test session starts ==============================
platform linux — Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 — /tmp/bench-venv/bin/python
cachedir: .pytest_cache
rootdir: /home/agent/coding-benchmark/test-codex
collecting … collected 13 items

test_todo.py::test_load_missing_file_returns_empty_list PASSED [ 7%]
test_todo.py::test_save_and_load_round_trip PASSED [ 15%]
test_todo.py::test_load_corrupt_json_returns_empty_list PASSED [ 23%]
test_todo.py::test_load_non_list_json_returns_empty_list PASSED [ 30%]
test_todo.py::test_add_todo_assigns_incrementing_ids PASSED [ 38%]
test_todo.py::test_mark_done PASSED [ 46%]
test_todo.py::test_mark_done_missing_id_returns_false PASSED [ 53%]
test_todo.py::test_delete_todo PASSED [ 61%]
test_todo.py::test_delete_missing_id_returns_false PASSED [ 69%]
test_todo.py::test_main_add_and_list PASSED [ 76%]
test_todo.py::test_list_shows_done_status PASSED [ 84%]
test_todo.py::test_main_done_and_delete PASSED [ 92%]
test_todo.py::test_main_missing_todo_returns_error PASSED [100%]

============================== 13 passed in 0.10s ==============================

Cline v3.0.49 — 24/24 (手动修复后)

Cline 生成的 todo.py (4751 bytes)

#!/usr/bin/env python3
"""A simple command-line todo app with JSON persistence.

Usage:
python todo.py add "Buy milk"
python todo.py list
python todo.py done 1
python todo.py delete 1

Tasks are stored in a JSON file (todo.json by default). Set the TODO_FILE
environment variable to use a different data file.
"""

import argparse
import json
import os
import sys

DEFAULT_DATA_FILE = "todo.json"

class TodoList:
"""A todo list backed by a JSON file.

Each task is a dict: {"id": int, "description": str, "done": bool}.
"""

def __init__(self, data_file=DEFAULT_DATA_FILE):
self.data_file = data_file
self.tasks = []
self._max_id = 0
self.load()

def load(self):
"""Load tasks from the JSON data file.

A missing or corrupt file simply yields an empty list so the app
never crashes on startup.
"""
try:
with open(self.data_file, "r", encoding="utf-8") as fh:
self.tasks = json.load(fh)
except (FileNotFoundError, json.JSONDecodeError, OSError):
self.tasks = []
self._max_id = max((task["id"] for task in self.tasks), default=0)

def save(self):
"""Persist tasks to the JSON data file."""
with open(self.data_file, "w", encoding="utf-8") as fh:
json.dump(self.tasks, fh, indent=2)

def _next_id(self):
"""Return the next task id: never reuse an id, even after deletion."""
self._max_id += 1
return self._max_id

def add(self, description):
"""Add a new task and return it."""
task = {"id": self._next_id(), "description": description, "done": False}
self.tasks.append(task)
self.save()
return task

def get(self, task_id):
"""Return the task with the given id, or None if it does not exist."""
for task in self.tasks:
if task["id"] == task_id:
return task
return None

def list_tasks(self):
"""Return a copy of all tasks in insertion order."""
return list(self.tasks)

def done(self, task_id):
"""Mark the task with the given id as done and return it.

Raises KeyError if no such task exists.
"""
task = self.get(task_id)
if task is None:
raise KeyError(f"no task with id {task_id}")
task["done"] = True
self.save()
return task

def delete(self, task_id):
"""Remove the task with the given id and return it.

Raises KeyError if no such task exists.
"""
for index, task in enumerate(self.tasks):
if task["id"] == task_id:
removed = self.tasks.pop(index)
self.save()
return removed
raise KeyError(f"no task with id {task_id}")

def format_task(task):
"""Render a task as a one-line string, e.g. '[x] 1. Buy milk'."""
marker = "[x]" if task["done"] else "[ ]"
return f"{marker} {task['id']}. {task['description']}"

def main(argv=None):
parser = argparse.ArgumentParser(
prog="todo", description="A simple CLI todo app with JSON persistence."
)
subparsers = parser.add_subparsers(dest="command", required=True)

add_parser = subparsers.add_parser("add", help="add a new task")
add_parser.add_argument("description", help="task description")

subparsers.add_parser("list", help="list all tasks")

done_parser = subparsers.add_parser("done", help="mark a task as done")
done_parser.add_argument("id", type=int, help="task id")

delete_parser = subparsers.add_parser("delete", help="delete a task")
delete_parser.add_argument("id", type=int, help="task id")

args = parser.parse_args(argv)

data_file = os.environ.get("TODO_FILE", DEFAULT_DATA_FILE)
todo = TodoList(data_file)

try:
if args.command == "add":
task = todo.add(args.description)
print(f"Added task {task['id']}: {task['description']}")
elif args.command == "list":
tasks = todo.list_tasks()
if not tasks:
print("No tasks yet.")
else:
for task in tasks:
print(format_task(task))
elif args.command == "done":
todo.done(args.id)
print(f"Marked task {args.id} as done.")
elif args.command == "delete":
task = todo.delete(args.id)
print(f"Deleted task {task['id']}: {task['description']}")
except KeyError as exc:
msg = exc.args[0] if exc.args else str(exc)
print(f"Error: {msg}", file=sys.stderr)
return 1

return 0

if __name__ == "__main__":
sys.exit(main())

Cline 生成的 test_todo.py (5061 bytes)

"""Unit tests for the todo CLI app."""

import json

import pytest

import todo

@pytest.fixture
def todo_list(tmp_path):
"""A TodoList backed by a temp JSON file."""
return todo.TodoList(tmp_path / "tasks.json")

@pytest.fixture
def cli(tmp_path, monkeypatch):
"""Point the CLI at a temp data file via the TODO_FILE env var."""
data_file = tmp_path / "tasks.json"
monkeypatch.setenv("TODO_FILE", str(data_file))
return data_file

# — Model: add —

def test_add_creates_task(todo_list):
task = todo_list.add("Buy milk")
assert task == {"id": 1, "description": "Buy milk", "done": False}
assert todo_list.tasks == [task]

def test_add_persists_to_disk(todo_list):
todo_list.add("Buy milk")
assert todo_list.data_file.exists()
saved = json.loads(todo_list.data_file.read_text())
assert saved == [{"id": 1, "description": "Buy milk", "done": False}]

def test_add_increments_ids(todo_list):
first = todo_list.add("First")
second = todo_list.add("Second")
assert first["id"] == 1
assert second["id"] == 2

# — Model: list —

def test_list_empty(todo_list):
assert todo_list.list_tasks() == []

def test_list_returns_all_tasks(todo_list):
todo_list.add("First")
todo_list.add("Second")
tasks = todo_list.list_tasks()
assert [task["description"] for task in tasks] == ["First", "Second"]

def test_list_returns_copy(todo_list):
todo_list.add("First")
listing = todo_list.list_tasks()
listing.clear()
assert len(todo_list.tasks) == 1

# — Model: done —

def test_done_marks_task(todo_list):
todo_list.add("First")
task = todo_list.done(1)
assert task["done"] is True
assert todo_list.get(1)["done"] is True

def test_done_persists(todo_list):
todo_list.add("First")
todo_list.done(1)
saved = json.loads(todo_list.data_file.read_text())
assert saved[0]["done"] is True

def test_done_unknown_id_raises(todo_list):
with pytest.raises(KeyError):
todo_list.done(99)

# — Model: delete —

def test_delete_removes_task(todo_list):
todo_list.add("First")
todo_list.add("Second")
removed = todo_list.delete(1)
assert removed["id"] == 1
assert [t["description"] for t in todo_list.list_tasks()] == ["Second"]

def test_delete_persists(todo_list):
todo_list.add("First")
todo_list.delete(1)
saved = json.loads(todo_list.data_file.read_text())
assert saved == []

def test_delete_unknown_id_raises(todo_list):
with pytest.raises(KeyError):
todo_list.delete(99)

def test_next_id_survives_delete(todo_list):
# Ids are derived from max existing id, so a deleted id is not reused.
todo_list.add("First")
todo_list.delete(1)
task = todo_list.add("Second")
assert task["id"] == 2

# — Model: persistence / loading —

def test_reload_from_disk(tmp_path):
path = tmp_path / "tasks.json"
todo.TodoList(path).add("Persisted")
reloaded = todo.TodoList(path)
assert reloaded.list_tasks() == [
{"id": 1, "description": "Persisted", "done": False}
]

def test_corrupt_file_starts_empty(tmp_path):
path = tmp_path / "tasks.json"
path.write_text("{not valid json")
store = todo.TodoList(path)
assert store.tasks == []

def test_missing_file_starts_empty(tmp_path):
store = todo.TodoList(tmp_path / "nonexistent.json")
assert store.tasks == []

def test_format_task():
assert todo.format_task({"id": 1, "description": "Buy milk", "done": False}) == (
"[ ] 1. Buy milk"
)
assert todo.format_task({"id": 2, "description": "Pay rent", "done": True}) == (
"[x] 2. Pay rent"
)

# — CLI end-to-end —

def test_cli_add(cli, capsys):
assert todo.main(["add", "Buy milk"]) == 0
assert "Added task 1: Buy milk" in capsys.readouterr().out

def test_cli_list_empty(cli, capsys):
assert todo.main(["list"]) == 0
assert "No tasks yet." in capsys.readouterr().out

def test_cli_list_shows_tasks(cli, capsys):
todo.main(["add", "First"])
todo.main(["add", "Second"])
assert todo.main(["list"]) == 0
out = capsys.readouterr().out
assert "[ ] 1. First" in out
assert "[ ] 2. Second" in out

def test_cli_done(cli, capsys):
todo.main(["add", "Task"])
assert todo.main(["done", "1"]) == 0
assert "Marked task 1 as done." in capsys.readouterr().out
todo.main(["list"])
assert "[x] 1. Task" in capsys.readouterr().out

def test_cli_delete(cli, capsys):
todo.main(["add", "Task"])
assert todo.main(["delete", "1"]) == 0
assert "Deleted task 1: Task" in capsys.readouterr().out
todo.main(["list"])
assert "No tasks yet." in capsys.readouterr().out

def test_cli_done_unknown_id_fails(cli, capsys):
assert todo.main(["done", "42"]) == 1
assert "Error: no task with id 42" in capsys.readouterr().err

def test_cli_state_persists_across_invocations(cli, capsys):
todo.main(["add", "Persist me"])
todo.main(["list"])
assert "1. Persist me" in capsys.readouterr().out

Cline pytest 复验输出

e_marks_task PASSED [ 29%]
test_todo.py::test_done_persists PASSED [ 33%]
test_todo.py::test_done_unknown_id_raises PASSED [ 37%]
test_todo.py::test_delete_removes_task PASSED [ 41%]
test_todo.py::test_delete_persists PASSED [ 45%]
test_todo.py::test_delete_unknown_id_raises PASSED [ 50%]
test_todo.py::test_next_id_survives_delete PASSED [ 54%]
test_todo.py::test_reload_from_disk PASSED [ 58%]
test_todo.py::test_corrupt_file_starts_empty PASSED [ 62%]
test_todo.py::test_missing_file_starts_empty PASSED [ 66%]
test_todo.py::test_format_task PASSED [ 70%]
test_todo.py::test_cli_add PASSED [ 75%]
test_todo.py::test_cli_list_empty PASSED [ 79%]
test_todo.py::test_cli_list_shows_tasks PASSED [ 83%]
test_todo.py::test_cli_done PASSED [ 87%]
test_todo.py::test_cli_delete PASSED [ 91%]
test_todo.py::test_cli_done_unknown_id_fails PASSED [ 95%]
test_todo.py::test_cli_state_persists_across_invocations PASSED [100%]

============================== 24 passed in 0.16s ==============================

Kilo Code v7.4.17 — 10/10

Kilo Code 生成的 todo.py (2985 bytes)

#!/usr/bin/env python3
"""A simple CLI todo app with JSON persistence."""

import argparse
import json
import sys
from pathlib import Path

DEFAULT_STORE = Path(__file__).with_name("todos.json")

class Todo:
def __init__(self, text, done=False, store_path=DEFAULT_STORE):
self.text = text
self.is_done = done
self.store_path = Path(store_path)

@property
def store(self):
if not self.store_path.exists():
return []
return json.loads(self.store_path.read_text() or "[]")

@store.setter
def store(self, items):
self.store_path.write_text(json.dumps(items, indent=2))

def _next_id(self, items):
return max((item["id"] for item in items), default=0) + 1

def add(self):
items = self.store
todo_id = self._next_id(items)
items.append({"id": todo_id, "text": self.text, "done": self.is_done})
self.store = items
return todo_id

def list(self):
return self.store

def done(self, todo_id):
items = self.store
for item in items:
if item["id"] == todo_id:
item["done"] = True
self.store = items
return True
return False

def delete(self, todo_id):
items = self.store
new_items = [item for item in items if item["id"] != todo_id]
if len(new_items) == len(items):
return False
self.store = new_items
return True

def main(argv=None):
parser = argparse.ArgumentParser(description="Manage a todo list.")
subparsers = parser.add_subparsers(dest="command", required=True)

add_parser = subparsers.add_parser("add", help="Add a new todo.")
add_parser.add_argument("text", help="The todo text.")

subparsers.add_parser("list", help="List all todos.")

done_parser = subparsers.add_parser("done", help="Mark a todo done.")
done_parser.add_argument("id", type=int, help="The todo id.")

delete_parser = subparsers.add_parser("delete", help="Delete a todo.")
delete_parser.add_argument("id", type=int, help="The todo id.")

args = parser.parse_args(argv)

todo = Todo("")
if args.command == "add":
todo.text = args.text
todo_id = todo.add()
print(f"Added todo {todo_id}")
elif args.command == "list":
for item in todo.list():
status = "x" if item["done"] else " "
print(f"{item['id']} [{status}] {item['text']}")
elif args.command == "done":
if todo.done(args.id):
print(f"Marked todo {args.id} as done")
else:
print(f"Todo {args.id} not found", file=sys.stderr)
return 1
elif args.command == "delete":
if todo.delete(args.id):
print(f"Deleted todo {args.id}")
else:
print(f"Todo {args.id} not found", file=sys.stderr)
return 1
return 0

if __name__ == "__main__":
sys.exit(main())

Kilo Code 生成的 test_todo.py (1569 bytes)

"""Unit tests for todo.py."""

import json

import pytest

from todo import Todo

@pytest.fixture
def todo(tmp_path):
return Todo("", store_path=tmp_path / "todos.json")

def test_add_creates_todo(todo):
todo_id = todo.add()
items = todo.list()
assert len(items) == 1
assert items[0]["id"] == todo_id
assert items[0]["text"] == ""
assert items[0]["done"] is False

def test_add_assigns_incrementing_ids(todo):
first = todo.add()
second = todo.add()
assert second == first + 1

def test_add_persists_to_disk(todo):
todo.add()
raw = todo.store_path.read_text()
assert json.loads(raw)[0]["text"] == ""

def test_list_empty(todo):
assert todo.list() == []

def test_list_returns_all_todos(todo):
todo.text = "first"
todo.add()
todo.text = "second"
todo.add()
assert [item["text"] for item in todo.list()] == ["first", "second"]

def test_done_marks_todo(todo):
todo_id = todo.add()
assert todo.done(todo_id) is True
items = todo.list()
assert items[0]["done"] is True

def test_done_missing_id_returns_false(todo):
assert todo.done(999) is False

def test_delete_removes_todo(todo):
todo_id = todo.add()
assert todo.delete(todo_id) is True
assert todo.list() == []

def test_delete_missing_id_returns_false(todo):
assert todo.delete(999) is False

def test_delete_only_removes_target(todo):
first = todo.add()
todo.delete(first)
assert todo.list() == []
second = todo.add()
assert [item["id"] for item in todo.list()] == [second]

Kilo Code pytest 复验输出

============================= test session starts ==============================
platform linux — Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 — /tmp/bench-venv/bin/python
cachedir: .pytest_cache
rootdir: /home/agent/coding-benchmark/test-kilo
collecting … collected 10 items

test_todo.py::test_add_creates_todo PASSED [ 10%]
test_todo.py::test_add_assigns_incrementing_ids PASSED [ 20%]
test_todo.py::test_add_persists_to_disk PASSED [ 30%]
test_todo.py::test_list_empty PASSED [ 40%]
test_todo.py::test_list_returns_all_todos PASSED [ 50%]
test_todo.py::test_done_marks_todo PASSED [ 60%]
test_todo.py::test_done_missing_id_returns_false PASSED [ 70%]
test_todo.py::test_delete_removes_todo PASSED [ 80%]
test_todo.py::test_delete_missing_id_returns_false PASSED [ 90%]
test_todo.py::test_delete_only_removes_target PASSED [100%]

============================== 10 passed in 0.06s ==============================

Crush vgo 编译版 — 13/13

Crush 生成的 todo.py (3654 bytes)

#!/usr/bin/env python3
"""A minimal CLI todo app with JSON persistence.

Commands:
add <description> Add a new todo
list List all todos (with done status)
done <id> Mark a todo as done
delete <id> Delete a todo
"""

import argparse
import json
import os
import sys

DEFAULT_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "todos.json")

def load_todos(path=DEFAULT_FILE):
"""Load todos from the JSON file, returning an empty list if missing."""
if not os.path.exists(path):
return []
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
except (json.JSONDecodeError, OSError):
return []
if not isinstance(data, list):
return []
return data

def save_todos(todos, path=DEFAULT_FILE):
"""Persist todos to the JSON file."""
with open(path, "w", encoding="utf-8") as f:
json.dump(todos, f, indent=2)

def next_id(todos):
"""Return the next available todo id."""
return max((t.get("id", 0) for t in todos), default=0) + 1

def add_todo(description, path=DEFAULT_FILE):
"""Add a new todo and return it."""
todos = load_todos(path)
todo = {"id": next_id(todos), "description": description, "done": False}
todos.append(todo)
save_todos(todos, path)
return todo

def list_todos(path=DEFAULT_FILE):
"""Return the list of todos."""
return load_todos(path)

def mark_done(todo_id, path=DEFAULT_FILE):
"""Mark a todo as done. Returns the todo, or None if not found."""
todos = load_todos(path)
for todo in todos:
if todo["id"] == todo_id:
todo["done"] = True
save_todos(todos, path)
return todo
return None

def delete_todo(todo_id, path=DEFAULT_FILE):
"""Delete a todo. Returns the deleted todo, or None if not found."""
todos = load_todos(path)
for i, todo in enumerate(todos):
if todo["id"] == todo_id:
removed = todos.pop(i)
save_todos(todos, path)
return removed
return None

def main(argv=None):
parser = argparse.ArgumentParser(prog="todo", description="A simple todo app")
sub = parser.add_subparsers(dest="command", required=True)

sub.add_parser("list", help="list all todos")

p_add = sub.add_parser("add", help="add a todo")
p_add.add_argument("description", help="todo description")

p_done = sub.add_parser("done", help="mark a todo as done")
p_done.add_argument("id", type=int, help="todo id")

p_del = sub.add_parser("delete", help="delete a todo")
p_del.add_argument("id", type=int, help="todo id")

args = parser.parse_args(argv)

if args.command == "add":
todo = add_todo(args.description)
print(f"Added todo #{todo['id']}: {todo['description']}")
elif args.command == "list":
todos = list_todos()
if not todos:
print("No todos.")
return 0
for todo in todos:
status = "[x]" if todo["done"] else "[ ]"
print(f"{status} {todo['id']}: {todo['description']}")
elif args.command == "done":
todo = mark_done(args.id)
if todo is None:
print(f"Todo #{args.id} not found.", file=sys.stderr)
return 1
print(f"Done: {todo['description']}")
elif args.command == "delete":
todo = delete_todo(args.id)
if todo is None:
print(f"Todo #{args.id} not found.", file=sys.stderr)
return 1
print(f"Deleted: {todo['description']}")
return 0

if __name__ == "__main__":
sys.exit(main())

Crush 生成的 test_todo.py (2861 bytes)

"""Unit tests for todo.py."""

import json
import os

import pytest

import todo

@pytest.fixture
def todo_file(tmp_path):
"""Return a path to a temp JSON file for a test."""
return str(tmp_path / "todos.json")

@pytest.fixture
def populated_file(todo_file):
"""Create a temp file pre-populated with two todos."""
data = [
{"id": 1, "description": "buy milk", "done": False},
{"id": 2, "description": "walk dog", "done": True},
]
with open(todo_file, "w", encoding="utf-8") as f:
json.dump(data, f)
return todo_file

def test_load_missing_file(todo_file):
assert todo.load_todos(todo_file) == []

def test_load_invalid_json(todo_file):
with open(todo_file, "w", encoding="utf-8") as f:
f.write("not json{")
assert todo.load_todos(todo_file) == []

def test_load_non_list(todo_file):
with open(todo_file, "w", encoding="utf-8") as f:
json.dump({"wrong": "shape"}, f)
assert todo.load_todos(todo_file) == []

def test_save_and_load_roundtrip(todo_file):
todos = [{"id": 1, "description": "x", "done": False}]
todo.save_todos(todos, todo_file)
assert todo.load_todos(todo_file) == todos

def test_next_id_empty():
assert todo.next_id([]) == 1

def test_next_id_continues(populated_file):
assert todo.next_id(todo.load_todos(populated_file)) == 3

def test_add_todo(todo_file):
added = todo.add_todo("buy milk", todo_file)
assert added == {"id": 1, "description": "buy milk", "done": False}
assert todo.load_todos(todo_file) == [added]