BenchFlow Launch Prep

Prepares a release: docs/labs/code review → CI gate → e2e smoke → CHANGELOG → version bump → PR.

Arguments passed: $ARGUMENTS

Dispatch

No args — show release state: current version (grep '^version' pyproject.toml), [Unreleased] section of CHANGELOG.md, branch and dirty files (git status --short). Recommend next step.

check — run CI gate, report pass/fail, no edits:

bash
1.venv/bin/ruff format --check src tests && .venv/bin/ruff check src tests
2.venv/bin/ty check
3.venv/bin/python -m pytest tests/ -q
4uv lock --check

patch / minor / major — follow Steps 0–6 below.

Step 0 — Pre-flight

1. Version: grep '^version' pyproject.toml, compute new version, ask user to confirm before proceeding. Use their value if they override.
2. Branch: git branch --show-current — if main, stop: "Switch to a feature branch or: git checkout -b release/v<NEW_VERSION>". Any non-main branch is fine.

Step 1 — Docs, labs, and code review (3 subagents in parallel)

A — user-facing docs: Read README.md and all docs/*.md. Evaluate as a first-time external user: complete install instructions, working examples, accurate CLI flags and API signatures, no broken relative links. Flag anything confusing or stale.

B — labs/: For each experiment: README matches code, no broken links, no stale src/benchflow API calls, no notebook tracebacks. Grep for TODO, FIXME.

C — src/ code smell: Grep src/ and tests/ for TODO, FIXME, HACK, XXX (blocker vs. debt). Grep for hardcoded secrets (api_key\s*=\s*["'], etc.). Check __init__.py for debug symbols.

Synthesize: Blockers / Polish / Clean. Stop on blockers.

Step 2 — CI gate

Mirrors .github/workflows/test.yml exactly. Stop on any failure.

bash
1.venv/bin/ruff format src tests       # note: ty check takes no path arg
2.venv/bin/ruff check src tests
3.venv/bin/ty check
4.venv/bin/python -m pytest tests/ -q
5uv lock --check

If ruff format changed files: git diff --name-only, then git add <those files only> — not git add ..

Step 3 — e2e smoke test

bash
1source .env 2>/dev/null || true
2.venv/bin/python -m pytest -m live tests/test_smoke.py -v

If Docker is unavailable, warn and ask to skip or abort — do not skip silently. Expected: test_hello_world_smoke passes with reward > 0 and non-empty trajectory.

Step 4 — CHANGELOG draft

If [Unreleased] is non-empty: show it, ask to confirm.

If empty: generate from git log —

bash
1git log $(git describe --tags --abbrev=0)..HEAD --oneline --no-merges

Map prefixes: feat: → Added, fix: → Fixed, chore:/refactor:/build: → Changed. Omit docs: and test: unless notable. Show draft and wait for user approval before writing anything.

Step 5 — Write CHANGELOG, pyproject.toml, and uv.lock

CHANGELOG: Insert approved entries as ## <NEW_VERSION> — $(date +%Y-%m-%d) immediately after ## [Unreleased], leaving [Unreleased] empty. Use Edit, not Write.

pyproject.toml + uv.lock: Edit only version = "..." under [project] (line 3). Do not touch target-version (ruff) or python-version (ty) — those are Python version pins. Then run uv lock so the editable benchflow package entry in uv.lock matches the new version. Do not edit __init__.py; it reads version from importlib.metadata automatically.

Step 6 — Commit and PR

bash
1git add CHANGELOG.md pyproject.toml uv.lock   # plus any ruff-formatted files from Step 2
2git commit -m "chore: release v<NEW_VERSION>"
3git push -u origin HEAD
4gh pr create --title "chore: release v<NEW_VERSION>" --body "$(cat <<'EOF'
5## Release v<NEW_VERSION>
6
7See CHANGELOG.md for details.
8
9## Checklist
10- [ ] CI gate passes
11- [ ] e2e smoke test passes
12- [ ] CHANGELOG updated
13- [ ] Version bumped in pyproject.toml
14- [ ] uv.lock refreshed
15- [ ] Merge and tag after review
16
17🤖 Generated with [Claude Code](https://claude.com/claude-code)
18EOF
19)"

Return the PR URL.

Step 7 — Post-merge (remind user, do not run)

bash
1git checkout main && git pull
2git tag v<NEW_VERSION> && git push origin v<NEW_VERSION>
3
4# GitHub release — substitute the actual version for <NEW_VERSION> in the awk pattern
5awk '/^## <NEW_VERSION> /{flag=1; next} /^## [0-9]/{flag=0} flag' CHANGELOG.md > /tmp/release_notes.md
6gh release create v<NEW_VERSION> --title "v<NEW_VERSION>" --notes-file /tmp/release_notes.md
7
8# PyPI publish — separate step (requires UV_PUBLISH_TOKEN or ~/.pypirc)
9uv build && uv publish