Taiga API

Query the hosted Taiga evaluation platform API for job results, transcripts, and problem runs.

IMPORTANT: Use Python, Not Shell

Always use Python for Taiga API requests. Shell has env var + pipe bugs that strip cookie values.

Python helper to load cookie:

python
1def get_cookie():
2    with open('/home/atondwal/dmodel/ant/taiga-worktree/.env') as f:
3        for line in f:
4            if line.startswith('TAIGA_IAP_COOKIE='):
5                return line.split('=', 1)[1].strip().strip('"')

IMPORTANT: Always Use Opus 4.5

When submitting jobs, ALWAYS use claude-opus-4-5-20251101 as the model. Never use Sonnet or other models unless explicitly requested.

Authentication

Cookie stored in ~/dmodel/ant/taiga-worktree/.env. Uses __Host- prefix (session-only). If auth fails, ask user to refresh from browser DevTools → Network → copy Cookie header.

Making Requests

python
1import urllib.request, json
2
3def taiga_get(endpoint):
4    cookie = get_cookie()  # see helper above
5    req = urllib.request.Request(f"https://taiga.ant.dev/api{endpoint}")
6    req.add_header('Cookie', cookie)
7    return json.loads(urllib.request.urlopen(req).read())
8
9# Example: get job problems
10data = taiga_get(f"/jobs/{job_id}/problems")

API Reference

Full docs at: https://taiga.ant.dev/api/docs

Jobs (Most Common)

Transcripts

Problem Runs

Environments

Problems

Docker Images

Common Workflows

Get Passrates for a Job

python
1job_id = "3c300cca-707a-4e92-ac71-5688165f9ae1"  # from URL ?id= param
2data = taiga_get(f"/jobs/{job_id}/problems")
3for r in data:
4    print(f"{r['problem_id']}: {r['final_score']}")

Aggregate Passrates

python
1from collections import defaultdict
2
3job_id = "YOUR_JOB_ID"
4data = taiga_get(f"/jobs/{job_id}/problems")
5
6problems = defaultdict(list)
7for r in data:
8    problems[r['problem_id']].append(r['final_score'])
9
10total_pass = total_runs = 0
11for pid, scores in sorted(problems.items()):
12    passed = sum(1 for s in scores if s == 1.0)
13    total = len(scores)
14    total_pass += passed
15    total_runs += total
16    print(f"{pid}: {passed}/{total} ({100*passed/total:.0f}%)")
17
18print(f"\nOverall: {total_pass}/{total_runs} ({100*total_pass/total_runs:.1f}%)")

Get Transcript

python
1problem_run_id = "118ed21a-9864-4c8c-b34b-d92428f1c22a"
2transcript = taiga_get(f"/transcript/{problem_run_id}")

List Jobs for Environment

python
1env_id = "8e646c11-1461-44a4-9e8d-e3800a02ba07"
2jobs = taiga_get(f"/jobs?environment_id={env_id}")
3for j in jobs:
4    print(f"{j['id']}: {j['status']}")

Check Job Status

python
1job = taiga_get(f"/jobs/{job_id}")
2print(f"Status: {job['status']}, Completed: {job.get('completed_count')}")

Create a Job

python
1import urllib.request, json
2
3with open('problems-metadata.json') as f:
4    problems = json.load(f)
5
6payload = {
7    "name": "my-job-name",
8    "problems_metadata": problems,
9    "n_attempts_per_problem": 10,
10    "api_model_name": "claude-opus-4-5-20251101"  # ALWAYS use Opus 4.5
11}
12
13cookie = get_cookie()
14req = urllib.request.Request(
15    "https://taiga.ant.dev/api/jobs",
16    data=json.dumps(payload).encode(),
17    headers={"Cookie": cookie, "Content-Type": "application/json"}
18)
19resp = json.loads(urllib.request.urlopen(req).read())
20print(f"Job ID: {resp.get('job_id')}")

Response Schemas

Problem Run

json
1{
2  "id": "118ed21a-...",
3  "problem_id": "sort-unique",
4  "attempt_number": 1,
5  "final_score": 1.0,
6  "status": "completed",
7  "subscores": {"matched_solution": 1.0},
8  "weights": {"matched_solution": 1.0},
9  "execution_time_ms": 467000,
10  "total_tokens": 34205
11}

Job

json
1{
2  "id": "3c300cca-...",
3  "status": "completed",
4  "environment_id": "8e646c11-...",
5  "api_model_name": "claude-opus-4-5-20251101",
6  "created_at": "2025-11-24T17:46:30Z"
7}

URL Patterns

From Taiga web UI URLs:

• Job page: https://taiga.ant.dev/job?id={job_id}&environmentId={env_id}
• Transcripts: https://taiga.ant.dev/transcripts?id={job_id}&problemId={problem_id}&...

The id parameter in URLs is the job_id.

Tips

1. Use Python with urllib.request - avoid shell due to env var bugs
2. Cookie expires periodically - refresh from browser if auth fails
3. /jobs/{id}/problems is the main endpoint for checking pass rates
4. For streaming large responses, use the /stream variants