EDD (Eval-Driven Development) Framework v2.64

Eval-Driven Development is a quality-first development pattern that enforces define-before-implement workflow with structured evaluations.

v2.88 Key Changes (MODEL-AGNOSTIC)

• Model-agnostic: Uses model configured in ~/.claude/settings.json or CLI/env vars
• No flags required: Works with the configured default model
• Flexible: Works with GLM-5, Claude, Minimax, or any configured model
• Settings-driven: Model selection via ANTHROPIC_DEFAULT_*_MODEL env vars

What is EDD?

EDD provides a systematic approach to software development with three phases:

1. DEFINE - Create structured eval specifications using TEMPLATE.md
2. IMPLEMENT - Build features according to eval definitions
3. VERIFY - Validate implementation against eval criteria

Check Types

Usage

bash
1# Invoke EDD workflow
2/edd "Define memory-search feature"
3
4# CLI script (if available)
5ralph edd define memory-search
6ralph edd check memory-search

Components

• TEMPLATE.md: Template for creating eval definitions
• edd.sh: CLI script for eval management
• /edd skill: Skill invocation from Claude Code
• ~/.claude/evals/: Directory for eval definitions

Template Structure

Each eval definition includes:

1. Capability Checks (CC-) - What the feature can do
2. Behavior Checks (BC-) - How the feature behaves
3. Non-Functional Checks (NFC-) - Performance, security, etc.
4. Implementation Notes - Technical guidance
5. Verification Evidence - Test results

Example: memory-search.md

markdown
1# Memory Search Eval
2
3**Status**: DRAFT
4**Created**: 2026-01-30
5
6## Capability Checks
7- [ ] CC-1: Search across semantic memory
8- [ ] CC-2: Support filtering by type
9
10## Behavior Checks
11- [ ] BC-1: Returns ranked results
12- [ ] BC-2: Handles empty queries gracefully
13
14## Non-Functional Checks
15- [ ] NFC-1: Search completes in <2s
16- [ ] NFC-2: Memory usage <100MB
17
18## Implementation Notes
19- Use parallel search for performance
20- Cache frequent queries
21
22## Verification Evidence
23- Test results attached

Integration with Orchestrator

EDD integrates with the orchestrator workflow to ensure quality-first development:

1. Clarify phase - Define evals
2. Plan phase - Review eval requirements
3. Implement phase - Build to eval specs
4. Validate phase - Verify against evals

Swarm Mode Integration (v2.81.1)

EDD framework now supports swarm mode for parallel evaluation across multiple check types.

Auto-Spawn Configuration

When invoked via /edd, the framework automatically spawns a specialized evaluation team:

yaml
1Task:
2  subagent_type: "general-purpose"
3  model: "sonnet"
4  team_name: "edd-evaluation-team"
5  name: "edd-coordinator"
6  mode: "delegate"
7  run_in_background: true
8  prompt: |
9    Execute Eval-Driven Development workflow for: $ARGUMENTS
10
11    EDD Pattern:
12    1. DEFINE - Create structured eval specifications
13    2. DISTRIBUTE - Assign check types to specialists
14    3. VERIFY - Validate against eval criteria
15    4. CONSOLIDATE - Merge findings from all evaluators

Team Composition

Swarm Mode Workflow

User invokes: /edd "Define memory-search feature"

1. Team "edd-evaluation-team" created
2. Coordinator (edd-coordinator) receives task
3. 3 Teammates spawned with check-type specializations
4. Eval definition distributed:
   - Teammate 1 → Capability Checks (CC-)
   - Teammate 2 → Behavior Checks (BC-)
   - Teammate 3 → Non-Functional Checks (NFC-)
5. Teammates work in parallel (background execution)
6. Coordinator monitors progress and gathers results
7. Findings consolidated into single eval specification
8. Final eval document returned

Parallel Evaluation Pattern

Each teammate focuses on their check type:

yaml
1# Teammate 1: Capability Checks
2CC-1: Feature can perform X
3CC-2: Feature supports Y configuration
4CC-3: Feature integrates with Z system
5
6# Teammate 2: Behavior Checks
7BC-1: Feature handles error case A gracefully
8BC-2: Feature returns expected response for B
9BC-3: Feature maintains state across C
10
11# Teammate 3: Non-Functional Checks
12NFC-1: Response time < 100ms
13NFC-2: Memory usage < 50MB
14NFC-3: Security vulnerability scan passes

Communication Between Teammates

Teammates use the built-in mailbox system:

yaml
1# Teammate sends finding to coordinator
2SendMessage:
3  type: "message"
4  recipient: "edd-coordinator"
5  content: "CC-3 defined: Feature integrates with auth system via OAuth2"

Task List Coordination

All teammates share a unified task list:

bash
1# Location: ~/.claude/tasks/edd-evaluation-team/tasks.json
2
3# Example tasks:
4[
5  {"id": "1", "subject": "Define Capability Checks", "owner": "teammate-1"},
6  {"id": "2", "subject": "Define Behavior Checks", "owner": "teammate-2"},
7  {"id": "3", "subject": "Define Non-Functional Checks", "owner": "teammate-3"},
8  {"id": "4", "subject": "Consolidate eval specification", "owner": "edd-coordinator"}
9]

Manual Override

To disable swarm mode:

bash
1/edd "Define feature X" --no-swarm

Output Location

bash
1# Evals saved to ~/.claude/evals/
2ls ~/.claude/evals/
3
4# View last eval
5cat ~/.claude/evals/latest.md

Testing

Test suite: tests/test_v264_edd_framework.bats (33 tests)

Run tests:

bash
1bats tests/test_v264_edd_framework.bats

Swarm Mode Tests

Additional tests for swarm mode integration:

bash
1# Test swarm team creation
2tests/edd/test-swarm-team-creation.sh
3
4# Test parallel evaluation
5tests/edd/test-parallel-evaluation.sh

Status

Current: Framework defined with swarm mode integration (v2.81.1) Note: TEMPLATE.md and evals directory structure ready for use

Version: v2.64 | Status: DRAFT | Tests: 33 passing

Action Reporting (v2.93.0)

Esta skill genera reportes automáticos completos para trazabilidad:

Reporte Automático

Cuando esta skill completa, se genera automáticamente:

1. En la conversación de Claude: Resultados visibles
2. En el repositorio: docs/actions/edd/{timestamp}.md
3. Metadatos JSON: .claude/metadata/actions/edd/{timestamp}.json

Contenido del Reporte

Cada reporte incluye:

• ✅ Summary: Descripción de la tarea ejecutada
• ✅ Execution Details: Duración, iteraciones, archivos modificados
• ✅ Results: Errores encontrados, recomendaciones
• ✅ Next Steps: Próximas acciones sugeridas

Ver Reportes Anteriores

bash
1# Listar todos los reportes de esta skill
2ls -lt docs/actions/edd/
3
4# Ver el reporte más reciente
5cat $(ls -t docs/actions/edd/*.md | head -1)
6
7# Buscar reportes fallidos
8grep -l "Status: FAILED" docs/actions/edd/*.md

Generación Manual (Opcional)

bash
1source .claude/lib/action-report-lib.sh
2start_action_report "edd" "Task description"
3# ... ejecución ...
4complete_action_report "success" "Summary" "Recommendations"

Referencias del Sistema

• Action Reports System - Documentación completa
• action-report-lib.sh - Librería helper
• action-report-generator.sh - Generador