Visual QA Pool

Orchestrate visual testing agents for a ticket with zero race conditions. Creates a visual run, populates entries, enumerates pending ones (filtering already-claimed/done), assigns each to a sub-agent by name using round-robin, then fires one claude process per visual test case. Each sub-agent claims its own specific test case via visual-run claim-next --name "TITLE" — no two agents get the same title, so no races.

Step 1 — Check and Prepare Config

bash
1TICKET_ID="<TICKET-ID>"
2MODE="${1:-baseline}"   # passed by calling agent: baseline or verification
3
4AGENTS=$(noob-tester qa-pool list --ticket "$TICKET_ID" --json)
5AGENT_COUNT=$(echo "$AGENTS" | jq 'length')
6
7if [ "$AGENT_COUNT" -eq 0 ]; then
8  echo "No QA pool agents configured for $TICKET_ID."
9  exit 1
10fi

If config found and user asked to update something:

bash
1ENTRY_ID=$(echo "$AGENTS" | jq -r '.[] | select(.agent_path | contains("<partial-path>")) | .id' | head -1)
2
3noob-tester qa-pool update "$ENTRY_ID" --target <new-target>       # if target changed
4noob-tester qa-pool update "$ENTRY_ID" --role <new-role>           # if role changed
5noob-tester qa-pool update "$ENTRY_ID" --agent <new-path>          # if agent changed
6noob-tester qa-pool update "$ENTRY_ID" --file <new-file>           # if file changed
7noob-tester qa-pool update "$ENTRY_ID" --launch-dir <new-dir>     # if launch dir changed
8
9AGENTS=$(noob-tester qa-pool list --ticket "$TICKET_ID" --json)
10AGENT_COUNT=$(echo "$AGENTS" | jq 'length')

If no config found, register it first:

bash
1noob-tester qa-pool add \
2  --ticket "$TICKET_ID" \
3  --agent <agent-path> \
4  --target <target-name> \
5  --role <role> \
6  --file <file-path> \
7  --launch-dir <directory>   # optional — defaults to pwd if omitted
8
9AGENTS=$(noob-tester qa-pool list --ticket "$TICKET_ID" --json)
10AGENT_COUNT=$(echo "$AGENTS" | jq 'length')

Step 2 — Get Latest Visual Run (by mode) and Enumerate Pending Entries

MAX_SPAWNS controls how many agents are launched. Default is 5. Override if user specifies a number.

bash
1MAX_SPAWNS=5   # default — override if user specified a number
2
3# Get the latest visual run for this ticket in the specified mode
4ALL_RUNS=$(noob-tester visual-run list --ticket "$TICKET_ID" --json)
5RECENT_RUN=$(echo "$ALL_RUNS" | jq --arg mode "$MODE" '.[] | select(.mode == $mode) | . // empty' | head -1)
6
7if [ -z "$RECENT_RUN" ] || [ "$RECENT_RUN" = "null" ]; then
8  echo "No existing visual run found for $TICKET_ID in mode '$MODE'. Create one first with noob-tester visual-run start."
9  exit 1
10fi
11
12VISUAL_RUN_ID=$(echo "$RECENT_RUN" | jq -r '.id')
13
14echo "Using visual run: $VISUAL_RUN_ID (mode: $MODE)"
15
16# Fetch all entries in the visual run
17RUN_ENTRIES=$(noob-tester visual-run get "$VISUAL_RUN_ID" --entries | jq '.entries // []')
18
19# Filter to pending entries only (exclude running, passed, failed, skipped)
20PENDING=$(echo "$RUN_ENTRIES" | jq '
21  [.[] | select(.status == "pending")]
22  | .[:'"$MAX_SPAWNS"']
23')
24
25PENDING_COUNT=$(echo "$PENDING" | jq 'length')
26
27if [ "$PENDING_COUNT" -eq 0 ]; then
28  echo "No pending visual test case entries in run $VISUAL_RUN_ID. Nothing to run."
29  exit 0
30fi
31
32echo "Dispatching $PENDING_COUNT pending visual test case entries (max: $MAX_SPAWNS)."

Step 3 — Assign Pending Entries Round-Robin and Pre-Claim

Distribute pending visual run entries round-robin across agent configs, then pre-claim each one.

bash
1# Build an array of agent configs for round-robin
2AGENT_PATHS=($(echo "$AGENTS" | jq -r '.[].agent_path'))
3AGENT_TARGETS=($(echo "$AGENTS" | jq -r '.[].target // ""'))
4AGENT_ROLES=($(echo "$AGENTS" | jq -r '.[].role // "default"'))
5AGENT_FILES=($(echo "$AGENTS" | jq -r '.[].file // ""'))
6AGENT_DIRS=($(echo "$AGENTS" | jq -r '.[].launch_dir // ""'))
7
8i=0
9LAUNCHES=()
10
11# Assign and pre-claim visual test case entries
12echo "$PENDING" | jq -c '.[]' | while read -r ENTRY; do
13  ENTRY_ID=$(echo "$ENTRY" | jq -r '.id')
14  IDX=$(( i % AGENT_COUNT ))
15
16  AGENT_PATH="${AGENT_PATHS[$IDX]}"
17  TARGET="${AGENT_TARGETS[$IDX]}"
18  ROLE="${AGENT_ROLES[$IDX]}"
19  FILE="${AGENT_FILES[$IDX]}"
20  DIR="${AGENT_DIRS[$IDX]}"
21
22  # ← PRE-CLAIM: Transition entry to claimed status
23  CLAIM_OUTPUT=$(noob-tester visual-run entry-claim --run "$VISUAL_RUN_ID" --entry "$ENTRY_ID" 2>/dev/null)
24  CLAIMED=$(echo "$CLAIM_OUTPUT" | jq -r '.claimed // false')
25
26  if [ "$CLAIMED" != "true" ]; then
27    echo "  Warning: Could not claim entry '$ENTRY_ID' — skipping"
28    i=$(( i + 1 ))
29    continue
30  fi
31
32  # Save claimed data to unique file for this agent
33  CLAIM_FILE="/tmp/pool-visual-claim-${i}.json"
34  echo "$CLAIM_OUTPUT" > "$CLAIM_FILE"
35
36  # Build invocation for visual test with claim file
37  INVOCATION="run visual $MODE test for ticket $TICKET_ID, run $VISUAL_RUN_ID entry $ENTRY_ID with agent @${AGENT_PATH}"
38  [ -n "$TARGET" ] && INVOCATION="$INVOCATION with target $TARGET"
39  [ -n "$ROLE" ] && [ "$ROLE" != "default" ] && INVOCATION="$INVOCATION and role $ROLE"
40  [ -n "$FILE" ] && INVOCATION="$INVOCATION and file $FILE"
41  INVOCATION="$INVOCATION and use claimed entry from $CLAIM_FILE"
42
43  echo "$DIR|$AGENT_PATH|$INVOCATION" >> /tmp/pool-visual-launches.txt
44  i=$(( i + 1 ))
45done
46
47# Read all launches
48if [ -f /tmp/pool-visual-launches.txt ]; then
49  mapfile -t LAUNCHES < /tmp/pool-visual-launches.txt
50  rm -f /tmp/pool-visual-launches.txt
51fi
52
53echo "Prepared ${#LAUNCHES[@]} agent invocations (round-robin assigned and pre-claimed)."

Step 4 — Launch Sub-Agents (Fire and Forget)

Each agent entry has its own launch_dir. The cd happens per-spawn in a subshell so agents can launch from different directories. Each spawn is recorded for tracking and management.

bash
1for LAUNCH in "${LAUNCHES[@]}"; do
2  DIR="${LAUNCH%%|*}"
3  REST="${LAUNCH#*|}"
4  AGENT_PATH="${REST%%|*}"
5  INVOCATION="${REST#*|}"
6
7  echo "→ Spawning @${AGENT_PATH} for: $(echo "$INVOCATION" | grep -o 'Test case: "[^"]*"')"
8
9  if [ -n "$DIR" ] && [ -d "$DIR" ]; then
10    (cd "$DIR" && claude -p "$INVOCATION" --agent "@${AGENT_PATH}") &
11  else
12    [ -n "$DIR" ] && echo "  Warning: launch_dir '$DIR' not found, using current directory"
13    claude -p "$INVOCATION" --agent "@${AGENT_PATH}" &
14  fi
15
16  # Record the spawn for tracking
17  AGENT_PID=$!
18  noob-tester pool-spawns record --ticket "$TICKET_ID" --agent "@${AGENT_PATH}" --pid $AGENT_PID --type visual-pool > /dev/null 2>&1
19done
20
21echo "All ${#LAUNCHES[@]} visual test agents spawned. Monitor at http://localhost:4040 → Visual Runs"
22echo "Manage spawned agents: noob-tester pool-spawns list --ticket $TICKET_ID --active"

Do not call wait — return to the user immediately after spawning. The agents run in the background and record their results to the database as they finish. Spawned agent tracking allows you to view and kill agents via the pool menu in the dashboard.

Step 5 — Report

Tell the user:

• Visual run ID and mode (baseline or verification)
• How many pending visual test case entries were dispatched
• How many agents were launched (capped at MAX_SPAWNS)
• Which agent configs were used (round-robin distribution)
• Whether config was pre-existing or newly registered
• Whether any fields were updated before running
• Dashboard URL to monitor: http://localhost:4040 → Visual Runs

Notes

• Reuse existing visual run — Step 2 retrieves the latest visual run for the ticket in the specified mode (baseline or verification, created via noob-tester visual-run start). If no run exists in that mode, the skill exits and asks the user to create one first.
• Mode parameter — passed by the calling agent (e.g., --mode baseline or --mode verification). The mode determines which visual run to use and is stored on the visual_runs table.
• Pending entries only — only entries with status pending are dispatched. Entries with status running, passed, failed, or skipped are skipped.
• Pre-claim atomic — each agent is assigned one pending entry via round-robin. The entry is atomically claimed (status → claimed) before the agent is launched, preventing race conditions.
• Claim file format — saved to /tmp/pool-visual-claim-${i}.json and contains the full entry data including visual_run_id, entry_id, test case details, and visual_steps config. Agents read this via the CLAIM_FILE path passed in invocation.
• Round-robin — distributes pending entries evenly across agent configs. With 2 configs and 6 pending entries: config[0] gets entries 0,2,4 — config[1] gets 1,3,5.
• Agent path — stored without @ in the DB; prepend @ in the claude invocation.
• Target — a named reference in the targets table resolved at runtime by the sub-agent (not a raw URL).
• Role — selects which credential set to inject from the secrets table for the given target.
• Missing agent file — if a .md file doesn't exist on disk, warn the user and skip that config entry.
• Mode semantics — baseline captures reference screenshots; verification captures + diffs against baseline. A baseline run must complete before verification can run.
• MAX_SPAWNS — caps how many agents are launched. Default 5. Remaining pending entries stay unclaimed for a subsequent /noob-visual-pool invocation.