AGILab runbook (Agent Skill)

Use this skill when you need repo-specific “how we do things” guidance in agilab/: launching Streamlit, regenerating run-config wrappers, debugging installs, or preparing releases.

AGILab working rules (repo policy)

• Use uv for all runs so dependencies resolve in managed envs:
  • uv --preview-features extra-build-dependencies run python …
  • uv --preview-features extra-build-dependencies run streamlit …
• No repo uvx: do not run uvx agilab from this checkout (it will run the published wheel and ignore local changes).
• Process ownership: treat existing terminals, Codex CLI sessions, dev servers, and other long-running processes as user-owned unless this turn started them. Do not use broad termination commands such as pkill, killall, pkill -f, or port-based kill pipelines that can match unrelated sessions. Stop only verified PIDs or tool sessions created for the active task. Do not use Codex CLI control shortcuts such as /stop, Esc interruption, or terminal-close actions to manage background terminals unless the terminal/session was created by this active task and its identity is verified. A status banner that says a background terminal is running is not ownership proof. If a port is busy, choose another port or ask before stopping its owner; do not try to "pause" another Codex CLI session from here.
• High-frequency shortcuts: prefer ./dev <shortcut> for repeated local validation loops. The top shortcuts are impact for impact validation, test for targeted pytest -q, flow for one or more workflow parity profiles, badge for the fresh coverage-badge guard, and docs for docs mirror sync plus stamp verification. impact tells you what must be validated, test runs the narrow pytest slice, flow matches local GitHub workflow profiles, badge catches stale coverage badges, and docs keeps the public mirror aligned. Add --print-only to inspect the expanded commands.
• Run config parity: after editing .idea/runConfigurations/*.xml, regenerate wrappers:
  • uv --preview-features extra-build-dependencies run python tools/generate_runconfig_scripts.py
• Local-first validation: do not jump to GitHub Actions when the same check can be run locally. Reproduce with the narrowest local command first: targeted pytest, isolated coverage commands, py_compile, Sphinx builds, badge generation, or publish dry-runs. Use CI only for GitHub-only behavior such as runner differences, OS/Python matrix coverage, permissions/secrets, or the final publish/deploy step.
• Impact triage first: for non-trivial diffs, run uv --preview-features extra-build-dependencies run python tools/impact_validate.py --staged before editing further or pushing. Use its output to decide:
  • whether the change is app-local or shared-core
  • which targeted tests are required
  • whether installer repros are mandatory
  • whether generated artifacts must be refreshed
• Clone policy: in the PROJECT page, keep two clone classes explicit:
  • temporary clones may share the source .venv by symlink for lightweight local experiments
  • working clones should detach .venv and rerun INSTALL before EXECUTE Do not treat a shared .venv clone as a durable environment, and do not leave renamed projects with .venv symlinks pointing at the old project path.
• Shared core approval gate: do not edit shared core technology without explicit user approval first. This includes src/agilab/core/agi-env, src/agilab/core/agi-node, src/agilab/core/agi-cluster, src/agilab/core/agi-core, shared installer/build/deploy code, and generic helpers reused across apps/pages. Prefer app-local fixes first. If a core edit looks necessary, stop and explain the required files, blast radius, and validation plan before making the change.
• Docs source of truth: edit docs in the sibling repo ../thales_agilab/docs/source (machine path: /Users/agi/PycharmProjects/thales_agilab/docs/source).
• Generated docs in this repo: treat docs/html (including docs/html/_sources) as build output only. Do not hand-edit files in docs/html; always edit source first and regenerate from ../thales_agilab/docs/source.
  • Canonical rebuild command: uv --preview-features extra-build-dependencies run --project ../thales_agilab --group sphinx python -m sphinx -b html ../thales_agilab/docs/source docs/html
• Streamlit API: do not add st.experimental_rerun(); use st.rerun.
• No silent fallbacks: avoid runtime “auto-fallbacks” between API clients or parameter rewrites; fail fast with actionable errors.
• Repository update requests: when the user asks to "update repos", "sync repos", or similar, first show the exact command plan before executing it. The plan should be a fenced bash block with concrete git -C <repo> commands for each targeted checkout. Use the fast path by default: status --porcelain=v1 --untracked-files=no, fetch --prune, rev-list --left-right --count HEAD...@{u}, then merge --ff-only @{u} only for repos that are actually behind. This avoids a redundant fetch from git pull and avoids slow untracked scans. Group independent repo checks and fetches in parallel when the tooling allows it. If a checkout has tracked dirty paths, do not merge it until the dirty paths are reported and the update plan is adjusted.

Git footprint maintenance

• Distinguish clearly between:
  • working-tree footprint (.venv, caches, build artifacts)
  • local Git footprint (.git/objects, .git/lfs)
  • remote repository history size
• If the user asks to reduce .git only, do not touch .venv.
• Measure before acting:
  • du -sh .git .git/objects .git/lfs
  • git count-objects -vH
  • git lfs prune --dry-run
• Prefer the safest local win first:
  • run git lfs prune when the dry-run shows meaningful reclaimable space
  • this reduces local .git/lfs without rewriting history
• For actual history reduction:
  • use git filter-repo, never ad hoc low-level object surgery
  • work in an isolated --mirror clone, not in the main checkout
  • create a backup bundle before rewriting: git bundle create /tmp/<repo>-pre-rewrite.bundle --all
  • preserve any uncommitted local files outside the checkout before realigning branches
  • rewrite only the intended refs/paths; avoid touching gh-pages or unrelated refs unless requested
  • after force-pushing rewritten refs, realign the local checkout to the new origin/* history and run:
    • git reflog expire --expire=now --all
    • git gc --prune=now
• Typical low-value history targets:
  • generated docs/html/**
  • .idea/shelf/**
  • obsolete legacy paths or duplicated archives
• Do not promise a smaller remote repository from local pruning alone. Local LFS prune and local GC only affect the clone on disk.

Common commands (from the runbook matrix)

• Impact triage:
  • cd "$PROJECT_DIR" && uv --preview-features extra-build-dependencies run python tools/impact_validate.py --staged
• Impact triage for planned paths:
  • cd "$PROJECT_DIR" && uv --preview-features extra-build-dependencies run python tools/impact_validate.py --files src/agilab/orchestrate_execute.py test/test_orchestrate_execute.py
• Dev UI: cd "$PROJECT_DIR" && uv --preview-features extra-build-dependencies run streamlit run src/agilab/About_agilab.py -- --openai-api-key "…" --apps-path src/agilab/apps
• Apps-pages smoke: cd "$PROJECT_DIR" && uv --preview-features extra-build-dependencies run python tools/smoke_preinit.py --active-app src/agilab/apps/builtin/flight_project --timeout 20
• Apps-pages regression (AppTest): cd "$PROJECT_DIR" && uv --preview-features extra-build-dependencies run pytest -q test/test_view_maps_network.py
• Publish dry-run (TestPyPI): cd "$PROJECT_DIR" && uv --preview-features extra-build-dependencies run python tools/pypi_publish.py --repo testpypi --dry-run --verbose
• Publish to PyPI: cd "$PROJECT_DIR" && uv --preview-features extra-build-dependencies run python tools/pypi_publish.py --repo pypi --verbose --git-tag --git-commit-version --git-reset-on-failure
  • Real PyPI publishes now require the GitHub CLI (gh) because tools/pypi_publish.py creates or updates the matching GitHub Release after pushing the tag.
  • Add --delete-former-github-release only when the public release page should keep a single current GitHub Release. This deletes the previous GitHub Release entry after the new one is created, but keeps the previous git tag and PyPI files.
  • Add --delete-pypi-release <version> only when a specific old PyPI version must be removed from the selected packages. This uses an exact pypi-cleanup --version-regex match, requires real PyPI web-login credentials in [pypi_cleanup], and cannot use API tokens or trusted publishing credentials.

CI and badge checks

• Prefer local reproduction before rerunning workflows:
  • if a failing step has a local command equivalent, run that first and fix locally
  • only rerun a workflow after the local equivalent is green or when the issue is GitHub-specific
• CI badge is pinned to main:
  • https://github.com/ThalesGroup/agilab/actions/workflows/ci.yml/badge.svg?branch=main
• When checking recent workflow state, prefer the GitHub Actions runs API:
  • uv --preview-features extra-build-dependencies run python - <<'PY' ... https://api.github.com/repos/ThalesGroup/agilab/actions/workflows/ci.yml/runs?per_page=10 ... PY
• Public job logs may not be directly retrievable without auth. Use the runs/jobs API first to identify the failing step, then reproduce that exact command locally.
• For AGILAB specifically, the GitHub README now uses a static, versioned PyPI badge committed under badges/:
  • https://raw.githubusercontent.com/ThalesGroup/agilab/main/badges/pypi-version-agilab.svg
• The live PyPI page can still lag until a new package is actually published; do not infer package publication from the GitHub badge alone.
• After a release, verify all four surfaces separately before trusting version state:
  • PyPI JSON: https://pypi.org/pypi/agilab/json
  • PyPI simple index: https://pypi.org/simple/agilab/
  • GitHub Release: gh release list --limit 5 and gh release view <tag>
  • GitHub static badge: https://raw.githubusercontent.com/ThalesGroup/agilab/main/badges/pypi-version-agilab.svg
• If the version changes, update the static badge and GitHub Release in the same commit series as the version bump so main, PyPI, the README, and release metadata stay aligned.

CI workflow lessons

• The root src/agilab/test step is more stable when run from the source tree instead of the full project environment:
  • PYTHONPATH='src' COVERAGE_FILE=.coverage.agilab uv --preview-features extra-build-dependencies run --no-project --with pytest --with pytest-cov --with toml --with packaging python -m pytest ... --ignore=src/agilab/test/test_model_returns_code.py src/agilab/test
• The integration-only src/agilab/test/test_model_returns_code.py should be ignored in CI collection, not merely deselected by marker, because import-time behavior can still break collection.
• Core package coverage steps are more reliable when each step uses an isolated no-project env with explicit editable core packages and test-only extras, instead of relying on the monorepo root env.
• agi-env tests need:
  • editable ./src/agilab/core/agi-env
  • editable ./src/agilab/core/agi-node
  • sqlalchemy
• Shared core tests (src/agilab/core/test) need:
  • editable ./src/agilab/core/agi-env
  • editable ./src/agilab/core/agi-node
  • editable ./src/agilab/core/agi-cluster
  • editable ./src/agilab/core/agi-core
  • sqlalchemy
  • pytest-asyncio
• Coverage combine/XML generation should use an isolated coverage toolchain too:
  • uv --preview-features extra-build-dependencies run --no-project --with coverage --with pytest-cov python -m coverage ...

Troubleshooting reminders

• Missing import: check both manager and worker pyproject.toml scopes (src/agilab/apps/<app>/pyproject.toml and src/agilab/apps/<app>/src/<app>_worker/pyproject.toml).
• Installer pip issue: run uv --preview-features extra-build-dependencies run python -m ensurepip --upgrade once in the target venv.
• Cluster inventory/status mismatch:
  • If the UI shows a worker as unreachable but ssh <user>@<ip> 'echo ok' works, reproduce the exact non-interactive probe path used by AGILAB before changing UI display code.
  • Check remote PATH and required tools with SSH, not an interactive shell: ssh <user>@<ip> 'printf "path=%s\n" "$PATH"; command -v python3; command -v nvidia-smi || true; uname -a'.
  • Validate the same account can reach the scheduler and shared storage from the worker: ssh <user>@<ip> 'ssh -o BatchMode=yes <scheduler_user>@<scheduler_ip> hostname' and the configured cluster-share mount/read-write sentinel.
  • Treat a display of "+ 1 worker unreachable" as an inventory/probe failure until the exact probe command succeeds; a bare SSH success only proves authentication.
• For a reinstalled cluster node, separate host-key repair from auth repair:
  • host key changed:
    • ssh-keygen -R <ip>
    • ssh-keyscan -H -t ed25519 <ip> >> ~/.ssh/known_hosts
  • user key missing on remote:
    • ssh-copy-id agi@<ip>
    • or recreate ~/.ssh/authorized_keys with 0700 / 0600 permissions
• If cluster mode depends on shared storage, restore the node’s .agilab/.env and remount the share before blaming AGILAB:
  • Linux node example:
    • AGI_CLUSTER_SHARE=/home/agi/clustershare
    • AGI_LOCAL_SHARE=/home/agi/localshare
    • sshfs agi@192.168.20.111:/Users/agi/clustershare /home/agi/clustershare
• For macOS SSHFS workers:
  • command -v brew can miss Intel Homebrew; check /usr/local/Homebrew/bin/brew before assuming Homebrew is absent.
  • Prefer an interactive package install of FUSE-T SSHFS or macFUSE plus SSHFS; the package step may need an admin password.
  • Confirm non-interactive SSH can find sshfs: ssh <user>@<worker> 'command -v sshfs'.
  • If sshfs is under /usr/local/bin but not found over SSH, add /usr/local/bin in the remote user’s ~/.zshenv.
  • Validate reverse SSH too: ssh <user>@<worker> 'ssh -o BatchMode=yes <manager_user>@<manager_ip> hostname'.
  • Fix worker-side known_hosts first, then add the worker public key to the manager account if reverse auth fails.
• After a reinstall, validate both directions explicitly before rerunning installs:
  • ssh agi@<ip> 'echo ok'
  • ssh agi@<ip> 'ssh -o BatchMode=yes agi@<scheduler_ip> hostname'