Experiment Loop for MD Home Care

Tracks content changes, measures their impact on traffic and rankings, and decides whether to keep, iterate, or revert. Runs weekly.

CRITICAL: Lag Times for YMYL Content

YMYL content (aged care, disability services) has longer lag times than SaaS content. Do not evaluate changes too early.

Step 1: Weekly Git Scan

Identify all content changes from the past week:

bash
1cd ~/Projects/mdhomecarebuild
2
3# All content changes in last 7 days
4git log --since="7 days ago" --name-only --pretty=format:"%h %s" -- "src/content/**/*.md" "src/content/**/*.mdx"
5
6# Summarize by type
7git log --since="7 days ago" --name-only --pretty=format:"" -- "src/content/blog/*.md" | sort -u | head -20
8git log --since="7 days ago" --name-only --pretty=format:"" -- "src/content/services/*.md" | sort -u | head -20
9git log --since="7 days ago" --name-only --pretty=format:"" -- "src/content/providers/*.md" | sort -u | head -20

Categorize each change:

• New page: Completely new content file
• Major edit: Structural changes (new sections, comparison tables, rewritten H1/H2)
• Minor edit: Small fixes (typos, link updates, frontmatter changes)

Step 2: Baseline Measurement

For each changed page, capture the pre-change baseline. If baseline was not captured before the change, use the previous period as proxy.

GSC Baseline

bash
1cd ~/Projects/mdhomecarebuild
2
3# For each changed page, get keyword data
4python3 src/scripts/advanced_gsc_analyzer.py --page "/services/[slug]"
5python3 src/scripts/advanced_gsc_analyzer.py --page "/blog/[slug]"

Record:

• Top 10 keywords by clicks
• Average position for primary keyword
• Total impressions and clicks (last 7 days)

PostHog Baseline

bash
1# Page traffic
2python3 src/scripts/posthog_analytics.py --page "/services/[slug]" --days 7
3
4# AI referral traffic
5python3 src/scripts/posthog_analytics.py --ai-referrals --days 7

Record:

• Total pageviews (last 7 days)
• AI referral visits to that page
• Traffic sources breakdown

Step 3: Post-Change Measurement

After the evaluation window has passed (see lag times table), measure again.

bash
1# GSC: same page analysis
2python3 src/scripts/advanced_gsc_analyzer.py --page "/services/[slug]"
3
4# PostHog: same page traffic
5python3 src/scripts/posthog_analytics.py --page "/services/[slug]" --days 7
6python3 src/scripts/posthog_analytics.py --ai-referrals --days 7

Step 4: Attribution and Decision

Compare Metrics

For each experiment, calculate:

Decision Framework

KEEP if:

• Organic clicks increased >10%
• OR average position improved by 2+ positions
• OR AI referral visits increased >20%
• OR impressions increased >15% (leading indicator)
• AND no negative impact on other pages (cannibalization check)

ITERATE if:

• Mixed signals (some metrics up, some flat)
• OR small positive movement (<10% clicks) that suggests potential
• OR evaluation window has not fully elapsed
• Action: Make targeted refinements and re-evaluate after another cycle

REVERT if:

• Organic clicks decreased >15%
• AND average position dropped by 3+ positions
• AND no compensating AI referral increase
• Action: Restore previous version via git, document what went wrong

WAIT if:

• Change is too recent (within lag window)
• Action: Re-evaluate next week

Step 5: Log to Playbook

Record every experiment result in PLAYBOOK.md:

markdown
1## [Date] - [Experiment Name]
2
3**Category:** [Service page optimization / Location page / Blog post / Comparison / Trust signal / FAQ]
4**Page:** [URL path]
5**Change:** [Brief description of what was changed]
6**Hypothesis:** [What we expected to happen]
7
8**Baseline (pre-change):**
9- Organic clicks (7d): X
10- Avg position (primary KW): X
11- AI referrals (7d): X
12
13**Result (post-change, measured [date]):**
14- Organic clicks (7d): Y (+/- %)
15- Avg position (primary KW): Y (+/- positions)
16- AI referrals (7d): Y (+/- %)
17
18**Decision:** KEEP / ITERATE / REVERT / WAIT
19**Lesson:** [What we learned]

Experiment Categories

Service Page Optimizations

• Adding comparison tables
• Rewriting H1/byline
• Adding trust signal sections
• Expanding FAQ sections
• Adding AI differentiation paragraphs

Location Page Creation

• New suburb-specific service pages
• Measure: local keyword rankings, location-specific traffic

Blog Post Publishing

• New informational content
• Template/download posts
• Provider comparison posts
• Measure: organic clicks, keyword coverage expansion

Provider Comparison Additions

• New comparison tables on existing pages
• New "vs" blog posts
• Measure: comparison keyword rankings, AI referral traffic

Trust Signal Enhancements

• Adding registration numbers
• Adding testimonials
• Adding clinical governance sections
• Measure: overall page authority signals, position changes

FAQ Additions

• New FAQ sections
• Expanding existing FAQs with PAA questions
• Measure: featured snippet captures, PAA appearances

Weekly Routine

Every week:

1. Run git scan (Step 1)
2. For changes past their evaluation window, measure results (Step 3)
3. Make keep/iterate/revert decisions (Step 4)
4. Log results to PLAYBOOK.md (Step 5)
5. Capture baselines for new changes (Step 2)

Usage

/experiment-loop

Runs the full weekly cycle: scan, measure, decide, log.

/experiment-loop --check "/services/sil-services"

Check status of a specific page experiment.