Understanding walkerOS Flow

Overview

walkerOS follows a Source → Collector → Destination(s) architecture for composable, modular event processing.

Core principle: Separation of concerns. Each component has one job. Components are composable and replaceable.

The Flow Pattern

Sources → [Pre-Transformers] → Collector → [Post-Transformers] → Destinations
(Capture)  (source.next)    (Processing) (dest.before)       (Delivery)

- Browser DOM              - Validation   - Validation        - Google Analytics
- DataLayer                - Enrichment   - Enrichment        - Meta Pixel
- Server HTTP              - Redaction    - Consent check     - Custom API
- Cloud Functions                         - Routing           - Data Warehouse

Key Concepts

Composability

A Flow combines components. You can:

• Use multiple sources feeding one collector
• Route events to multiple destinations
• Swap components without changing others

The Flow Type

See packages/core/src/types/flow.ts for the canonical interface.

typescript
1// Conceptual structure (see source for full type)
2interface Flow {
3  sources?: Record<string, Source>;
4  transformers?: Record<string, Transformer>;
5  destinations?: Record<string, Destination>;
6  collector?: Collector.InitConfig;
7}

Universal Push Interface

All components communicate via push functions:

The elb() function is an alias for collector.push - used for component wiring.

startFlow Helper

See packages/collector/src/flow.ts for the startFlow function.

typescript
1import { startFlow } from '@walkeros/collector';
2
3const { collector, elb } = await startFlow({
4  sources: {
5    /* ... */
6  },
7  transformers: {
8    /* ... */
9  },
10  destinations: {
11    /* ... */
12  },
13});

Separation of Concerns

Transformer Chains

Transformers run at two points in the pipeline, configured via next and before:

Pre-Collector Chain

Runs after source captures event, before collector processing:

Bundled mode (flow.json):

json
1{
2  "sources": {
3    "browser": {
4      "package": "@walkeros/web-source-browser",
5      "next": "validate"
6    }
7  },
8  "transformers": {
9    "validate": {
10      "package": "@walkeros/transformer-validator",
11      "next": "enrich"
12    },
13    "enrich": {
14      "package": "@walkeros/transformer-enricher"
15    }
16  }
17}

Integrated mode (TypeScript):

typescript
1sources: {
2  browser: {
3    code: sourceBrowser,
4    next: 'validate'
5  }
6},
7transformers: {
8  validate: {
9    code: transformerValidator,
10    config: { next: 'enrich' }
11  },
12  enrich: {
13    code: transformerEnrich
14  }
15}

Note: In flow.json, next is at the reference level. The CLI bundler automatically moves it into config.next for runtime - you don't need to handle this yourself.

Post-Collector Chain

Runs after collector enrichment, before destination receives event:

Bundled mode (flow.json):

json
1{
2  "destinations": {
3    "gtag": {
4      "package": "@walkeros/web-destination-gtag",
5      "before": "redact"
6    }
7  },
8  "transformers": {
9    "redact": {
10      "package": "@walkeros/transformer-redact"
11    }
12  }
13}

Integrated mode (TypeScript):

typescript
1destinations: {
2  gtag: {
3    code: destinationGtag,
4    before: 'redact'
5  }
6},
7transformers: {
8  redact: {
9    code: transformerRedact
10  }
11}

Chain Resolution

• source.next → starts pre-collector chain
• transformer.next (flow.json) or transformer.config.next (runtime) → links transformers
• destination.before → starts post-collector chain per destination

Related Skills

• walkeros-understanding-events - Event model
• walkeros-understanding-sources - Source interface
• walkeros-understanding-destinations - Destination interface

Package READMEs:

• packages/collector/README.md - Collector details

Source Files:

• packages/collector/src/flow.ts - startFlow implementation

Documentation:

• Website: Flow - Flow concept
• Website: Collector - Collector docs