Push Workflow Design Notes

This document describes the architecture for the segment/journey push workflow in the SDK layer.

Overview

The push workflow handles syncing local YAML files (segments and journeys) to Treasure AI. Previously, most logic was in segment-push-command.ts (~2200 lines). This refactoring moves ~90% of business logic to the SDK layer.

Architecture

Command Layer (~150-200 lines)        SDK Layer (~1500 lines)
┌──────────────────────────────┐     ┌─────────────────────────────────────┐
│ 1. Parse CLI args            │     │ push-workflow.ts                    │
│ 2. Resolve source directory  │────▶│ ├── preparePushSegments()           │
│ 3. Call preparePush()        │     │ │   ├── fetchExistingData()         │
│ 4. Display summary & diffs   │◀────│ │   ├── parseLocalFiles()           │
│ 5. Handle --dry-run          │     │ │   ├── prepareSegmentFiles()       │
│ 6. Confirm with user         │     │ │   ├── prepareJourneyFiles()       │
│ 7. Call executePush()        │────▶│ │   │   └── buildComparisonYaml()   │
│ 8. Display results           │◀────│ │   └── findDeletions()             │
│ 9. Save session context      │     │ └── executePushSegments()           │
└──────────────────────────────┘     │     ├── createFolders()             │
                                     │     ├── pushSegmentsWithActivations()│
                                     │     ├── pushJourneys() (3-step)     │
                                     │     └── deleteResources()           │
                                     └─────────────────────────────────────┘

Key Interfaces

Dependencies

interface PushWorkflowDeps {
  segment: SegmentSDK;
  journey: JourneySDK;
  user: UserSDK;
  connection: ConnectionSDK;
}

Preparation

interface PreparePushOptions {
  sourceDir?: string;
  singleFilePath?: string;
  journeysOnly?: boolean;
  includeDelete?: boolean;
  verbose?: boolean;
  onProgress?: (phase: string, detail?: string) => void;
}

interface PushPrepareResult {
  parentName: string;
  parentId: string;
  sourceDir: string;
  rootFolderId?: string;

  segmentFiles: SegmentPushFile[];
  journeyFiles: JourneyPushFile[];
  foldersToCreate: string[];
  segmentsToDelete: SegmentResponse[];
  journeysToDelete: JourneyListItem[];

  // Pre-built context to avoid re-fetching during execution
  executionContext: PushExecutionContext;

  counts: PushCounts;
  hasChanges: boolean;
}

Execution

interface ExecutePushOptions {
  verbose?: boolean;
  onFolderCreated?: (path: string) => void;
  onSegmentCreated?: (name: string, id: string) => void;
  onSegmentUpdated?: (name: string) => void;
  onSegmentDeleted?: (name: string) => void;
  onActivationCreated?: (segment: string, activation: string) => void;
  onActivationUpdated?: (segment: string, activation: string) => void;
  onActivationDeleted?: (segment: string, activation: string) => void;
  onJourneyCreated?: (name: string, id: string) => void;
  onJourneyUpdated?: (name: string) => void;
}

interface PushExecuteResult {
  segments: { created: number; updated: number; deleted: number };
  journeys: { created: number; updated: number; deleted: number };
  activations: { created: number; updated: number; deleted: number };
  folders: { created: number };
}

File Structure

src/sdk/segment/
├── push-workflow.ts           - Main entry points (preparePush, executePush)
├── push-preparation.ts        - Segment/journey file preparation
├── push-journey-comparison.ts - Complex journey server-to-YAML comparison
├── push-execution.ts          - Execution logic (create/update/delete)
└── ... other existing files

Complex Areas

Audience Kind

An audience is either complete or composable, and the two live in separate tables whose ids are indistinguishable by shape (see sdk/segment/audience-context.ts). Push therefore resolves a ResolvedAudience — id, name, kind, root folder — and carries it on PushExecutionContext. Every read and write that has a composable counterpart branches on audience.kind, and nothing downstream re-resolves or infers it from an id.

Two consequences worth knowing before changing this code:

• A journey write must carry a folder. POST /entities/journeys requires relationships.parentFolder, and the folder is what classifies the journey's audience kind server-side — a folder-less create on a composable audience lands on the complete controller. So a create with no resolvable folder is refused (JOURNEY_FOLDER_REQUIRED) rather than sent with an empty id.
• An empty listing is never assumed. The complete-only GET /audiences/:id fallback used to resolve a root folder 404s for a composable audience, and that 404 was swallowed into an empty journey list — which push reads as "every journey is new", forking a duplicate of each. The listings take the audience kind and refuse that fallback for a composable audience.

Journey Comparison Logic

The most complex part is comparing local journey YAML with server state. This requires:

1. Segment Resolution: Server segments have auto-generated names like journey_entry_criteria_<uuid>. We match them to local segments by comparing expanded rule content (not names).

2. Expanded Rule Hashing: Rules can reference other segments. We expand these references recursively to create a canonical representation for comparison.

3. Multi-Version Handling: Journeys can have multiple versions. We fetch all versions and build a merged YAML for comparison.

4. Activation Resolution: Journey activations are embedded and need ID-to-name resolution.

3-Step Journey Creation

New journeys require a 3-step workflow:

1. Create Journey: Without entry criteria (segments don't exist yet)
2. Create Embedded Resources: Create journey-local segments and activations with journeyId
3. Update Journey: Add entry criteria and activation references with real IDs

This is necessary because:

• Journey-local segments need journeyId to be properly associated
• Activations need to be created before they can be referenced in steps
• Entry criteria reference segment IDs that don't exist until step 2

Migration Phases

Phase	Description	Complexity
1	Extract execution context building	Low
2	Extract segment preparation	Low
3	Extract journey comparison logic	High
4	Extract journey preparation	Low
5	Extract execution logic	Medium
6-7	Create entry points + SDK wrapper	Low
8	Refactor command to thin wrapper	Low

Usage Example (After Migration)

// In command
const result = await tdx.segment.preparePush(parentName, {
  sourceDir,
  onProgress: (phase) => console.error(`  ${phase}...`),
});

// Display and confirm...

const execResult = await tdx.segment.executePush(result, {
  onSegmentCreated: (name) => console.error(chalk.green(`✔ Created '${name}'`)),
  onJourneyCreated: (name, id) => {
    console.error(chalk.green(`✔ Created journey '${name}'`));
    console.error(`  ${getJourneyUrl(id)}`);
  },
});

Related Files

• pull-workflow.ts - Similar pattern for pull operations
• resolution-context.ts - Folder ID/path mapping utilities
• child-segment-yaml.ts - Segment YAML conversion
• journey-yaml.ts - Journey YAML conversion

References

• PR #544: Pull workflow refactoring (established the pattern)
• PR #545: Initial push workflow extraction