YAML Pull/Push Model Architecture

Date: 2025-12-23 Status: Planned

Summary

Refactor the YAML-based pull/push flows for segments, journeys, and agents to have clear transformation layers with proper model class naming.

Current Problem: "YamlXXX" naming is misleading - these are TypeScript objects, not YAML. ID resolution happens inline during conversion, making it hard to separate concerns.

Target Flow:

Local YAML file ←→ LocalModel (no IDs) ←→ ResolvedModel (with IDs) ←→ ApiModel
                   "portable"              "resolved"                 "server"
                   human-readable          ID-referenced              API format

Current Design Analysis

Affected Resources

Resource	Files	Current Types
Child Segment	sdk/segment/child-segment-yaml.ts	ChildSegmentYaml, YamlRule, YamlCondition, YamlOperator, YamlActivation
Parent Segment	sdk/segment/parent-segment-yaml.ts	ParentSegmentYaml, YamlColumn, YamlJoin, YamlSource, YamlAttributeGroup, YamlBehavior
Journey	sdk/segment/journey-yaml.ts	JourneyYaml, YamlStep, YamlJourneyStage, YamlEmbeddedSegment, YamlJourneyCriteria
Agent	sdk/agent/agent-yaml.ts	AgentYaml, YamlAgentTool, YamlAgentOutput, YamlAgentVariable, KnowledgeBaseYaml

Current Issues

1. YamlXXX naming implies YAML format, but these are TypeScript objects representing portable configs
2. Missing intermediate layer - Resolution happens during YAML→API conversion, not as separate step
3. Context object inconsistency - Different naming: ActivationConversionContext, ApiToYamlContext, YamlToApiContext
4. Resolution timing varies - Child segment resolves after, Journey/Agent resolve during conversion

Proposed Architecture

Naming Convention: Def/Resolved Pattern

Replace YamlXXX with XXXDef (definition - portable, human-readable):

Current	Proposed
ChildSegmentYaml	SegmentDef
YamlRule	RuleDef
YamlCondition	ConditionDef
YamlActivation	ActivationDef
ParentSegmentYaml	ParentSegmentDef
JourneyYaml	JourneyDef
YamlStep	StepDef
AgentYaml	AgentDef
YamlAgentTool	ToolDef

Resolved models (with IDs) use ResolvedXXX prefix when needed:

• ResolvedRule (rule with segment IDs resolved)
• ResolvedStep (step with segment/activation IDs)
• ResolvedToolRef (tool with KB/agent IDs)

Resolution Strategy: Inline when trivial

• Parent segment (no ID refs) can use direct defToApi() / apiToDef() without separate resolution step
• Complex cases (child segment, journey, agent) use explicit resolve() / dereference() steps

Transformation Layers

                    PULL (Server → Local)
┌─────────────────────────────────────────────────────────┐
│  ApiModel  →  ResolvedModel  →  XXXDef     →  YAML     │
│  (server)     (with IDs)        (names)       (file)   │
│                                                         │
│  fetchApi()   dereference()     serialize()            │
└─────────────────────────────────────────────────────────┘

                    PUSH (Local → Server)
┌─────────────────────────────────────────────────────────┐
│  YAML     →  XXXDef     →  ResolvedModel  →  ApiModel  │
│  (file)      (names)       (with IDs)        (server)  │
│                                                         │
│  parse()     resolve()       toApiModel()              │
└─────────────────────────────────────────────────────────┘

Simplified flow for parent segment (no ID resolution needed):

PULL: ApiModel → ParentSegmentDef → YAML
PUSH: YAML → ParentSegmentDef → ApiModel

Context Objects (Unified)

// Resolution context (names → IDs)
interface ResolutionContext {
  resolveSegmentId(name: string): Promise<string>;
  resolveActivationId(name: string): Promise<string>;
  resolveJourneyId(name: string): Promise<string>;
  resolveKnowledgeBaseId(name: string): Promise<string>;
  resolveAgentId(name: string): Promise<string>;
  // ... other resolvers
}

// Dereference context (IDs → names)
interface DereferenceContext {
  resolveSegmentName(id: string): Promise<string>;
  resolveActivationName(id: string): Promise<string>;
  resolveJourneyName(id: string): Promise<string>;
  resolveKnowledgeBaseName(id: string): Promise<string>;
  resolveAgentName(id: string): Promise<string>;
  // ... other resolvers
}

Migration Phases

Phase 1: Define Def Types (Non-Breaking)

Create new type definitions alongside existing ones:

1. Create sdk/segment/types/segment-def.ts

   • SegmentDef, RuleDef, ConditionDef, OperatorDef, ActivationDef

2. Create sdk/segment/types/parent-segment-def.ts

   • ParentSegmentDef, ColumnDef, JoinDef, SourceDef, AttributeGroupDef

3. Create sdk/segment/types/journey-def.ts

   • JourneyDef, StageDef, StepDef, CriteriaDef, EmbeddedSegmentDef

4. Create sdk/agent/types/agent-def.ts

   • AgentDef, ToolDef, OutputDef, VariableDef

5. Create sdk/types/resolution-context.ts

   • ResolutionContext, DereferenceContext interfaces

Phase 2: Implement Transformation Functions

For each resource type, create clear transformation layers:

Child Segment:

// Parse: YAML string → SegmentDef
parseSegmentDef(yaml: string): SegmentDef

// Resolve: SegmentDef → ResolvedSegment (names → IDs)
resolveSegment(def: SegmentDef, ctx: ResolutionContext): Promise<ResolvedSegment>

// ToApi: ResolvedSegment → CreateSegmentRequest
toCreateRequest(resolved: ResolvedSegment): CreateSegmentRequest

// FromApi: Segment → ResolvedSegment
fromApiSegment(api: Segment): ResolvedSegment

// Dereference: ResolvedSegment → SegmentDef (IDs → names)
dereferenceSegment(resolved: ResolvedSegment, ctx: DereferenceContext): Promise<SegmentDef>

// Serialize: SegmentDef → YAML string
serializeSegmentDef(def: SegmentDef): string

Parent Segment (simplified - no separate resolution):

parseParentSegmentDef(yaml: string): ParentSegmentDef
defToApi(def: ParentSegmentDef): CreateParentSegmentRequest
apiToDef(api: ParentSegmentFull): ParentSegmentDef
serializeParentSegmentDef(def: ParentSegmentDef): string

Journey:

parseJourneyDef(yaml: string): JourneyDef
resolveJourney(def: JourneyDef, ctx: ResolutionContext): Promise<ResolvedJourney>
toCreateJourneyRequest(resolved: ResolvedJourney): CreateJourneyRequest
fromApiJourney(api: Journey): ResolvedJourney
dereferenceJourney(resolved: ResolvedJourney, ctx: DereferenceContext): Promise<JourneyDef>
serializeJourneyDef(def: JourneyDef): string

Agent:

parseAgentDef(yaml: string): AgentDef
resolveAgent(def: AgentDef, ctx: ResolutionContext): Promise<ResolvedAgent>
toCreateAgentRequest(resolved: ResolvedAgent): CreateAgentRequest
fromApiAgent(api: AgentFull): ResolvedAgent
dereferenceAgent(resolved: ResolvedAgent, ctx: DereferenceContext): Promise<AgentDef>
serializeAgentDef(def: AgentDef): string

Phase 3: Migrate Existing Code

Update existing YAML handling to use new transformation layers:

1. Update child-segment-yaml.ts

   • Rename types: ChildSegmentYaml → SegmentDef
   • Extract resolution logic to resolveSegment()
   • Extract dereference logic to dereferenceSegment()

2. Update parent-segment-yaml.ts

   • Rename types: ParentSegmentYaml → ParentSegmentDef
   • Simpler migration (direct defToApi/apiToDef, no separate resolution)

3. Update journey-yaml.ts

   • Rename types: JourneyYaml → JourneyDef
   • Refactor YamlToApiContext to use ResolutionContext
   • Refactor ApiToYamlContext to use DereferenceContext

4. Update agent-yaml.ts

   • Rename types: AgentYaml → AgentDef
   • Align with unified context interfaces

Phase 4: Update Commands & SDK

1. Update commands to use new type names
2. Update SDK public interfaces
3. Update tests

File Changes Summary

File	Changes
sdk/segment/types/segment-def.ts	NEW - Def types for child segment
sdk/segment/types/parent-segment-def.ts	NEW - Def types for parent segment
sdk/segment/types/journey-def.ts	NEW - Def types for journey
sdk/agent/types/agent-def.ts	NEW - Def types for agent
sdk/types/resolution-context.ts	NEW - Unified context interfaces
sdk/segment/child-segment-yaml.ts	Refactor to use Def types
sdk/segment/parent-segment-yaml.ts	Refactor to use Def types
sdk/segment/journey-yaml.ts	Refactor to use Def types
sdk/agent/agent-yaml.ts	Refactor to use Def types

Decisions Made

1. Naming convention: XXXDef (e.g., SegmentDef, JourneyDef, AgentDef)
2. Resolution timing: Inline when trivial (parent segment skips separate resolution)
3. Scope: All resources at once (child segment, parent segment, journey, agent)

Verification

After each phase:

• npm run typecheck
• npm run lint
• npm test
• Manual test: tdx segment pull/push, tdx journey pull/push, tdx agent pull/push