TDX Codebase Refactoring Plan

Summary

Reorganize utilities and business logic to achieve:

• Feature-specific utilities move to SDK modules
• Pure formatters move to sdk/util/
• Commands become thin wrappers over SDK
• FileSystem abstraction for testability
• Config/profile stays in core/

Target Directory Structure

src/
├── cli.ts                    # CLI entry point
├── index.ts                  # Package exports
├── commands/                 # Thin CLI wrappers
│
├── core/                     # CLI infrastructure (keeps config)
│   ├── command.ts           # BaseCommand
│   ├── command-output.ts    # CLI output (paging, less, file write) - MOVED from utils
│   ├── constants.ts         # MOVED from utils (CLI-specific limits/defaults)
│   ├── config.ts            # API key & config (stays)
│   ├── profile.ts           # Profile management (stays)
│   ├── global-context.ts    # Context layer merging (stays)
│   ├── session.ts           # Session (stays)
│   ├── keychain.ts          # Credentials (stays)
│   └── ...                  # Other core files stay
│
├── sdk/
│   ├── index.ts             # Main TDX exports
│   │
│   ├── client/              # MOVED from src/client/
│   │   ├── http-client.ts   # Base HTTP client
│   │   ├── td-api-client.ts # TD API client
│   │   ├── cdp-api-client.ts # CDP API client
│   │   ├── llm-api-client.ts # LLM API client
│   │   ├── trino-api-client.ts # Trino API client
│   │   ├── workflow-client.ts # Workflow API client
│   │   ├── engage-api-client.ts # Engage API client
│   │   └── rate-limiter.ts  # Rate limiter
│   │
│   ├── util/                # NEW: SDK utilities
│   │   ├── formatters.ts    # MOVED: table/JSON/JSONL/TSV formatters
│   │   ├── format-detector.ts # MOVED: format auto-detection
│   │   └── tree-formatter.ts  # MOVED: tree formatting
│   │
│   ├── io/                  # NEW: FileSystem abstraction
│   │   ├── index.ts
│   │   ├── types.ts         # FileSystem interface
│   │   └── node-filesystem.ts # Node.js implementation
│   │
│   ├── segment/             # Segment SDK (enhanced)
│   │   ├── index.ts
│   │   ├── child-segment-yaml.ts
│   │   ├── parent-segment-yaml.ts
│   │   ├── journey-yaml.ts
│   │   ├── segment-ref-parser.ts    # MOVED from utils
│   │   ├── segment-path-parser.ts   # MOVED from utils
│   │   ├── journey-utils.ts         # MOVED from utils
│   │   ├── fields-display.ts        # MOVED from utils
│   │   └── tdx-config.ts            # MOVED from utils
│   │
│   ├── agent/               # Agent SDK (enhanced)
│   │   ├── index.ts
│   │   ├── agent-yaml.ts
│   │   ├── sync.ts                  # NEW: Extracted from sdk/llm.ts (pull/push/clone)
│   │   ├── agent-ref-parser.ts      # MOVED from utils
│   │   └── agent-config.ts          # MOVED from utils
│   │
│   ├── llm/                 # NEW: LLM module folder
│   │   ├── index.ts         # MOVED from sdk/llm.ts
│   │   ├── model-aliases.ts # MOVED from utils
│   │   ├── model-factors.ts # MOVED from utils
│   │   ├── chat-cache.ts    # MOVED from utils
│   │   └── llm-backup-utils.ts # MOVED from utils
│   │
│   ├── workflow/            # NEW: Workflow module folder
│   │   ├── index.ts         # MOVED from sdk/workflow.ts
│   │   └── workflow-utils.ts # MOVED from utils
│   │
│   ├── query/               # NEW: Query module folder
│   │   ├── index.ts         # MOVED from sdk/query.ts
│   │   └── table-ref-parser.ts # MOVED from utils
│   │
│   └── connection/          # Connection SDK
│       ├── index.ts
│       └── connection-validator.ts
│
├── utils/                   # General-purpose only
│   ├── async.ts
│   ├── colors.ts
│   ├── diff.ts
│   ├── json-loader.ts
│   ├── option-validation.ts
│   ├── port.ts
│   ├── process.ts
│   ├── prompt.ts
│   ├── spinner.ts
│   ├── sql-parser.ts
│   ├── sse-parser.ts
│   ├── string-utils.ts
│   ├── tdx-logger.ts
│   └── version-check.ts
│
├── proxy/                   # LLM Proxy (unchanged)
└── types/                   # Types (unchanged)

Migration Phases

Phase 1: Create Infrastructure

1. Move client/ to sdk/client/

   • Move src/client/ → src/sdk/client/
   • Update all imports across the codebase

2. Create sdk/io/ (FileSystem abstraction)

   • src/sdk/io/types.ts - FileSystem interface
   • src/sdk/io/node-filesystem.ts - Node.js implementation
   • src/sdk/io/index.ts - Exports

3. Create sdk/util/ (SDK formatters)

   • Move src/utils/formatters.ts → src/sdk/util/formatters.ts
   • Move src/utils/format-detector.ts → src/sdk/util/format-detector.ts
   • Move src/utils/tree-formatter.ts → src/sdk/util/tree-formatter.ts
   • Create src/sdk/util/index.ts

4. Move CLI-specific files to core/

   • Move src/utils/command-output.ts → src/core/command-output.ts
   • Move src/utils/constants.ts → src/core/constants.ts
   • Update imports to use sdk/util/formatters

Phase 2: Segment Module

Move to src/sdk/segment/:

• src/utils/segment-ref-parser.ts
• src/utils/segment-path-parser.ts
• src/utils/journey-utils.ts
• src/utils/fields-display.ts
• src/utils/tdx-config.ts

Update src/sdk/segment/index.ts with exports. Update all imports in commands.

Phase 3: Agent Module

Move to src/sdk/agent/:

• src/utils/agent-ref-parser.ts
• src/utils/agent-config.ts

Extract sync logic from sdk/llm.ts to sdk/agent/sync.ts:

• pullProject() (~290 lines) - Project pull with diff calculation
• pushProject() (~490 lines) - Project push with dependency resolution
• pullAgent() (~107 lines) - Single agent pull
• pushAgent() (~95 lines) - Single agent push
• cloneProject() (~346 lines) - Clone from local or remote

Create AgentSync class in sdk/agent/sync.ts that:

• Accepts LLMApiClient and FileSystem via constructor
• Contains all pull/push/clone methods
• Uses agent-yaml.ts for parsing/serialization

Update LLMSDK to delegate to AgentSync:

// sdk/llm/index.ts
import { AgentSync } from '../agent/sync.js';

class LLMSDK {
  private agentSync: AgentSync;

  pullProject(...) { return this.agentSync.pullProject(...); }
  pushProject(...) { return this.agentSync.pushProject(...); }
}

Update src/sdk/agent/index.ts with exports. Update imports.

Phase 4: LLM Module

Create src/sdk/llm/ folder:

• Move src/sdk/llm.ts → src/sdk/llm/index.ts
• Move src/utils/model-aliases.ts
• Move src/utils/model-factors.ts
• Move src/utils/chat-cache.ts
• Move src/utils/llm-backup-utils.ts

Update imports.

Phase 5: Workflow Module

Create src/sdk/workflow/ folder:

• Move src/sdk/workflow.ts → src/sdk/workflow/index.ts
• Move src/utils/workflow-utils.ts

Update imports.

Phase 6: Query Module

Create src/sdk/query/ folder:

• Move src/sdk/query.ts → src/sdk/query/index.ts
• Move src/utils/table-ref-parser.ts

Update imports.

Phase 7: FileSystem Integration

Update YAML managers to accept FileSystem interface:

• src/sdk/segment/child-segment-yaml.ts
• src/sdk/segment/parent-segment-yaml.ts
• src/sdk/segment/journey-yaml.ts
• src/sdk/agent/agent-yaml.ts

Pattern:

import { FileSystem, nodeFileSystem } from '../io/index.js';

export async function parseYamlFile(
  path: string,
  fs: FileSystem = nodeFileSystem
): Promise<Config> {
  const content = await fs.readFile(path);
  return parseYamlString(content);
}

Phase 8: Command Refactoring

Update commands to use SDK file operations instead of direct fs calls:

• src/commands/segment-pull-command.ts
• src/commands/segment-push-command.ts
• src/commands/journey-command.ts
• src/commands/agent-command.ts
• src/commands/parent-segment-command.ts
• src/commands/workflow-command.ts

Files to Move Summary

From	To
client/*	sdk/client/*
utils/formatters.ts	sdk/util/formatters.ts
utils/format-detector.ts	sdk/util/format-detector.ts
utils/tree-formatter.ts	sdk/util/tree-formatter.ts
utils/command-output.ts	core/command-output.ts
utils/constants.ts	core/constants.ts
utils/segment-ref-parser.ts	sdk/segment/segment-ref-parser.ts
utils/segment-path-parser.ts	sdk/segment/segment-path-parser.ts
utils/journey-utils.ts	sdk/segment/journey-utils.ts
utils/fields-display.ts	sdk/segment/fields-display.ts
utils/tdx-config.ts	sdk/segment/tdx-config.ts
utils/agent-ref-parser.ts	sdk/agent/agent-ref-parser.ts
utils/agent-config.ts	sdk/agent/agent-config.ts
utils/model-aliases.ts	sdk/llm/model-aliases.ts
utils/model-factors.ts	sdk/llm/model-factors.ts
utils/chat-cache.ts	sdk/llm/chat-cache.ts
utils/llm-backup-utils.ts	sdk/llm/llm-backup-utils.ts
utils/workflow-utils.ts	sdk/workflow/workflow-utils.ts
utils/table-ref-parser.ts	sdk/query/table-ref-parser.ts
sdk/llm.ts	sdk/llm/index.ts
sdk/workflow.ts	sdk/workflow/index.ts
sdk/query.ts	sdk/query/index.ts

Code Extraction Summary

Extract From	To	Methods
sdk/llm.ts	sdk/agent/sync.ts	pullProject, pushProject, pullAgent, pushAgent, cloneProject (~1,300 lines)

FileSystem Interface

// src/sdk/io/types.ts
export interface FileSystem {
  readFile(path: string): Promise<string>;
  writeFile(path: string, content: string): Promise<void>;
  exists(path: string): Promise<boolean>;
  mkdir(path: string, options?: { recursive?: boolean }): Promise<void>;
  readdir(path: string): Promise<string[]>;
  stat(path: string): Promise<{ isDirectory(): boolean; isFile(): boolean }>;
  unlink(path: string): Promise<void>;
}

export interface FileSystemSync {
  readFileSync(path: string): string;
  writeFileSync(path: string, content: string): void;
  existsSync(path: string): boolean;
  mkdirSync(path: string, options?: { recursive?: boolean }): void;
}

Verification After Each Phase

1. npm run typecheck - No type errors
2. npm run lint - No new lint errors
3. npm test - All tests pass
4. npm run build:ts - Builds successfully

Key Decisions

• API clients to SDK: client/ moves to sdk/client/ - SDK becomes self-contained
• Config stays in core/: Config/profile management remains in core/ as CLI infrastructure
• Constants to core/: constants.ts moves to core/ (CLI-specific limits/defaults)
• FileSystem abstraction: YAML managers will accept FileSystem interface for testability
• SDK util folder: Pure formatters go to sdk/util/, not sdk/formatting/
• No re-exports: Update all imports in one pass, no deprecated re-exports
• CLI output in core/: command-output.ts (paging, less, file writing) moves to core/
• Agent sync extraction: Pull/push/clone logic (~1,300 lines) extracted from sdk/llm.ts to sdk/agent/sync.ts