Migrate to monorepo structure

[jasonkuhrt]

Migrate to Monorepo Structure

Overview

Transform Graffle from a single package into a monorepo with 17 focused packages. This will:

• ✅ Enable independent versioning and releases
• ✅ Separate generic GraphQL utilities (@graffle/grafaid) from Graffle-specific code
• ✅ Allow extensions to be used independently
• ✅ Improve tree-shaking and bundle sizes
• ✅ Enable better code organization and maintainability
• ✅ Claim the @graffle npm scope

Package Structure (17 packages)

Core Packages (5)

1. @graffle/core

Graffle-specific shared infrastructure

Contains:

• Context system (src/context/)
• Extension base class (Extension)
• GlobalRegistry
• TypeFunction
• ContextFragments
• RequestResult types
• Docpar (document parser)
• SchemaDrivenDataMap
• Shared Graffle data structures

Used by: client, generator, all extensions

---

2. @graffle/grafaid

Generic GraphQL utilities (not Graffle-specific)

Contains:

• Schema types & definitions (from src/types/Schema/)
• GraphQL document utilities
• Schema introspection & analysis
• HTTP helpers
• Execution helpers
• Tree-shakeable, reusable GraphQL tools

Design Goal: Grafaid should be usable standalone for any GraphQL tooling, not just Graffle.

Used by: client, generator

---

3. @graffle/client

GraphQL Client

Contains:

• Client implementation (src/client/)
• Request pipeline (src/requestPipeline/)
• Static helpers (src/static/gql)
• Client methods
• Export path: graffle/utilities-for-generated (re-exports from core + client for generated code)

Depends on: @graffle/core, @graffle/grafaid

---

4. @graffle/generator

Code Generator & CLI

Contains:

• Generator (src/generator/)
• CLI (src/cli/)
• Code generation logic

Depends on: @graffle/grafaid, @graffle/core

---

5. graffle

Main Convenience Package

The "batteries included" package.

• Re-exports @graffle/client and common extensions
• Default entry point for users: npm i graffle

---

Extension Packages (8)

Each extension is independently publishable and versioned:

• @graffle/extension-document-builder
• @graffle/extension-introspection
• @graffle/extension-throws
• @graffle/extension-opentelemetry
• @graffle/extension-schema-errors
• @graffle/extension-transport-http
• @graffle/extension-transport-memory
• @graffle/extension-upload

All depend on: @graffle/core

---

Preset Packages (3)

Pre-configured extension bundles:

• @graffle/preset-bare
• @graffle/preset-basic
• @graffle/preset-minimal

---

Dependency Graph

@wollybeard/kit (external)
          ↓
    @graffle/core
          ↓
    ┌─────┴─────────────┐
    ↓                   ↓
@graffle/grafaid   Extensions (8)
    ↓                   ↓
@graffle/client    Presets (3)
@graffle/generator      ↓
    ↓                   ↓
  graffle (meta package)

Key Design Decisions

1. Grafaid = Generic, Core = Graffle-specific

• @graffle/grafaid: Generic GraphQL utilities, tree-shakeable, usable standalone
• @graffle/core: Graffle-specific infrastructure (extensions, context, etc.)

2. Replace Internal Kit with @wollybeard/kit

• Remove src/lib/anyware, src/lib/tex, src/lib/config-manager
• Use @wollybeard/kit for these utilities

3. Keep utilities-for-generated as Export Path

• Not a separate package
• Export path in @graffle/client: graffle/utilities-for-generated
• Generated code imports: import * as $$Utilities from "graffle/utilities-for-generated"

4. Independent Versioning

• Each package can be released independently
• Use Changesets for version management

5. Schema Types in Grafaid

• Move src/types/Schema/ into @graffle/grafaid
• Rely on tree-shaking to keep client bundle small

Tooling Stack

Build & Task Running

• pnpm workspaces - Package management
• Turborepo - Build orchestration & caching (temporary)
• TypeScript project references - Fast, incremental builds
• Vite/Rolldown - Bundling

Future Migration

• Migrate to Vite+ when available (early 2026)
• Vite+ includes built-in monorepo task runner with intelligent caching

Versioning & Publishing

• Changesets - Independent versioning & changelog generation
• Each package publishes to @graffle/* scope (except main graffle package)

Directory Structure

graffle/                           # Monorepo root
├── packages/
│   ├── core/                      # @graffle/core
│   ├── grafaid/                   # @graffle/grafaid
│   ├── client/                    # @graffle/client
│   ├── generator/                 # @graffle/generator
│   ├── graffle/                   # graffle (meta package)
│   │
│   ├── extensions/
│   │   ├── document-builder/      # @graffle/extension-document-builder
│   │   ├── introspection/         # @graffle/extension-introspection
│   │   ├── throws/                # @graffle/extension-throws
│   │   ├── opentelemetry/         # @graffle/extension-opentelemetry
│   │   ├── schema-errors/         # @graffle/extension-schema-errors
│   │   ├── transport-http/        # @graffle/extension-transport-http
│   │   ├── transport-memory/      # @graffle/extension-transport-memory
│   │   └── upload/                # @graffle/extension-upload
│   │
│   └── presets/
│       ├── bare/                  # @graffle/preset-bare
│       ├── basic/                 # @graffle/preset-basic
│       └── minimal/               # @graffle/preset-minimal
│
├── examples/                      # Example projects (workspace, not published)
├── website/                       # Documentation site (workspace, not published)
├── tests/                         # Integration/e2e tests
├── scripts/                       # Build/dev scripts
├── turbo.json                     # Turborepo config
├── pnpm-workspace.yaml            # pnpm workspace config
└── package.json                   # Root package.json

Migration Strategy

Phase 1: Setup Infrastructure

• Create monorepo structure
• Configure pnpm workspaces
• Set up Turborepo
• Configure TypeScript project references
• Set up Changesets

Phase 2: Split Core Packages

• Extract @graffle/core
• Extract @graffle/grafaid (includes Schema types)
• Split @graffle/client
• Split @graffle/generator
• Update internal imports

Phase 3: Extract Extensions

• Move each extension to packages/extensions/*
• Update extension imports and dependencies
• Test extensions independently

Phase 4: Extract Presets

• Move presets to packages/presets/*
• Update preset dependencies

Phase 5: Create Meta Package

• Create packages/graffle/ as main entry point
• Re-export client + common extensions
• Update documentation

Phase 6: Replace Kit Dependencies

• Replace src/lib/anyware → @wollybeard/kit
• Replace src/lib/tex → @wollybeard/kit
• Replace src/lib/config-manager → @wollybeard/kit
• Remove internal lib utilities

Phase 7: Testing & Publishing

• Run full test suite across all packages
• Update CI/CD for monorepo
• Publish initial versions
• Update documentation and migration guides

References

• Related: Extract extensions to own packages #1124 (Extract extensions to own packages)
• npm scope @graffle has been claimed

Open Questions

• Should examples/ and website/ be in monorepo or separate repos? Decision: Keep in monorepo
• Versioning strategy for extensions - lock-step or independent? Decision: Independent
• Breaking change migration path for users

[jasonkuhrt]

Detailed Monorepo Migration Plan - Updated 2025-10-28

Strategy: Big Bang Migration with Independent Versioning

Approach: Complete monorepo transformation releasing 14 packages simultaneously with independent versioning.

Key Decisions:

• @graffle/grafaid → @graffle/graphql (clearer name for generic GraphQL utilities)
• New @graffle/document package (docpar + document utilities extracted from core)
• Presets bundled in graffle meta package (not separate packages)
• 14 total packages (5 core + 8 extensions + 1 meta)
• Build tool: Turborepo (temporary, migrate to Vite+ in 2026)
• Versioning: Independent from day one using Changesets
• User migration: Manual with comprehensive documentation

---

Package Structure (14 packages)

Core Packages (5)

1. @graffle/graphql - Generic GraphQL utilities (was grafaid)
2. @graffle/document - Document parsing (docpar, SDDM, Select, etc.)
3. @graffle/core - Graffle infrastructure (context, extensions)
4. @graffle/client - GraphQL client
5. @graffle/generator - Code generator & CLI

Extension Packages (8)

6. @graffle/extension-document-builder
7. @graffle/extension-introspection
8. @graffle/extension-throws
9. @graffle/extension-opentelemetry
10. @graffle/extension-schema-errors
11. @graffle/extension-transport-http
12. @graffle/extension-transport-memory
13. @graffle/extension-upload

Meta Package (1)

14. graffle - Batteries included (includes presets as export paths)

---

Dependency Graph

@wollybeard/kit (external)
          ↓
   @graffle/graphql (GraphQL utilities, schema types)
          ↓
   @graffle/document (docpar, SDDM, Select)
          ↓
   @graffle/core (context, extensions)
          ↓
    ┌─────┴─────────────┐
    ↓                   ↓
@graffle/client    Extensions (8)
@graffle/generator      
    ↓                   ↓
  graffle (meta + presets)

---

Phase 1: Monorepo Infrastructure Setup

Goal: Establish monorepo tooling

1.1 Initialize Workspace Structure

packages/
├── graphql/                    # @graffle/graphql
├── document/                   # @graffle/document
├── core/                       # @graffle/core
├── client/                     # @graffle/client
├── generator/                  # @graffle/generator
├── extensions/
│   ├── document-builder/
│   ├── introspection/
│   ├── throws/
│   ├── opentelemetry/
│   ├── schema-errors/
│   ├── transport-http/
│   ├── transport-memory/
│   └── upload/
└── graffle/                    # Meta package

1.2 Configure pnpm Workspaces

• Create pnpm-workspace.yaml
• Move root dependencies to appropriate packages

1.3 Setup Turborepo

• Create turbo.json with pipeline configuration
• Configure caching for build outputs

1.4 TypeScript Project References

• tsconfig.base.json for shared config
• Each package: tsconfig.json + tsconfig.build.json
• Configure composite builds

1.5 Setup Changesets

• Install @changesets/cli
• Configure for independent versioning
• Add version/publish scripts

1.6 Update CI/CD

• GitHub Actions for monorepo
• Turbo for builds
• Changeset automation

Dependencies: None
Validation: Empty packages build

---

Phase 2: Extract @graffle/graphql (Foundation)

Goal: Generic GraphQL utilities, zero internal dependencies

• Move src/lib/grafaid/** → packages/graphql/src/
• Move src/types/Schema/** → packages/graphql/src/schema-types/
• Setup tree-shakeable exports
• Dependencies: External only (@0no-co/graphql.web, @wollybeard/kit, peer graphql)

Dependencies: Phase 1
Validation: Builds and tests independently

---

Phase 3: Extract @graffle/document (Document Layer)

Goal: Document parsing and selection set utilities

• Move src/docpar/** → packages/document/src/
• Setup exports for SDDM, Select, InferResult
• Dependencies: @graffle/graphql

Dependencies: Phase 2
Validation: Builds independently, imports from @graffle/graphql work

---

Phase 4: Extract @graffle/core (Infrastructure)

Goal: Graffle-specific shared infrastructure

• Move src/context/** → packages/core/src/context/
• Move src/types/RequestResult/**, GlobalRegistry/**, ContextFragment.ts
• Move src/utils.ts
• Extract Extension base class
• Resolve circular dependencies: Extract HandleOutput type to core

Dependencies: Phase 3
Validation: Core builds, no circular dependency errors

---

Phase 5: Extract @graffle/client (Client)

Goal: GraphQL client implementation

• Move src/client/**, requestPipeline/**, static/**, select/**
• CRITICAL: Create utilities-for-generated export:
  • Re-exports from core, document, graphql
  • Maintains API for generated code
• Dependencies: @graffle/core, @graffle/document, @graffle/graphql

Dependencies: Phase 4
Validation: Client builds, utilities-for-generated exports correctly

---

Phase 6: Extract @graffle/generator (Generator)

Goal: Code generator and CLI

• Move src/generator/**, cli/**
• Move src/lib/fsp.ts, import-first.ts, typescript-formatter.ts
• Update generated code templates to use @graffle/* imports
• Dependencies: @graffle/core, @graffle/document, @graffle/graphql

Dependencies: Phase 5
Validation: Generator outputs correct imports

---

Phase 7: Extract Extensions (8 Packages)

Migration Order:

1. Simple: throws, opentelemetry, upload, transport-memory
2. Transport: transport-http
3. Complex: introspection
4. Most complex: document-builder, schema-errors

For each extension:

• Create package structure
• Move source from src/extensions/{Name}/**
• Setup exports (runtime + gentime for SchemaErrors)
• Update dependencies and imports
• DO NOT EDIT generated fixtures

Special handling:

• DocumentBuilder circular dependency: Extract graffleMappedResultToRequest to @graffle/core/utilities

Dependencies: Phase 6
Validation: Each builds independently

---

Phase 8: Create Meta Package with Presets

Goal: Batteries-included package with bundled presets

• Move src/exports/presets/*.ts → packages/graffle/src/presets/
• Create re-exports for client and common extensions
• Setup export paths matching old structure:
  • graffle → main
  • graffle/presets/basic → basic preset
  • graffle/extensions/* → extensions (backward compatible)
• Dependencies: @graffle/client + common extensions

Dependencies: Phase 7
Validation: Meta package installs, imports work

---

Phase 9: Regenerate Test Fixtures

Goal: All generated code uses new structure

• Regenerate all test schemas
• Regenerate SchemaErrors/DocumentBuilder fixtures
• Regenerate website schemas (Pokemon, Graffle)
• Regenerate examples
• Run full test suite: turbo run test

Dependencies: Phase 8
Validation: All tests pass, generated code correct

---

Phase 10: Documentation & Migration

Goal: User migration guide

Migration Guide

Old → New:

// Old
import { Graffle } from 'graffle'
import { SchemaErrors } from 'graffle/extensions/schema-errors'

// New (Option 1: Meta package - easiest)
import { Graffle } from 'graffle'
import { SchemaErrors } from '@graffle/extension-schema-errors'
// OR keep: import { SchemaErrors } from 'graffle/extensions/schema-errors'

// New (Option 2: Specific packages - smaller bundles)
import { Graffle } from '@graffle/client'
import { SchemaErrors } from '@graffle/extension-schema-errors'

// Presets unchanged
import { create } from 'graffle/presets/basic'

Dependencies: Phase 9
Validation: Docs complete, examples work

---

Phase 11: Publishing Prep

• Verify all package.json metadata
• Run publint on all packages
• Local testing with pnpm link
• Create initial changeset (major version, all 14 packages)
• Verify @graffle npm scope access
• Dry run publish

Dependencies: Phase 10
Validation: Publint passes, dry-run succeeds

---

Phase 12: CI/CD & Release

• Update GitHub Actions workflows
• Test CI pipeline
• Merge monorepo PR to main
• Review and merge version PR
• Automated publish to npm
• Verify all 14 packages published

Dependencies: Phase 11
Validation: Successful publish

---

Risk Mitigation

1. Circular dependencies - Resolved in Phase 4 by extracting shared types to core
2. Generated code breaking - Thorough testing in Phase 9
3. Import path changes - Backward compatible paths in meta package + migration guide
4. Publishing failures - Dry-run testing in Phase 11

Rollback Plan: Work in feat/monorepo branch, only merge when fully tested

---

Success Criteria

• ✅ All 14 packages build successfully with Turborepo
• ✅ All tests pass in monorepo structure
• ✅ Generated code uses correct imports
• ✅ Examples work with new package structure
• ✅ All packages published to npm under @graffle scope
• ✅ CI/CD pipeline works with changesets
• ✅ Documentation updated and migration guide complete
• ✅ No circular dependency errors
• ✅ Independent versioning works with changesets

---

Estimated Effort

• Phase 1: 1-2 days (infra)
• Phase 2: 1 day (graphql)
• Phase 3: 1 day (document)
• Phase 4: 2 days (core + circular deps)
• Phase 5: 1-2 days (client)
• Phase 6: 1-2 days (generator)
• Phase 7: 3-4 days (8 extensions)
• Phase 8: 1 day (meta + presets)
• Phase 9: 1-2 days (regenerate)
• Phase 10: 1-2 days (docs)
• Phase 11: 1 day (publish prep)
• Phase 12: 0.5 day (release)

Total: ~14-20 days focused work