Extension: Caching support

[jasonkuhrt]

Summary

Add a caching extension to provide optional caching capabilities for Graffle.

Context

While Graffle excels as a general-purpose GraphQL client, users building UI applications may benefit from caching capabilities. Currently, we position clients like Apollo and Urql as better fits for UI applications primarily due to their caching features. However, caching could be implemented as a Graffle extension, maintaining our philosophy of keeping the core lean while offering powerful opt-in features.

Potential Approaches

1. Simple response cache - Cache responses by query + variables
2. Normalized cache - Document-based normalized caching similar to Apollo/Urql
3. Adapter pattern - Allow integration with existing cache solutions

Benefits

• Maintain Graffle's simplicity while offering caching as opt-in
• Users could use Graffle for UI applications without switching clients
• Follows our extensibility philosophy (like OpenTelemetry, Upload extensions)
• Could start simple and evolve based on user needs

Considerations

• Cache invalidation strategies
• Memory management
• TypeScript support for cached data
• Performance implications
• Bundle size impact (extension should be tree-shakeable)

Related

This came up during comparison documentation work - currently we direct UI app developers toward Apollo/Urql primarily for caching, but an extension could address this gap while maintaining Graffle's design principles.

[jasonkuhrt]

Caching Extension Research Summary

Overview

Researched options for adding normalized caching support to Graffle, including existing libraries, implementation approaches, and query reduction strategies.

---

Existing Cache Libraries Analysis

All are MIT Licensed ✅

• Apollo Client (): MIT
• URQL Graphcache (): MIT
• Relay Runtime: MIT
• graphql-norm: MIT

All can be legally forked/vendored.

---

Modularity & Code Quality Assessment

graphql-norm (Recommended Starting Point)

• Size: 108KB, ~8 files, ~500 LOC total
• Dependencies: ZERO (only peer dep on graphql)
• Last update: 2020 (stale but algorithm is sound)
• Code quality: Simple, pure functions, easy to understand
• Core API:

  normalize(query, variables, data, getObjectId)   // Response → flat cache
  denormalize(query, variables, cache)              // Cache → response
  merge(cache1, cache2)                             // Combine caches

Pros:

• Clean, understandable implementation
• No framework coupling
• Easy to fork and extend
• Perfect for server-side use (no UI cruft)

Cons:

• Unmaintained (5 years old)
• Basic features only (no optimistic updates, etc.)
• Would need to extend for TTL, LRU eviction, etc.

URQL Graphcache Store

• Size: 523KB total, but core Store is much smaller
• Dependencies: , (observables),
• Key discovery: Store class is independent of wonka (synchronous)
• Code quality: Well-architected, battle-tested, actively maintained

Core modules:

/store/store.ts       - Main Store class
/store/data.ts        - InMemoryData structures
/operations/query.ts  - Query operations  
/operations/write.ts  - Write/normalize operations

Store API (synchronous):

store.resolve(entity, field, args)     // Read specific field
store.invalidate(entity, field, args)  // Invalidate cache
store.inspectFields(entity)            // List cached fields
store.updateQuery(query, updater)      // Update query results
store.readQuery(query, variables)      // Read cached query

Pros:

• Rich feature set (garbage collection, field-level APIs)
• Better APIs for cache interrogation
• Actively maintained
• Good TypeScript types

Cons:

• More complex extraction (~20-30 files needed)
• Need to replace URQL-specific imports
• Includes client-side features not needed server-side (optimistic updates, layers, reactivity)

Apollo InMemoryCache

• Size: 10.2MB package
• Assessment: Too tightly coupled to Apollo ecosystem, not recommended for extraction

Relay Store

• Size: 1.3MB + fbjs dependencies
• Assessment: Extremely coupled to Relay compiler, not suitable for extraction

---

Query Reduction Discussion

What is Query Reduction?

Taking a query like:

query {
  user(id: "123") {
    id
    name
    email
    address
  }
}

When cache has {id, name}, reduce it to:

query {
  user(id: "123") {
    email
    address
  }
}

Why Major Clients Don't Do This

From Relay Classic → Relay Modern migration:

• Relay Classic had "fat queries" for dynamic query generation
• Removed in Modern for being unpredictable and adding overhead
• Result: 80% smaller bundle, 900ms faster TTI on mobile

Key issues identified:

1. Atomicity/Consistency: Merging data from different timestamps creates inconsistent snapshots
2. HTTP/2 Reality: Modern multiplexing makes multiple requests cheap
3. Complexity vs Benefit: AST manipulation overhead often exceeds network savings
4. Static > Dynamic: Compile-time optimization beats runtime manipulation

When Query Reduction MIGHT Be Worth It

Server-side use cases:

• ✅ Long-running services (amortize optimization cost)
• ✅ Expensive backend queries (reducing DB hits matters more)
• ✅ High request volume (optimization pays off at scale)
• ✅ Controlled environment (you control both ends)

Client-side:

• ❌ Mobile constraints (battery, CPU)
• ❌ Bundle size concerns
• ❌ HTTP/2 makes network cheap

Solving the N+1 Problem with Relay Node Interface

The issue: Nested queries with partial cache hits can fragment into multiple requests.

Solution using Relay node interface:

Original query:

query {
  user(id: "123") {
    id
    name
    organization {
      id
      name
      address  # Missing from cache
    }
  }
}

Transform to:

query {
  user(id: "123") {
    id
    name
    organization { id }
  }
  _org456: node(id: "Organization:456") {
    ... on Organization {
      address
    }
  }
}

Then stitch results back. Keeps everything in one request.

Requirements:

• Server implements Relay node interface
• Cache tracks field-level data with arguments
• Stitching logic to reconstruct original shape

---

Recommendations

Phase 1: Basic Normalized Cache (Start Here)

Fork graphql-norm and adapt:

1. ~8 files, manageable codebase
2. Add Graffle-specific features:
   • TTL/expiration
   • LRU eviction
   • Integration with anyware system
3. Exact query deduplication (cache hit = return immediately)
4. Simple, proven algorithm

Timeline: 1-2 weeks
Value: Deduplication for identical queries, solid foundation

Phase 2: Query Reduction (Optional, Measure First)

Only if measurements show benefit:

1. Implement partial cache hit detection
2. Add query reduction for servers with Relay node interface
3. Make it optional via configuration:

   NormalizedCache({
     queryReduction: {
       enabled: true,
       nodeInterface: true  // Requires server support
     }
   })

4. Track metrics: % of partial hits, latency improvements

Timeline: Additional 2-3 weeks
Risk: Complex, limited real-world benefit (measure first!)

Anyware Integration

Graffle's anyware system is already powerful enough:

Extension.create('NormalizedCache')
  .requestInterceptor(async ({ pack }) => {
    // Before exchange: check cache
    const cached = cache.get(query, variables)
    if (cached) return cached
    
    // After exchange: update cache
    const { unpack } = await exchange()
    const result = await unpack()
    cache.set(query, variables, result)
    return result
  })

No core changes needed - the exchange step is perfect for caching.

---

Conclusion

Start simple:

1. Fork graphql-norm (~500 LOC, proven algorithm)
2. Integrate with anyware for request interception
3. Add TTL, eviction, and other server-side features
4. Ship basic normalized cache

Measure before advancing:

• Track cache hit rates
• Measure partial hit frequency
• Determine if query reduction would actually help

Query reduction is not a bad idea - but it's complex and the benefits are situational. The atomicity trade-off and implementation complexity should be weighed against measured performance gains in real-world usage.

The normalized cache foundation (phase 1) is valuable on its own for exact query deduplication. Query reduction can be added later if data shows it's worth the complexity.

[jasonkuhrt]

Caching Extension Research Summary

Overview

Researched options for adding normalized caching support to Graffle, including existing libraries, implementation approaches, and query reduction strategies.

---

Existing Cache Libraries Analysis

All are MIT Licensed ✅

• Apollo Client (@apollo/client/cache): MIT
• URQL Graphcache (@urql/exchange-graphcache): MIT
• Relay Runtime: MIT
• graphql-norm: MIT

All can be legally forked/vendored.

---

Modularity & Code Quality Assessment

graphql-norm (Recommended Starting Point)

• Size: 108KB, ~8 files, ~500 LOC total
• Dependencies: ZERO (only peer dep on graphql)
• Last update: 2020 (stale but algorithm is sound)
• Code quality: Simple, pure functions, easy to understand
• Core API:

  normalize(query, variables, data, getObjectId)   // Response → flat cache
  denormalize(query, variables, cache)              // Cache → response
  merge(cache1, cache2)                             // Combine caches

Pros:

• Clean, understandable implementation
• No framework coupling
• Easy to fork and extend
• Perfect for server-side use (no UI cruft)

Cons:

• Unmaintained (5 years old)
• Basic features only (no optimistic updates, etc.)
• Would need to extend for TTL, LRU eviction, etc.

URQL Graphcache Store

• Size: 523KB total, but core Store is much smaller
• Dependencies: @urql/core, wonka (observables), @0no-co/graphql.web
• Key discovery: Store class is independent of wonka (synchronous)
• Code quality: Well-architected, battle-tested, actively maintained

Core modules:

/store/store.ts       - Main Store class
/store/data.ts        - InMemoryData structures
/operations/query.ts  - Query operations  
/operations/write.ts  - Write/normalize operations

Store API (synchronous):

store.resolve(entity, field, args)     // Read specific field
store.invalidate(entity, field, args)  // Invalidate cache
store.inspectFields(entity)            // List cached fields
store.updateQuery(query, updater)      // Update query results
store.readQuery(query, variables)      // Read cached query

Pros:

• Rich feature set (garbage collection, field-level APIs)
• Better APIs for cache interrogation
• Actively maintained
• Good TypeScript types

Cons:

• More complex extraction (~20-30 files needed)
• Need to replace URQL-specific imports
• Includes client-side features not needed server-side (optimistic updates, layers, reactivity)

Apollo InMemoryCache

• Size: 10.2MB package
• Assessment: Too tightly coupled to Apollo ecosystem, not recommended for extraction

Relay Store

• Size: 1.3MB + fbjs dependencies
• Assessment: Extremely coupled to Relay compiler, not suitable for extraction

---

Query Reduction Discussion

What is Query Reduction?

Taking a query like:

query {
  user(id: "123") {
    id
    name
    email
    address
  }
}

When cache has {id, name}, reduce it to:

query {
  user(id: "123") {
    email
    address
  }
}

Why Major Clients Don't Do This

From Relay Classic → Relay Modern migration:

• Relay Classic had "fat queries" for dynamic query generation
• Removed in Modern for being unpredictable and adding overhead
• Result: 80% smaller bundle, 900ms faster TTI on mobile

Key issues identified:

1. Atomicity/Consistency: Merging data from different timestamps creates inconsistent snapshots
2. HTTP/2 Reality: Modern multiplexing makes multiple requests cheap
3. Complexity vs Benefit: AST manipulation overhead often exceeds network savings
4. Static > Dynamic: Compile-time optimization beats runtime manipulation

When Query Reduction MIGHT Be Worth It

Server-side use cases:

• ✅ Long-running services (amortize optimization cost)
• ✅ Expensive backend queries (reducing DB hits matters more)
• ✅ High request volume (optimization pays off at scale)
• ✅ Controlled environment (you control both ends)

Client-side:

• ❌ Mobile constraints (battery, CPU)
• ❌ Bundle size concerns
• ❌ HTTP/2 makes network cheap

Solving the N+1 Problem with Relay Node Interface

The issue: Nested queries with partial cache hits can fragment into multiple requests.

Solution using Relay node interface:

Original query:

query {
  user(id: "123") {
    id
    name
    organization {
      id
      name
      address  # Missing from cache
    }
  }
}

Transform to:

query {
  user(id: "123") {
    id
    name
    organization { id }
  }
  _org456: node(id: "Organization:456") {
    ... on Organization {
      address
    }
  }
}

Then stitch results back. Keeps everything in one request.

Requirements:

• Server implements Relay node interface
• Cache tracks field-level data with arguments
• Stitching logic to reconstruct original shape

---

Recommendations

Phase 1: Basic Normalized Cache (Start Here)

Fork graphql-norm and adapt:

1. ~8 files, manageable codebase
2. Add Graffle-specific features:
   • TTL/expiration
   • LRU eviction
   • Integration with anyware system
3. Exact query deduplication (cache hit = return immediately)
4. Simple, proven algorithm

Timeline: 1-2 weeks
Value: Deduplication for identical queries, solid foundation

Phase 2: Query Reduction (Optional, Measure First)

Only if measurements show benefit:

1. Implement partial cache hit detection
2. Add query reduction for servers with Relay node interface
3. Make it optional via configuration:

   NormalizedCache({
     queryReduction: {
       enabled: true,
       nodeInterface: true  // Requires server support
     }
   })

4. Track metrics: % of partial hits, latency improvements

Timeline: Additional 2-3 weeks
Risk: Complex, limited real-world benefit (measure first!)

Anyware Integration

Graffle's anyware system is already powerful enough:

Extension.create('NormalizedCache')
  .requestInterceptor(async ({ pack }) => {
    // Before exchange: check cache
    const cached = cache.get(query, variables)
    if (cached) return cached
    
    // After exchange: update cache
    const { unpack } = await exchange()
    const result = await unpack()
    cache.set(query, variables, result)
    return result
  })

No core changes needed - the exchange step is perfect for caching.

---

Conclusion

Start simple:

1. Fork graphql-norm (~500 LOC, proven algorithm)
2. Integrate with anyware for request interception
3. Add TTL, eviction, and other server-side features
4. Ship basic normalized cache

Measure before advancing:

• Track cache hit rates
• Measure partial hit frequency
• Determine if query reduction would actually help

Query reduction is not a bad idea - but it's complex and the benefits are situational. The atomicity trade-off and implementation complexity should be weighed against measured performance gains in real-world usage.

The normalized cache foundation (phase 1) is valuable on its own for exact query deduplication. Query reduction can be added later if data shows it's worth the complexity.