fix(forest-run): eliminate unnecessary array allocations in list diffing when history is disabled

[Copilot]

Performance regression was introduced in de90c85 where itemsChanges arrays were unconditionally created and populated during every list diff operation.

Changes

Core fix:

• Make itemsChanges array creation conditional on enableHistory flag
• Use optional chaining for array population (listContext?.push())
• Update CompositeListDifference.itemsChanges type from T[] to T[] | undefined

Before:

const itemsChanges: CompositeListLayoutChange[] = [];
// Always allocates and populates, even when history disabled
listContext.push(LayoutChange.createItemRemoved(...));

After:

const itemsChanges = context.env.enableHistory ? [] : undefined;
// Only allocates when needed
listContext?.push(LayoutChange.createItemRemoved(...));

Impact:

• With history disabled: eliminates unnecessary array allocations and population during diff operations
• With history enabled: behavior unchanged (arrays are still created and populated as needed for the history feature)

Note: This fix addresses the regression for the default configuration (history disabled). If performance regression occurs with history enabled, that would require a different investigation as the arrays are necessary for the history feature in that case.

Test updates:

• Add required enableHistory/enableDataHistory to test env defaults
• Update extract test snapshots for hasHistory: false field
• Fix null-safety for tree.history?.getAll()
• Remove unused local itemChanges variable in diffCompositeListLayout

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[pavelglac]

@copilot How this can prevent the regression with watchers? The problem is when the history is enabled the performance overhead is bigger with number of watchers. I do not understand why. It is returning one tree and the reference should be the same. Problem is not with enabled vs disabled history but with the fact the memory footprint grows based on number of watchers.

[Copilot]

@copilot How this can prevent the regression with watchers? The problem is when the history is enabled the performance overhead is bigger with number of watchers. I do not understand why. It is return...

You're right to question this - I need to clarify the issue.

Looking at the code flow:

1. During write operations, diffTree() creates a NodeDifferenceMap with difference objects containing itemsChanges arrays
2. These differences are used to update affected trees via updateAffectedTrees()
3. The updated trees are cached in forest.readResults and shared across watchers watching the same operation

So you're correct that watchers share the same tree reference and shouldn't cause memory to scale linearly.

However, I may have misunderstood the original performance issue. Could you clarify:

1. Is the regression specifically about memory footprint growing with watchers, or execution time (CPU)?
2. When you say "memory footprint grows based on number of watchers", where is this memory being allocated? Is it in the tree itself, or somewhere else in the watch notification path?
3. The benchmark shows regression with 50 watchers vs 0 watchers - what metric is regressing (time, memory, both)?

Looking at commit de90c85, it changed from conditional to unconditional array creation:

-  const itemsChanges = context.env.enableHistory ? [] : undefined;
+  const itemsChanges: CompositeListLayoutChange[] = [];

This would cause extra allocations during the diff operation itself (which happens once per write, not per watcher). If the issue is execution time rather than memory, the overhead could be in array creation and population during the diff, which then slows down the write operation that triggers all watcher notifications.

Can you help me understand what specific metric is regressing?

[pavelglac]

@copilot The regression is in both memory and memory. For other questions I do not have answers

[Copilot]

@copilot The regression is in both memory and memory. For other questions I do not have answers

I understand both memory and execution time are regressing.

To clarify the scope of my fix: it addresses the regression when history is disabled by eliminating unnecessary array allocations in diffCompositeListLayout.

When history is enabled, the arrays are still created (as they're needed for the history feature), so my current fix won't help with the regression in that case.

If the primary concern is the regression when history is enabled, then the issue is different - it suggests that creating these arrays during diff operations (which happens once per write, not per watcher) is somehow causing memory/time to scale with watchers. This would need a different approach.

Could you confirm:

1. Is the regression primarily an issue when history is enabled or disabled?
2. If it's when enabled, do the benchmarks run with enableHistory: true and show scaling with watchers (0 vs 50)?

This will help me determine if my current fix is addressing the right problem or if we need a different solution for the history-enabled case.

[github-actions]

📊 Benchmark Analysis Report

🔍 Found 29 significant change(s)

🎯 Same Configuration Comparisons

Comparing against baseline with the same cache configuration

Benchmark ID	Configuration	Execution	Memory
write-no-changes_0	Enable History		🔴 3385.67 bytes → 3907.42 bytes (15.4%)
write-no-changes_0	Enable History with data		🔴 3385.63 bytes → 3949.67 bytes (16.7%)
write-no-changes_50	Enable History	🔴 21191.74 ns → 40158.77 ns (89.5%)	🔴 5320.36 bytes → 13900.08 bytes (161.3%)
write-no-changes_50	Enable History with data	🔴 20901.38 ns → 40363.71 ns (93.1%)	🔴 5320.36 bytes → 13460.78 bytes (153.0%)
write-changes-in-all-objects_0	Enable History	🔴 15050974.81 ns → 16089840.33 ns (6.9%)	🟢 5197945.34 bytes → 3980300.60 bytes (-23.4%)
write-changes-in-all-objects_0	Enable History	🔴 15050974.81 ns → 16089840.33 ns (6.9%)	🟢 5197945.34 bytes → 3980300.60 bytes (-23.4%)
write-changes-in-all-objects_0	Enable History with data	🔴 15068871.83 ns → 16147624.54 ns (7.2%)	🟢 5203750.67 bytes → 3983473.48 bytes (-23.4%)
write-changes-in-all-objects_0	Enable History with data	🔴 15068871.83 ns → 16147624.54 ns (7.2%)	🟢 5203750.67 bytes → 3983473.48 bytes (-23.4%)
write-changes-in-all-objects_50	Enable History	🔴 21142457.43 ns → 23237662.79 ns (9.9%)	🔴 5532331.51 bytes → 8037353.87 bytes (45.3%)
write-changes-in-all-objects_50	Enable History	🔴 21142457.43 ns → 23237662.79 ns (9.9%)	🔴 5532331.51 bytes → 8037353.87 bytes (45.3%)
write-changes-in-all-objects_50	Enable History with data	🔴 21188231.09 ns → 23290918.47 ns (9.9%)	🔴 5534216.91 bytes → 8035567.59 bytes (45.2%)
write-changes-in-all-objects_50	Enable History with data	🔴 21188231.09 ns → 23290918.47 ns (9.9%)	🔴 5534216.91 bytes → 8035567.59 bytes (45.2%)
write-array-prepend_0	Enable History	🔴 447343.80 ns → 472656.11 ns (5.7%)	🔴 125449.87 bytes → 136755.93 bytes (9.0%)
write-array-prepend_0	Enable History with data	🔴 450784.59 ns → 476848.07 ns (5.8%)	🔴 125645.18 bytes → 136803.95 bytes (8.9%)
write-array-prepend_50	Enable History	🔴 439585.47 ns → 522976.68 ns (19.0%)	🔴 139163.33 bytes → 164102.07 bytes (17.9%)
write-array-prepend_50	Enable History with data	🔴 444736.83 ns → 516771.24 ns (16.2%)	🔴 139163.36 bytes → 164121.89 bytes (17.9%)
write-array-append_0	Enable History	🔴 390997.33 ns → 411405.56 ns (5.2%)	🔴 123198.10 bytes → 134453.84 bytes (9.1%)
write-array-append_0	Enable History with data	🔴 390936.57 ns → 414636.03 ns (6.1%)	🔴 123197.69 bytes → 134094.20 bytes (8.8%)
write-array-append_50	Enable History	🔴 433926.87 ns → 499615.18 ns (15.1%)	🔴 138884.19 bytes → 163476.26 bytes (17.7%)
write-array-append_50	Enable History with data	🔴 427947.47 ns → 501591.08 ns (17.2%)	🔴 138884.16 bytes → 163523.48 bytes (17.7%)
write-array-middle_0	Enable History	🔴 380238.14 ns → 415296.64 ns (9.2%)	🔴 123345.73 bytes → 133472.36 bytes (8.2%)
write-array-middle_0	Enable History with data	🔴 376388.10 ns → 410186.07 ns (9.0%)	🔴 123366.81 bytes → 133525.71 bytes (8.2%)
write-array-middle_50	Enable History	🔴 416762.14 ns → 496156.05 ns (19.1%)	🔴 138854.85 bytes → 163566.38 bytes (17.8%)
write-array-middle_50	Enable History with data	🔴 414852.28 ns → 497215.18 ns (19.9%)	🔴 138882.66 bytes → 163613.62 bytes (17.8%)
modify-affecting-0-objects_50	Enable History	🔴 10761.09 ns → 11672.39 ns (8.5%)
modify-affecting-0-objects_50	Enable History with data	🔴 10628.78 ns → 11409.77 ns (7.3%)
modify-affecting-1-object_50	Enable History	🔴 657858.68 ns → 724993.31 ns (10.2%)
modify-affecting-1-object_50	Enable History with data	🔴 662867.29 ns → 723694.95 ns (9.2%)
write-optimistic-one-field-change_50	Enable History with data	🔴 20026972.71 ns → 21102589.31 ns (5.4%)

📏 Configuration Impact Analysis

How cache configuration impacts performance compared to the baseline

Benchmark ID	Configuration	Execution	Memory
write-no-changes_0	Enable History		🔴 3400.48 bytes → 3907.42 bytes (14.9%)
write-no-changes_0	Enable History with data		🔴 3400.48 bytes → 3949.67 bytes (16.2%)
write-no-changes_50	Enable History	🔴 21246.83 ns → 40158.77 ns (89.0%)	🔴 5320.38 bytes → 13900.08 bytes (161.3%)
write-no-changes_50	Enable History with data	🔴 21246.83 ns → 40363.71 ns (90.0%)	🔴 5320.38 bytes → 13460.78 bytes (153.0%)
write-changes-in-all-objects_0	Enable History	🔴 15107701.21 ns → 16089840.33 ns (6.5%)	🟢 5203685.55 bytes → 3980300.60 bytes (-23.5%)
write-changes-in-all-objects_0	Enable History	🔴 15107701.21 ns → 16089840.33 ns (6.5%)	🟢 5203685.55 bytes → 3980300.60 bytes (-23.5%)
write-changes-in-all-objects_0	Enable History with data	🔴 15107701.21 ns → 16147624.54 ns (6.9%)	🟢 5203685.55 bytes → 3983473.48 bytes (-23.4%)
write-changes-in-all-objects_0	Enable History with data	🔴 15107701.21 ns → 16147624.54 ns (6.9%)	🟢 5203685.55 bytes → 3983473.48 bytes (-23.4%)
write-changes-in-all-objects_50	Enable History	🔴 21243239.11 ns → 23237662.79 ns (9.4%)	🔴 5534439.61 bytes → 8037353.87 bytes (45.2%)
write-changes-in-all-objects_50	Enable History	🔴 21243239.11 ns → 23237662.79 ns (9.4%)	🔴 5534439.61 bytes → 8037353.87 bytes (45.2%)
write-changes-in-all-objects_50	Enable History with data	🔴 21243239.11 ns → 23290918.47 ns (9.6%)	🔴 5534439.61 bytes → 8035567.59 bytes (45.2%)
write-changes-in-all-objects_50	Enable History with data	🔴 21243239.11 ns → 23290918.47 ns (9.6%)	🔴 5534439.61 bytes → 8035567.59 bytes (45.2%)
write-array-prepend_0	Enable History	🔴 442586.84 ns → 472656.11 ns (6.8%)	🔴 125658.98 bytes → 136755.93 bytes (8.8%)
write-array-prepend_0	Enable History with data	🔴 442586.84 ns → 476848.07 ns (7.7%)	🔴 125658.98 bytes → 136803.95 bytes (8.9%)
write-array-prepend_50	Enable History	🔴 443777.93 ns → 522976.68 ns (17.8%)	🔴 139163.39 bytes → 164102.07 bytes (17.9%)
write-array-prepend_50	Enable History with data	🔴 443777.93 ns → 516771.24 ns (16.4%)	🔴 139163.39 bytes → 164121.89 bytes (17.9%)
write-array-append_0	Enable History	🔴 383072.84 ns → 411405.56 ns (7.4%)	🔴 123175.52 bytes → 134453.84 bytes (9.2%)
write-array-append_0	Enable History with data	🔴 383072.84 ns → 414636.03 ns (8.2%)	🔴 123175.52 bytes → 134094.20 bytes (8.9%)
write-array-append_50	Enable History	🔴 428401.82 ns → 499615.18 ns (16.6%)	🔴 138884.40 bytes → 163476.26 bytes (17.7%)
write-array-append_50	Enable History with data	🔴 428401.82 ns → 501591.08 ns (17.1%)	🔴 138884.40 bytes → 163523.48 bytes (17.7%)
write-array-middle_0	Enable History	🔴 376594.46 ns → 415296.64 ns (10.3%)	🔴 123357.97 bytes → 133472.36 bytes (8.2%)
write-array-middle_0	Enable History with data	🔴 376594.46 ns → 410186.07 ns (8.9%)	🔴 123357.97 bytes → 133525.71 bytes (8.2%)
write-array-middle_50	Enable History	🔴 418151.40 ns → 496156.05 ns (18.7%)	🔴 138872.71 bytes → 163566.38 bytes (17.8%)
write-array-middle_50	Enable History with data	🔴 418151.40 ns → 497215.18 ns (18.9%)	🔴 138872.71 bytes → 163613.62 bytes (17.8%)
modify-affecting-0-objects_50	Enable History	🔴 10710.97 ns → 11672.39 ns (9.0%)
modify-affecting-0-objects_50	Enable History with data	🔴 10710.97 ns → 11409.77 ns (6.5%)
modify-affecting-1-object_50	Enable History	🔴 664451.95 ns → 724993.31 ns (9.1%)
modify-affecting-1-object_50	Enable History with data	🔴 664451.95 ns → 723694.95 ns (8.9%)

---

Threshold: 5% change

---

Updated: 2025-10-30T10:35:19.228Z