Problem statement

Storybook has the ability to generate story files for components, but it's quite hard to know whether the generated story will just work. We need to be able to (1) measure the success of an approach, (2) understand the failure points, and (3) gracefully fallback to a manual setup when the approach fails.

Milestones

M1: Solidifying story creation API

Owner: @yannbf
Complete by: @date

• Create a single, reusable Node API responsible for generating stories
• Use the API in the current story generation logic in "CreateNewStory" flow

M2: Automating the API usage

Owner: @yannbf
Complete by: @date

• Add a new (possibly temporary) CLI command to generate stories based on a file glob
• Add a heuristic to detect "suitable components sample"
• Add a step to set up and run Vitest tests against the newly created stories
• Add a step to process and report test results
• Extend the Storybook vitest plugin to run additional tests

M3: Virtual story tests

• Investigate on running story tests on virtual files

M4: Integrating the API

• Experiment with the newly worked API in the CreateNewStory flow
• Extend telemetry metadata to include theming packages

M5: Automating the research

Owner: @yannbf
Complete by: @date

• Create a GitHub action to automate generating stories, testing and reporting results from any given repo
• Select at least 20 different repos to run tests at
• Gather research findings and think about preview error categorization

========= Everything below this line is strictly nice-to-have =========

Related issues that may be resolved by this project

• Preview error categorization
• Storybook error UX improvements
• Considering to ship the experimental CLI flag