Storybook was originally implemented with PagePreviews, and then later extended to Vector with some component stories and skin stories that mocked both the legacy and modern Vector skin. We continued to use it for early Desktop Improvements features like user links and sticky header. However, it was not used for later features, and the existing cases quickly became out of date or broken.
Conversation
editJon: where did we land on storybook. Are we ready to remove it from Minerva and/or Vector ? https://phabricator.wikimedia.org/T310004 https://phabricator.wikimedia.org/T321102
<lots of yeses>
Jan: I’m down too but I think it’d be good to reflect on why Storybook wasn’t very useful in these repositories. At the skin level, there’s an immense amount of integrations that we’re dealing with in terms of the skin, extensions, gadgets, user modes, etc. The extensions alone like ULS, VisualEditor, Echo introduce a tonne of variants that we have to account for. We also have feature flags that introduce their own complexity, as well as permission levels like anon vs logged-in that need to be dealt with differently. Not to mention templates. So although I’m a big fan of Storybook, with this level of integration, I don’t think looking at the components in isolation provides a lot of value since the skin is so context dependent.
Bernard: agreed, i think storybook is more suited for something like codex, a component library where we want stories for isolated components. what we need is covered better by a combination of jest, jest snapshots, and pixel.also, updating templates and template data in jest is already a good amount of work, it’s pretty time consuming to update both jest and storybook
Nick: I have similar thoughts.I think storybook works better in situations that are capable of being client-side rendered, where you’re not having to tell storybook how to render the story (e.g. needing to call mustache.render and having to import the necessary styles). That makes me lose confidence that it resembles production, and it also requires a fair amount of work in setting up each story as a result. On the other hand, I find the storybook inside Popups more valuable because much of the rendering logic in each story is encapsulated inside Popups itself
Bernard: should we incorporate page previews into jest/pixel then?
Jon: I agree with Nick that storybook in Popups is definitely valuable (because it’s the content + component we’re testing not the component). I doubt jest or Pixel would add much value
Storybook = does the preview show correctly in a variety of ways.
Bernard: could we use content provider so that we can get previews to use in pixel? pixel allows us to specify the root document to run visual regressions on, so hypothetically couldnt we specify just the page preview?
Jon: the problem with the content provider is content is live so edge cases we identify now, might evolve with content improvements.Generating a static page of examples would likely be a better approach since we have the CSS and JS to create the previews handy.
Jan: Visual diffs for Popups are definitely possible, Storybook has a Visual Test feature but it requires a subscription to Chromatic. I do recall struggling with Storybook when it came to setting languages. The interface language vs. content language paradigm is very MediaWiki specific, so it might be valuable to run Popup in Mediawiki with Pixel just for that.
Regarding the content provider, I just checked it looks like it does support proxying article revisions, so we could peg the tests to a specific revision (oh wait the popup content itself would still be dynamic…) never mind.
Recommendations
edit- Remove Storybook usage in Skins in favor of a combination of Pixel and Jest
- Continue using Storybook for PagePreviews
- Still useful in PagePreviews because it's the content + component that's tested, and because the rendering logic in each story is encapsulated inside Popups itself
Current Challenges
edit- Storybook not suited for skin level coverage
- Skins have an immense amount of complexity with extensions (ULS, Echo), gadgets, user modes (anon vs logged in), and feature flags that we need to account for. Skins are too context dependent and have too many integrations.
- Time consuming to update
- Not updating template data and rendering is the most frequent reason why Storybook became outdated/broken. This is also duplicative effort with Jest, which is at least is integrated into our CI.
- Storybook works better in situations that are capable of being client-side rendered, where you don't have to tell storybook how to render the story (e.g. needing to call
mustache.renderand having to import the necessary styles).