Reading/Web/Projects/A frontend powered by Parsoid/Notes

Notes of the research project through the weeks, and the planning sessions.

Things left to tackle

loot-ui & loot
- Cache invalidation strategy (on loot api and loot-ui client)
Reports
- Html
  - Wikipedia parser analysis (analogous to Restbase analysis)
- Webpagetest
  - Define how to (and do) measure
Slides

Done Fix a couple of regressions with cached site and broken lazy loaded images
In progress Expand report to include other transformations like superficial markup and ids
Done Defer loading of initial web app data (to minimize initial HTML payload)
Lazy load references in loot-ui
Run Webpagetest to generate reports in browser rendering of wp web, wp mobile web, restbase and loot-ui

Done Add CI running tests & lint on master & pull requests
Done Handle 404s and link tag redirects from restbase
Done Show last modified bar on articles.
Done Gather data about HTML size with different transformations individually and make report (task T120504)
To do Add tests for UI components (server side testing of browser components and its interactions) (delayed)

Research Service worker & learn (Reading list)
Done Document current architecture and code structure in loot-ui (link to docs)
Done Create application Chrome .
Done Start adding search feature.

Client side data caching
- Use indexedDB + fallbacks and cache in the client the state and fetched data to avoid future round trips.
- Visit an article, click a link. Navigate back. Refresh the page as if you visited later, click on the same link, load should be instant.
Lazy loading images
- Improve transformation of images markup in loot.
- Lazy load images in viewport in loot-ui.
- Visit an article, you should see images in viewport loaded. As you scroll you'll see the rest of the images appear instead of all at once blocking the browser and using bandwidth.
Sync with Services team
- Chat about Apps - Api driven frontend - Loot
- Shared basic services composed in aggregation services.

Web application experience : Deployed version
- Web application scaffold (routing, state management, view layer, styles & asset pipeline)
  - Webpack + React + Redux + React-Router
- Move Server Side rendering to web app infrastructure ^
- Pick up navigation & rendering in client after first render
- Basic caching infrastructure (assets & api requests)
Structured article output from API
- Move from basic content HTML response to richer JSON structure with content + other entities (references, images, etc)
Sync with Services team & Bernd about parsoid html findings
- Sync with Bernd Notes (12 Nov)

Clean up loot server produced in offsite's hackathon.
- Move to more efficient parsing library (libxmljs).
- Add more transforms than the basic ones.
- Add basic sectioning.
Reproduce research and produce output report.
Deploy loot to vm in wmflabs and set up git deployments.
Plan next steps, html only version.

Offsite

Hacked a middleman node server to test parsoid html performance with & without stripping content.
Anecdotal results from hacking time show very interesting data.
Plan to formalize the server and write up a more informed report.