Reading/FY2016-2017 Architecture Needs

The WMF fiscal year 2016-2017 (FY16-17) starts July 1, 2016 and ends June 30, 2017.

What shape does the architecture Reading wants take on for FY16-17? Granted, strategy processes are in flight, but based on what information exists today and looking into the crystal ball what do we think?

Please sub-bullet comment inline with the customary MediaWiki signature. If there are additional areas we really need to think hard about, please add a new bullet.

More specifically:

Do we plan to begin the migration to a Node.js server side composited experience with distinct page components or do we instead plan to do most web things exclusively in MediaWiki PHP?
- I think initially yes the MediaWiki PHP API is the answer and there are no plans at this stage, unless we do a big rewrite like the research web app which right now doesn't look likely within this FY. I suspect the challenge now is to find places where REST services would be more performant e.g. can we prove that using a REST base service would be more efficient than the existing PHP MobileFormatter? If we can a plan will form. Jdlrobson (talk) 23:45, 11 February 2016 (UTC)[reply]
- The web is very entrenched in the mediawiki way and I don't think there's any other option than working on the mediawiki php and api. Maybe the apps would actually benefit from consuming services with distinct page components and composing/caching in the client, since they have the freedom to innovate in those matters. JHernandez (WMF) (talk) 17:45, 2 March 2016 (UTC)[reply]
Do we anticipate creation of more RESTbase microservices (e.g., aggregators or what roughly amount to versioned proxies to ensure backward compatibility) or do we anticipate defining canonicalized shared URL formats (less likely alternative: edge side canonicalizer) and using smaxage for scaling for api.php endpoints?
- On the Android team we're finding the Node.js/REST API approach really promising and we'd keep moving forward in this area with additional (micro)services (e.g., for thumbnails, related pages...) MHolloway (WMF) (talk) 18:13, 10 February 2016 (UTC)[reply]
- On iOS we will be looking into services for feed related work - and may (will) involve other teams. So its not clear whether this will be Node - but its likely. CFloyd (WMF) (talk) 19:14, 11 February 2016 (UTC)[reply]
- On web I'd hope that in certain places these APIs would make a lot more sense e.g. Hovercards but we'd need convincing they can scale to our traffic. Jdlrobson (talk) 23:45, 11 February 2016 (UTC)[reply]
  - I'd say proof/testing that the services can handle the traffic instead of convincing :p JHernandez (WMF) (talk) 17:45, 2 March 2016 (UTC)[reply]
- In any case, having a canonical way to construct API urls for all platforms would be really interesting to maximize cache usage when using the bare mediawiki api. JHernandez (WMF) (talk) 17:45, 2 March 2016 (UTC)[reply]
How much do we foresee a need to grow the memcached object pool?
How much do we foresee a need to grow the Varnish object pool?
How much do we foresee a need to grow the Cassandra object pool?
How much do we foresee a need to grow the parser cache pool?
- The current discussion of splitting the parser cache for mobile to support lazy images and references will have an impact on PC storage needs. --BDavis (WMF) (talk) 16:08, 10 February 2016 (UTC)[reply]
  - Depends on progress in https://phabricator.wikimedia.org/T125899 - would definitely be worth talking to wikidata and their needs Jdlrobson (talk) 23:45, 11 February 2016 (UTC)[reply]
  - +1, it's hard to predict how much right now though. JHernandez (WMF) (talk) 17:45, 2 March 2016 (UTC)[reply]
What sort of notifications architecture do we want and for what purposes?
- What does this mean? Jdlrobson (talk) 23:45, 11 February 2016 (UTC)[reply]
- Is this related to push messages at the bottom? JHernandez (WMF) (talk) 17:45, 2 March 2016 (UTC)[reply]
- I think a push notification infrastructure (Google Cloud Messaging for Android, Apple Push Notification service for iOS) might be useful in the future for trending article notifications, and if we ever do anything with Echo. As a minor thing we could also replace some of the polling we have for remote config, Article of the Day Android widget, Alpha updates but it's not urgent since the polling intervals for these are quite large. BSitzmann (WMF) (talk) 00:32, 3 March 2016 (UTC)[reply]
What sort of trending and hybrid algorithm/human curator endpoints and rule engine do we want for things like Trending?
- On iOS we have been discussing how we can use the curated data on Main pages in the app. These are early discussions, but anything like "in the news", "picture of the day", "article of the day" would be prime candidates for end points CFloyd (WMF) (talk) 19:14, 11 February 2016 (UTC)[reply]
- If we are serious about this Ihink this is a big chunk of work and I think we'll need to dedicate resources to a custom algorithm rather than use something like the page view. The answer is the more datapoints we have feeding into this algorithm the better. Jdlrobson (talk) 23:45, 11 February 2016 (UTC)[reply]
What sorts of generalized independent object storage for new experiences might be desired?
- For the Android app we need a key/value store mechanism to save meta information about saved pages and reading lists in general. BSitzmann (WMF) (talk) 00:32, 3 March 2016 (UTC)[reply]
What sort of additional analytics hosting needs would we ideally have fulfilled?
What do we need to support push services?
- On iOS we would like to deliver updates to users for events like edits or new feed items. These could be push notifications or just background updates so the app is more responsive when it is launched CFloyd (WMF) (talk) 19:14, 11 February 2016 (UTC)[reply]
- Do we want to do this on web? It would be interesting to explore using this for fundraising. I'd imagine a Special:PushMessageUsers which allows you to schedule a push notification to send to subscribers. I see this as being an important part of our future so it would be valuable to start putting architecture in place. Jdlrobson (talk) 23:45, 11 February 2016 (UTC)[reply]
- The options here are varied, we can use node.js servers, we can do it in php, or we can use whatever other language we want. Also it may be wise to consider using a 3rd party service depending on the volume we'll need to handle and the pricing vs the cost and maintenance of doing it ourselves. Definitely it seems like it is something that could be tackled this next FY and it would be interesting to take it into account in the planning (for all 3 platforms). JHernandez (WMF) (talk) 17:45, 2 March 2016 (UTC)[reply]

Note:

Various budget was already requested in the "core narrative" for testing devices, software/SaaS licensing, and known contracting needs
No budget was requested for an additional Mac Mini in the "core narrative" (no input was received on this page).