Wikimedia Product Infrastructure team/Push Notifications Infrastructure

Group:	Product Infrastructure
Start:	2020-01
Team members:	Bernd Sitzmann, Mateus Santos, Michael Holloway, Yiannis Gianellos
Backlog:	#Push-Notification-Service

Background

The Wikimedia product teams have a long-standing desire to leverage push notifications to drive user engagement and retention. Product Infrastructure is working on designing and building a push notifications platform for Wikimedia. The goal of the project is to use push technology to provide MediaWiki notifications to Wikimedia web and app users.

Push Notifications (disambiguation)

Push Notifications (capability; the ability to)
Push Notifications (SaaS: The full software stack that provide the push capability)

How to get started

I want to contribute to the project

You can find more information on how start contributing to the project in the project repository homepage.

I want to add a new push notification to my app

Our push notifications implementation is heavily based on Echo, so in order to integrate your app with push you need to also interact with Echo.

You can register new tokens from your users devices querying the echopushsubscriptions on MediaWiki action API.
- For example: /api.php?action=help&modules=echopushsubscriptions
Echo sends a checkEchoV1 push notification to the registered devices when a new EchoEvent is created.
- Here you can find more information on how to register a new Echo notification.
When a new checkEchoV1 push notification is received on the device, you can query the MediaWiki action API to retrieve more information about the event.
- For example: /api.php?action=query&meta=notifications

In case there are limitations on the integration and using EchoEvent is not possible, you might need to create a new push notification type other than checkEchoV1 and orchestrate sending notifications manually.

Version Notes

Release Date	Version	Features
09/29/2020	v1.0.0	- Echo can send messages to the user via native Wikipedia apps for Android and iOS - App users can subscribe and unsubscribe to receive Echo notifications - The system can limit the rate and volume of messages to prevent server overload - Invalid subscriptions are automatically removed from the system upon notification request failure

Upcoming Project Phases

Initial R&D: Q3 FY 2019-2020 (January - March 2020)

Initial product use cases and cross-team prioritization [Link to google doc, or move google doc table to wiki?]
Technical planning and investigation
Technology department consultation
TechCom RFC submitted

Phase 1: Native Echo

Initial use cases are mentioned in the TechCom RFC and revolve around forwarding Echo notifications as push notifications for the native apps.

Clients supported:

Wikipedia iOS App
Wikipedia Android app

Use cases supported:

Echo can push existing messages received by a user to the user via native apps for Android and iOS
App users can subscribe and unsubscribe to receive Echo notifications
App and web users can subscribe and unsubscribe to be pushed specific message types (adding a column to delivery preferences for "Push Notifications")
App users can manage their subscriptions via the native apps
The system can limit the rate and volume of messages to prevent server overload
The platform has a Push Notification service for managing these needs in a shared platform competent
Users who can no longer be reached after repeat attempts are removed from the service on a regular cycle
Product Infrastructure and SRE can monitor basic service usage via a standard dashboard

Estimated Timeline

April 2020

Incorporate RFC feedback
Finalize Phase 1 implementation plans
Start repository and begin exploratory implementation of the required services

May 2020

Initial service development

June-July 2020

Service deployed to Beta Cluster
Client engineers develop prototypes against initial implementations on staging

August 2020

Technology reviews (DBA/Security/Performance) & follow-ups

Sept-Oct 2020

Early-mid September: initial service launch date on k8s
Target public launch date: Sept 21
Post-launch bug fix iteration
Beginning app client adoption of production push notifications

Technology approvals required:

SRE machine allocation (for launching on k8s)
Performance review
Security review
DBA review

Phase 2: Open Echo

Phase 2 is focused on extending the push system to allow pushes to browsers via the Web Notifications API

Clients supported:

Wikipedia iOS App
Wikipedia Android App
Browsers implementing the Push and Notifications APIs

Use cases supported:

As a web user I can receive my Echo notifications as browser notifications
As a web user I can be prompted to enable notifications in context where that in meaningful (ie. not just because I land on our domain)
As a web feature team I can implement subscription prompts in context where that in meaningful (ie. not just because I land on our domain)
App and web users can subscribe and unsubscribe to be pushed specific message types (adding a column to delivery preferences for "Push Notifications")
As an Echo developer I can add a new notification type and have it also be delivered via push

Components

There is currently no estimated timeline for Phase 2 and beyond. Please check for future iterations on this page once those plans become firmer. However, we do know the basic components that require development for this phase, and a rough order of operations for their implementation.

Initial case defined by a web feature team (eg. Editing or Growth)
Define initial client browser target and extend push service to push to the selected platform(s)
Web service worker implementation
Implement/update sign-up and subscription UIs for web

Phase 3: Other Than Echo

The final envisioned stage of development is the creation of new types of notifications by client teams to serve specialized use cases. Example use cases include:

Re-engagement and encouragement of new or "lapsed" editors ("We haven't seen you in a while")
New feature and software update announcements
Campaigns and fundraising announcements ("Add a reference today to be part of #1Lib1Ref!")
Updates and recommendations for topics or portals of interest to a reader
Other News or Current events related content suggestions
Notifications for anonymous users

The order and time-frame for these will depend on the needs of the teams and their target users.

Architectural Overview

The push notification system consists of two components:

Push subscription management and notification request support in the Echo extension; and
A standalone service (push-notifications) for batching requests for submission to third party push providers.

In the initial (apps-only) implementation, to mitigate the user privacy and security risks associated with sending potentially sensitive information to users via third-party systems, pushed messages will contain only a short descriptor (e.g., checkEchoV1) to prompt the apps to retrieve message content from Wikimedia servers (e.g., by request to the Action API's notifications module). Further, the push-notifications service will collect push notification requests and send them in periodic batches in order to increase the difficulty of drawing inferences between on-wiki actions and messages traversing the public internet.

Echo extension

The Echo extension has been updated to handle storage of push subscriptions and to provide an API interface for clients to manage them. Push subscriptions are stored by central user ID in the echo_push_subscription table, and can be added or removed by clients using the Action API echopushsubscriptions module.

Additionally, the PushNotifier class has been created to handle sending MediaWiki notifications by push. When an Echo event occurs and the user has chosen to be notified about events of that type via push, an EchoPushNotificationRequest job is scheduled to retrieve all stored push subscriptions associated with the user's central ID and send a request to the push-notification service that a message be pushed via the relevant service for each subscription.

Push-notifications service

A new Node.js service, push-notifications, has been created to process push notification requests and submit them to platform-provided push APIs including Firebase Cloud Messaging (FCM) and Apple Push Notification Service (APNS). The push-notifications service is boostrapped from the Wikimedia Node.js service template and written in TypeScript. Interactions with FCM and APNS are supported by Google's official firebase-admin module and the popular community-developed node-apn module.

A primary function of the service is to collect and batch push notification requests before submitting them to platform push APIs. The number of messages to batch before submission and the time to wait between submissions can be defined in the service configuration. Additionally, the time to wait before submitting can be randomized within configurable lower and upper bounds.

Contact the team

IRC #product-infrastructure