Readers/Web/Team/Pushing production code from beta to stable

The reading web team use Extension:BetaFeatures and the built in MobileFrontend beta mode ($wgMFEnableBeta = true;) to test, validate code against real users and gain feedback. When we are comfortable with a new change we push it from the beta to production environment.

Benefits

• Via EventLogging get a sense of how a feature performs
• Allow time for translators to translate a feature
• Test features against real data / edge cases.
• Allow time for real users to provide feedback

Developing features behind feature flags or for beta

• Keep code sandboxed from stable. Use ResourceLoader modules and the SkinMinervaBeta class for mobile (or hooks for desktop). This will minimise regressions to production code.
• Feature flag features. All features should be feature flagged, even those in deployment. As a code reviewer and code author it is your duty to ensure this happens!
• Allow time for i18n messages to be collected. Make sure they are in the repository and feature flagged for at least 2 weeks

Pushing to stable

• Make sure when pushing code to stable that browser tests are in place
• When replacing existing features be sure to remove old browser tests
• Avoid changing any message keys or ResourceLoader module names to avoid hitting caching problems (see below)

Optimise for allowing reverts

All features, regardless of whether they are new or are replacing existing ones should be behind feature flags.

When a feature is flipped on, it should always be assumed that something may go wrong post deployment and that it may need to be reverted back.

With PHP only changes reverting is usually trivially and nothing in particular needs to be done. However, for any feature which includes i18n messages and ResourceLoader modules special care must be taken. A removed i18n message will lead to the removal of all translations in that repository via an update by translatebot. ResourceLoader modules that are removed will report themselves as `missing` in some cached ResourceLoader resources which can lead to JavaScript errors. Due to these known problems, it is essential that any removed feature should be retained in the repostitory for a period of time. It is up to the tech lead as part of sign off to determine this time frame and  ensure that the old feature is eventually removed from the repository.

Process for deployment

• Allow a possible revert path for any change, regardless of how certain you are it is working.
• The Product manager, Quality assurance (if available) and designer should be engaged as soon as a feature is pushed to production. A card should be signed off in the current sprint before the close of play on Wednesday, the day before it rolls out to production for all users.

Signing off changes

• Retain any old code for at least 2 weeks, to aid any necessary reverts, then remove from the repository.
• Remove EventLogging within a month.
  • Ideally we should be able to collect all data we need within a month.