Growth/Engineering/Incident tracking
< Growth | Engineering
Please do not mark this page for translation yet. It is still being drafted, or it contains incomplete translation markup which should be fixed before marking for translation. |
This page tracks "incidents", meaning notable, user-facing bugs, for code in GrowthExperiments. We are tracking these to assess frequency of issues and decide what/how we need to adjust practices in response, not to cast judgment or blame.
Criteria for adding tasks:
- would have blocked the train, but fixed in advance
- blocked the train
- incorrect/missing analytics for Growth features
- significant performance change
Criteria for low/medium/high:
- high: blocked train, caused outage in production
- medium: ?
- low: ?
| Date | Task | Tagged with #wikimedia-production-error | Train rollback | How the issue was found | Lessons / Follow-up task(s) | Impact | Severity |
|---|---|---|---|---|---|---|---|
| 2022.06.23–2022.07.04 | T311916: "Add an image" structured edits add a blank line instead of an image | no | no | Patroller on cswiki informed Martin, Martin informed the team. | end-to-end tests would have prevented the problematic patch in VisualEditor from being merged in the first place. | all edits saved with the AddImage plugin contained a blank line, rather than an image and caption. | high |
| 2022.06.23- 2022.06.29 | T311467: [wmf.17-mobile] "Suggestions" label has no padding | no | no | Manual QA by Elena | Visual regression or e2e testing could catch this. | No close button for mobile users of structured tasks, semi-broken UI | medium |
| 2022.03.10 – 2022.05.25 | T309152: [regression-wmf.13] Add image: Missing "View image details" link | no | no | Found by Marshall and Kirsten in manual review of the feature |
|
"View image details" button was missing on caption step of "add image" plugin, which makes writing captions more difficult. | medium |
| 2022.03.08 | T303251 Error: exception of type Error: Cannot unpack array with string keys | no | no | apitesting-test jobs failed for GrowthExperiments. | task T303255 perhaps core api-testing job should also run api-testing tests for extensions/skins; that would have caught this in the core patch that caused the breakage. | betalabs was broken for some help panel guidance message requests for a few hours. If undetected, this would have been a wmf.26 train blocker. | low |
| 2022.02.24 | T302746 Post-edit dialog does not appear for newcomer task edits | no | no | Rita and Marshall noticed on testwiki, and commented in Slack on February 24. | We currently have no end-to-end tests verifying the presence of the post-edit dialog. (A patch pending review will this test.) That wouldn't have caught the problem in VisualEditor's patch approval, but would have when GrowthExperiments patches would be checked. | Newcomer experience degraded, miscellaneous bugs due to post-edit dialog not loading, | medium |
| 2022.02.10 | T301486 Add an Image: back/next buttons in onboarding screens are not instrumented | no | no | Morten, while doing analysis on add image. | missed during code review and QA, not a regression. | Clicks on back/next buttons in onboarding dialog were not instrumented, so we don't have data to help better understand how users interact with this dialog. | low |
| 2022.02.06 | T301062 Error: Call to a member function getId() on null | no | no | @Sunny00217 on zh betawiki | side-effect of the MentorProvider refactoring, where an important check was unintentionally removed. Follow-up patch adds a regression test. | zh betawiki was broken for new user accounts. This would have been a train blocker, similar to the December 15 2021 incident. | low |
| 2022.01.27 | T300336 Add an image: Mobile-only quality gate shown to desktop users | no | no | Marshall and Elena, after the feature flag was enabled in production. | A small subset of newcomers who are opted into imagerecommendation variant would have experienced a confusing dialog when attempting to do image suggestion tasks on desktop. | medium | |
| 2021.12.15 | T297827 Error: Call to a member function getId() on null | yes | yes | Antoine, after train promotion to group1 | Newcomers were unable to login on cawiki for about 25 minutes. group1 wikis were rolled back to previous train version. | high |