Wikimedia Platform Engineering/MediaWiki Core Team/Quarterly review, March 2013

What we have done since September 2012

Projects:

• Lua scripting
  • Tim, Brad, Chad, Aaron
  • Profiler
  • Richer Lua environment implemented:
    • mw.ustring
    • mw.language
    • mw.uri
    • mw.site
  • Deployment to English Wikipedia and 13 other wikis!
• TemplateSandbox
  • Brad
• Gerrit
  • Chad, Christian
  • Upgraded to Gerrit 2.5, 2.6
  • Fixed various bugs
  • Set up Github replication
• Test infrastructure
  • Antoine
  • Beta cluster
  • Continuous integration
    • Zuul
    • Extension unit tests
• Datacenter migration
  • (whole team effort)
  • Removed NFS dependency from Math, Timeline, ConfirmEdit, ExtensionDistributor
  • Read-only mode tested
  • Ceph support
• Media storage
  • Aaron, Jan
• Job queue
  • Aaron
  • Performance
  • Duplicate removal
  • Refactoring
  • Redis backend
  • External DB backend
• Admin tools
  • Chris, Aaron
  • AbuseFilter global rules
  • ConfirmAccount admin notification
  • CentralAuth mass account locking
• Lucene search maintenance
  • Ram, Tim, Aaron
  • RMI timeouts
• Wikivoyage migration
  • Chris, Sam
  • Extension review
  • Account migration
• Wikidata deployment
  • Tim, Sam, Chad, Chris
  • Code review
  • Deployment

Projects and activities that will continue into next quarter (March–June 2013)

Perpetual

• Deployment
  • Core deployments
  • External team deployments
    • Wikidata
    • Board elections
• Shell bugs
• Test infrastructure (Beta cluster and continuous integration)
• MediaWiki operations (performance, debugging, ops team support)
• Large code review projects (e.g. Wikidata)
• Security issue response
• Git/Gerrit improvement
• Incremental architectural improvements
• Volunteer contributed deployments
• Admin tools
• Multimedia maintenance

Projects with ends in sight

• Lua (March deployment plus education and wrapup)
• Search
• Media storage
• Current large code review projects
  • EducationProgram
  • Wikidata
  • API redesign
  • E3 analytics API
  • Fundraiser 2013
  • Wikimania scholarship forms (later this year)

New projects starting next quarter (March–June 2013)

Major projects

• OAuth
  • Chris's estimate: 2 weeks
  • Rob's estimate: 3 people for a month plus a full-team sprint for 1 week
• Volunteer contributed deployments (Deployment queue)
  • Blame map extension
  • Category collation improvements (bugzilla:44667, bugzilla:43799)
  • VipsScaler extension (Package in Ubuntu http://packages.ubuntu.com/search?keywords=libvips-tools )
  • Score extension
• Search
  • Legacy search improvements
  • Solr investigation

As time allows

• Incremental architectural improvements
  • JobQueue:
    • Move to redis to reduce master load? (https://gerrit.wikimedia.org/r/#/c/39174/)
    • Implement immediate priority jobs
    • Aaron's estimate: Switch to Redis in 2 days full time plus 1 week monitoring
  • Caching improvements:
    • Redirect to canonical encoding (bugzilla:27935)
    • When a commons image is updated, update the pages that use it (bugzilla:22390)
    • Include version in thumbnail URL (bugzilla:17577)
• Admin tools development
  • Working down the list
• OpenID Provider

Back burner

• git-deploy
• Configuration database
• Central code repo

Outcomes of potential projects

Project	Developer encouragement	Features	Bug fixes	Performance
OAuth		Y
Admin tools		Y
Caching improvements			Y
Volunteer contributed project deployment	Y	Y		Y
Job queue		Y		Y
Configuration database	Y

Breakdown of activities by person

• Aaron Schulz
  • OAuth, media storage, incremental architectural improvements, volunteer contributed deployments (VipsScaler, Score)
• Antoine Musso
  • Test infrastructure, OAuth sprint
• Brad Jorsch
  • OAuth, Lua, API reviews, volunteer contributed deployments
• Chris Steipp
  • OAuth, security releases, Wikidata reviews, other security reviews, volunteer contributed deployments (Blame Map review), admin tools development
• Chad Horohoe
  • Git/Gerrit, Search, OAuth sprint
• Greg Grossmeier
  • Release management
• Munagala Ramanath (Ram)
  • Search, OAuth sprint
• Sam Reed
  • Releases, shell bugs, Education Program review, OAuth sprint, volunteer contributed deployments
• Tim Starling
  • OAuth, Lua, volunteer contributed deployments, incremental architectural improvements
• Christian Aistleitner
  • Gerrit
• Jan Gerber
  • TimedMediaHandler

Questions for the group

• Deployments/releases
  • Should we aspire to a faster cycle than once every 2 weeks?
  • How active of a role should we play in support of non-Wikimedia uses of MediaWiki?
• Test infrastructure
  • What role should beta play in the deployment cycle?
  • Do we make people deploy to beta before production, period?
• Code review
  • What statistics should we track, and how should we go about getting those numbers?
    • Talk to Max, he started making some reports using the new REST API

Meeting summary

Summary for the meeting.