Wikimedia Engineering/Report/2014/October
Major news in October include:
- the removal of support for SSL 3.0 on Wikimedia sites to protect users against the "POODLE" vulnerability;
- the release of the first native “Nearby” functionality in the Android and iOS Wikipedia apps;
- the launch of the Structured Data project, an initiative to augment Wikimedia Commons using the technology developed for Wikidata.
Engineering metrics in October:
- 150 unique committers contributed patchsets of code to MediaWiki.
- About 27 shell requests were processed.
Upcoming events
editThere are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.
For a more complete and up-to-date list, check out the Project:Calendar.
Date | Type | Event | Contact |
---|---|---|---|
3 November 2014 | Tech Talk: The MediaWiki Content Translation Extension 1800-1900 UTC in #wikimedia-office connect. | Joel Sahleen | |
14 November 2014–16 November 2014 | Dutch GLAM and Wikidata hackathon (Amsterdam, Netherlands) |
Personnel
editAre you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- Senior Software Engineer - Services
- Software Engineer - Maps & Geo - Mobile
- Software Engineer - Mobile - iOS
- Release Engineer
- Technical Writer
- Full Stack Developer - Analytics
- Research Analyst
- Agile Coach/ScrumMaster - Team Practices Group
- Senior Technical Product Manager
- Community Liaison
- Community Liaison (PT Contract)
- Operations Security Engineer
- UX Senior Designer
- UX Senior Design Researcher
- UX Visual Design Fellowship
- Mobile Partnerships Regional Manager
Announcements
edit- Marcel Ruiz Forns joined the Analytics Development team (announcement).
- Guillaume Paumier joined the Product and Strategy team as Senior Analyst (announcement).
- Bartosz Dziewoński joined Wikimedia as a Features Contractor (announcement).
- Moriel Schottlender converted to full time staff member status as a Software Engineer in Features Engineering (announcement).
- Marielle Volz joined the Wikimedia Foundation as a Software Engineer for the Editing team (announcement).
Technical Operations
editDallas data center
- In October we completed the initial stage of our data center deployment in Dallas; finishing the setup of essential infrastructure, and making sure all essential project data is being replicated and/or backed up in Dallas. The Ops team is now moving towards the next phase; making
codfw
capable of serving the wikis independently ofeqiad
, ideally using HHVM.
Tampa data center
- On the first day of October we shutdown nearly all remaining servers in Tampa, allowing some room to resolve unexpected problems should they occur. On October 6, we started removing all equipment from their racks, and shipped equipment with remaining value to Dallas. By October 12, we had vacated the facility completely, marking the end of 10+ years of Wikipedia's home in Tampa.
Labs metrics in October:
- Number of projects: 145
- Number of instances: 413
- Amount of RAM in use (in MBs): 1,992,192
- Amount of allocated storage (in GBs): 20,395
- Number of virtual CPUs in use: 975
- Number of users: 4,109
Tool metrics:
- Number of tools: 947
- Number of tool maintainers: 550
Wikimedia Labs
- Labs suffered a brief, partial outage on October 7 due to a presumed out-of-memory failure on a virtualization node. Everything was back up and running after about 30 minutes; Andrew rebalanced instance load to forestall similar such failures.
- New hardware has been ordered to expand Labs VM capacity. With luck this will be in place within the next month or so.
- Marc redesigned the default base image for labs instances. /var/log is now in its own partition, and we support some limited partition resizing.
- Yuvi has made a good start on setting up instance state monitoring.
- Marc added an Ubuntu Trusty bastion and exec node to Tool Labs. Some work is still in progress here due to package renames and dependency changes.
- Marc wrote many pages of operator documentation for Tool Labs.
Editor retention: Editing tools
editIn October, the team working on VisualEditor improved some features, prepared the way for table structure editing, and fixed over 60 bugs and tickets.
The keyboard shortcuts for items in the toolbar's menus now show on the right hand side, to make them easier to find. We made a few improvements to the display and editing of templates. Firstly, template fields can now have a value automatically set, like today's date for clean-up notices. Links inside templates now show in red if their targets don't exist, like for the editing surface. Last month's change to warn against saving a template with missing required fields was improved by highlighting them, as is the warning about trying to make an image 0 pixels wide.
The initial tools to edit the structure of tables were finalised and will be made available next month. We made progress on providing a new auto-filled citations tool, and improvements to the link editing and media searching tools, all of which will be coming in the near future.
The deployed version of the code was updated five times in the regular release cycle (1.25-wmf3, 1.25-wmf4, 1.25-wmf5 and 1.25-wmf6).<indicator>
s, a way to show icons which tell the user something about the page in MediaWiki. These features are now supported in the Vector, Monobook, Cologne Blue and Modern skins, and will replace the need for the local CSS hacks that some wikis have used to approximate this feature. The TemplateData extension now wraps its contents in a <noinclude>
tag unless editing a sub-page, and the help link's target can now be set to a page other than the one on MediaWiki. The last vestiges of the CiteThisPage extension were removed from the Cite extension, as part of a complex manoeuvre to simplify the naming of the system; it now has its own home. The team's more significant work, on front-end standardisation, is now reported under that project.Core Features
editFront-end
editFront-end libraries standardization
- In October, the Front-End Standards Group continued their work to extend and adopt the OOjs UI library. This included the finalisation of the initial version of the MediaWiki theme in collaboration with the Design team, which can be explored in the online demo; this will be deployed into MediaWiki's use of OOUI in the next few weeks. The library now has a parallel, PHP language port. This will allow pages to be created server-side and later as a progressive enhancement made rich for JavaScript-supporting clients. This PHP port will be made available inside MediaWiki soon, and will be used to provide the framework for the future skin architecture. The plan is to demonstrate it in rebuilding a small number of the core interfaces into OOUI by the end of the calendar year. Other improvements to the library include support for toolbar menus to only show a few items initially with option to expand later, and showing keyboard shortcuts, expansion of the developer documentation, and fixing a number of bugs. Two new widgets were created – a widget for a button that's also an input, and a widget for a progress bar, which will be used as part of the work to convert the new PDF generation system to use the OOjs UI framework.
MediaWiki Core
editSpecial:GlobalRenameRequest is the form that users who are renamed post-finalization can fill out to request a new username and submit it to Stewards and global renamers. Special:GlobalRenameQueue allows Stewards and global renamers to manage rename requests in a simplified interface form for post-finalization requests to be sent to from Special:GlobalRenameRequest. Lastly, Special:GlobalUserMerge is in very early testing stages. This tool will allow Stewards to merge the accounts of two or more usernames owned by one person who may have multiple account names due to conflicting usernames.
Once these tools are tested out and deployed, outreach will begin by talk page contact and by email where possible with all accounts that will be renamed due to SUL finalization.Library infrastructure for MediaWiki
An investigation into the possibility of using a package manager for JavaScript libraries in MediaWiki closed with the consensus opinion that we are not ready to choose a package manager at this time. The frontend standards group will revisit this decision in three to six months. It was agreed however that as far as possible JavaScript libraries should follow the guidelines for library development that are being worked on for PHP code.
Initial work has begun on a Profiler implementation that uses XHProf to collect information about the runtime costs of MediaWiki code. This approach to profiling will enable collection of information on code running on Wikimedia servers without relying on explicit wfProfileIn() and wfProfileOut() calls. This in turn will make splitting code out of MediaWiki core easier.Security auditing and response
Quality Assurance/Browser testing
Multimedia
editIn October 2014, the multimedia team started work on the Structured Data project, co-hosting a weeklong bootcamp in Berlin with community members and the Wikidata team. Participants developed first ideas for data models and user interface designs, as well as a high-end API prototype to read and write machine-readable data on Wikimedia Commons. These first ideas are now being documented here on Commons and in these project slides, for discussion purposes. If you would like to get involved, we invite you to sign-up for the newsletter and join the discussions on the Structured data hub. You can also learn more about the related metadata cleanup drive in this blog post. Our next office hours chat on Structured Data will take place on Thursday, November 20 at 19:30 UTC, on #wikimedia-office connect (Freenode IRC).
The team also developed and released more improvements to Media Viewer, based on feedback from the recent community consultation and user research. These improvements include: an easier way to disable Media Viewer for personal use, as well as re-enable it from a file page, pre-render thumbnails on backend for new uploads and make MediaViewer text larger in Monobook. Our last improvement for this release will be a caption or description right below the image, to be launched in November. In parallel, we also continued our code refactoring for the UploadWizard, and are now analyzing metrics for the upload funnel analysis, to gradually fix the most critical bugs that cause error messages during upload.
Lastly, the team hosted a quarterly project review for multimedia and planned its activities for this quarter, as shown in these slides. For more information about our work, join the multimedia mailing list.Volunteer coordination and outreach
We created a number of reports on mobile readership, including: daily snapshots of pageviews by device class and access method; a report on OS version breakdown for iOS app users and overall app usage; updated stats on the impact of the tablet redirect.
We started the documentation on mobile microcontributions and provided instrumentation requirements and experimental design support for a series of tests planned by the Mobile team to be rolled out in this quarter. We also started background research on missing claims in Wikidata as a function of item classes and language, to help inform the design of new types of microcontributions.
We hosted our monthly research showcase, with a presentation by Aaron Halfaker on Wikipedia as a socio-technical system and a guest talk by BarcelonaMedia's David Laniado on sentiment analysis of editor discussions.
We made substantial progress on defining reader engagement metrics. We also supported several external requests for data, including a request to extract a complete set of references with PubMed Identifiers and a dump of revert metadata for the English Wikipedia.
We completed a preliminary analysis of HHVM's effect on newly registered editors.The Kiwix project is funded and executed by Wikimedia CH.
- We released a minor version (1.91) of Kiwix for Android fixing a few annoying issues. Preparation to welcome Google Code-in junior developers has started.
The Wikidata project is funded and executed by Wikimedia Deutschland.
- In October the Wikidata team focused on performance, bugfixing and planning for structured data support for multimedia files. This included a week-long workshop with volunteers and the WMF multimedia team in Berlin. They made progress on statements on properties, simple queries and usage tracking and rolled out first partial changes to the user interface that are part of the new design. They answered questions in an office hour on IRC about structured data for multimedia files and are looking for input on how to improve references and constraints violation reports. And to top it off the project celebrated its second birthday.
Future
edit- The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.