Wikimedia Engineering/Report/2014/September
Major news in September include:
- a call for candidates for the Free and Open Source Software Outreach Program for Women;
- a roundtable discussion between the Language engineering team and editors from the Catalan language Wikipedia, focusing on the Content Translation tool.
Engineering metrics in September:
- 151 unique committers contributed patchsets of code to MediaWiki.
- About 27 shell requests were processed.
Upcoming events
editThere are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.
For a more complete and up-to-date list, check out the Project:Calendar.
Date | Type | Event | Contact |
---|---|---|---|
1 October 2014–3 October 2014 | SMWCon Fall 2014 (Vienna, Austria) | Conference talk | |
6 October 2014 | Tech Talk: The Dashboarding Problem 1900-2000 UTC in #wikimedia-office connect. | Nuria Ruiz | |
8 October 2014–8 October 2014 | Collection/Book/PDF Bug Day (exact date pending) | Talk | |
22 October 2014 | Tech Talk: Design Research in Product Development 1900-2000 UTC in #wikimedia-office connect. | Abbey Ripstra | |
22 October 2014 | Meetup: Exploratory Testing for Complex Software; Lessons from Cloud Foundry 0130-0300 UTC in #wikimedia-office connect. |
Personnel
editAre you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- Senior Software Engineer - Services
- Software Engineer - Maps & Geo - Mobile
- Software Engineer - Mobile - iOS
- Release Engineer
- Technical Writer
- Full Stack Developer - Analytics
- Research Analyst
- Agile Coach/ScrumMaster - Team Practices Group
- Senior Technical Product Manager
- Community Liaison
- Community Liaison (PT Contract)
- Operations Security Engineer
- UX Senior Designer
- UX Senior Design Researcher
- UX Visual Design Fellowship
- Mobile Partnerships Regional Manager
Announcements
edit- Damon Sicore joined the Wikimedia Foundation as Vice President of Engineering (announcement).
- Rachel Farrand joined the Engineering Community Team as Events Coordinator (announcement).
- Jeff Hobson joined the Wikipedia Zero engineering team (announcement).
- Daisy Chen joined the UX Research team as Associate Design Researcher (announcement).
Technical Operations
editDallas data center
- In September we have setup (backup) replication of most project data, including core databases and external storage. Work on Swift images and system backups was still ongoing into October. Essential system infrastructure such as an installation server, DNS, LVS, NTP etc. has been deployed as well.
Tampa data center
- We started the last push to get the remaining services & systems out of our Tampa data center, with a deadline for shutdown of all systems on October 1st. The remaining services included PDF generation, mail servers,
noc.wikimedia.org
and LDAP.
Labs metrics in September:
- Number of projects: 146
- Number of instances: 415
- Amount of RAM in use (in MBs): 1,996,288
- Amount of allocated storage (in GBs): 20,435
- Number of virtual CPUs in use: 977
- Number of users: 4,083
Wikimedia Labs
- Wikitech (the Labs web interface) is now managed via the standard WMF deployment system. This should allow for more frequent MediaWiki updates and overall greater stability.
- The last historic remaining dependencies on our old Tampa datacenter (e.g. LDAP and Labs DNS backup servers) were finally stamped out and replaced with dependencies on Dallas hardware.
- One of the labs virtualization hosts (virt1006) was suffering intermittent problems, so all affected instances were migrated to other hosts in order to stave off possible future disaster. Consequently, Labs is a bit short on virtualization space, but new hardware procurement is under way.
- Several long-unused instances and projects were cleaned up in order to free up more space.
- The last of the ToolLabs replica DB servers was upgraded to MariaDB 10.
Editor retention: Editing tools
editUsers of Internet Explorer 10, who we were previously preventing from using VisualEditor due to some major bugs, will now be able to use VisualEditor; this follows on from Internet Explorer 11 support last month. When editing a template with a required field, VisualEditor now warns you to avoid leaving it blank, and you can now create auto-numbered links using VisualEditor.
Improvements and updates were made to a number of interface messages as part of our work with translators to improve the software for all users, based on feedback from users and user testing. We made progress on table structure editing and auto-filled citations, both of which will be coming soon.
The deployed version of the code was updated five times in the regular release cycle (1.24-wmf20, 1.24-wmf21, 1.24-wmf22, 1.25-wmf1 and 1.25-wmf2).The team's work on front-end standardisation is focussed on improving libraries and infrastructure, and in particular, the OOjs UI library. This included the creation of a MediaWiki theme in collaboration with the Design team, which can be explored in the online demo; this will be deployed into MediaWiki's use of OOUI in the next few weeks. A number of bugs were fixed, including working around window and popup sizing, over-flow item placement, and working around some browser bugs in Firefox and Safari. The code documentation has a number of minor issues corrected, and the build process was extended to create a minified distribution. The OOjs library was updated to fix a minor bug in oo.Compare
, with a new version (v1.1.1) released and pushed downstream into MediaWiki, VisualEditor and OOjs UI.
autovalue
" parameter property, a wikitext value that a parameter can be set to have inserted by default if desired. Also, the specification for TemplateData was re-written to be clearer and more consistent. Next month the TemplateData GUI editor will be made available on all Wikimedia wikis.Services
editWork on secondary index updates continued at full steam, and is now close to being merged.
Core Features
editIn September, the Growth team shut down, with workflows shifting into the mainstream of other teams.
Language engineering communications and outreach
Notable improvements include:
- a basic formatting toolbar (for Chrome);
- more accurate warnings for unchanged machine translated content;
- design improvements for the top bar and progress bar;
- bi-directional support for Spanish-Portuguese machine translation;
- link adaptation improvements.
MediaWiki Core
editSecurity auditing and response
Multimedia
editThese improvements aim to make Media Viewer easier to use by readers and casual editors, with these features: a more prominent "More Details" button, linking to the File: page; separate icons for "Download" and "Share or Embed" features; and an easier way to enlarge images by clicking on them. Next, we plan to work on an easier way to disable Media Viewer for personal use and a caption or description right below the image. We would like to thank all the community members who suggested these improvements. Our research suggests that they offer a better user experience, that is both clearer and simpler.
This month, we also ramped up the Structured Data project, in collaboration with community members and the Wikidata team: in October, we will start developing a first prototype for a high-end API that can read and write machine-readable data on Wikimedia Commons, to be followed by a wider deployment in coming months. In parallel, the foundation is also launching a file metadata cleanup drive to add machine-readable attributions and licenses on files that lack them, spearheaded by Guillaume Paumier. To learn more, join our Structured Data Q&A on Thursday, October 16 at 18:00 UTC, for an office hours chat on #wikimedia-office connect (Freenode IRC).
We also continued our code refactoring for the UploadWizard, and started to collect metrics for an upload funnel analysis, to find out how many users drop out at each step of the upload and where failure is occurring, so we can prioritize bug fixes. For more information about our work, join the multimedia mailing list.Work was done on the following metrics:
- Rolling New Active Editor - Implemented
- Rolling Surviving New Active Editor - Implemented
- Pages Created and Edits - Updated to include reporting configuration to include changes to deleted pages (this is a default).
- Metrics with ‘Namespaces’ as a parameter let you specify “All Namespaces.” Leave the input field blank to do so.
- Rolling recurring old active editor is implemented, but does not perform sufficiently rapidly for us to enable it on the production servers.
- The status of the implementation of Standardized Metrics defined by the Research Team is here: https://meta.wikimedia.org/wiki/Research:Metrics_standardization/Implementation
A terrific weekly summary is posted to the Analytics mailing list with a summary at the top of each email. Here are the links to related posts in the archives.
Analytics/Editor Engagement Vital Signs
We completed the definitions, documentation and requirements for a new set of metrics to be implemented in Vital Signs.
We completed a first draft of a page view definition, which is currently being discussed. We supported the mobile team with baseline traffic reports for Apps and Mobile Web.
We participated in the preparatory sessions for the design of an open consultation led by the Community Liaison team as well as in regular meetings to support the strategy consultation process.
We held our Q1-2015 quarterly review, reviewed the team's progress against Q1 goals and posted our proposed Q2 goals.The Kiwix project is funded and executed by Wikimedia CH.
- We made progress in our work with our partner Bookeen to get an e-ink reader able to read Wikipedia offline. We managed to get a first version of the device firmware working, and it will be tested in the field as part of the Malebooks pilot deployment.
- As a consequence of a bug fixing sprint with Parsoid and Wikisource developers at Wikimania, we were finally able to generate usable ZIM files from Wikisource (example with fr.wikisource.org).
- Work on the offline project Gutenberg continued and we are now almost ready to release. A few ZIM files are in testing, for example in German and in Spanish.
- Kiwix was represented at the Selenium conference where we held a 2-day bug hunting session: 120 bugs were reported, of which 50% were fixed.
- mwoffliner was improved to make it easier to use for everyone, in particular to make ZIM files for only a selection of articles. As a demonstration, we prepared a ZIM files containing all Wikipedia articles about medicine.
- After many years, a new version of a tool to generate a live CD including Kiwix and Wikipedia content was released.
The Wikidata project is funded and executed by Wikimedia Deutschland.
- In September, the Wikidata team focused on improving performance, doing groundwork for the new user interface design, and making it possible to track where data from Wikidata is used. Next to that, they worked on tests and prepared for a week-long meeting with the WMF multimedia team and volunteers to discuss and plan structured data support for Wikimedia Commons.
Future
edit- The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.