NOLA Hackathon 2011/Saturday

Berius hartorutus



Markus Glaser wants to talk with some people about making it easier for 3rd party devs.

  • get more "core hacks" upstream
  • reduce need for "core hacks"
  • avoid work being done twice

FileRepo / File refactor


attending: Tim, Ben(MS), Russ, Brion, Aaron, Ben(WMF), robla

For cleaning up Swift & related repositories

Requests for comment/Refactor on File-FileRepo-MediaHandler <- some earlier notes

                              File     <- have this do the metadata part
                                |------ <----- FileBackend goes here?
Foreign...File     SwiftFile      ForeignApiFile

most things in FileRepo will move to FileBackend

  • (FileRepo will be mostly empty for BC)

Things like ForeignDBRepo, ForeignAPIRepo, SwiftRepo will mostly move to FileBackend alternate classes

Need a multi-write FileBackend (to distribute write requests to both swift & classic filesystem)

  • Pass config arrays on to the other backends etc...

FileRepo will have a file store, it won't be a file store

ForeignAPIRepo -> should never need to call a FileBackend

ForeignDBRepo -> still does.

^ this'll help us see which to separate

Example LocalRepo::cleanupDeletedBatch -- mixes raw file work like unlink() with DB manipulation. Russ's poking at separating that to the FileBackend (for the file bit) and the FileRepo (for the DB bit) cooperating.

Assigned to Aaron: cf last night's notes: NOLA Hackathon/Friday#Swift media .26 stuff

(Tim & Ben(MS) chatting on various details of splitting up FileRepo, RepoGroup etc)

Also see the FileBackend notes.



<- add some info on chad's work?

new server mostly working as of 12:30pm local -- yay! Try clicking around!

We are seeing intermittent phpunit test failures which we don't quite understand:

  • they look like {{brace expansions}} not working (some templates, some parser functions)

Haven't been able to repro yet on Ubuntu 10.04 separate install or our own local installs Checking if can repro on same machine outside of jenkins...

We fixed this, we were running a beta version of phpunit rather than the stable copy. Downgrading resolved the issue.

From Developer Intro


The Manual:Title.php doc should be changed to not use factory method ~ Title.fromURL per the doc string on the method, this is not the function your looking for.



Brion looked over some of this w/ Kevin

Recommended splitting the record & generate feed portion from the spider/archiver portion

  • makes the record/feed code cleaner, easier to review & deploy
    • feed seems fairly clean, has an id-based continutation which should index fine
    • runs out through API; format looks sane enough; already has code to spider that format
  • is prepared to spider based on the feed, so we're not in a hurry to run our own archiver

Recording stuff for the feed generation can fetch currently-stored external links before the ArticleSaveComplete and compare against the ParserOutput to get a differential of just 'added' links (more or less)

Spider/archiver should probably run its own logic for 'this page has been seen before', which simplifies the feed generator logic.

Third-party committers help


What can we do to make things easier for 3rd-party devs to upstream stuff? Do we want to? ;)

TL;DR summary key items:

  • check perf implications of empty wfRunHooks() and sprinkle more hooks around
  • continue refactoring things that are more flexible with factory functions & subclassing
  • clean up documentation, all pitch in on it
  • skin discussions: -> get people talking towards a final plan
  • MW core devs: communicate in support of markus & friends when these things come up! Ping robla if necessary, he'll poke us.

Some issues/pressures:

  • sometimes projects have requirements that don't seem to fit well with core
    • ^-- we could probably actually have a nice central point ('getDisplayName' etc) so it's easier to hook, even if we don't need the same feature
    • in this case there's also input / name collision issues...
  • More hooks!
    • being able to replace entire functions/methods is often also usefulish
      • hooks in inner loops can be slowish -- may be a concern? (is this a real problem? can we measure some?) -> check the actual performance overhead of bunches of extra callbacks
  • OO parts with good factories can be really easy to subclass
    • do an audit over non-factory usage? see if we have to kill more raw usages to make subclassing easier
  • Better outreach? Ask some more 3-rd party devs what they need merged back or refactored
    • ex: we're working with Wikia to slowly merge in more of their hooks & hacks
    • It's a two way street. Core devs can be more accomodating, 3rd parties can come forward with what they need.
    • bump up communication on mediawiki-l? or a new low-traffic list?
      • start asking what peoples' needs are so we can start working on them
      • invite more folks to next hackathons etc?
      • do a launch meeting with some of the biggies? (special hackathon? :)
  • There's lots of duplication of effort
    • eg: Wikia, WMF and others have all implemented "image insertion dialogs"
  • documentation update week!
    • some stuff is out of date or confusing -- running into old docs can confuse the heck out of people
    • may have to mellow out some of our older documentation! ("never ever try to restrict read access or your dog will die!" -> "let's make sure things are factored nicely for this to work")
    • try to encourage active merging as people come across stuff
  • [markus] would like to encourage an atmosphere of extension developers reviewing each others' code
    • looking at other examples has helped him -- it's a win-win
    • the best documentation is written while working on the code
  • Need to improve how we communicate about upcoming changes (eg resourceloader coming in, and what you would need to do and *how important it is* to deal with it)
  • Versioning for core vs extensions -- how to handle extensions that want to work on multiple core versions? Core maintainers tend to update extensions on trunk that are only compatible with the latest.
    • the current 'make a branch of all extensions at the time of core change' feels awkward; they're very hard to maintain in SVN
      • probably easier to maintain each extensions' back-compat branches in git?
    • maintaining compat with multiple core versions in the current trunk can be really awkward too
    • seeing core devs' changes break your code on the old version at least lets you know a change is coming ;)
    • warnings with deprecation... try to keep a couple versions before dumping things to let ext devs have time to migrate to new usages.
      • markus indicates that the deprecation warnings in logs can be useful for him
  • git will also make it easier to maintain your own back-compat branches without them having to be in central master repos (or doing ad-hoc branch names/descriptions)
  • SKINS aaaaiiiiieeeeeee
    • there has been talk :) -> see Daniel Friesen's threads on wikitech-l
    • still needs to be finalized; some other folks working on related issues per robla
      • Wikia - Owen Taylor (oasis etc)
        • modular pieces that get assembled into a skin with headers, sidebars, banners, widgets, footers etc
      • WMF - Patrick Reilly (MobileFrontend system has some similarities in some parts)
        • think XSLT-like transformation rules on top of a simple "base layer" skin?
    • per chad - we should probably go ahead and boil the oceans; when new sensible skin system comes in let all the old ones die :)
    • we all need to push to go ahead and find and implement a solution here -- if done by a 3rd-party need to be sure we all make it work for us all!

1.18 regression triage


From Etherpad

Bug triage for 1pm local time (New Orleans, Louisiana, USA) 15 October 2011

  1. wikimedia-dev on FreeNode IRC

(Part of NOLA Hackathon)

  • bugzilla:30961: Run refreshImageMetadata.php on wmf wikis sometime after 1.18 deploy
  • bugzilla:31060: [Regression] Sortable tables: "unsortable" should also work for rows outside the table heading
  • bugzilla:31239: inputbox with 1.18 let break=no not work, but put the buttons on a next line
  • bugzilla:31249: Special:BrokenRedirects broken since 1.18
  • bugzilla:31361: Some messages on Special:GlobalBlock are broken
  • bugzilla:31362: Lost CSS styles in pages transcluding pages from the Special:
  • bugzilla:31388: mysql caching is consuming more disk space on 1.18 wikis due to change in parser cache keys
  • bugzilla:31391: Aspect ratio incorrect for rotated images uploaded prior to 1.18 deployment
  • bugzilla:31398: Revision information not rendered in edit notices
  • bugzilla:31403: Autoblocks popping up for long-since blocked users frequently
  • bugzilla:27858: Block notice when editing talk page of blocked user doesn't go away after block expires
  • bugzilla:31404: flickering while vertical position in editbox is restored
  • bugzilla:31405: Block reason no longer pre-loads
  • bugzilla:31408: tags for edits changed
  • bugzilla:31409: Arrow-down-icon in Vector no longer centered in 1.18
  • bugzilla:31410: ArticleFeedback widget displays at top of page in colognblue skin
  • bugzilla:31412 Changed behavior of back button
  • bugzilla:31413: No prompt when entering a blank edit summary in my user space
  • bugzilla:31431: no such special page
  • bugzilla:31438: wider table columns causing problems in IE
  • bugzilla:31441: allow sysops to add/remove new user group "autoreview"
  • bugzilla:31442: Math not rendering
  • bugzilla:31490: lcfirst and ucfirst parser functions do not work on Turkish wikis
  • bugzilla:31492: Collapse/expand buttons in extended watchlist are misaligned
  • bugzilla:31495: Missing external link problem on IE6
  • bugzilla:31502: TOC is missing on Special:EditWatchlist
  • bugzilla:31511: Unable to add/remove buttons from (classic) toolbar from a gadget after MW update
  • bugzilla:31514: mw-collapsible does not work together with sortable anymore
  • bugzilla:31527: jquery.tablesorter's data-sort-type feature not working for wikitext tables on WMF
  • bugzilla:27478: Enable $wgHtml5 on Wikimedia wikis (*)
  • bugzilla:31543: Ratings panel only partially displayed
  • bugzilla:31545: Some AJAX features seem to have stopped working in 1.18
  • bugzilla:31547: Category links need less spacing and better wrapping
  • bugzilla:31550: Galleries in categories don't have newlines before/after page links.
  • bugzilla:31560: Mediawiki is showing text from MediaWiki:<Message>/<lang1> also on MediaWiki:<Message>/<lang1>/<lang2>
  • bugzilla:31561: List at Special:EditWatchlist/raw is empty but content can be seen in HTML source
  • bugzilla:31566: toolbar and header/footer from proofread page on wikisource
  • bugzilla:31567: Ensure consistent position of the "Watch this page" checkbox
  • bugzilla:27860: Do not mark edits as minor by default on new section or new page creation
  • bugzilla:31576: Magic words are considered to be (non-existing) templates
  • bugzilla:31602: Do not use native datalist support in mediawiki.legacy.mwsuggest for Firefox
  • bugzilla:31636: Commons: Special:Upload&uselang=deownwork shows "Erlaubte Dateitypen: $1."
  • bugzilla:31638: Special:GlobalUsers pagination broken (incorrectly links to Special:ListUsers)
  • bugzilla:31643: Upload preview thumb hangs Firefox 7+
  • bugzilla:31644: Global usage should use protocol relative links
  • bugzilla:31650: (Outdated) New Page Patrolling issues in ruwiki
  • bugzilla:31651: Can't click on you have a new message on cs.wikipedia RC page
  • bugzilla:31654: Selecting a suggestion in monobook doesn't go to the article
  • bugzilla:31602: Do not use native datalist support in mediawiki.legacy.mwsuggest for Firefox (*)
  • bugzilla:31656: AbuseFilter should extract old_wikitext from the last revision, not from the edited revision
  • bugzilla:31670: Make collapsible boxes more usable by having the whole header row clickable
  • bugzilla:31673: JavaScript Error: "Object expected" makes "Special Characters" section of (enhanced) toolbar of MW 1.18 unusable on IE 8
  • bugzilla:31674: can't edit watchlist if it contains a page in the "Special" space
  • bugzilla:31680: [Regression] Thumbnail cache should be invalidated on re-upload
  • bugzilla:31682: MW1.18 broke Edittools for Special:Upload
  • bugzilla:31724: Missing CSS on AbuseFilter (the server responds with a status of 403 - Forbidden)