Meetings/General Engineering/2010-09-14

These are the tidied notes from the September 14, 2010 meeting of the General Engineering Team. Also included are the text of Rob Lanphier's emails to coordinate the meeting.

Participants

• Rob Lanphier in his role as EPM for the General Engineering Team led the meeting.
• ...

Agenda

The the stuff we should actually talk about in our first meeting (and in email beforehand):

•  Status reports
  •  Office wiki?
  •  Problems/questions with general format?
•  What you all would like from User:RobLa in his role as EPM for the General Engineering Team
  • Help integrating a full view of all the mediawiki related analytics
  • projects that exist/how we can improve/build upon them
•  Inventory of all of the stuff we're working on:
  • Analytics (OWA) (Open-source Wit' Attitude)
  • Data Warehousing
  • Code Review Process
    • If we get multiple people involved (e.g. Roan, Trevor), divide by author
  • MW Release
    • 1.17 is going to be branched soonish (next month or two?). Major things still needing review/cleanup: ResourceLoader, new installer,???
    • Last branch point was r62817 so there's some proportion of 10k revs to review
  • EPM Processes
  • Process Improvement Tools
  • Tech Writing
  • Hiring Bugmeister
  • QA
    • Selenium/phpUnit/CruiseControl Testing
    • Development sandboxes for testing
    • UI testing process
  • Pending Changes / Flagged Revs
  • Open Street Maps (new)
  • Community-driven development
    • Google Summer of Code
    • Installer
    • Extension manager
  • Calcey / outsourced testing
  • API Development
  • Anything missing?
•  Meeting times?  Rotate to share pain, or maintain consistency?
•  Meeting frequency?
•  Public vs private communication for the group

Weekly Status

This week's status:

RobLa: Here's the summary of the Achievements & Objectives Reports I received, plus some extrapolation based on what I know from you:

• Nimish: continuing work getting OWA slaves to write to master in various configs
• Chad: Finishing up EZProxy, shifting to FlaggedRevs
• Erik Z: Streamline the Squid log processing
• RobLa: Building schedule for "Pending Changes v2", in SF Tues-Thurs
• Peter Adams is remove load balancer requirement from OWA by moving to database queues, and writing a memcache class
• Siebrand: http://translatewiki.net/

RobLa: Grokking the rest from conversations with you all:

• Sam is travelling for the first part of the week
• Priyanka is working on Selenium deploy, and getting prototype Bugzilla configured for project management
• Zak has finished up his doc plan, starting execution
• Tim is wrapping up work on udp2log reliability tracking
• Aryeh has wrapped up his summer contract and is headed back to school
• Aaron working on Pending Changes "reject" button along with refactoring to accommodate it before heading back to school
• Mark H is recuperating

Meeting Notes

robla: How is the weekly report format. Does it work for everyone? Is it useful?

-- general assent that the format is good --

robla

very distributed team. Less than half are full-time. Many are part-time working in some cases 1 or 2 days a week. Gives a different feel for things. We have feature engineering that focuses on high-profile stuff. We're focused on keeping the dev train rolling.

Ensuring that we have QA, infrastructure for automated testing, releases, tech writing, etc.  Will get short-schrifted if we let it.

To deal with this, we need to make sure that we're make good progress on key areas. We need to be sure that we're not seen as just free resources. This meeting is focused on figuring out how we can be really effective and coordinated.  We want to figure out how we can help each other out, where we have areas that we can shift staff, so that we can address gaps in coverage, what are some of the - those are the types of issues that we need to deal with.

That's the general context of what this meeting is about.

robla

with that in mind, lets figure out what you need from me. I've been in close contact with some of you and not with some of the rest of you?

-- no real discussion here :) --

robla

let's review the list of issues, starting with analytics

Peter and Nimish are working closely with Thomasz here. A lot of this work was closely connected with fund-raising and it made sense to let him run with the ball here.

Tim's done some work on UDP to log that's let us know when we are losing packets. We have been in a situation where we have not known many of those packets get lost in the process. That is one of the aspects of logging improvements.

OWA - open web analytics is a project that peter is working on for us and other clients. working on making this work for fundraising and other areas.  nimish is working on this as well.

Data warehousing is another area that is within our purvue.

Code review is something that Tim owns and that Mark was working on before. Roan also does some code review as well.

mediawiki release process is Tim. We talked about 1.17 release. I am not as much on top of this as I could be. We might cover that later. Chad provided some information above.

process improvement tool - Priyanka, Guillaume, Alolita and Robla have been working on this. There is a process of scoping out the various project management tools. As a foundation, we know that we need to have more transparency here. Having a tool out there where people can see more raw data to support the reports we have out there as well. the EPM process ...

Zak has a documentation plan published on http://mediawiki.org/ and is working on docs now.  - User:Zakgreant/MediaWiki Technical Documentation Plan

There is a job description for bugmeister.

QA hire by end of year.

selenium/phpunit/cruisecontrol infrastructure - one of the things that we are trying to do with that. priyanka's driving that. Marcus (not a WMF staffer) has been doing the heavy lifting.  Ryan Lane has alsobeen working to get the infrastructure in place.

There is development sandboxes for testing.

Pending changes is the oddball on this list.  This would usually fall under Alolita's group, but we now have it. Chad and Aaron.

Open Street Maps was added to Robla's list by Erik recently. I have some notion of what we want to work on here, but I am not really aware of what is going on on this front.  I need to talk to Erik and get more informations.

community-driven development - our group needs to figure out how we can better do this. It is a job for everyone in the foundation, but it is really a part of our focus.

Chad mentored Brian Wolff in Google Summer of Code (GSoC), worked with image metadata. New installer is pet project.

The extension manager is something that is part of GSoC, written by Jeroen De Dauw.

API development is Sam

Anything that i'm missing?

General questions

robla: is the installer the marquee feature for 1.17?

chad: yes.

robla: and the resource loader

chad: yes. but it's not a sexy for end users. So the key feature is the installer.

robla: will a sysadmin be excited about the resource loader?

chad: i don't know

Audio conf

robla: just as an aside on our audio conf technology. As i type in one of the channels. The next one of these, we'll try to do something other than webex. However, that will need software installs...

robla: hopefully, we can use an OSS technology called Mumble that we can use. xxx will send out info so that we can try this out before we hold any important meetings with it.

MW Release 1.17

robla: last branch point was ... 10k of revisions to review.

tim: these are mostly general changes that need to be reviewed.  Just the usual stuff.

robla: with mark out of commission - how should we deal with this - how can we get through even a fraction of these reviews? tim - what is your review pace.

tim: i don not remember the figures. If we can get some experienced devs like Roan and Trevor to review, that will be helpful.

nimish: can we use Roan - extend his hours - to help work on this?

robla: there are many demands on Roan's time. There are a lot of demands on Trevor's time as well, but it should be more viable. He might welcome the break from feature development at this point. It is not my call. I'll push for that. Danese hinted that this might be a possibility today.

robla: Danese mentioned that she's getting asked about the 1.17 release because of some bug fix that might be in there.  People are asking about the release.

robla: Chad, you'd mentioned that you were focused on a small set of things to try and get the installer out there.  tim, i think you'd agree.

tim: we can just push a particular fix into 1.16.1

robla:  let's put an action item thing ... mumble. (see below end of notes for items and format)

Pending changes

robla: i want to give an update on pending changes. There is a public thread where Jimmy has taken the position that we need to keep pending changes on. 65% of people want to turn it on, but 35% is significant dissent.  We need to polish the features. We cut some corners to get the feature out the door - like the reject button.  We need to add features, clean up the UI, etc. over the next week, i'll be working on this. I don't think that i'll need to rope in anyone who has not already been roped in - but i might be wrong. Chad, you are still the guy on this..

Nimish: is it just UI stuff or are there other features ?

robla: history gets pretty messy, etc. performance is something near and dear to everyone's heart here.  pending changes is tickling parts of the core that don't get much attention.  It is not that the feature is slow, but... For example, pending changes looks at and parses old revisions.  Do we need to cache old revisions? We're still scratching our heads about what to do.

...: if we are changing things around for pending changes, we should also think about the rate this article feature.

robla: we can take some of this stuff offline. I wanted to bring this to people's attention.

Logistics

robla: let's work out some of the logistic stuff now, as i don't want to keep everyone on the call to long.

robla: lets sort out the logistics. How often should we do this kind of call. We could introduce everyone. let's do that. In order of status review.

Participants self introductions

nimish: i'm nimish. i'm working on integrating owa into our infrastructure and building a min web analytics cluster.

chad: working on flag revs and pending changes. Workin on side project on a reference database called credo.

erik: doing data analysis. Refactoring squid logs processing scripts to make them automateable.

robla: been talking the whole time.

peter: i'm working on Nimish with making web analytics set up so that it can be used everywhere.

siebrand: i am the http://translatewiki.net/ guy. I do i18n, a lot of L10n, and translatewiki.net translator community management.

priyanka: i am working on process and tools, working with Alolita and Rob ...

zak: i'm working

tim: working on code review lately, though i've got a few other projects going.   I have been working on a ...  I have certainly got plenty to do.

brandon: stalking you all *snort*

Meeting frequency

robla: how often should we do these - once a month?

priyanka: more often would be nice.

erik: is this the format that we want to use. This seems to mirror the status reports.

robla: let's use email for reporting and try to focus these on discussing things like the mediawiki 1.17 discussion that we had.  We should try to cover the types of things that people may not be aware of.  maybe next time we talk about calcey and the ...  The kind of stuff that doesn't get enough coverage

nimish: should we use this as the time to talk about the direction of projects and such?

robla: we should make sure to do a combination of public communication and this meeting. we don't want to make people wait for this meeting. if we do this more frequently, then we won't be involving the community as much any more.  i'd also like to hit the sweet spot between positive anticipation of the meeting and not being a burden.

siebrand?: ( i didn't catch what he said here - something about trunk? --zak)

robla: that is a good thing to bring up, even if we can't do anything about it.

robla: also, mumble should allow for more adhoc meetings. this shouldn't be the only meeting that this team should have. there should be other meetings as needed. this meeting should be about getting the bigger picture.

Etherpad public ?

Brandon: I've been wondering if we shouldn't make etherpad urls public.  there's a lot of "transparency" issues that might be solved with that. (not on the call, just an injection.)

robla: i don't have an objection to this.  we haven't talked about something private, but we might need to.   is there anything secret that we've talked about?

nimish: maybe we shouldn't publish the url, but we can publish the notes.

robla:

erik: people will find the notes themselves, as there is a consistent format to the url.

I don't necessarily agree; the url format changes between meetings

and teams.  Features has a differnt url format; our scrums for the article assement pilot have even different ones.

Anything secret goes into the secret channel, non? (this is a thing for me lately is all, i'm big on trying to be less opaque)

Next meeting

robla

is there anything else that we should cover?  the next one of

these, we could to concurrent with the developer meeting that danese and chad are organizing for DC scheduled for the weekend of october 22nd. can we do the meeting at this time.

priyanka: i like the idea of having it during the meeting.  Fewer time zones to deal with.

robla: we'll have to figure out how to rope people in. we won't be able to get everyone in. I suggest rope. Simple. Does the job. Made of hemp.  Duct tape? If we don't have rope. Six of one, half dozen the other.

robla: next meeting will be mid-october. should we try to move the times?

chad: i don't think that everyone in DC will want to be on at 2am

robla: with the November meeting, should we try to keep things consistent

siebrand? should we flip it 12 hours each time.

nimish: i think that we should move it around.  It might be that we find a time that works best for everyone.

robla: there's an online gizmo for managing this that Danese and Alolita were talking about. maybe i'll try one of these things and see what preferences people have. Once we figure out what the attendance of the DC meeting will be  ...

Mailing list ?

robla: one other quick question - should we have a mailing list?

siebrand: no

erik:  I'd favor using offline status reporting, can be public

siebrand:

zak: favours announce-only list. public list for all else.

priyanka: happy without a new list

chad: is with Priyanka

...

robla: let's forego a mailing list for now. we'll forget people, but that's a small cost. better to force ourselves to use a public list.

Last questions

robla: since we're over time, is there anything we should cover?

Siebrand: can someone do some custom profiling over the next few weeks.

tim: i can probably do that

robla:  thanks everyone!

(beep beep beep, as phones disconnect)

Action items

•  RobLa: Follow up on request for 1.17...can we backport and cut 1.16.1?
•  Robla: Have these meetings once a month
•  RobLa: post to a public wiki