Wikimedia Developer Summit/2017/Lessons Learned

This page is a combination of feedback from a survey sent to participants, debriefs, physical help desk suggestions collected at the event, and ideas from the organizers.

It focuses on improvements for future Wikimedia run technical events and specifically focuses on things that went poorly and should be changed. Because we are focusing on "improvables", please keep in mind while reviewing that 94.1% of the people that responded to the feedback survey strongly agreed or agreed that the event was worth their time and 91.2% would like to attend a similar event next year. This is an attempt to improve!

Suggested Changes for the next Developer Summit

• Better organization of the unconference
• Speakers need to be more prepared for their sessions & clearly communicate expectations upfront
• Etherpads should be created ahead of time
• Better spaces for quiet room and informal conversations
• Juice should be available
• Review this task during planning phase: https://phabricator.wikimedia.org/T154925

New things to keep

• Keep "Capture problems as they come" Phab task
• Venue & shuttles in both directions

Feedback Survey

Background on the Feedback Survey

• rfarrand created the feedback survey form making some changes from last year.
• The feedback survey began accepting responses during the last day of the Developer Summit, January 11.
• Participants received two emails asking / reminding them to fill out the survey. The importance of completing the survey was also mentioned during the summit multiple times.
• The deadline for filling out the feedback survey was February 2nd, the feedback survey was closed on February 3, 2017.
• 68 out of 157 (43%) participants filled out the feedback survey.
• Multiple in person debriefs with various parts of the organizing team were scheduled, tracked on a Phabricator task, and incorporated into the top section of this page.

Considerations for next year

This section is based on the fill-in-the-blank/comment sections of the feedback form. Some of these comments will be contradictory. I included common themes and issues that were mentioned by more than one or two people. This section focuses more heavily on areas that have room for improvement instead of things that went well. This section is a bit more subjective than the Wikimedia Developer Summit 2017/Lessons Learned#Data section below.

Day 1 & 2: Schedule and Documentation Questions

What was the most valuable session that you attended and why?

Most mentioned:

(add links)

• Platforms are Products Too
• Ward's Keynote
• Annotations
• Multi-Content Revisions
• What should an AI do you for you?

Are there any changes to the first two days that you would recommend for next year?

Common Themes

• The most agreed upon point here was the that unconference was hard to understand, unorganized, and seemed to be pretty random. More work is needed to make a better process. Some of the unconferenec sessions were under-attended.
• Multiple people mentioned that the speakers should be more prepared. Slides should be better, better for big rooms (no small font), and speakers should have a better understanding of the purpose of the session in advance and set expectations in advance.
• A few people people mentioned that sessions should be shorter (reminder that in one of the data points below that 67.6% of participants liked the session lengths and 23.5% wanted shorter sessions lengths)
• There should be more hackathon related activities, those events are more valuable and missing in North America

Suggestions for next time

• Some people want more focus in sessions on next steps and some people want less focus on it.
• Lighting talks should be included
• Don't spend time discussing management issues when the entire audience is not from WMF
• Find better solutions to judge interest levels of sessions
• Specific requests from one person:
• Talk from community member (why they want a feature)
• Infrastructure track
• More Small group discussions in sessions
• More session recording

What would you like to change on the use of the combination of etherpads, Phabricator and wikis to coordinate the sessions? How did it go?

Common Themes

• The good news is that we have improved a lot. The majority of comments here were people liking the process or commenting on its improvement.
• Etherpads should be created ahead of time, also etherpad formatting does not allow easy copy paste onto wikis.
• There were multiple comments about needing more clarity on the process of how information is stored and move around before and after sessions. Process may still be to complex.
• Everybody needs to follow the process
• Better note taking should always be prioritized. Realize that people taking notes need a back-up and also wont be able to participate as easily.
• Some comments: Do we really need to use the wiki? Other comments: Do we really need to use Phab?

Suggestions for next time

• Update Phab tasks when sessions are scheduled.
• Make sure all notes end up with links to slides and recordings
• Remote IRC participation should have more direction

Day 3: Get Stuff Done Day Questions

Are there any changes to the Get Stuff Done Day that you would recommend for next year?

Common Theme

• Find a way for people to better understand what others are doing.

Suggestions for next time

• Power on the tables, not under them
• Allow people to have sessions
• Ask for people to focus on Developer Wishlist

What did you do during the Get Stuff Done Day?

Copy / Paste, deleted about 6 responses that said something like "meetings" or "I didn't attend"

• Spoke with people and worked on a prototype patch for an idea discussed in an unconference session.
• I worked on a gadget that was requested on the wishlist survey.
• * a lot of Code-Review * Bringing my changes in Gerrit up-to-date and tried to find persons who can merge/review them * Discussing "bigger" changes and ideas with other people (who are mostly more difficult to reach otherwise)
• Code review!
• Making a draft change for the DNS configuration to be able to generate automatically DNS names for the service discovery.
• Chatted with folks, did some code reviews, planned some of my work.
• Talked to a lot of people about Wikidata and discussed plans with them.
• Write some patches. Had useful conversation with other people to remove blockers for future work. Had useful conversion with other people to suggest solutions to their problems.
• Got help from Aaron with ORES. Helped someone else write an extension. Worked on a labs tool.
• I worked on a service discovery system that we talked about the day before.
• I discussed my plan with a designer I discussed my plan with an engineer I wrote some code to advance my project I started the bot approval process and got pretty far with it
• hack, hack, hack and actually was able to create functional code that people were very happy with.

• Attended spill-over meetings and got some work done.
• Ended up working with a volunteer on a bot that were developing. It was fun!
• i18n related things
• Worked on getting a project running on WMFLabs. Knowing that there was people around whom I could ask if I broke something definitely increased the confidence with which I tried doing these things for the first time.
• Well I did get some stuff done actually; I had to get stuff done around being able to get paid as a contractor! So yay for that. Also made two connections for projects I'm working on ("I'll trade you reviews on X if you help me with Y") :-) No dead time whatsoever.
• Mostly just talked to people. Updated a few patches.
• Discussed with people I cannot sit with for a longer talk normally. Write on code with people I normally only contact online.
• I paired with a FR-Tech member to work on an off-goals project they had been waiting for a long time, and we managed to get something working at the end of the day, which was cool!

• A new feature for Visual Editor, with excellent help from Thalia and Ed. Unfortunately I couldn't complete it. If there were two days, I would already do it. I'll definitely complete it, but it will take me more time without their direct help.
• Cleaned the backlog of configuration for the wikis
• Two meetings about upcoming projects, prepare to mail in my broken computer
• I had one meeting commitment at the venue and wasn't involved in the other stuff due to a competing calendar commitment with another team. I like the concept of more hackathon and do things. I'd mainly like to see us be more present with SF Bay Area technology providers and tech-oriented advocacy groups and pulling them in to help us. They have valuable insights and we could grow a more equipped volunteer and partner base.
• Prepared for a presentation I had and had impromptu conversations with people (very useful for at least my next two quarters of work).
• I spent time talking with people I needed to synchronize with for future projects, and participated as a helper to build a proof-of-concept of banner impressions data loaded into a new analytics tools (project currently being productionised).
• I got roped into a number of discussions with other people, and in between those I managed to track down a bug in the interaction between VisualEditor and the Translate extension.
• Socialized with users I've engaged with online and met for the first time.
• Worked with Yaron on adding features to Page Forms
• Codin' & patchin' & note-writing of stuff from the previous two days.
• Dug into the way Wikimedia uses OAuth and single-sign on.
• Socializing + some ordinary work.
• Hacked with others I rarely meet in person. Had several small discussions, that I didn't find the opportunity for on the days before.

Logistics Questions

Are there any changes to the logistics that you would recommend for next year?

Common themes

• The quiet room was not quiet enough, noise from venue could be heard
• More places and time for informal discussion

Suggestions for next time

• Various drink requests: juice, more kinds of soda, real coffee, cold brewed coffee, sparkling water
• People were happy with the food, some comments: made-to-order sandwiches, make sure veg food is not spicy, hot food for lunch, more vegan food options, no beans, nothing greasy,
• Add ice breakers to parties
• Add a shuttle to the hotel
• There are no cheap accommodation options for people who are paying their own way to attend
• A few people wanted an earlier start a few people wanted a later start

Anything else?

Who should be at the next Wikimedia Developer Summit who wasn't at this one?

Mentioned a lot

• More volunteer developers and community members
• More 3rd Party Developers

Mentioned Twice

• Newcomers
• Template Developers
• Bay Area Organizations (generally)

Mentioned Once

• All WMF Devs with no exceptions
• Organizations mentioned: Google, OSC, CC, Sandstorm
• Groups mentioned: Bot developers, the research team, regular users, module developers
• Individuals mentioned: Yuvi, Multichill, Kunal, Magnus M, Krenair, Isarra

Are there different ways we should do outreach before or during the conference next year?

• More outreach to local-to-SF organizations
• Email previous years participants
• Give this problem more thought
• Clearer solicitation of topics

What did you like most about the summit?

Mentioned a lot

• Meeting developers
• Social in-person time
• Listening and learning
• The venue

Mentioned

• Unconference
• Ward / Keynote
• Get Stuff Done Day
• Consensus on complicated and important issues
• Deep technical discussions

• Unstructured
• Welcoming atmosphere
• Seeing managers participating
• Sessions focusing on community
• Experimenting 

What did you like least about the summit?

• People did not like the rain / weather
• The commute for locals was difficult
• This year the sessions were not as well organized by the main speakers
• Two people said there were sessions that were a complete waste of time with no outcomes
• Two people did not like the food

Individual suggestions

• Don't pretend this is a community event when it is not. Either make it one and bring the community or have internal discussion internally
• Don't have the same sessions every year, that means we are not solving problems
• Move the event outside of SF
• Political discussions (like technical debt, etc.) should not be done once a year. They should be talked about regularly among WMF staff.

Data

Day 1 & 2: Schedule and Documentation Questions

How satisfied were you with the pre-scheduled sessions?

Very satisfied	15	22.1%
Satisfied	44	64.7%
Neither satisfied or dissatisfied	5	7.4%
Dissatisfied	1	1.5%
Very Dissatisfied	0	0%
Did not attend	3	4.4%

How satisfied were you with the keynote by Ward?

Very satisfied	24	35.3%
Satisfied	27	39.7%
Neither satisfied or dissatisfied	7	10.3%
Dissatisfied	6	8.8%
Very Dissatisfied	0	0%
Did not attend	4	5.9%

How satisfied were you with the Q & A by Wes and Victoria?

Very satisfied	8	11.8%
Satisfied	27	39.7%
Neither satisfied or dissatisfied	27	39.7%
Dissatisfied	3	4.4%
Very Dissatisfied	0	0%
Did not attend	3	4.4%

How satisfied were you with the unconference sessions?

Very satisfied	16	23.5%
Satisfied	45	66.2%
Neither satisfied or dissatisfied	5	7.4%
Dissatisfied	1	1.5%
Very Dissatisfied	0	0%
Did not attend	1	1.5%

How satisfied were you with the unconference scheduling process?

Very satisfied	24	35.3%
Satisfied	29	42.6%
Neither satisfied or dissatisfied	8	11.8%
Dissatisfied	2	2.9%
Very Dissatisfied	1	1.5%
Did not attend	4	5.9%

How satisfied were you with the TPG Coaching Clinic?

Very satisisfied	5	7.9%
Satisfied	5	7.9%
Neither satisfied or dissatisfied	4	6.3%
Dissatisfied	0	0%
Very Dissatisfied	0	0%
Did not attend	49	77.8%

How did you like the length of the sessions? (70 minutes)

They should have been longer	0	0%
They should have been shorter	16	23.5%
They were the right length	46	67.6%
I don't know	6	8.8%

Day 3: Get Stuff Done Day Questions

How satisfied were you with the Get Stuff Done Day?

Very satisfied	16	23.5%
Satisfied	23	33.8%
Neither satisfied or dissatisfied	15	22.1%
Dissatisfied	2	2.9%
Very Dissatisfied	0	0%
Did not attend	12	17.6%

Logistics Questions

This event overall [Ratings of the Summit logistics]

Very Satisfied	33	48.5%
Satisfied	32	47.1%
Neither dissatisfied or satisfied	2	2.9%
Dissatisfied	1	1.5%
Very Dissatisfied	0	0%
N/A	0	0%

Overall organization of the summit [Ratings of the Summit logistics]

Very Satisfied	43	63.2%
Satisfied	24	35.3%
Neither dissatisfied or satisfied	1	1.5%
Dissatisfied	0	0%
Very Dissatisfied	0	0%
N/A	0	0%

Organized social activities [Ratings of the Summit logistics]

Very Satisfied	26	38.2%
Satisfied	28	41.2%
Neither dissatisfied or satisfied	8	11.8%
Dissatisfied	1	1.5%
Very Dissatisfied	0	0%
N/A	5	7.4%

Availability of space for informal discussions [Ratings of the Summit logistics]

Very Satisfied	19	27.9%
Satisfied	31	45.6%
Neither dissatisfied or satisfied	15	22.1%
Dissatisfied	3	4.4%
Very Dissatisfied	0	0%
N/A	0	0%

Food served at the event [How would you rate:]

Excellent	20	29.4%
Good	29	42.6%
Fair	10	14.7%
Poor	8	11.8%
Very bad	0	0%
N/A or didn't attend	1	1.5%

Wifi quality at Golden Gate Club [How would you rate:]

Excellent	37	54.4%
Good	24	35.3%
Fair	2	2.9%
Poor	2	2.9%
Very bad	1	1.5%
N/A or didn't attend	2	2.9%

Meeting Room equipment [How would you rate:]

Excellent	28	41.2%
Good	35	51.5%
Fair	1	1.5%
Poor	0	0%
Very bad	0	0%
N/A or didn't attend	4	5.9%

Anything else?

Attending this event was worth my time [Please tell us how much you agree or disagree with the following statements.]

Strongly Agree	46	67.6%
Agree	18	26.5%
Neither agree or disagree	3	4.4%
Disagree	1	1.5%
Strongly Disagree	0	0%

The opportunity to meet fellow developers was valuable to me [Please tell us how much you agree or disagree with the following statements.]

Strongly Agree	46	67.6%
Agree	20	29.4%
Neither agree or disagree	2	2.9%
Disagree	0	0%
Strongly Disagree	0	0%

I would like to attend this event again next year. [Please tell us how much you agree or disagree with the following statements.]

Strongly Agree	44	64.7%
Agree	18	26.5%
Neither agree or disagree	4	5.9%
Disagree	2	2.9%
Strongly Disagree	0	0%

Related Links

Capture problems as they come task (review before next event): https://phabricator.wikimedia.org/T154925

Organization Debrif(s) task: https://phabricator.wikimedia.org/T155597

The task for this survey/and summary: https://phabricator.wikimedia.org/T153996

Remote Participation

This information is from the feedback survey that went out to people who participated remotely at the Wikimedia Developer Summit.

Survey background and statistics

• 13 attendees filled out the survey form
• As of now (a week after the summit) two of the main sessions: "Ward Cunningham- Has our success made it hard to see your own contribution?" and "Wikimedia Foundation Technology and Product Q&A" received 218 and 53 views on YouTube each.
• 53.8% said they heard about the remote participation through an email received from WMF, 23.1% through WMF's social media channel, and 23.1% through other sources.
• For 53.1% watching the YouTube video, for 15.4% reading the session notes, and for 15.4% participating in discussion on IRC channels were the most useful components of remote participation
• 92.3% respondents said they would be interested in participating remotely in the future events and 69.2% opted-in to receive updates about the same

Inspiring quotes:

"It was awesome to be able to watch all those sessions. And the notes were invaluable, saved so much time. It was great to have both, on some sessions I preferred reading the notes, on others watching the video"

"Keep doing this, if you can, for it opens up possibilities that would not otherwise be open"

Things to improve for next time:

• • Audio quality
  • Camera placement (was more on the audience, and less on the speaker/ slides)
  • Check ahead of time what the formats for live-streamed sessions are going to be like. If a session is intended to be a breakout group style around whiteboards, it may not be that useful for the remote audience to participate
  • Some speakers were good at explaining the roles (such as note-taker, facilitator) at the beginning of their session, others were not. More efforts to recruit people for specific roles ahead
  • Streamed URLS changed (this happened as we made last minutes plans to stream and not to stream some sessions). Maybe consider unique YouTube links for all the sessions
  • Backchannel for organizers and remote advocates to discuss things like YouTube link updates, additional conference sessions, and information for participants in one place

Other Links

Dev Summit Debrief info: https://phabricator.wikimedia.org/T155597