Wikimedia Developer Summit/2016/Lessons Learned

This page is a combination of feedback from a survey sent to participants and ideas from the organizers.

It focuses on improvements for next time and things that went poorly. However keep in mind that 94.1% of the people that responded to the feedback survey strongly agreed or agreed that the event was worth their time. This is an attempt to improve.

Suggested Changes for the next Developer Summit

Research changes of location. Ideally we will find a location with cheaper food and open ports. https://phabricator.wikimedia.org/T132398
Keep a hand written (because of unconference) schedule up at the registration desk, also have a large map of the venue and room locations
Spend some time rethinking the use of the combination of Phab / Etherpad / Wiki. https://phabricator.wikimedia.org/T132400
More professional signage
Better food for vegetarians
If the event is at Mission Bay, have a shuttle bus to the MBC in the morning as well.
Change the wrap-up session to be more interesting and useful
Help people find each other and projects during the unscheduled day
Don't hold this event near the holidays

Things to keep

Roles / role cards
Un-conference
Hackathon / Unscheduled 3rd day
Small group dinners
Pre-event discussion of sessions and session topics

Background on Feedback Survey

rfarrand created the feedback survey form during the Developer Summit.
The feedback survey began accepting responses during the last day of the Developer Summit, January 6.
Participants received two emails asking / reminding them to fill out the survey. The importance of completing the survey was also mentioned during the summit multiple times.
The deadline for filling out the feedback survey was January 18th, the feedback survey was closed on January 19th.
84 out of 155 (54%) participants filled out the feedback survey.
An in personal retrospective will be scheduled for the week of January 25th, 2016. Notes from that meeting will be posted.

Considerations for Next Year

This section is based on the fill-in-the-blank/comment sections of the feedback form. Some of these comments will be contradictory. I included common themes and issues that were mentioned by more than one or two people. This section focuses more heavily on areas that have room for improvement instead of things that went well. This section is a bit more subjective than the #Data section below.

Day 1 & 2 Schedule and Content

Pre-scheduled Sessions

Common Themes

There was a good range of topics and it was hard to choose between sessions.
The session descriptions could have used a lot of work. They were too long or unclear and it was hard to understand what the session was going to be about.
It was helpful to assign roles
The sessions were not friendly to people without technical backgrounds
The process for scheduling these sessions was "unclear," "overwhelming," and "messy." The process changed at the last minute and and confused everything further.
The overview of how the sessions were going to work was helpful in the beginning. The language rules of engagement should have been more internationally targeted, different things are offensive or not in different cultures.
The larger sessions were less productive and often without resolution. Sometimes the purpose of the session was completely unclear.
Presenters were well prepared

Suggestions for next time

All sessions should have a clear "goal" at the top of its description.
Session scheduling process should be worked out at least a month in advance and not changed.
Pre-event IRC or hangout with information on what to expect in the prescheduled sessions so it is easy to choose.
Wrap-up session was not a good use of time, change it and shorten it. Questions from wrap up should have been discussed on a mailing list.
Keynote should be improved

Unconference Sessions

Common Themes

People liked them and found them to be engaging and useful. The smaller unconference sessions were lead by passionate speakers and resulted in great knowledgeable conversations.
It was hard to choose between sessions
Roles cards were useful but people did not always stick to using them. More facilitation to make sure people don't forget next time.
The scheduling was unfair. It was stated that you could not schedule in advance, but some people did and got to keep their spot.
Mixing scheduled and unconference is weird. It also makes it hard to pre plan your time at the Summit.

Suggestions for next time

Encourage more community members to run sessions
Ensure enforcement of roles during sessions
Sessions should have better agendas created in advance
Unconference should be better explained in advance for those not familiar.
Define unconference. To some people it means needing to vote on sessions in advance. To others it means more fluidity and encouraging people to get up and move between sessions.
Build in "you can leave now" breaks in the middle?

Most valuable session

Most mentioned:

(TODO: add actual session names / associated tasks + send positive feedback to session leaders)

Real time collaboration
Unconference Day
Code Review
Community Wishlist
Content format
Community Relations
Shadow Namespaces
Mediawiki Stakeholders

Use of the combination of etherpads, Phabricator and wikis to coordinate the sessions

In general the reception was positive. 20 people liked the combination of the three tools the way we used them, however there were multiple other mentions of feeling disjointed and overwhelmed with the massive amount of information and places to look for it. Next time we should officially designate an IRC channel. Multiple complaints throughout that wikis and tasks were created at the last minute. Wikipages, tasks, and etherpad links should be created in advance.

Suggestions for Wiki use

Pretty positive overall reaction to the use of wikis, some people felt they were redundant.

All relevant links should always be on one wiki page.
Keep the overall schedule on wiki
Name wiki pages after session topic and not Phabricator ticket number
Improve wikitable display on phones
If we continue to use wikis, find a way to ensure that they are kept up to date

Suggestions for Phabricator use

Phabricator seems to divide the people who were able to put time into focus on the summit in advance and those who showed up and wanted to get a good idea of what was going on. People who were able to participate in the discussions in advance were grateful to have all of the information in one place on the specific topic. People who were jumping in and trying to get involved on a particular issue at the summit felt overwhelmed and missed key pieces of information that were buried in the task. The use of phabricator was polarizing but more people liked its use than not. However there is still a lot of room for improvement.

All Phab tasks should be updated before a session with a TLDR. Speakers should also give that TLDR. How can newcomers easily understand the discussion.
Phabricator tickets are not a substitute for schedules or session description
More follow up tasks should have been created during the event.
Each session should have one task. It was too confusing to link three tasks for one session.
Requiring the pre-event phab activity in order to schedule a session paid off.
Phab is best for history and complete documentation about a topic.
Format to consider: Brian Wolff's session: devised in phab:T122987, implemented in phab:T114419

Suggestions for Etherpad use

People really liked the use of Etherpads at the Developer Summit. Newcomers were impressed and found it easy to use and jump in and start helping.

Ensure that useful information and action items from etherpads is transfered to phab and wikis later. Assign someone?
Etherpads should be set up in advance, even when using unconference style
Whenever possible etherpads should be projected
People should identify themselves in the etherpads
Someone suggested audio recording the sessions and then going back and checking the etherpad against the recording, we could suggest that and leave it up to the session leaders?
Etherpads encouraged collaboration

Overall suggestions for next time

Designate one ultimate source of truth, the use of all three tools was too much for a few people
More constantly use tools in the same ways and cross link them in the same ways
There was too much duplication and redundancy between Phab and Wiki

Day 3: Unscheduled Day

The majority of comments about the unscheduled day were that it was valuable, should be kept and that no changes are recommended.

Common Themes and Changes for next time

Don't be so strict with rules "door physically open" and "no scheduling at all"
Create a better method for people to track what others are doing. Use IRC, whiteboards, small "what am I working on" signs, or etherpads.
It made the previous two days more valuable and it was easier to focus during the previous two days knowing that an unschedule / informal day was going to follow.
Have a designated quite space
Good emphasizes on the importance of extra time for getting to know volunteers and getting some hacking work done
Figure out a system for people who want to find projects or want to learn about ongoing projects.
Could we have monthly or quarterly designated "hacking days" for people to continue the work and discussions of the Developer Summit?
Collect a short list of projects that could use some work for people who are feeling uninspired, have more support for newcomers.

What did people do

This is going to be a bit of an abstract list because it is compiled from fill-in-the-blank answers putting it together involved some interpretation of responses. Next time we will use the responses bellow in a multiple choice checkbox format to get an ever better idea. The things below are listed in order of number of times it was mentioned. If someone mentioned multiple things, everything they mentioned was counted separately.

Meetings with others continuing dev summit discussions (23)
Socializing / small discussions (17)
Hacking (13)
Talked to people fro my team, team meetings (10)
Collaborated on projects / discussions that needed face to face time (10)
Helped a newcomer / helped someone else with a problem / Taught something (10)
Some Regular Work (8)
Collaborative decision making (7)
Code review / bug fixing / low priority backlog stuff (5)
Did not attend / Overlapping team offsite (5)
Learned from someone (5)
Introduced people to each other (4)
All-Hands Prep (3)
Worked with others to solve a problem identified at the summit (3)
Spoke with others about projects unrelated to my day-to-day job (1)

Logistics

Food

Food quality should be improved
Better options for vegetarians
More types of food, more options of hot food vrs snack food
More caffeine free and sugar free food

Transportation

Muni was confusing and difficult
The Hotel is too far from the venue
Have busses to the venue

Wifi

If you can't help having closed ports, let people know in advance

Other

Don't have the event near the holidays
Identify organizers with different badges
Help newcomers identify people and what they work on
Include directions from the venue to the nearest coffee shop / small store

Anything else?

In the consensus-building process, some facilitation training and tools might be helpful.
Acknowledge first time attendees
Clarify purpose of the event
Wrap-up sessions was confusing and disconnected
There was a small tension between the two overarching themes of "running a project" and "building software"
It would be interesting to have more non-WMF people attending the event. We could benefit a lot from their input
Add lighting talks
Speakers should publish slides on commons

What did people like most

Listed in terms of times it was mentioned

In person discussions / meeting people in person (13)
Talking to people / Social (11)
Unconference (6)
WMF and volunteers meeting (5)
Atmosphere (4)
Scheduled sessions (4)
Learning (4)
Unscheduled day (3)
Decisions that were made / Problem Solving (3)
Constructive (2)

Everything else was mentioned once

Logistics
Quiet room
Seeing teammates
Having the opportunity to present my ideas

What did people liked least

Listed in terms of times it was mentioned

Sessions scheduled at the same time, hat to choose (4)
The Start time was too early (3)
Arguments, off topic and boring sessions in the main room (3)
Public transportation / venue location (2)
Food (2)
Wes's keynote (2)
The wrap-up (2)

Everything else was mentioned once

Undefined Event
Intro about what is OK to say and what is not
Participants should have prepared better
People is sessions did not pay attention
Cozy place to rest was missing
Closed ports
Staff attitudes and culture around development
The message that you have to be planning in advance to attend or you should not go
More community
Unconference scheduling

Data

Day 1 & 2 Schedule and Content Questions

How satisfied were you with the pre-scheduled sessions?

	#	%
Very satisfied	12	14.3%
Satisfied	58	66.7%
Neither satisfied or dissatisfied	12	14.3%
Dissatisfied	4	4.8%
Very dissatisfied	0	0%

How satisfied were you with the keynote?

	#	%
Very satisfied	2	2.5%
Satisfied	20	25%
Neither satisfied or dissatisfied	38	47.5%
Dissatisfied	17	21.3%
Very Dissatisfied	3	3.8%

How satisfied were you with the unconference sessions?

	#	%
Very satisfied	28	34.1%
Satisfied	42	51.2%
Neither satisfied or dissatisfied	12	14.6%
Dissatisfied	0	0%
Very Dissatisfied	0	0%

Day 3: Unscheduled Day Questions

How satisfied were you with the unscheduled day?

	#	%
Very satisfied	32	40%
Satisfied	34	42.5%
Neither satisfied or dissatisfied	12	15%
Dissatisfied	2	2.5%
Very Dissatisfied	0	0%

Do you think the unscheduled day was a useful addition to the Developer Summit

	#	%
Yes	66	81.5%
No	2	2.5%
Unsure	13	16%

Logistics Questions

This event overall [Ratings of the Summit logistics]

	#	%
Very satisfied	29	35.4%
Satisfied	49	59.8%
Neither satisfied or dissatisfied	1	1.2%
Dissatisfied	3	3.7%
Very Dissatisfied	0	0%

Overall organization of the summit [Ratings of the Summit logistics]

	#	%
Very satisfied	33	40.2%
Satisfied	44	53.7%
Neither satisfied or dissatisfied	2	2.4%
Dissatisfied	3	3.7%
Very Dissatisfied	0	0%
N/A	0	0%

Organized social activities [Ratings of the Summit logistics]

	#	%
Very satisfied	29	35.4%
Satisfied	41	50%
Neither satisfied or dissatisfied	8	9.8%
Dissatisfied	0	0%
Very Dissatisfied	0	0%
N/A	4	4.9%

Availability of space for informal discussions [Ratings of the Summit logistics]

	#	%
Very satisfied	43	52.4%
Satisfied	32	39%
Neither satisfied or dissatisfied	6	7.3%
Dissatisfied	1	1.2%
Very Dissatisfied	0	0%
N/A	0	0%

Food served at the event [How would you rate:]

	#	%
Excellent	28	33.7%
Good	34	41%
Fair	19	22.9%
Poor	1	1.2%
Very Bad	1	1.2%
N/A or didn't attend	0	0%

Wifi quality at Mission Bay [How would you rate:]

	#	%
Excellent	16	19.3%
Good	37	44.6%
Fair	12	14.5%
Poor	15	18.1%
Very Bad	2	2.4%
N/A or didn't attend	1	1.2%

Wifi quality at WMF [How would you rate:]

	#	%
Excellent	47	58%
Good	28	34.6%
Fair	4	4.9%
Poor	0	0%
Very Bad	0	0%
N/A or didn't attend	2	2.5%

Meeting Room equipment [How would you rate:]

	#	%
Excellent	27	32.9%
Good	42	51.2%
Fair	6	7.3%
Poor	0	0%
Very Bad	0	0%
N/A or didn't attend	7	8.5%

Anything else?

Attending this event was worth my time [Please tell us how much you agree or disagree with the following statements.]

	#	%
Strongly Agree	46	54.8%
Agree	33	39.3%
Neither agree or disagree	1	1.2%
disagree	4	4.8%
strongly disagree	0	0%

The opportunity to meet fellow developers was valuable to me [Please tell us how much you agree or disagree with the following statements.]

	#	%
Strongly Agree	58	69%
Agree	21	25%
Neither agree or disagree	5	6%
disagree	0	0%
strongly disagree	0	0%

I would like to attend this event again next year. [Please tell us how much you agree or disagree with the following statements.]

	#	%
Strongly Agree	51	60.7%
Agree	24	28.6%
Neither agree or disagree	7	8.3%
disagree	2	2.4%
strongly disagree	0	0%