Topic on Talk:WMF product development process

It is not waterfall. The main concerns shared in this discussion have been addressed. Further action may come in the form of edits or new topics about specific problems.

(Reposting, since the original was lost in the conversion to flow)

The process as written here seems to imply that a feature, even a large one like VE, would go through all the phases, roughly in order, as a single effort. Basically it would culminate in a "big bang" release of the whole thing, months or possibly years after the initial conceptual work was done. That contrasts sharply with the agile and FOSS approach of "release early and often".

I believe (as do most agile and open source practitioners) believe that whenever possible, small increments of work (at most a few developer-weeks of effort) should move through all the phases, including release, so that real end-user feedback can inform later increments of the same major feature. I hope that is how this was intended to be used, but if so, it needs a fair bit of clarifying text. If the intent is to shift to a more waterfall-style development process, then that's a whole other discussion we should have.

I wanted to jump back in and say: Having worked pretty closely with Wes, I'm quite confident that he did not intend to propose a waterfall-like process. He is very familiar and comfortable with agile development, and my perception is that he is an advocate of it.

So despite how this page was initially worded, I am confident that our work will continue to be more on the agile side.

Hmmm - I was looking at it differently, but I was focusing at the top of the table where it has iterations of bugs and feedback, so I was thinking of "Release" as being an iterative cycle. Can you describe more about what this would look like to make it seem more agile?

Without context, this page pretty much exactly aligns with a multi-year waterfall process. Even in waterfall, there are some backtracks, exactly as the two "bugs and feedback" arrows indicate.

My main suggestion would be that this page should specifically advocate for splitting bigger features up into smaller ones. And to advocate iterative releases of relatively small bits of functionality, rather than one "big bang" release. (That is, when iterative releases are possible...which they almost always are.)

The page should also warn against performing extensive work in the Concept phase, or any other phase, before the first release. Because anything you dream of up front might end up invalidated by user feedback. You want that feedback loop to be as short as possible, to avoid wasted effort.

The best diagram I could find in 5 minutes was this one. The current wording of this page seems to advocate a waterfall approach, as opposed to the approach labelled "spiral" here: https://en.wikipedia.org/wiki/Software_development_process#In_practice (although I think the quadrants on that spiral diagram are in an odd sequence).

Yes to splitting bigger features and develop on small increments, but there needs to be a definition of general checkpoints where community review is encouraged / required. In a regular agile project the team members will decide next steps on a bi-weekly basis or so, and "real end-user feedback" will start coming as soon as alpha/betas exist. In our context though, we cannot ask our communities to participate in every sprint planning/review, and we need their involvement already in the planning phase. How to solve this equation in a way that works for developer teams and our communities?

@KSmith (WMF) we added some additional text to help articulate things further. Thanks for the input.

Thanks Wes. It still feels more waterfall-friendly than I would prefer, and doesn't emphasize "release early and often" as much as I would like. But the changes look good, and I definitely I like that it now mentions agile.

Per Quim's comment, and in other ways, it's unclear how this process would actually interact with an agile process like scrum or kanban, which many teams are already using. I look forward to seeing this evolve.

@KSmith (WMF) - do you happen to have a table or some sort of imagery to this effect?

+1 to @KSmith (WMF). It might help to show example features and how they move through this process.

+1 to @KSmith (WMF). @Rdicerb (WMF) this imagery is a quick and dirty visualization of iterative cycles vs waterfall cycles:

Waterfall

https://upload.wikimedia.org/wikipedia/commons/e/e2/Waterfall_model.svg

Iterative

https://upload.wikimedia.org/wikipedia/commons/3/39/Iterative_development_model.svg

Also, I was surprised to see categories under "Maintain" and not see TPG in there:

"Retrospective," "Feedback," "Improvements."

Seems right up TPG's alley.

I like those diagrams, Max, but they don't connect to each other. The waterfall one, like the table on this wiki page, could be used to represent 1 week or 2 years. Similarly, the second one has some implication of a faster cycle, but doesn't explicitly make that point. It too could apply to a project with releases every year or two.

I sent a sketch to Rdicerb which tries to show how phases work within an iterative process. Hopefully something like that can be incorporated here, or we can find a (far) better diagram online.

i agree - i see the process represented as linear with maintenance as an end state. i would prefer to see a cycle / spiral of continuous improvement. i see some teams are doing this.

I like the spiral idea to help convey better and I need to figure out how I can do that in the html effectively.

here are some cycle diagrams c:Category:PDCA or c:Category:Spiral_model_of_Boehm

Actually, the agile/iterative image like this one is the one that the product team is thinking about adopting. Does this look like it makes sense around an agile methodology? If you think about a product that is launched in some projects and in Beta in others - say VisualEditor - It's been in use for some years, but Citoid was developed earlier this year. There are languages that it is not ready for that the team has not yet started working on.

Does this image help to clarify?

As the original creator of that image, I feel like it does communicate the point, but in a very crude and non-artistic way. I would love if someone with more design+art talent could do a better job. The image also overstates the maintenance burden over time--when done properly, a given feature really shouldn't require substantial "maintenance". (Users might request additional features, but that would be tracked as new functionality.)

If a better image isn't available, I think adding this image to the article would be a net positive. As long as it has a couple sentences describing what it means.

Is there anything actionable left here or can be consider this topic resolved?

Personally, I think this page still *looks* like a waterfall process. The one mention of agile is nice, but for me doesn't overcome the impression left by the rest of the page. So for me the issue is definitely not resolved. I thought there was going to be another image or two which would help make the page more clearly non-waterfall.

We need to conciliate the agile idea of constant iterations with a reality where hundreds of communities cannot be expected to be involved in a sprint process. The agile process is described in detail at WMF product development process/Proposal, and that is the document that aims to define the daily practice of WMF teams. Here we aim to provide an overview that should allow everybody to understand the different activities and stakeholders involved in the development process.

I think the emphasis of this overview should be put on checkpoints, the first time a project aims to enter a new stage and the requirements to pass each checkpoint. Once a project passes a checkpoint, then they can iterate as much as needed within that stage and the previous ones.

Let's take "Release" as an example for a checkpoint. No matter how agile a project is, it will require several iterations before thinking seriously about releasing a new product to real users. A requirement for that checkpoint could be a minimum release plan. No matter how agile a project is, it would not enter the release stage without a release plan publicly available.

This is not waterfall (in a waterfall you cannot swim back), but there is a sequence. If you allow the comparison, this is more like an adventure game, where a team has to solve problems in order to enter a new room, but they can go back to previous rooms at any time.  :)

@KSmith (WMF) I added a version of the time scale as a chart but it needs the explanation. Could you expand please? @Slowking4 I added an example cycle table to help illustrate the point. Thanks for the input.