In the Metawiki proposal to remove Flow, you suggested treating it as a test for the TCG.
I don't think that case is really much use as an effective test. I think the collaboration guideline is about to face a real test. I wanted to let you know in advance, and I invite any thoughts you have on it.
The situation:
The WMF decided to build a new Wikitext Editor. It is currently available as an opt-in beta test feature. According to the plan, in a few months it would become the default wikitext editor. The current editor would temporarily remain as an opt-in option, but eventually it would be removed.
There are currently a big pile of bugs on the project. And of course that's normal and fine. I'm not concerned about them. I have full confidence that our devs will do an excellent job addressing the routine issues. I'm also confident that the WMF isn't going to try to move it out of beta until those routine issue are resolved to a high quality.
However I believe there are two "non-routine" issues. Several editors have told the WMF that these are significant issues. 100% of the (non-staff) comments on the issues agree that they are significant issues. Most of those comments explicitly identified the issues as severe enough to disqualify the project from deployment as the default editor.
I opened a topic on each issue, each topic explicitly stating that the issue should be categorized as an actionable blocker. That statement was in the body of the comment. In retrospect, I probably should have put that in the title of the topic, and I probably should have explicitly mentioned that they were intended as TCG-actionable-blockers. Staff on the project do not seem to have made the connection, and they do not seem to have placed any special significance on them.
Staff have acknowledged the basic validity of the concerns. However the two issues would be either very difficult, or virtually impossible to resolve, given the fundamental plan and design of the project. The response of the staff has basically amounted to "oh well, we can't/won't fix them, and we don't consider them very important".
This is where things always run into trouble. The WMF is great at bugfixes and enhancement requests. But when an issue can't be resolved within the pre-defined project plan, staff deal with it as politely as they can and they just go back to work. And if it's just one person with a dumb issue, that's fine. But then a second editor says this issue disqualifies the project from deployment, and they are politely dismissed. Then the third editor is politely dismissed. Then the fourth editor is politely dismissed. And the project just keeps rolling forwards. Then phases like "talking to a brick wall" start to become common vocabulary in the community.
When you have random individuals showing up with assorted comments, I understand it can be hard to sort out the important issues from the unimportant-individual-fluff. I think part of the problem here is that, for most staff, the only time they encounter an RFC is when it looks like a giant catastrophic meteor falling from the sky. In the community, RFCs are an important, useful, and routine tool. If someone posts on a project page saying they think a project shouldn't be deployed because (issue), I think it would be great if the response was something like this:
That's a hard issue. I don't think that's something we can/will fix. But we will certainly escalate the issue if you point us to a community consensus saying it's really that severe.
Result: The person knows they were heard. They were offered a constructive avenue to pursue their concern. And they have effectively been told to go away.... in the most positive and polite possible meaning of that phrase. Either they drop the non-issue, or the community shoots down their non-issue, or they come back with a consensus saying yes, this really is something important.
The last thing most staff want to do is invite someone to run an RFC, but it really is an excellent way to deal with the community. If the person is a pest ranting about a non-issue, the community will deal with them. If the person is genuinely representing the community view, if the issue really is as important as they're claiming, then that is an issue the project team needs to deal with whether they like it or not. Effectively inviting an RFC in the early stages is far better than a meteor-from-the-sky at deploy phase.
I got rather sidetracked in the last few paragraphs.... but I think it was a good sidetrack. Anyway, back to the two Actionable Blocker issues at hand:
Issue 1: Previews
The New Wikitext Editor does not provide genuine article previews. The "new editor" is actually being designed as a new mode within the Visual Editor extension. The "new editor" actually uses the VE software-stack to generate previews. This means that there's a vast number of things that render differently in the preview than when the page is actually saved.
Project team's perspective (as I understand it)
There have always been all sorts of things that don't display correctly in VE, but it's never been a major issue before so long as it gets most things right. VE is awesome, it's the Future Of Wikipedia, it is the long term plan, everything should move in that direction.
Community perspective (as I understand it)
VE is a secondary editor, used by almost no one. (About 95% of all edits are wikitext edits.) VE's goal was to make editing easier and bring in new people. VE failed that goal. The WMF's own research shows that VE helped an additional 0% of new users to successfully make a first edit. The research also shows it produced a 0% increase in editor retention, and 0% increase in total contributions. Almost everyone uses wikitext because it's simply a better tool for the job.
No one complains about rendering issues in VE because it's a secondary editor, because we understand VE was a difficult project to build, and because anyone who chooses to use VE is choosing to accept any warts that come with it.
We already have an excellent wikitext editor. It already gives 100% accurate previews. The essential purpose of previews is accuracy. Preview errors will frustrate experienced editors and confuse new editors. Replacing accurate previews with broken previews is an obvious downgrade. The team has stated that some of the rendering errors are CANTFIX/WONTFIX. There are currently a ton of serious rendering issues the team does intend to fix, but even if the team fixes a hundred bugs, even if the errors only show up in rare or weird cases, it's just plain dumb to break previews. Many people won't care if the project is aiming for "99+%" accuracy. The blocker here is that the the WMF stop trying to play whack-a-mole fixing most of the endless preview-bugs. The blocker is to provide genuine article views, using the genuine article view engine. That fixes 100% all at once.
Many people are going to be hostile. The view will be that the WMF is breaking our PRIMARY editor... and people will think it's because the WMF's obsession with VE is becoming pathological.
I know everyone at the WMF has good intentions, but the WMF's perspective and priorities sometimes don't quite parallel the perspectives, priories, and experience of editors. Good intentions tend to fall to the wayside when it seems like things are being broken for bad reasons.
Issue 2: Load time
With the current wikitext editor it only takes three seconds before I can begin editing a large and important article like United States.
With the current wikitext editor it only takes five seconds before I can begin editing our largest article.
The New Wikitext Editor is actually a new mode in VE. It requires far more code, it's a memory hog, and I think(?) that it has to spend time analyzing the wikitext before it can do anything.
With the New Wikitext Editor I have to wait thirty seconds before I can begin editing the United States article.
When I try to edit our largest article with the New Wikitext Editor, I'm stuck sitting here staring at my clock for one hundred and twenty seven seconds before I can begin editing.
I don't know if my time measurements will be typical for other people. However the issue is obvious, even if most people get better times than I do.
The perspectives on this issue will be essentially the same as before. For the VE team, slow load times are an accepted fact. No one ever screamed that it was a fatal problem for VE. For the community, slow load times are simply one of the warts that comes along with VE. If you don't like the VE's load time, then don't use VE.
And here too, people are going to get hostile. An editor that takes 30 seconds to load is BAD, an editor that can take 127 seconds to load is absolutely obscene. People will ask why is the WMF screwing up our primary editor? Some people will feel that the WMF's obsession with VE has become pathological, some people will feel that the WMF's disdain for wikitext is bordering on deliberate sabotage.
I first raised the preview issue three months ago, when the very first prototype was available on the test servers. Someone else first raised the load time issue two months ago. Multiple editors have said these issues BLOCK deployment. I explicitly said they should be categorized as "actionable blockers". So we're at the next step. I am in the middle of drafting an RFC to file a formal block against deployment (subject to finding a resolution, of course). I anticipate the RFC is going to pass overwhelmingly. Based on what I know about the project, I don't think these issues can be fully resolved without substantially scrapping the design.