Talk:Wikimedia Technical Committee/MediaWiki Platform Architecture Principles/Draft 9-2018

Standards docs usually use not just SHOULD, but MUST and MAY. :) We should (ha!) consider which of the principles are so important they "MUST NOT" be violated, and which are more suggestions ("SHOULD") or things that are permissible but not necessarily a majority case ("MAY"). This could help us zoom in on the key items.

I agree, but I did not find anything that would warrant a strict MUST, not anything that could be reduced to a mere MAY. Do you have suggestions?

You suggested to have horizontal scalability as a MUST. But a LOT of things we do don't match that. Anything that's in MySQL doesn't. I don't think it's fair to call this a must.

This stands out as one of the few principles that has negative cost, since CI is effectively a declaration that we're not going to bother doing manual testing of our software before deployment. It's based on the myth that all bugs can be found automatically. Tests are run after every commit, which means that they have to run quickly, which leads to the ironic situation of having a "continuous integration" system in which very little integration testing is actually done. Instead we run unit tests, in which units are tested against the assumptions of the developers who wrote them. External dependencies are mocked out, replaced by stubs which work the way the developer thinks they should work, instead of the way they actually work. As a result, severe bugs are often noticed by users after deployment.

However, I'm all for cheap things. The reason CI works for us is because in our particular environment, mistakes can be quickly fixed, and nobody dies if we get it wrong. Both users and developers appreciate the fast deployment of features and bug fixes. I'm just unsure about elevating it to the status of an architectural principle. It seems to me that it is a development methodology, not an architectural principle. It's a methodology that is fairly specific to software that we develop and deploy in-house. Nobody is suggesting that we continuously integrate upstream changes from Debian Unstable — it is only feasible when development and deployment are closely tied together. It's fine as a statement of fact for MediaWiki, but it seems too broad to apply to all "other Wikimedia engineering endeavours".

I agree - having unit tests is a requirement in my mind, but they are not a replacement for integration tests or system tests. We need to have all of these.

"good test coverage" is an engineering practice, but not really an architecture principle. design for testability is., but we already have that on another line.

In the old version of the document, we had a section called "Engineering Process Considerations". I'll revive that and move the points about testing there.

I think being more general about interacting with users is a good thing. I think splitting interaction into content and editing subtly hints that these should be treated differently in code, when that's not necessarily the healthiest option for the project in the long term. So I think this is not just sufficient but a better overall direction:

Software that interacts with users SHOULD be designed to make key functionality available on devices with a variety of capabilities and restrictions, as well as potentially limited connectivity.

Ok, dropping the "designed to present the content" line.