Talk:Requests for comment/Configuration database 2

DefaultSettings.json?

Context: In an IRC conversation, ^d suggested turning DefaultSettings.php, which becomes a huge metadata file, into a JSON file

^demon, I thought about this a bit and don't think it'll work. The defaults need to be backend-independent, which means it can't be JSON. Legoktm (talk) 03:22, 9 November 2013 (UTC)

We do a lot of stuff like after extract globals:

if ( $wgDBname === 'testwikidatawiki' ) {
 // do blah!
}

and there even are some hooks and such written into WMF common settings, like SkinCopyrightFooter to set the CC-0 license for Wikidata data namespaces. Important that there is a way to do this stuff in a new system. Aude (talk) 10:12, 20 November 2013 (UTC)

Okay looks like maybe that hook is not actually used but there are numerous such hooks for specific things. Aude (talk) 10:17, 20 November 2013 (UTC)

$wgDBname can't be changed on-wiki since it's needed to figure out which setting to use in the extraction of globals. The loading order will be LocalSettings.php first, and then LocalSettings.json. I've clarified this. Legoktm (talk) 18:41, 20 November 2013 (UTC)

All configuration settings? I hope not.

There are a lot of configuration settings that really don't need to be messed with by anyone except sysadmins. For example, there's no possibility that on-wiki configuring of $wgScriptPath is going to make any sense. A lot of other settings have security or performance implications that are not necessarily obvious. Personally, I think I'd rather see a whitelist of configuration settings that can be managed via this mechanism rather than an assumption that things will be managed this way by default. And that whitelist would, of course, not be able to whitelist itself. Anomie (talk) 15:15, 20 November 2013 (UTC)

IMO everything that can technically be configurable on-wiki should be. The user rights are flexible enough so that if a preference shouldn't be messed with by an administrator, then just restrict the rights so it can't be touched by anyone not in the 'sysadmin' group. I think it will be important to come up with sane defaults for a simple MediaWiki installation. Legoktm (talk) 18:22, 20 November 2013 (UTC)

Before we can proceed, do we need a list of what config settings would be available, by default, to be edited by which user groups, or is it enough if we can agree on some basic principles that will guide this default-setting process? For starters, how about saying that the ones listed at Manual:List of MediaWiki configuration settings containing sensitive data will not even be viewable by anyone but the site owner? The site owner should be in a group that doesn't yet exist, above even bureaucrats. Maybe it's time to bring back the old "developer" group. The developers would be at the top of the hierarchy. Leucosticte (talk) 18:31, 20 November 2013 (UTC)

Well all of them would be available to the configuration API backend/storage. What would then be configurable (on a per site basis, I'd imagine) is which ones can be configured via the web, and by whom. ^demon^{[omg plz]} 22:26, 20 November 2013 (UTC)

And yet the proposal includes an "edit everything" right, which it sounds like should never be given to anyone ever except for maybe the people who could edit LocalSettings.php anyway. Another fun one, you can't give web editability for $wgGroupPermissions, because anyone who could edit that could edit it to give their group "edit everything" and then edit everything. Anomie (talk) 14:02, 21 November 2013 (UTC)

Viewing settings

I hope it will still be possible for non-'configure'-group users to view config settings, as they are presently able to on wikis (all three of them!) that have Extension:ViewFiles installed. Leucosticte (talk) 18:54, 20 November 2013 (UTC)

That is currently the plan. Unless the configuration option has 'private' or 'hidden' set, anyone can view it. Legoktm (talk) 19:37, 20 November 2013 (UTC)

This is actually an improvement with reference to transparency, then, because most wikis haven't made LocalSettings.php viewable by users, but this will make most of the settings viewable by default. Good; hopefully this will lead to more active participation by users in making suggestions for config setting changes and will allow wiki system administrators to learn from and copy other wikis' settings. Leucosticte (talk) 20:17, 20 November 2013 (UTC)

Permissions nightmare waiting to happen

I think "flexible permissions system" and the nature of the configure user right are way too vaguely defined. How would the permission be assigned? If it's bundled with adminship, local admins are not elected with any technical knowledge required at all, and generally tend to be disconnected from the MediaWiki technical community (either by ignorance, choice, or language barriers). To me that spells fights and dumb changes to configuration waiting to happen. If it's bundled with a similar right such as bureaucrat or steward, it suffers from the same problems. If it's not bundled with a currently existing user right, there are two options: either elections locally or globally, which create even more bureaucratic overhead nobody wants, or the configure right is appointed by fiat somehow, which I don't think anyone would like either. Basically I think it's a no-win situation when it comes to deciding how to give out a new configuration right, which suggests to me the idea is fundamentally flawed. Steven Walling (WMF) • talk 21:54, 21 November 2013 (UTC)

I'm not sure I follow your logic here.

For Wikimedia wikis: what's wrong with using stewards? Stewards were always envisioned as the roots of Wikimedia. Stewards will implement requests from local communities just as shell users do now. The fact that logo changes and the like currently require a shell user is bad for many reasons, not the least of which are that it's expensive and doesn't scale.

For MediaWiki wikis: it's long been a request to have a sane mechanism by which to change settings similar to that of any modern computer program. (What desktop application doesn't have a preferences area? Why wouldn't MediaWiki?) Requiring users to edit messy PHP files is obviously a poor design.

This particular proposal does have some user rights issues, to be sure. The proposal seems so flexible that we could end up with hundreds of new user rights, which we obviously want to avoid. Is this your concern? --MZMcBride (talk) 05:27, 22 November 2013 (UTC)

I likewise don't follow your logic. MediaWiki has a Preferences area. Like most applications for the Web and for desktop, this allows to you to change preferences at the user level. Comparing to the preference system in a desktop app doesn't make a whole lot of sense in general, since what we're talking about is adding the ability for sysops or Stewards to change the settings for all other users. That's not preferences, it's site configuration. The only applications I know of that do major site configuration work through a GUI are CMS systems like WordPress. These are not a good corollaries. These systems operate this way because they are designed to be downloaded, hosted, configured, and run by users who cannot do configuration outside of a GUI. On these CMS's, the user who needs to do site config through a GUI also typically is the sole decision maker. There aren't really any good examples of other systems as large as ours, where end users with superuser privileges can do site configuration for thousands of other users who work in a shared collaborative environment. This is probably because it's a recipe for madness. Steven Walling (WMF) • talk 22:06, 24 November 2013 (UTC)

Then I guess we'll be pioneers (again). :-) It's important that we implement a configuration interface in order to reduce wasted shell user time, to make for a much better user experience for anyone administering a MediaWiki wiki (there are thousands and thousands of third party users with very small sites), and to give stewards the capability of implementing site-wide or global (i.e., all Wikimedia wikis) changes as a means of putting power into the hands of those elected to hold it (cf. this mailing list post and m:Stewards/History).

I honestly don't believe adding a graphical configuration interface to MediaWiki is a controversial idea, though perhaps it needs further discussion? I thought there was general agreement that, if possible, everything short of the database configuration information itself (DB username, DB password, DB hostname, and DB database name) should be in a graphical user interface. I think madness is the current situation that requires hand-editing PHP files. --MZMcBride (talk) 06:13, 25 November 2013 (UTC)

Be wary of assumptions. You say that a "graphical" configuration interface would be "a much better user experience for anyone administering a MediaWiki wiki". It is likely that exactly the opposite would be true for experienced *nix sysadmins who are used to editing configuration files, so not really "anyone". Anomie (talk) 14:33, 25 November 2013 (UTC)

Steven, I think there is an important distinction between Wikimedia wikis, and general MediaWiki software. If you're running your own personal wiki or a corporate wiki, just give the permissions to whoever has shell access. On a Wikimedia wiki, as MZ says above, they could be given out to Stewards, or maybe even local crats. That's a different discussion though ;) Legoktm (talk) 16:47, 25 November 2013 (UTC)

It seems obvious that you think we should enable such a GUI on Wikimedia wikis. I don't see a point in developing such as a system if we can't figure out how the permission would be handed out in a sane way. Plus, you should not be designing the system until you decide specifically who the users are intended to be. Steven Walling (WMF) • talk 21:43, 25 November 2013 (UTC)

No point figuring out who to give what permissions if those permissions don't exist ;)

The users for this functionality are any people who currently administrate (in the both the technical and social sense) and run MediaWiki wikis.

I would like to see this eventually enabled on Wikimedia wikis. I don't think its my place to figure out what rights are assigned to what groups though, that's up to each individual community. Legoktm (talk) 03:07, 26 November 2013 (UTC)

Will this change get rid of any capabilities? If a wiki wanted to keep using LocalSettings as it exists now, would that be possible? If that is the case, then these governance issues need not be discussed now, because changing anything is totally optional, and the communities can reject enabling any of the new capabilities. But if changes will be mandatory, then perhaps a migration plan should be set out. Leucosticte (talk) 03:39, 26 November 2013 (UTC)

No. LocalSettings.php will work just fine, and still needed if you want to add extensions, add custom hook functions, or do other non-configuration related stuff. Legoktm (talk) 20:48, 27 November 2013 (UTC)

Accountability and avoiding accidental breakages

I notice not all of the arguments from the old RFC were copied into this one. It would still be accurate to say, wouldn't it, that this change will improve accountability (by logging who changes what) and help avoid accidental breakages, e.g. if someone forgets to put a semicolon before they save LocalSettings.php? Also, won't it be more secure, since it doesn't require giving out shell access to a bunch of people who don't necessarily need the capability of breaking anything and everything in potentially hard-to-fix ways? Leucosticte (talk) 23:09, 24 November 2013 (UTC)

Probably. Feel free to copy them over. I'll also take another look.

Security is somewhat arbitrary, someone could easily set $wgGroupPermissions['*']['editinterface'] = true; (or a lot of other bad config settings) and screw up your wiki. Legoktm (talk) 16:55, 25 November 2013 (UTC)

Yeah, but shell access lets people delete the whole database, file system, etc.; or do any/all of the other malicious kinds of stuff that hackers do when they get shell access, depending on whether the person is root. Bad edits to the interface are reversible. Letting someone make a limited set of configuration changes is still a big deal, but it seems like it would be less of a big deal than granting shell access. Leucosticte (talk) 17:08, 25 November 2013 (UTC)

Title (currently: "RFC/Configuration database 2")

I think the title is perhaps too specific. Lego: what do you think about "RFC/Graphical configuration interface"? Or perhaps you have a better idea? --MZMcBride (talk) 06:15, 25 November 2013 (UTC)

Yeah, that sounds like a better name, it's more clearly focused on what the goal is. Legoktm (talk) 03:17, 26 November 2013 (UTC)

Done. Sharihareswara (WMF) (talk) 20:35, 29 April 2014 (UTC)

Requirements

During the RFC review, Tim said that we should develop clear requirements, to make this easier to figure out what to implement. There's already a short list on the RFC, which I've copied below:

• Performant: should not slow down the site
• Backwards compatible: should work fine with existing code and extensions
• Farm support: Work for both single-wiki installations and large farms like Wikimedia
• Cross-wiki support: should be able to access another wiki's settings
• Flexible permissions system

Some of the other ideas mentioned during the RFC review:

• local UI and global UI for changing settings

What else? Legoktm (talk) 17:03, 25 November 2013 (UTC)

Backend

One of the other main things discussed in the RFC review was what backend to use for storage. There are a few different options that have been discussed, which I'll list below. Legoktm (talk) 17:18, 25 November 2013 (UTC)

JSON/CDB

• Basically uses key-value storage to store options
• <TimStarling> neither JSON nor CDB are really appropriate backends for a web interface (Can someone expand on why? Yuvipanda (talk) 02:41, 23 December 2013 (UTC))

• I'm not sure what Tim's objection is exactly, but using JSON is really not a good idea since the JSON must be parsed on each web request. On sites like Wikipedia, that just introduces overhead that isn't necessary. -- ☠MarkAHershberger☢(talk)☣ 14:44, 4 January 2014 (UTC)

Database (MySQL, Postgres, etc.)

• New table with config option, value, wikis that it applies to...
• For cross-wiki, one local database and one global one?

On-wiki ContentHandler

• Already has history/diffs built-in
• Can't store private settings (not very many of those anyways)

• Could we not expand Manual:$wgNamespaceProtection to also protect against viewing protected namespaces? It could be useful elsewhere anyway. --MarkTraceur (talk) 23:10, 4 December 2013 (UTC)

• Not easily. See Security issues with authorization extensions for some of the challenges. It's about time someone does fix that, but I don't want to mix that up with this RfC. Legoktm (talk) 03:55, 5 December 2013 (UTC)

• Can use API to get settings for another wiki
• Uses database underneath
• Has edit protection basically built-in, too

Discussion

I thought a bit about this today, and I'm really liking the idea for doing it on-wiki. A new namespace, similar to how messages are stored in the MediaWiki namespace would work, with some kind of caching layer. More thoughts on this would be appreciated. Legoktm (talk) 03:16, 26 November 2013 (UTC)

One of the disadvantages with that is that it's hard to keep config settings secret if you do it that way. But it's probably time to fix these security issues anyway. The other thing is that it's yet another namespace. I try to just use the MediaWiki: namespace if I'm only going to be adding a few more pages that will require access restrictions. Leucosticte (talk) 03:45, 26 November 2013 (UTC)

Right, you can't store private settings. There aren't that many so I'm willing to make that tradeoff. The MediaWiki namespace already is filled with interface messages, so I think this should use it's own namespace. The editing interface would also need to be different to work with arrays and such, plus you would want different access permissions for MediaWiki namespace and configuration settings. Legoktm (talk) 04:00, 5 December 2013 (UTC)

How many pages will this new namespace likely have? Do we anticipate that more will likely be added later? Leucosticte (talk) 07:15, 5 December 2013 (UTC)

A major downside with a new namespace is that you don't remove the requirement that the person configuring the extension actually knows the "proper syntax" -- it'd be almost just as bad as editing PHP files, the only difference being they can't also execute arbitrary code. Also, you say there "aren't that many" private settings. That means that there are some, and extensions usually also add more. Where do these private settings get configured? In LocalSettings.php? If so, then instead of unifying configuration to an interface that makes sense, you a) split it up seemingly ad hoc between the old way and the new way, and b) without increasing usability of the new interface besides making history and diffs more easy (still need to know syntax and exactly which page names to edit). In all, I don't think wiki pages are good for this. Special page, yes, wiki pages, no. --Skizzerz 17:12, 19 December 2013 (UTC)

Why would they need to know the proper syntax? We could have custom edit interfaces like how Wikibase does. We would still have a special page listing the various configuration settings so people don't need to know the specific variable name (the equivalent of Special:allmessages but with more information/help links). I'm not sure if there is a good way to fix the private settings issue though. Legoktm (talk) 00:48, 23 December 2013 (UTC)

Private vars would still need to be set in LocalSettings.php, yes, and this does complicate things, but we can use this as the motivation for really working on the privacy/security aspects of MW as there will be (some) motivation to get this done within the WMF since there would be an actual need for this sort of thing. -- ☠MarkAHershberger☢(talk)☣ 15:07, 4 January 2014 (UTC)

There would be a new page for each $wg global, and pages would be added/removed based on whenever new configuration globals are added or removed. Legoktm (talk) 00:48, 23 December 2013 (UTC)

Resetting indent and some comments from IRC (for some reason I can't indent a <pre> and get it to work right...):

<Skizzerz> legoktm: the moment you insert a custom edit interface, you're already at special page status effectively
<legoktm> Skizzerz: but the backend of where it gets saved is what's important
<Skizzerz> sure, and I argue that pages are not the appropriate place for it
<Skizzerz> 1) config changes should go in their own log, instead of a generic edit on RC
<Skizzerz> 2) how do you configure private settings while keeping them private?
<Skizzerz> 3) you're now grabbing content from the (greatly bloated) text table instead of a table designed for config vars, making stuff slower
<Skizzerz> in addition to all the other indirection between page/text via page_id so you need to add joins to the mix
<legoktm> Skizzerz: don't we already do that for MessageCache?
<Skizzerz> we need config variables before we can run most of the wiki code
<Skizzerz> we don't need messages to do that

Additional thoughts: if we need a special page to track all the variables on this custom namespace anyway, why bother having said custom namespace? Making things work like the MediaWiki namespace is not necessarily a good thing (in fact, I'd argue as well that the MediaWiki namespace was a kludge back in the day and all the interface messages should have a special page to configure them as well instead of being wiki pages). The good things about making them wiki pages (history and talk pages are the only two things I can think of. Standard diffs are worthless if you have a special editing interface since you'd need a special diff engine per setting type anyway) are easily enough re-implemented or done elsewhere with a special page (tracking histories can be done with another table, the changes themselves can be done as log entries for a special configuration log, and discussion can happen on a village pump type page or wherever the community talks about things). --Skizzerz 02:19, 23 December 2013 (UTC)

I like the idea of a configuration namespace. The MediaWiki namespace may have started as a kludge, but it enables some very nice abilities -- allowing the users to change the appearance and behavior of the UI -- that demonstrate the need for a configuration namespace. -- ☠MarkAHershberger☢(talk)☣ 15:22, 4 January 2014 (UTC)

I started drafting a schema for it being in the database as a custom table (assuming a special page frontend), it's available at http://skizzerz.net/configdbschema.pdf -- I probably won't have any extra time to work on it before the summit, but if it's decided at the summit that a special page is the way to go, it may come in handy :) --Skizzerz 05:50, 5 January 2014 (UTC)

Revamp

With some input from Skizzerz, I've revamped the RfC to include a wikiset editor to solve the wiki farm/WMF issue, and decided upon using a database table to store settings. Hopefully this will get discussed at the Architecture Summit so we can start implementing this. Legoktm (talk) 09:05, 19 January 2014 (UTC)

discussed in architecture meeting

We discussed this RfC in an RfC review/architecture meeting on 12 March 2014.

I think one of the authors agreed to spec out requirements further.

Also see http://lists.wikimedia.org/pipermail/wikitech-l/2014-April/076255.html . Sharihareswara (WMF) (talk) 20:58, 29 April 2014 (UTC)

I've reverted the changing of the RfC title for now, I was going to split the RfC into "sub-RfCs" which would cover the 3 major things in the RfC: 1) configuration interface (currently in process), 2) the actual configuration database, and 3) the graphical configuration interface. Since the RfC still covers all 3, the page move was premature in my opinion, and should be done when it's more focused towards that. Legoktm (talk) 08:55, 10 May 2014 (UTC)

Extra permissions to make open config work better

First off, this is a really good idea, especially for third parties (wikifarms will suddenly become a lot more customizable!). Virtually every other widely used large piece of software for running websites with things to configure has had this for years (forums, other wikis, CMSs, blogging sofware, etc). It'll be great if wikiapiary starts collecting information on how wikis are configured. I can see two cases where adding an extra permission to MediaWiki would help smooth out opening up access and make this even more useful.

1. A way to exclude giving some rights from the general "userrights" power and the general power to edit usergroup rights. Perhaps "dangerousrights" as an array, with users only able to add those rights to a usergroup or add users to a usergroup with that right if they have the "dangerousrights" user right (which would be given by default to the developer group). In a lot of cases a site owner will want to grant someone access to everything they safely can in one go, and the implementation described with just one configure everything right plus a possibly large number of specific groups (with more potentially added by extensions) does not cover this well.

2. A way to hide specific config settings from the public. I'm all for transparency, but one of the biggest wins for smaller wikis from the creation of this will be allowing semi-trusted users to contribute an endless supply of capatcha questions. You need to be able to hide the capacha questions from the spambots. An array of config variables which can't be seen unless you have the power to edit them would be good.--etesp (talk) 22:01, 11 May 2014 (UTC)

Status?

Hi - what's the status on this RfC and can we help move it forward? Thanks! Sharihareswara (WMF) (talk) 20:14, 21 July 2014 (UTC)

Blocked due to only having 24 hours in a day ;) Right now I want to focus on extension registration which is a sub-part of the configuration overhaul. Legoktm (talk) 07:36, 2 August 2014 (UTC)

Easier wiki farm installs

In the 2018 dev summit session on 3rd party users of mediawiki, the topic of configuration came up. It was mentioned that our configuration is currently this strange mix of executable code in LocalSettings.php and DB entries (for Main Page content, various messages, etc). This RFC seems to veer hard in the "all configuration should be in the DB" direction... but then we need to make it straightforward to programmatically initialize the DB, update it, etc. Consider installing and then upgrading the configuration of a farm of 100 or so wikis. cscott (talk) 19:13, 2 February 2018 (UTC)