It would be great to provide a mechanism for disabling extensions on the fly - as far as I know, Composer doesn't provide a way to un-include an extension without actually uninstalling it. The only solution I can think of is to have a global variable called something like $wgDisabledExtensions, settable within LocalSettings.php, that would hold an array of extension names. Any extension included using Composer would be expected to check that array right in the beginning, and, if its name was included in the array, to exit out immediately. What do people think?
Talk:Requests for comment/Extension management with Composer
+2 I also strongly believe that it must be possible to prevent the autoloading of extensions installed via Composer. I do not know what the best implementation of this will be, but Yaron's idea sounds good to me.
As I have already indicated on the SMW mailing list, I think this approach is conceptually incompatible with dependency management. Not loading something that is installed undermines the safety of the system. It's like installing packages via apt-get and then disabling it manually somehow in such a fashion apt-get does not know it is no longer installed.
I do not want to repeat directly what was written on the mailing lists, but they provide an idea of the issues and concerns raised:
[1] thread of the smw developer mailing-list
[2] thread of the smw user mailing-list (starting with egel at Jan 18, 2014; 10:10pm)
Jeroen - could you clarify in what way safety would be undermined, or give an example?
Install A, disable it. Install B that depends on A. No problem is detected since A is "installed". Wiki breaks because B misses its dependency.
Ah, I see. Well, I imagined that extensions would also check for their dependencies - so that an extension that required Semantic MediaWiki (the obvious example) would also exit gracefully if SMW were included in the "disabled" array.
Dependencies are transative. Do you want SMW to check for all its dependencies? That breaks encapsulation, and gets really ugly as soon as you use components you do not maintain yourself.
Also, in case someone then runs into an error because their installation is broken (rather than a tool catching the issue before breaking the install), what are they going to do about it? If you have the disabled thing, I see some users trying to do a manual install. Or perhaps install a wrong version.
If you break all that security, you can forget about relyable automation. So bye bye idea of having a GUI extension installer on top of the existing Composer support.
Well, I probably should have been more detailed with my proposal. I'd imagine that only "user-facing" (if that's the right term) extensions would be disable-able; so if someone added, say, Validator/ParamProcessor to the the array of disabled extensions (one of SMW's dependencies), that would simply be ignored. I don't think it would be that hard to have extensions handle this variable in a smart way.
I also don't see the connection between this and security and automation; although maybe that's no longer relevant, given the above.
> I don't think it would be that hard to have extensions handle this variable in a smart way.
What you are saying is that $wgDisabledExtensions becomes intelligent or the extensions that handles this variable should attach some logic in order to handle dependency knowledge?
While I could agree to have $wgDisabledExtensions as debugging tool to temporary disable a extension, I surely can't see $wgDisabledExtensions as tool that would circumvent the intention of Composer namely the management of dependencies.
Removing a package from the composer.json and therefore from the deployment tree is the safest method because it ensures that dependencies are properly resolved and not just disabled.
I guess I'm saying the latter - I just mean that a simple logic needs to be in place: if the current extension is listed in the array, exit; if a "real" (non-library) extension that this extension requires is in the array, exit; otherwise, don't exit. What's unsafe about this approach?
Yaron: can you make a clear case for why it is problematic to manage what is installed purely via Composer? I simply fail to see the real problem there. And without that any undermining of the current system is not going to seem worth it.
Sure, I can think of two obvious examples: running a wiki farm, and debugging one's own installation. In both cases, the admin might want to disable extensions in any combination - well, almost any, barring of course extensions that depend on another. As far as I know, there's no way to do that currently with Composer; so for the debugging, one would have to keep re-installing different combinations, while for the wiki farm you'd have to literally have a different set of code for each possible combination - annoying if you have more than one extension, untenable if you have more than, say, three. That's not to mention the difficulty it would place on maintaining any local changes to the code.
Also, can you explain in what way my proposed system would undermine things?
> running a wiki farm, and debugging one's own installation.
I do not see the actual problem in either case. You are guessing some stuff will be very hard or not possible. However if you know how to use this tool I think you can do everything you need to do without to much bother. Some things will get a bit more cumbersome, though at the same time others become less so.
> so for the debugging, one would have to keep re-installing different combinations
In some cases you might want to do this yes. Though is removing the thing from the composer file and running composer update really that hard? It's slightly more work than commenting out an extension. Though at the same time it will also stop loading not needed dependencies automatically, which you'd have to do manually otherwise.
Also, disabling extensions is one thing you might want to do during debugging (though not something you do every minute I'd hope). Another thing is switching between different versions. That one is actually easier with Composer.
> wiki farm you'd have to literally have a different set of code for each possible combination
Yeah so? Disk space is a non-issue. I personally would prefer this approach if I ran a wiki farm, it's way less scary than running different wikis with different config and even different extensions via "the same" code.
> That's not to mention the difficulty it would place on maintaining any local changes to the code.
If you do not manage your changes via git, then yeah, things could get difficult. (Though in many cases you could still do what you want to do without to much hassle). The only place where this gets really annoying is if you indeed have a farm and need the changes at all places. In which case you can just stuff them in git and have much more solid control over your farm than you'd otherwise have as well.
> Also, can you explain in what way my proposed system would undermine things?
I have already done so in a previous post.
Come on, you can't seriously suggest that someone running a wiki farm set up 32, 64 etc. separate installations, one for each combination of extensions you might want to disable, is a workable solution? What if the code includes some custom changes? It's a nightmare just to think about it.
Similarly, if someone is uninstalling extensions to do debugging, and any of their extensions contain custom changes, that becomes quite a bit of work as well - and I would think they easily run the risk of deleting important work.
I know you explained the undermining thing before, but I'm pretty sure that was based on an incorrect understanding of my proposal, as I noted. Could you try explaining it again?
I think you both have very valid points.
Having every extension check a config variable and disable itself is not a solution in my opinion. Some will do it, some won't. It will get messy. And it will get really messy once you get to dependency trees that are more than one or two levels deep. The whole point of dependency management is to not have to check if everything is in place. Take the Lingo extension. It depends on ext-dom (not a very good example, i know). Now suppose somebody switched that off. Lingo would consequently not load. However, SemanticGlossary would, because all its dependencies are there. And yes, you could put all sorts of logic into maintaining that config var of switched off extensions, but doing that in each and every extension is just wrong.
Would it be possible (both technically and "politically") to change composer, so that you could load and install a library, but switch it off, so that the library itself and all depending libraries are not included in the autoloader? This way you would only need one set of libraries for all wikis of a farm and only the autoloading scripts need to be different.
F.trott - yes, that's not a great example: I'm not even sure what ext-dom is, but I'm assuming it's some sort of library, not meant to be used standalone, so you couldn't even disable it under my proposed scheme. However, you do maintain another extension, Semantic Forms Inputs, that does fit the bill. And for that sort of extension (and maybe for Semantic Glossary as well, if I'm wrong about it), I would say (a) it's not a huge deal to have extensions check for all their dependencies, if they have more than one, and (b) even if they don't check for all dependencies, it's not a big problem: if the admin disables an extension but forgets to disable an extension that depends on it, they'll get an error message - which is the same thing that happens right now if an admin comments out one extension but forgets to comment out the extensions that depend on it. So the situation won't be any worse.
If there were a true Composer-based solution, that would be ideal, of course.
> (a) it's not a huge deal to have extensions check for all their dependencies
The extension shouldn't have to check dependencies because that's the job of the Composer otherwise you have two components that do similar things, the Composer and the extension.
> it's not a big problem: if the admin disables an extension but forgets to disable an extension that depends on it, they'll get an error message
You arbitrarily disable a component (or package) knowingly at the cost of incompatibilities or even instability. That's a questionable approach!
Yaron, I don't say that your scheme will not work. It may, technically, in an ideal world. It requires that each and every extension developer plays along nicely and (1) checks all their dependencies and (2) turns their own module¹ off if it's in the config var. Fat chance, really.
I mean, as a developer why would I check for dependencies, if I know that's taken care of by composer. After all I carefully crafted that blasted uncommentable² json. And why would I turn my extension off for you? What's my benefit? Heh, some developers will not even know that that was required. And they won't notice either, all the intended functionality would be there.
No, IMHO you need a solution that is enforcible, i.e. something that does not require extensions to play along. If you wanted you could probably write some central manager that needs to be called by extensions for them to be enabled. And that central manager could then decide to not enable an extension based on some criterion.
I'd rather try getting a patch into composer than trying to get some extension manager into MediaWiki.
We can not be the only ones with this issue, can we? How do other frameworks handle this?
¹ Yes, module. I don't buy the "only visible extensions" part. It's an arbitrary definition. ² You actually can misuse undefined key-value-pairs to put comments in the composer.json.
MWJames, F.trott - I disagree with some of your specific objections, but I agree with both of you that my proposed solution has weaknesses. However, it's the only solution I could think of - if there's a better one, whether it's directly part of Composer or not, I'd be glad to switch to that one instead.
Come on, you can't seriously suggest that someone running a wiki farm set up 32, 64 etc. separate installations, one for each combination of extensions you might want to disable, is a workable solution? What if the code includes some custom changes? It's a nightmare just to think about it.
Each wiki would have their own composer.json file. And if you have wikis that have the same config, they could share the file. All you then need to do is keep the composer.json files up to date (which is likely easier than managing this in one huge file with switches in it). And run "composer update" in each wiki dir. That does not sound very hard to me. That includes handling custom changes you made.
So yes, I seriously suggest this. And I suggest you learn how this tool works and can be used, before you conclude it cannot fit in a particular workflow.
If you ran a wiki farm with 100 wikis in it, and you wanted to make a local change to some Composer-based extension, would you need to make the change once, or 100 times? (Or somewhere in between, if you tried to group together extensions that had the same configuration of extensions?)
You'd need to make it once.
Can you document this setup you're talking about somewhere, or at least explain it in more detail? I don't understand how you can have multiple composer.json files modifying the same set of code. (If I understand even that part correctly.)
Indeed, an explanation would be nice. I do understand the management of dependencies of composer, and I can see that wiki farm configurations must be handled in a new manner with composer. But I cannot see how one could split the extensions of a farm into two sets: commonly used extensions and individually used extensions added to the commonly used extensions. Any idea for solving the following problem?
- wiki farm with 100 wikis
- all wikis share a basic set of extensions
- some wikis add further extensions which are not activated on the whole farm (example: BreadCrumbs)
Solution with LocalSettings.php
- one directory with all extensions (including BreadCrumbs)
- one file
globalExtensions.php(containing therequire_once-statements for the common set of extensions) - individual files
localExtensions.php(containing therequire_once-statements for individually chosen extensions) - individual
localSettings.phpfiles (including the filesglobalExtensions.phpandlocalExtensions.php) - adding/removing of a globally used extension:
- adding/deletion of
require_once-statement inglobalExtensions.php - adding/deletion of extension from the global extensions directory
- adding/deletion of
- adding/removing of a locally used extension:
- adding/deletion of
require_once-statement inlocalExtensions.php - adding/deletion of extension from the global extensions directory
- adding/deletion of
?? Solution with Composer (assuming that in the long run all extensions will use Composer)
- one directory with all extensions (including BreadCrumbs)
- one file
composer.json(not including BreadCrumbs) - link of individual wikis to the global
composer.jsonfile - ?? BUT my question is: how can individual wikis ADD BreadCrumbs to this scenario?
- adding/removing of a globally used extension: ok - treatment via composer
- adding/removing of a locally used extension ???
I am aware of the fact, that one could argue, that there might be dependencies between globally and locally used extensions - but adhering strictly to this credit would mean that wikis on a farm cannot be indiviualized, unless all possible combinations of individualizations are spelled out in separate composer.json files and each wiki links to a different composer.json file - which makes the idea of maintaining a wiki farm rather odd
PS I have chosen BreadCrumbs as an example of an extension which changes the appeareance of a wiki if it is activated
Does anybody have a solution for this scenario? Does composer have an inheritance mechanism?
This post was posted by Kappa~mediawikiwiki, but signed as Kappa.
Useful summary, despite the blanks about composer: can you integrate it into Manual:Wiki family? Thanks.
I can come with a POC for wiki farms – I tested it with my wikifarm extension – but it’s very hacky. For the context, let’s say you have a single directory with MediaWiki code ($IP) and two separate directories, one for each wiki ($D1 and $D2), and you want to install SemanticMediaWiki with Composer only for the first wiki (but you have to execute Composer for both wikis given MW core has Composer dependencies).
$ cd $IP $ vi composer.json (and add in config: "vendor-dir": "$D2/vendor") $ composer update --no-dev $ vi composer.json (and change in config: "vendor-dir": "$D1/vendor") $ vi composer.json (and add in require: "mediawiki/semantic-media-wiki": "*") $ composer update --no-dev $ mkdir vendor $ vi vendor/autoload.php (and add your logic of selecting the wiki followed by require_once "$D1/vendor/autoload.php" or "$D2/vendor/autoload.php")
At the end, you have the real Composer data in $D1/vendor and $D2/vendor. When executing MediaWiki, the only visible difference (that I see) compared to a classical installation is MediaWiki doesn’t show Composer libraries in Special:Version.
This type of hack fullfills the issue with dependencies said by Jeroen since each Composer execution is kept and hence is a coherent set, and it solves the basic need for a wiki farm. Obviously it stays hacky and it needs some work and tests to integrate it in a real wiki farm. An issue is you have to execute the “biggest set” (here MW+SMW) in last, else SMW will be removed from extensions/ (I know, “biggest” is badly defined since the order is not total).
EDIT: in fact, I now see Composer librairies in Special:Version. Don’t know what happened.
@Nemo bis @Yaron Koren @Jeroen De Dauw @F.trott @MWJames @Kghbln @MarkAHershberger @Mglaser @Cindy.cicalese @Egel @Nicolas NALLET: I revive this thread and ping you since I just did, I dare to think, a breakthrough on this issue, after thinking and working during some months about it.
I wrote a summary of the issue (from this thread and personal ideas and experiments) on Requests for comment/Extension management with Composer/Farms and Composer; and I just finished a proper implementation of what I think is the better compromise to manage extensions with Composer in a farm (4th method in the summary page). This allows to activate and deactivate the Composer-installed extensions on a per-wiki basis in a farm, and there is almost no overhead, though it has to be checked in a production environment. I’m interested in any comment you have about the summary/methods and/or the MediaWikiFarm extension.
Although this method works, this stays fragile; and I personally greatly prefer non-Composer-installed extensions, or at least when the activation is independent from Composer, like did PageForms 4.0 and SemanticFormsSelect 2.0 (for the some extensions I’m aware).
It would also be helpful to get input from @Legoktm
This talk is mentioned at Requests for comment/Improving extension management
The approach currently used by SMW and some other extensions does not allow specifying which MW versions are supported as far as I know. Using a meta package into which both MW and extensions are installed would solve the issue, though hardly seems like a viable approach at this point. (This means a big workflow change and requires all global scope assumptions in core to be fixed first.) I'm investigating how to do this in a way that can be used in the near future. If anyone figured this out, I'd be very interested in the details.
I had a long chat with simensen on #composer about this. We came to the conclusion that having "provide": { "mediawiki/mediawiki": "$wgVersion" } in the root composer.json. This then allows extensions to specify which MW versions they are compatible with. Still figuring out how to best implement this.
Actually putting this code into the file is somewhat problematic, as it needs to be updated whenever the MW version changes. Having users do this manually is clearly problematic. Thus we either need to automatically update this, or not actually have the info be in the file itself, but rather added problematically via a Composer hook.
I tried using the Composer scripts system, which unfortunately gets initialized too late to add things to the provide section. Then I tried using the plugin mechanism and having the handling code in MediaWiki itself. This also did not work, as the plugin system apparently requires using a dedicated package. My third try was updating the version on run of update.php, for which I have code here. This is also not going to work, as MediaWiki is already loaded when this script runs, and would thus require a re-run or some other silliness.
A regular plugin (in its own package) could presumably be used, though it'd first need to be installed. This thus requires running composer install before adding any extensions to composer.json. That sucks.
I'll further poke people about this. Perhaps a new hook in Composer is needed, or an enhancement to the plugin system.
What about fixing composer so that it can install things via {composer} install $package_name without requiring they be present in the composer.json?
This would allow you to have a proper composer.json committed to git with all the provide, dependency, etc... information already written in it.
By all means, create a feature request on the Composer project.
I have filed an issue on the Composer issue tracker for this.
The issue has been resolved, so the "provide" approach is now viable. Implementation in this commit.
I have proposed a patch at Gerrit change 162367 to restore the Composer hook integration to MediaWiki's composer.json file.
I asked Reuben Smith from wikiHow:
- Would Composer work in your MediaWiki environment?
and he replied:
- I don’t think we have any comments about Composer at this time. We use Composer for a few other packages, so we support the development effort.
and gave me permission to mention that here.
We'll be discussing related RfCs on Tuesday 29 April, 2200-2300 UTC and I thought people interested in this RfC might want to participate, by commenting on the other proposals, and/or by coming to the IRC meeting.
Summary of discussion: We have some open questions for Markus regarding Composer, and for administrators of MediaWiki installations. Would Composer work in your environment?
- LINK: https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer (TimStarling, 22:19:37)
- LINK: http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/ (sumanah, 22:24:03)
- <bd808> The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster). The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs. (sumanah, 22:31:33)
- <parent5446> Technically it should be trivial to convert most extensions to a composer library (sumanah, 22:31:50)
- Wikidata team has some Composer experience per aude's explanation (sumanah, 22:32:48)
- discussion of whether to deal with multiple packaging systems or just one, use something like https://github.com/derrabus/composer-webui for a web UI, CLI vs web (doing web UI securely is ?) (sumanah, 22:33:38)
- IDEA: question for composer-for-extensions: best way to fit in with our source-control-based deployment? (brion, 22:37:11)
- IDEA: question for composer-for-extensions: web ui possible for small installations? (compare with wordpress’s installer & extension manager) avoid chmod 777 (brion, 22:37:11)
- IDEA: question for composer-for-extensions: avoid proliferation of package managers, but make sure we can still install extensions manually if need be or for migrations (brion, 22:37:12)
- <bd808> A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md (sumanah, 22:40:59)
- ACTION: sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page (sumanah, 22:42:16)
- ACTION: sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate (sumanah, 22:43:01)
This is the IRC log of today's discussion - this happened in between two other RfC discussions so look at the full log for context.
- 22:19:45 <andrewbogott> Yeah, I was going to say: I don't think Ryan_Lane is fine with it :)
- 22:19:47 <sumanah> but since Markus isn't here and JeroenDeDauw needs to head off, it's suboptimal. Ah well
- 22:20:22 <Ryan_Lane> composer, as currently implemented, means you need to run composer on every node you deploy to. it also makes it impossible to disable any individual extension
- 22:20:30 <TimStarling> I don't think there's any reason to use composer in production
- 22:20:38 <aude> Ryan_Lane: no
- 22:20:44 <aude> you make a build
- 22:20:57 <aude> with all the stuff installed
- 22:20:59 <aude> run tests on it
- 22:21:00 <bd808> The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster).
- 22:21:05 <aude> then deploy that
- 22:21:14 <aude> all the stuff checked in, including composer.lock
- 22:21:17 <Ryan_Lane> what about other people using similar deployment methods?
- 22:22:06 <mwalker> what aude has described is basically how we're having to deploy Node.JS apps
- 22:22:08 <aude> it's also possible to have an internal satis (packagist) system
- 22:22:24 <bd808> The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs.
- 22:22:24 <aude> that only knows about our own stuff
- 22:22:26 <TimStarling> I'd rather not have any kind of external dependency for source in production -- just our own git servers
- 22:22:33 <Ryan_Lane> so, does this assume that all WMF extensions will also have composer data?
- 22:22:48 <brion> can composer use private repos?
- 22:22:50 <TimStarling> the wikidata thing is to run composer locally and then push the result into git, right?
- 22:22:51 <aude> probably our own satis + a build would work
- 22:22:54 <aude> brion: yes
- 22:22:57 <parent5446> Technically it should be trivial to convert most extensions to a composer library
- 22:22:59 <brion> yes looks like good
- 22:23:00 <brion> :D
- 22:23:14 <aude> TimStarling: yes
- 22:23:28 <brion> so what does it look like to composer-ize an extension? what sort of migration path would we be looking at?
- 22:23:38 <aude> including it's important to carefully check composer lock and the diff
- 22:23:40 <brion> and would this just handle fetching files, or would it also handle installation, enabling, etc?
- 22:23:45 <parent5446> You make a composer.json file that tells composer to include the main extension file
- 22:23:47 <mwalker> brion, http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/
- 22:23:54 <aude> plus all our tests run and then only after they pass, it gets merged
- 22:23:59 <TimStarling> composer fetches files and handles class autoloading
- 22:23:59 <brion> thanks
- 22:24:03 <sumanah> #link http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/
- 22:24:36 <TimStarling> it has its own autoload system, and does clever things like scanning .php files on install for class definitions
- 22:24:41 <aude> some of a problem will be when some other extension wants to use same library that wikidata uses
- 22:24:58 <TimStarling> in addition to PSR-0 of course
- 22:25:01 <aude> would be nice to have it registered in a shared place
- 22:25:01 <brion> related question:
- 22:25:12 <brion> obviously small installations would love an extension installation manager.
- 22:25:23 <brion> would tying to composer force us to require CLI for extension setup?
- 22:25:37 <brion> or would it allow for a web control panel as well, given appropriate permissions on server?
- 22:25:42 <sumanah> ok, it sounds like there is a lot of interest in Composer usage. Ryan_Lane does this interest you enough to make you wanna collaborate on investigating things after today, working on the RfC with Markus Glaser or whoever else wants to work on this? JeroenDeDauw maybe?
- 22:25:43 <aude> TimStarling: we also use composer option to dump a classmap (helps optimize)
- 22:25:54 <sumanah> because we're not gonna figure all this out and approve it today
- 22:25:57 <aude> so in the end it is like AutoLoader.php
- 22:25:58 <TimStarling> we can support any number of pacakge management systems in parallel
- 22:26:01 <parent5446> brion: Yes, unless we make a PHP interface for composer, which we can
- 22:26:13 <brion> parent5446: that works for me
- 22:26:13 <TimStarling> we don't need to support composer exclusively
- 22:26:20 <parent5446> ^that as well
- 22:26:26 <brion> hmmmmmmmmmmm
- 22:26:30 <brion> i hate mixing package managers though
- 22:26:38 <mwalker> +1 to brion
- 22:26:40 <aude> brion: a php interface would be cool, but not aware of one
- 22:26:45 <TimStarling> as long as we document dependencies as well as putting them in composer.json, people can keep on installing things the old way
- 22:26:50 <Ryan_Lane> sumanah: I can't really spend time on this. I need this as a user of mediawiki, though
- 22:26:50 <Dantman> I also have doubts about being able to do a web interface around composer.
- 22:26:50 <aude> probably would be fairly simple
- 22:27:00 <Dantman> Especially a secure one.
- 22:27:01 <sumanah> Ryan_Lane: totally fair - just wanted to check
- 22:27:27 <brion> Dantman: secure is the hard part. wordpress updates for instance seem to jump through weird hoops to ssh into itself or some weird shit :D
- 22:27:32 <brion> it scares me
- 22:27:38 <brion> but is an oh-so-useful feature
- 22:28:04 <TimStarling> it's a bit more secure than just using the web user for installation
- 22:28:25 <brion> chmod 777
- 22:28:26 <TimStarling> at least after installation, the source tree is not writable by the webserver
- 22:28:31 <brion> yeah :D
- 22:28:35 <parent5446> Wordpress also allows using FTP to update
- 22:28:39 <Dantman> Most of the important metadata is still in PHP. So you can't read it without first fetching and installing the extension.
- 22:28:41 <brion> *shudder*
- 22:28:45 <bd808> Google found https://github.com/derrabus/composer-webui for me
- 22:29:36 <parent5446> Looking at the code for what bd808 just posted, you can basically just include composer itself, and use it's internal interface
- 22:29:36 <sumanah> hexmode: you talk to Markus a lot, I assume :-) Do you know whether Markus has time to respond to any of these points soon?
- 22:29:43 <sumanah> (since he proposed Composer-y stuff)
- 22:30:11 <Dantman> It's also a dead end if you can't do shell execution from php. (Don't we still sort of support that?)
- 22:30:12 <brion> sounds like we’ve got a lot of questions and ideas to queue up for that :D
- 22:30:30 <sumanah> brion: help me summarize for the notes?
- 22:30:36 <Dantman> Side note on composer itself.
- 22:30:44 <parent5446> You wouldn't need shell access to use composer from PHP
- 22:31:00 <Dantman> Hmmm... right.
- 22:31:33 <sumanah> #info <bd808> The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster). The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs.
- 22:31:44 <Dantman> Composer is similar to other package managers that install things locally.
- 22:31:50 <sumanah> #info <parent5446> Technically it should be trivial to convert most extensions to a composer library
- 22:31:52 <Dantman> Like npm and bower.
- 22:32:37 <Dantman> However it's unique (in a flawed way) at being the only one that requires that every installed package be present inside the .json file.
- 22:32:48 <sumanah> #info Wikidata team has some Composer experience per aude's explanation
- 22:33:32 <Dantman> Both bower and npm can install packages locally without adding them to the .json file you commit. (If you want to add it you use --save)
- 22:33:36 <TimStarling> how would registration be handled?
- 22:33:38 <sumanah> #info discussion of whether to deal with multiple packaging systems or just one, use something like https://github.com/derrabus/composer-webui for a web UI, CLI vs web (doing web UI securely is ?)
- 22:33:45 <bd808> You can add arbitrary libraries using Composer at the command line, but it writes them into the lock file
- 22:34:06 <legoktm> [15:28:39] <Dantman> Most of the important metadata is still in PHP. So you can't read it without first fetching and installing the extension. <-- Yeah, I was thinking about that as well. I had an idea that we could just put all the extension registration stuff (hooks, classes, basically mw:User:Legoktm/extension_registration) into a JSON file, so anything can read it without actually installing the extension
- 22:34:21 <bd808> Which is very similar to using a require.txt file with pip in python or ruby's bundler
- 22:35:14 <TimStarling> would composer installation somehow register the extension? or would registration require a line to be added manually to LocalSettings.php?
- 22:35:32 <aude> TimStarling: can be done wither way
- 22:35:59 <aude> i prefer manually registration for prodution
- 22:35:59 <TimStarling> that may be a question for discussion on the RFC page then
- 22:36:20 <parent5446> TimStarling: If you set the autoloader to include the main file, it would not need lines added to LocalSettings
- 22:36:25 <aude> so we can enable something for one wiki but not an other (e.g.)
- 22:36:35 <Dantman> My point is, if it weren't for that flaw in composer. It would have never been necessary to delete the composer.json that we used to have in core.
- 22:36:36 <aude> like we do w/o composer
- 22:36:45 <bd808> A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md
- 22:37:11 <brion> #idea question for composer-for-extensions: best way to fit in with our source-control-based deployment?
- 22:37:11 <brion> #idea question for composer-for-extensions: web ui possible for small installations? (compare with wordpress’s installer & extension manager) avoid chmod 777
- 22:37:11 <brion> i hope i’m using the right magic tags
- 22:37:12 <brion> #idea question for composer-for-extensions: avoid proliferation of package managers, but make sure we can still install extensions manually if need be or for migrations
- 22:37:12 <brion> brb
- 22:37:38 <sumanah> brion: yeah, you are :-) https://wiki.debian.org/MeetBot #idea works
- 22:37:46 <brion> ok ...
- 22:37:53 <brion> did we get the key points?
- 22:38:05 <sumanah> also #info #action #help #link
- 22:38:11 <TimStarling> maybe we should have our own admin console for registration
- 22:38:21 <brion> whoa colloquy just dropped the last 5 minutes on me in a batch :D
- 22:38:37 <TimStarling> and it would detect extensions installed by composer for the purposes of displaying the list
- 22:38:53 <bd808> Dantman: Part of the problem is that core is both a library and an application. Composer doesn't grok that.
- 22:38:55 <TimStarling> that way you can disable them, but it is still easy to use
- 22:38:57 <brion> TimStarling: i like that
- 22:39:08 <brion> composer for physical installation, but we have a separate activation as we do now
- 22:39:19 <brion> this works well for multi-site farms as well as for shipping defaults etc
- 22:40:36 <TimStarling> you could pull the list of enabled extensions out of the DB or some other web-writable place
- 22:40:46 * sumanah invited folks from Wikia, ShoutWiki, & wikiHow to come talk about this; will try to get them to talk on the talkpage
- 22:40:59 <sumanah> #info <bd808> A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md
- 22:41:01 <TimStarling> iterate, invoke some autoloaded static function for registration
- 22:42:16 <sumanah> #action sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page
- 22:42:21 <aude> so, i guess... with wikidata, we disabled autoloading of wikibase client and repo by default
- 22:42:39 <aude> those are then loaded via CommonSettings based on config
- 22:42:54 <TimStarling> should we talk about parent5446'
- 22:42:55 <aude> once one of those is loaded, all the dependencies are autoloaded
- 22:42:59 <TimStarling> should we talk about parent5446's RFC now?
- 22:43:01 <sumanah> #action sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate
- 22:43:06 <sumanah> I think so TimStarling
I don't think this should be done until the Config revamp is finished. I'm not the first to say that either, according to this ;)
What has the "Config revamp" to do with the this RFC? (the link you are referring to as source doesn't contain any information relevant to this RFC).
I think the Extension manager should be merged with the configuration panel of the extension. Dunno.
The wikitech-l archives got broken a while back, the email I'm referring to is .
It might indeed be nice to have a UI that has stuff for both tasks. However, this RFC is not about the UI of extension management at all. I see not reason why this RFC should be blocked on it.
I didn't see this mentioned in the RfC, so: is the plan to have a hard dependency upon composer? And if yes, why?
Well, I'm not sure I understand your question as to what you are referring to as "hard dependency upon composer" but I think the RFC itself made it clear that it is about "Extension management with Composer" which means that the Composer is an integral part of the RFC.
Let me rephrase my question: Will composer be required to install extensions?
+1 to this question. I will vehemently oppose this RFC if composer is a requirement/prerequisite to installing extensions, as we cannot guarantee that everyone has it (especially on shared hosts where the end user has incredibly limited access, and no, telling them to host elsewhere is not a viable option especially if our software currently works on their host without composer).
If composer is an optional (but recommended) component that makes extension management easier, then I'm looking forward to see how this develops, but only then.
I might have misread the RFC but it states clearly as "Extension management with Composer" not without.
Also, I far as I understand, this RFC is not about an obligation (or "hard dependency") but rather a general proposal as to how dependencies can be specified and integrate within MediaWiki.
If you think that you have a better way of defining dependencies that do not require Composer, I would suggest to open a similar RFC to this one that would allow to make dependency management within a MediaWiki environment "hassle free" because until now (after 10+ years development) MW has not been able to provide an viable solution for extension or extension developers.
It is not inteded that Composer be a hard requirement. It will be on top of the existing way of installing extensions. So if you want do it manually, you still can and the process will not get any more complicated. But if the RfC is implemented, you can take the alternative route via the extension installer.
Also, I'd like to see the architecture of the extension manater built in such a way that it's possible to change the mechanism that handles dependency management. So it should be possible to adapt the extension manager to a different "engine".
However, I think we should agree on one engine (hence Composer) and not plan for a whole bunch of solutions intially, to keep things within a reasonable scope and timeframe.
I would agree that Composer is a sane choice here -- it's in use in quite a few other places (user familiarity), it's developed by a dedicated team (well tested and supported), and it should be pretty easy to build additional UI on top of it (for whenever we get around to an Extension manager). Rolling our own dependency manager "just cause" is something that should be avoided when good FOSS alternatives already exist.
We also need to establish how an extension gets from our git to the repo. One idea is to use git tags as a basis.
I do not understand the first sentence. "the repo"? What is that?
Regardless of if Packagist or a custom package repository is used, Composer works with tags to indicate versions. This has already been standardized and adopted by many projects. For more details, see the Composer docs on versioning packages.
I rephrased the RfC to make that somewhat clearer.
No global scope: all global variables need to be declared first. Jeroen proposed to use $GLOBALS['wgNAME'].
This is oddly phrased. Global scope assumptions need to be removed. How one does this is really not of much consequence for composer usage, hence the second sentence serves little purpose. I outlined this in more detail in this blog post.
Ok, I rephrased it accordingly in the RfC.
Both James and I have poked at some UI stuff for managing extensions. See this email thread.
There is already a first draft of an extension manager: [4]
The Extension Installer is meant to provide people using MW < 1.22 with a way to install extensions via Composer. It is not meant to provide a GUI, and any UI tool created would be quite disjoint from what this extension is doing.
Thanks for clarification. I took out that reference in the RfC.
There are no older topics