Architecture meetings/RFC review 2014-07-23
22:00-23:00 UTC on Wednesday 23 July 2014, at #wikimedia-office connect.
Requests for Comment to review
edit- Requests for comment/Composer managed libraries for use on WMF cluster. We approved this RfC in this meeting.
Summary and logs
edit- LINK: https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-23 (sumanah, 22:00:27)
- Future of RFC/architecture stuff (sumanah, 22:00:38)
- I'm going on vacation in mid-August, and am going to be stepping away from running these meetings and from coordinating the process overall - including after I come back from vacation. I'm gonna work with Rachel Farrand, Quim Gil, and the architecture committee to make sure stuff still happens. (sumanah, 22:00:50)
- Next week we'll be talking about Recent Changes & rcstream, https://www.mediawiki.org/wiki/Requests_for_comment/Publishing_the_RecentChanges_feed , PubSubHubbub vs WebSockets vs rcstream, and so on. (sumanah, 22:01:07)
- thanks to sumanah for lots of great work on the process so far! (brion, 22:01:22)
- Composer managed libraries for use on WMF cluster (sumanah, 22:02:06)
- Today we're talking about revamping how we manage some libraries on the WMF cluster, and using Composer. (sumanah, 22:02:15)
- LINK: https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster (sumanah, 22:02:20)
- LINK: http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077619.html - Bryan Davis's note to wikitech-l (sumanah, 22:02:39)
- LINK: https://git.wikimedia.org/summary/mediawiki%2Fcore%2Fvendor (aude, 22:16:07)
- LINK: https://git.wikimedia.org/tree/mediawiki%2Fcore%2Fvendor (aude, 22:16:14)
- AGREED: general agreement that this approach makes much more sense than running composer on deployment nodes :) (brion, 22:20:30)
- some uncertainty about mediawiki/vendor vs mediawiki/core/vendor (brion, 22:20:51)
- ACTION: bd808 to get repo renamed from mw/core/vendor to mw/vendor (bd808, 22:23:20)
- LINK: https://security.sensiolabs.org/check or such might help? (aude, 22:30:33)
- LINK: https://igor.io/2013/01/07/composer-versioning.html Some discussion of how composer versioning (sumanah, 22:46:10)
- <andrewbogott> I just hope we have some plan to avoid accidentally forking forever (sumanah, 22:48:50)
- ACTION: Look into https://security.sensiolabs.org/check and https://github.com/sensiolabs/security-advisories for vulnerability tracking (bd808, 22:53:56)
- ACTION: csteipp or similar person to do security update planning with Ops, Bryan, and other foresaken souls (sumanah, 22:54:45)
- AGREED: general acceptance of the basic plan (brion, 22:54:52)
- rename tracked in https://bugzilla.wikimedia.org/show_bug.cgi?id=68485 (bd808, 22:57:41)
- ACTION: RfC accepted, with future actions to improve the security response plan (brion, 22:58:52)
- AGREED: RFC accepted! (sumanah, 22:59:01)
Meeting ended at 23:00:00 UTC.
Action items
edit- bd808 to get repo renamed from mw/core/vendor to mw/vendor
- Look into https://security.sensiolabs.org/check and https://github.com/sensiolabs/security-advisories for vulnerability tracking
- csteipp or similar person to do security update planning with Ops, Bryan, and other foresaken souls
- RfC accepted, with future actions to improve the security response plan
Action items, by person
edit- bd808
- bd808 to get repo renamed from mw/core/vendor to mw/vendor
- **UNASSIGNED**
- Look into https://security.sensiolabs.org/check and https://github.com/sensiolabs/security-advisories for vulnerability tracking
- csteipp or similar person to do security update planning with Ops, Bryan, and other foresaken souls
- RfC accepted, with future actions to improve the security response plan
People present (lines said)
edit- bd808 (68)
- sumanah (47)
- aude (40)
- brion (38)
- TimStarling (32)
- andrewbogott (22)
- hoo (13)
- mwalker (11)
- RoanKattouw (9)
- robla (8)
- legoktm (6)
- wm-labs-meetbot (5)
- marktraceur (2)
- MaxSem (2)
- Krenair (2)
- YuviPanda (1)
- awjr (1)
- mark (0)
Generated by MeetBot 0.1.4 (http://wiki.debian.org/MeetBot)
Full log
editMeeting logs |
---|
::22:00:08 <sumanah> #startmeeting Composer-managed libraries on WMF cluster | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours ::22:00:08 <wm-labs-meetbot> Meeting started Wed Jul 23 22:00:08 2014 UTC and is due to finish in 60 minutes. The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot. ::22:00:08 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. ::22:00:08 <wm-labs-meetbot> The meeting name has been set to 'composer_managed_libraries_on_wmf_cluster___channel_is_logged_and_publicly_posted__do_not_remove_this_note___https___meta_wikimedia_org_wiki_irc_office_hours' ::22:00:12 <sumanah> #chair sumanah brion TimStarling ::22:00:12 <wm-labs-meetbot> Current chairs: TimStarling brion sumanah ::22:00:20 * brion waves ::22:00:22 <sumanah> #chair sumanah brion TimStarling mark ::22:00:22 <wm-labs-meetbot> Current chairs: TimStarling brion mark sumanah ::22:00:27 <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-23 ::22:00:31 * bd808 waves to the crowd ::22:00:33 <sumanah> First off - administrative note. ::22:00:38 <sumanah> #topic Future of RFC/architecture stuff ::22:00:40 * aude wave ::22:00:41 * awjr waves ::22:00:43 * legoktm waves ::22:00:49 * mwalker waves ::22:00:50 <sumanah> #info I'm going on vacation in mid-August, and am going to be stepping away from running these meetings and from coordinating the process overall - including after I come back from vacation. I'm gonna work with Rachel Farrand, Quim Gil, and the architecture committee to make sure stuff still happens. ::22:01:07 <sumanah> #info Next week we'll be talking about Recent Changes & rcstream, https://www.mediawiki.org/wiki/Requests_for_comment/Publishing_the_RecentChanges_feed , PubSubHubbub vs WebSockets vs rcstream, and so on. ::22:01:10 <aude> sumanah: enjoy! :) ::22:01:13 <sumanah> Thanks :) ::22:01:22 <brion> #info thanks to sumanah for lots of great work on the process so far! ::22:01:33 <bd808> +1 ::22:01:36 <sumanah> so legoktm Krinkle|detached Lydia_WMDE you want to maybe reply on wikitech-l to help make that discussion happen :) ::22:01:43 * legoktm nods ::22:01:59 <sumanah> Thanks brion & bd808 :) ::22:02:06 <sumanah> ok, now on with today's programme ::22:02:06 <sumanah> #topic Composer managed libraries for use on WMF cluster ::22:02:15 <sumanah> #info Today we're talking about revamping how we manage some libraries on the WMF cluster, and using Composer. ::22:02:20 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster ::22:02:31 <sumanah> Bryan bd808, perhaps you would like to start by giving an overview of how the RFC has evolved in the last 2 weeks? ::22:02:37 <sumanah> or of what you'd like today ::22:02:39 <sumanah> #link http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077619.html - Bryan Davis's note to wikitech-l ::22:02:46 <bd808> Thanks sumanah ::22:02:50 <bd808> I'm looking for a general consensus that managing 3rd party libraries with Composer is sane. The RFP is really about a procedure for managing those dependencies in a way that will work for the WMF beta and prod clusters, but if Composer is a crap idea then it's moot. ::22:03:01 <bd808> On the proposed procedures, I'd like input on possible use cases that aren't covered. ::22:03:07 <bd808> I'd also like help filling in sketchy bits like how we will track upstream security issues and ensure that they are resolved. ::22:03:26 <brion> i feel like it’s a more general solution than pulling things in with git submodules / svn externals like the olden days ::22:03:37 <aude> generally looks sane to me ::22:04:00 <hoo> +1 ::22:04:08 <TimStarling> it mostly seems pretty similar to what I was supporting on wikitech-l, except for one minor detail ::22:04:17 <bd808> I hoped aude would like it. I think it's a lot like what wikidata is doing ::22:04:20 <brion> bd808: my main concern is whether this is going to be wmf-deployment-specific or whether we’d expect similar usage outside of wmf? ::22:04:20 <hoo> aude or I can give an overview of how the Wikidata team uses composer for WMF production ::22:04:22 <TimStarling> which is mediawiki/core/vendor versus mediawiki/vendor for the gerrit project name ::22:04:48 <legoktm> brion: the repository is set up so if people don't want to use composer, they can clone our vendor repo and use it in place ::22:05:07 <bd808> TimStarling: I think changing the repo name is trivial. I can ask to have that done. ::22:05:17 <brion> nice ::22:05:19 <legoktm> so it's wmf-deployment specific, but can easily be reused ::22:05:36 <TimStarling> I think that for developers, it is nice to be able to check out the whole gerrit hierarchy even if you want to use composer directly ::22:05:45 <andrewbogott> bd808: Sorry if this is already expressed in the RFC. Are you talking about having one repository per composer-managed extension? Or one big repository that contains the current state of all extensions and dependencies? ::22:06:00 <TimStarling> even if we recommend that all developers use the submodules, there'll still be a need for testing composer ::22:06:08 <bd808> andrewbogott: One big pile of the things that are blessed for prod deploy ::22:06:28 <brion> what’s the alternative for getting those deps? manual installation? or git submodules? ::22:06:31 <andrewbogott> So if I need special extensions for wikitech, this doesn't help me, huh? ::22:06:32 <bd808> andrewbogott: But the upstream development will be from other repos ::22:06:49 <andrewbogott> Or would the wikitech extensions be rolled into the repo but just not pulled in for normal prod? ::22:07:07 <bd808> andrewbogott: You are asking because of SMW? ::22:07:11 <MaxSem> for third parties, should update.php also update composer dependencies? ::22:07:47 * aude also notes that vagrant now supports composer ::22:07:51 <TimStarling> andrewbogott: it's not for extensions, it's for core dependencies ::22:08:04 <aude> and think there is progress for jenkins, but don't know the details ::22:08:13 <andrewbogott> yes, SMW ::22:08:19 <aude> (the vagrant support goes a long way for us providing a wikibase role) ::22:08:22 <bd808> brion: individual submodules could be done or something like PEAR where the code lands in the search path somehow as an alternative. ::22:08:24 <aude> composer* ::22:08:32 <TimStarling> extensions can implement an analogous mechanism ::22:08:48 <bd808> brion: But I think both of those options have issues. Versioning across branches is one. ::22:08:51 <brion> *nod* i’ve just had bad experiences with hidden dependencies in extensions requiring, say, a PEAR installation in include_path ::22:09:41 <andrewbogott> Uhoh, I maybe don't understand the difference between 'extension' and 'core dependency'. ::22:09:52 <bd808> TimStarling: andrewbogott is asking because installation via Composer has become the (only?) way to install SMW ::22:10:06 <sumanah> oh! ::22:10:10 <sumanah> I didn't realize that ::22:10:12 <hoo> same goes for Wikibase, btw ::22:10:55 <TimStarling> wikibase has mostly moved to github, which seems like something that we maybe should have discussed more broadly at the time ::22:10:57 <andrewbogott> Of course I could follow your pattern but manage my own repo that pulls in wikitech dependencies. ::22:11:20 <brion> imo extensions should have their own dependency trees probably? hmm ::22:11:32 <hoo> TimStarling: That's not the point... even when more stuff was in gerrit, we started to not really support non-Composer installs ::22:11:32 <brion> ugh that gets ugly fast though with shared deps ::22:11:47 <bd808> andrewbogott: We should really talk more about the needs for wikitech. Maybe start an email thread somewhere on it. ::22:11:51 <hoo> if the number of components is to high, manually managing them isn't an option anymore ::22:12:07 <RoanKattouw> [15:10] TimStarling wikibase has mostly moved to github, which seems like something that we maybe should have discussed more broadly at the time ::22:12:15 <andrewbogott> bd808: It's the same problem as the one you're talking about, right? Just with a different set of dependencies? ::22:12:19 <RoanKattouw> Ahm, don't we have like a /policy/ of how all deployed code is *required* to be mastered in Gerrit? ::22:12:33 <bd808> andrewbogott: yes. I think it is. ::22:12:37 <hoo> RoanKattouw: No ::22:12:40 <TimStarling> RoanKattouw: well, we deploy wikibase, so I'm not sure how that can be ::22:12:45 <RoanKattouw> (Not exactly on-topic for Composer, I realize) ::22:12:52 <aude> RoanKattouw: we make a build that is on gerrit ::22:12:53 <hoo> let's get back to composer, yeah ::22:13:03 <sumanah> RoanKattouw: feel free to bring that up onlist later? ::22:13:08 <hoo> point is: We use to composer to pull everything together ::22:13:12 <sumanah> good question though imo ::22:13:16 <hoo> and then we push that to gerrit and deploy it ::22:13:19 <hoo> in a nutshell ::22:13:25 <bd808> We keep the code that goes into prod in gerrit. But how it gets there is an open question I think. ::22:13:28 <aude> the build is deployed, similar to what bd808 proposes for the core libraries ::22:13:34 <sumanah> MatmaRex: AaronSchulz http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140723.txt in case you want to catch up. ::22:13:38 * aude doesn't want to derail... ::22:13:40 <bd808> And I'm asking to bring in libraries that are "not invented here" ::22:13:41 <brion> *nod* it all makes some kind of sense :) ::22:14:23 <TimStarling> does anyone other than me have an opinion on mediawiki/vendor versus mediawiki/core/vendor? ::22:14:42 <brion> i have no opinion on the repository naming :) ::22:15:04 * aude no opinion ::22:15:11 <RoanKattouw> What is the vendor repo? ::22:15:15 <TimStarling> because IIRC there were no opinions given on wikitech-l, and no comments here other than bd808 "easy enough to implement" ::22:15:16 <RoanKattouw> The RFC page doesn't seem to explain this ::22:15:18 <legoktm> if it only has core dependencies, I think "mediawiki/core/vendor" makes more sense, but I don't really care tbh ::22:15:21 <sumanah> I am mildly in favor of concision. Like, super mildly ::22:15:30 <sumanah> as in, less typing ::22:15:37 <brion> RoanKattouw: that’s the git repo that contains the stuff that composer checked out, basically ::22:15:41 <aude> RoanKattouw: at teh moment, just stuff for logging ::22:15:43 <hoo> I see it like legoktm ::22:15:44 <aude> monolog, psr ::22:16:01 <RoanKattouw> Oh it's where all the 3rd party stuff we import lives? ::22:16:07 <aude> https://git.wikimedia.org/summary/mediawiki%2Fcore%2Fvendor ::22:16:10 <brion> right ::22:16:14 <aude> https://git.wikimedia.org/tree/mediawiki%2Fcore%2Fvendor ::22:16:15 <RoanKattouw> Or rather, the composer infra for pulling it in ::22:16:21 <brion> then in production, we only have to check out that repo instead of running composer on every node or something ::22:16:32 <hoo> brion: yes ::22:16:33 <aude> brion: exactly ::22:16:36 <bd808> RoanKattouw: Yes. "vendor" is the name Composer uses by default for libraries it manages. ::22:16:47 <aude> composer really isn't a thing for deployment ::22:16:58 <aude> just to package / assemble the dependencies ::22:17:18 <aude> not a dpeloyment tool ::22:17:19 <hoo> Yeah, we use it to pull stuff together, then we "freeze" that into a git repo which can be deployed just like any other ::22:17:21 <aude> bah ::22:17:32 <bd808> brion: We are actually already doing it (deploying the vendor directory to prod) ::22:17:34 <MaxSem> aude, you can always dsh composer update :P ::22:17:41 <brion> excellent ::22:18:26 <bd808> It is branched in the make-branch script and added as a submodule to the wmf branch just like an extension or skin ::22:19:30 <RoanKattouw> Yeah ::22:19:34 <brion> so any open issues remaining with this rfc? ::22:19:49 <RoanKattouw> I'd mildly favor mw/core/vendor over mw/vendor because of the way it's submodule-d in, but I have no strong opinion either ::22:20:00 <mwalker> we should talk about making sure we're up to date ::22:20:02 <sumanah> TimStarling: brion: can you summarize with #info or #agreed on what we've decided here? ::22:20:30 <brion> #agreed general agreement that this approach makes much more sense than running composer on deployment nodes :) ::22:20:36 <TimStarling> RoanKattouw: fair point ::22:20:37 <bd808> mwalker: Yes. Especially for security fixes ::22:20:51 <brion> #info some uncertainty about mediawiki/vendor vs mediawiki/core/vendor ::22:21:05 <TimStarling> is there definitely a submodule in master? the RFC was unclear about this ::22:21:31 <TimStarling> it said that there is a submodule in the deployment branch ::22:21:32 <legoktm> there's no submodule in master, just in wmf deployment branches ::22:21:36 <bd808> The submodule is not in master. We only add it to the 1.24wmfX branches ::22:21:41 * YuviPanda wonders about git subtrees instead of submodules ::22:22:12 <TimStarling> right ::22:22:20 <bd808> Adding the submodule to master would really be stepping on the toes of anyone using Composer independently. ::22:22:20 <TimStarling> I take it back then, unfair point ;) ::22:22:41 <bd808> TimStarling: I'll do the leg work to rename. It's easy enough. ::22:22:49 <TimStarling> thanks bd808 ::22:23:20 <bd808> #action bd808 to get repo renamed from mw/core/vendor to mw/vendor ::22:23:38 <bd808> sumanah: was that how I log that sort of thing? ::22:23:51 <sumanah> yep! ::22:24:20 <bd808> So ... tracking security updates? ::22:24:32 <bd808> How do we do that in general? ::22:24:49 <brion> apt-get update? :) ::22:24:55 <bd808> Basically each 3rd party lib we add could have vulnerabilities ::22:25:15 <brion> is that core’s problem? or ops’ problem? to watch for those… ::22:25:18 <TimStarling> we often have bugs in our bugzilla that are proxies for upstream bugs ::22:25:50 <bd808> brion: csteipp wanted just to ensure that it wasn't his problem :) ::22:25:55 <brion> :D ::22:26:25 <bd808> One idea he and I talked about was that each library should have a "sponsor" ::22:26:38 <brion> *nod* ::22:26:44 <bd808> But keeping up with this over time is obviously problematic ::22:26:55 <bd808> We already have extensions that want for maintainers ::22:27:01 <Krenair> We actually have a keyword for bugs in our bugzilla which are just tracking upstream ::22:27:09 <Krenair> Except people keep using it for things that haven't been reported upstream yet ::22:27:13 <TimStarling> we are talking about the work of watching vendor release notes for security updates? ::22:27:13 <bd808> And the platform core team can only take on so much ::22:27:49 <bd808> TimStarling: I think so. And at least poking folks to update our copies ::22:28:21 <mwalker> there are deeper issues too -- because dependencies of dependencies might have updates that are not reflected in the release notes; and what do we do about dependencies that go unmaintained ::22:28:28 <TimStarling> I don't think it should be spread over too many people ::22:29:14 <bd808> mwalker: True enough. We would need to track all the way down for things we are using. ::22:29:16 <TimStarling> we'll lose track and updates will fall through the cracks ::22:29:28 <TimStarling> unless there is quite a heavyweight process on top of it ::22:29:47 <TimStarling> I mean, I'm sure things fall through the cracks at the moment ::22:29:53 <sumanah> Oh hi lilatretikov! ::22:30:08 <mwalker> there are automated services for nodejs that will look at your repo and see if recursively everything is up to date -- I just searched and didn't find anything for composer but my googlefoo has been failing me today ::22:30:11 <sumanah> lilatretikov: we're discussing https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster ::22:30:33 <aude> https://security.sensiolabs.org/check or such might help? ::22:30:42 <aude> there must be a way to help automate this ::22:30:50 <marktraceur> sumanah: We're just getting her set up with IRC now :) ::22:30:53 <bd808> aude: Nice! ::22:30:57 <sumanah> nice :) ::22:31:19 <andrewbogott> So… surely there's a top-level composer call that will just update every dependency and every dependency's dependency, etc. right? ::22:31:25 <TimStarling> aude: do you know if they compile that vulnerability list themselves? or is there a standard way to report security vulnerabilities? ::22:31:29 <andrewbogott> Which would make us one big patch for our repo with every change? ::22:31:31 <bd808> Maybe we could have a jenkins job to check that daily/weekly and email "someone" ::22:31:41 <aude> TimStarling: i would have to look into this more to say for sure ::22:31:46 <mwalker> andrewbogott, yep -- that exists ::22:31:57 <bd808> andrewbogott: Yes. -- https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster#Adding.2Fupdating_a_library ::22:32:10 <TimStarling> oh, it comes from https://github.com/sensiolabs/security-advisories ::22:32:13 <andrewbogott> So, is the current proposal to /not/ do that, but instead only selectively update libraries as needed? ::22:32:22 <TimStarling> which they apparently compile ::22:32:26 <TimStarling> nice that it is public though ::22:32:33 <mwalker> I think the problem is that we dont trust it -- but it's gotten to the point in the pdf renderer that I basically just have to trust it (I have 1M lines of code from dependencies, I cant look at them all) ::22:32:35 <TimStarling> so we can build our own tool on top of that git repository ::22:32:57 <aude> should be possible ::22:33:36 <TimStarling> mwalker: if you can't review then you should sandbox ::22:34:14 <mwalker> *nods* -- as much as possible, I have an apparmor profile that's a stand in for that ::22:35:01 <bd808> andrewbogott: The proposal is just about how to prepare the "big commit" mostly ::22:35:33 <bd808> We don't want to do it daily/weekly, we just want to pull in new and/or updated things when we need them. ::22:35:43 * andrewbogott nods ::22:36:06 <bd808> And we want to version "this lib went with this deploy branch" ::22:36:34 <andrewbogott> If when we do need an update we're pulling in everything, then... ::22:36:55 <andrewbogott> well, that sounds to me like an argument for frequent updates. Because otherwise a serious emergency bug will force us to review an epic patch in a rush ::22:37:11 <bd808> Composer supports "pinning" verisons so we can know what we are getting. ::22:37:22 <andrewbogott> commit message: "Fix <x> bug and also pull in 4 month's worth of associated updates in a million packages" ::22:37:23 <brion> ah that should help ::22:38:02 <aude> regular updates seems good to me ::22:38:07 <bd808> I'm currently being very specific -- https://github.com/wikimedia/mediawiki-core-vendor/blob/master/composer.json ::22:38:22 <bd808> And I would suggest we continue that pattern ::22:38:26 <aude> there is also the potential for some incompatibility between some version of a library and core ::22:38:53 <aude> easier to deal with / minimize that when we make smaller, more frequent updates ::22:38:53 <bd808> Yes. It's the class "DLL hell" problem ::22:39:06 <bd808> *classic ::22:39:49 <bd808> But nobody wants to update a library like monolog every 2-4 weeks. Maybe quarterly? ::22:39:56 <aude> maybe ::22:40:48 <aude> the more stable the library is, less often is ok ::22:40:50 <bd808> When we use Ubuntu LTS releases we basically commit to only getting major updates once every 2 years ::22:40:59 <aude> and hope we use only nice stable things, preferably ::22:41:19 <bd808> The less stable the library is the less I want to see it used for core, but that's another debate. ::22:41:32 <aude> yep ::22:42:00 <aude> exception might be our own stuff (e.g. the oojs stuff) ::22:42:02 * bd808 excludes his own code from the stability debate :) ::22:42:12 <aude> which is updated all the time ::22:42:19 <andrewbogott> If we're committed to blind-merging these big composer patches then this isn't a big deal. It's only the prospect of having to actually read them that would argue for frequent tiny updates. ::22:42:21 <aude> outside of composer ::22:42:41 <andrewbogott> Well, and also general CI values :) ::22:42:42 <robla> we could queue up updates in gerrit automatically as if it were pushed. it might be healthy to have gerrit implicitly nagging us to upgrade as often as the upstream changes. ::22:43:06 <mwalker> ^ +1 to robla ::22:43:51 * aude looks at the diff for wikidata builds but also the tests are quite important ::22:43:53 <bd808> I'd be ok with that within a major version. API bumps are a different story ::22:44:03 <aude> particular attention to the composer lock file ::22:44:22 <bd808> We don't deploy the latest jquery regularly do we? ::22:44:23 <mwalker> bd808, does composor support semantic versioning? (e.g. I want to only update within the 4.x series?) ::22:44:32 <bd808> mwalker: Yes it demands it ::22:45:10 <robla> bd808: nope, we don't, though we used to, and I was the one of the main detractors of frequent updates there...so, I'll fess up to being wildly inconsistent there :-) ::22:45:37 <robla> I wouldn't suggest that we automatically +2 upstream updates ::22:45:53 <bd808> mwalker: Some discussion of how composer versioning works at -- https://igor.io/2013/01/07/composer-versioning.html ::22:46:10 <sumanah> #link https://igor.io/2013/01/07/composer-versioning.html Some discussion of how composer versioning ::22:46:22 <bd808> robla: We just need to write a security review bot :) ::22:46:58 <sumanah> oh dear ::22:47:09 <sumanah> Chris is away for ONE DAY and you start replacing him ::22:47:12 <andrewbogott> If we don't automatically +2 upstream versions, then I almost don't understand how this will work. What happens if we review upstream changes and find them wanting? Do we need a system to declare 'upstream freeze' and 'upstream unfreeze' ::22:47:54 <aude> andrewbogott: it's as much also about ensuring core is updated to be compatible with the changes ::22:47:56 <bd808> andrewbogott: We could fork or freeze I think -- https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster#Hotfix_for_an_external_library ::22:47:58 <robla> andrewbogott: basically....we discuss what to do on a case-by-case basis ::22:48:11 <aude> per robla about changes we don't want ::22:48:38 <andrewbogott> I just hope we have some plan to avoid accidentally forking forever ::22:48:50 <sumanah> #info <andrewbogott> I just hope we have some plan to avoid accidentally forking forever ::22:48:55 <sumanah> (I think that is important) ::22:48:56 <brion> andrewbogott: i think ‘get tired of forking and fix it’ may handle that ::22:49:09 <brion> but it’s all about coefficient of static friction :) ::22:49:13 <aude> or contribute upstream :) ::22:49:26 <aude> to fix wahtever issue ::22:49:40 <robla> if the upstream goes silly, we have a problem no matter what mechanism we choose ::22:49:45 <bd808> andrewbogott: You can sort of look at this as us being our own Debian/Ubuntu to gate php libraries into our environment. ::22:49:46 <andrewbogott> ok, so that's a reasonable answer. If the upstream is sketchy then we fork + scramble to to resolve the upstream problem so we can unfork. ::22:50:12 <TimStarling> you know we're finally unforking librsvg for trusty ::22:50:14 <robla> sometimes it's ok to live with the fork/freeze. it's kind of what we're doing with Lua, for example. ::22:50:40 <brion> oh librsvg. my old nemesis ::22:50:51 <TimStarling> after what, 6 or 8 years? ::22:50:57 <brion> :) ::22:51:08 <bd808> None of these issues are new to this method of importing libraries. ::22:51:21 <robla> bd808: agreed ::22:51:25 <bd808> What I'm trying to propose is just a method for the madness ::22:51:33 <brion> so do we need to work this out in more detail or are we pretty happy ith the notions so far? ::22:52:13 <bd808> I think security update tracking needs more work, but maybe it's not a blocker to the larger RFC? ::22:52:19 <andrewbogott> Yep, I didn't mean express an objection, just hoping for a roadmap when things go wrong. ::22:52:38 <bd808> andrewbogott: Sure. The questions are good. ::22:52:40 <brion> *nod* ::22:52:53 <sumanah> ok, we have 8 min left ::22:53:04 <sumanah> any wrapup #info or #agreed or #action items? ::22:53:41 <brion> should we push the security update plannning to a future action and agree on the rest? ::22:53:43 <brion> or anythign else left? ::22:53:56 <bd808> #action Look into https://security.sensiolabs.org/check and https://github.com/sensiolabs/security-advisories for vulnerability tracking ::22:54:02 <brion> i think we’re pretty agreed on everything else ::22:54:27 <TimStarling> all good ::22:54:45 <sumanah> #action csteipp or similar person to do security update planning with Ops, Bryan, and other foresaken souls ::22:54:52 <brion> #agreed general acceptance of the basic plan ::22:54:56 <sumanah> ok, that may have been over the top ::22:55:00 <brion> hehe ::22:55:42 <sumanah> mwalker: (as long as I have you - who is taking over https://www.mediawiki.org/wiki/Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy ?) ::22:56:09 <mwalker> unknown -- fundraising is getting a contractor to deal with centralnotice over the next couple of months ::22:56:14 <mwalker> and it is still on the services team roadmap ::22:56:37 <sumanah> ok, brion bd808 TimStarling mark anything else that needs deciding or marking here? ::22:56:44 <marktraceur> Heh, marking. ::22:56:46 <sumanah> bd808: and are you at least vaguely satisfied? ::22:57:11 <TimStarling> nope ::22:57:21 <bd808> sumanah: I'm ecstatic! ::22:57:34 <robla> \o/ ::22:57:39 <brion> :DD ::22:57:41 <bd808> #info rename tracked in https://bugzilla.wikimedia.org/show_bug.cgi?id=68485 ::22:57:55 <sumanah> bd808: I totally want an animated gif depicting your current state of joy from https://andtheniwaslike.co/ ::22:57:57 <brion> can we mark it accepted then? ::22:58:36 * andrewbogott is sad that this won't magically fix wikitech ::22:58:52 <brion> #action RfC accepted, with future actions to improve the security response plan ::22:58:53 <bd808> andrewbogott: We can work on that. wikitech needs love too ::22:59:01 <sumanah> #agreed RFC accepted! ::22:59:07 <brion> \o/ ::22:59:13 <sumanah> omg this is great ::22:59:16 <andrewbogott> bd808: yep, this will be useful anyway ::22:59:53 <sumanah> OK, this is the end of the meeting then! reminder: next week, rcstream, pubsubhubbub, websockets, et alia ::22:59:56 <bd808> achievement unlocked ::23:00:00 <sumanah> #endmeeting ::[23:00:52] <brion> https://www.mediawiki.org/w/index.php?title=Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster&diff=1075394&oldid=1075386 \o/ ::[23:01:06] <brion> well done everybody ::[23:01:11] <sumanah> brion: MARVELOUS. Thank you ::[23:01:12] <sumanah> thanks all ::[23:01:12] <brion> good points raised ::[23:01:29] <brion> :D ::[23:01:59] <brion> see y'all next time or yknow in all those other channels ;) |