[21:00:26] <sumanah> #startmeeting RFC review of MVC & structured logging proposals | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE)
[21:00:27] <wm-labs-meetbot`> Meeting started Wed Mar 19 21:00:26 2014 UTC and is due to finish in 60 minutes. The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot.
[21:00:27] <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
[21:00:27] <wm-labs-meetbot`> The meeting name has been set to 'rfc_review_of_mvc___structured_logging_proposals___channel_is_logged_and_publicly_posted__do_not_remove_this_note_'
[21:00:27] <sumanah> #chair brion TimStarling
[21:00:27] <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-03-19
[21:00:27] <wm-labs-meetbot`> Warning: Nick not in channel: brion
[21:00:28] <wm-labs-meetbot`> Current chairs: TimStarling brion sumanah
[21:00:42] <OwynD> hello
[21:00:46] <sumanah> Ah, there's OwynD!
[21:00:51] <sumanah> OK, OwynD you're first :)
[21:00:56] <sumanah> #topic MVC proposal
[21:00:56] <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/MVC_framework
[21:01:01] <OwynD> so... i was under the mistaken impression that this was an in-person meeting, so i'm at the foundation office. :/
[21:01:21] <OwynD> the office staff is very confused
[21:01:44] <sumanah> OwynD: I apologize! I am sorry for not explicitly specifying in my mention to you that it was IRC ONLY
[21:02:01] <TimStarling> #new-montgomery
[21:02:05] <sumanah> ha
[21:02:15] <bd808> Somebody should show OwynD the transporter pad in the closet that leads to Tim's house
[21:02:28] <sumanah> oh if only!
[21:02:28] <robla> OwynD: where exactly are you?
[21:02:41] <OwynD> sitting in the 6th floor lobby
[21:02:46] <sumanah> !link https://www.mediawiki.org/wiki/Talk:Requests_for_comment/MVC_framework
[21:03:04] <robla> OwynD: I'll come upstairs and get you :-)
[21:03:10] <OwynD> gabriel is here
[21:03:30] <OwynD> i'll come down to 3rd then we will start
[21:04:11] <sumanah> OK
[21:04:31] <sumanah> (I mentioned that it was IRC, but I didn't say IRC only)
[21:05:31] <gwicke> just arrived downstairs
[21:05:35] <OwynD> ok, let's begin then. what's the process?
[21:05:50] * aude waves�
[21:05:55] <sumanah> ok hi OwynD
[21:05:56] <TimStarling> still no brion?
[21:06:29] * rdwrer waves�
[21:06:29] <sumanah> OwynD: Well, I usually ask the RFC authors what they'd like out of today's discussion
[21:07:02] <sumanah> such as a decision on a particular subtopic, or an approval of the whole RfC
[21:08:17] <sumanah> OwynD: So I ask that of you :)
[21:08:45] <TimStarling> in terms of templating systems, nirvana seemed to be the odd one out, out of those presented at the architecture summit
[21:09:00] <TimStarling> it was coming from a very different design direction to the others
[21:09:03] <OwynD> well, the main difference that i see between the proposals is that one is a more specific solution for building special pages (nirvana)
[21:09:10] <sumanah> !link https://www.mediawiki.org/wiki/Architecture_Summit_2014/HTML_templating#Nirvana_Templates
[21:09:14] <OwynD> and the other is a much more general purpose templating solution that could be used elsewhere
[21:10:02] <OwynD> nirvana was also an attempt to isolate some of the wikia code from the core mediawiki code (ended up re-implemeting some stuff, like an ajax entry point) but the templating part of it can be separated.
[21:10:34] <gwicke> OwynD, where do you see the strengths of Nirvana vs. something like Handlebars or Knockoff?
[21:10:59] <OwynD> and the other goal with making the proposal was just for wikia to get more involved in the process as a way of learning about it, which i think has been achieved.
[21:11:24] <sumanah> #link http://www.gossamer-threads.com/lists/wiki/wikitech/442532 current wikitech-l thread - summary of "what are HTML templates systems anyway"
[21:11:28] <OwynD> well, one "advantage" is that it's PHP. :)
[21:11:44] <TimStarling> presumably it's fast
[21:11:50] <MaxSem> OwynD, we're looking for PHP and JS engine
[21:11:53] <gwicke> we should add it to https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#Performance
[21:12:24] <OwynD> the other advantage from my perspective was that it allowed (but doesn't really enforce) some separation of business logic from template/formatting logic.
[21:12:30] <gwicke> the LightnCandy Handlebars implementation (which compiles templates to PHP code) is currently the fastest PHP library in the tests
[21:12:37] <TimStarling> there's been a lot of talk about security properties of various template engines
[21:12:38] <robla> is Knockoff our cutesy name for our Knockout workalike, or is that just a persistent thinko?
[21:12:42] <OwynD> our custom skin was getting totally unweildy.
[21:12:51] <TimStarling> and I gather nirvana is on a par with QuickTemplate in that respect
[21:12:51] <OwynD> so, nirvana was designed as a framework for building our new Skin
[21:13:00] <gwicke> robla, https://github.com/gwicke/knockoff
[21:13:03] <TimStarling> i.e. it relies a lot on the template author being aware of security
[21:13:18] <OwynD> nirvana is basically QuickTemplates++ yes
[21:13:26] <gwicke> robla: so yes, cutesy
[21:13:36] <robla> :-)
[21:14:03] <sumanah> #action gwicke add Nirvana to add it to https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#Performance
[21:14:29] <OwynD> the framework also allowed us to develop new parts of the skin as extensions and "plug" them in really easily.
[21:14:34] <gwicke> heh, maybe I can get OwynD to add it to https://github.com/gwicke/TemplatePerf ?
[21:14:55] <OwynD> but in that sense it's not really a part of the framework, it's just a way of organizing the massive amounts of code / developers we have all working on the same codebase.
[21:15:06] <OwynD> oh sure, i can do a performance thing
[21:17:07] <TimStarling> I would like to know if anyone from WMF is in favour of the concept, compared to other proposals
[21:17:49] <OwynD> basically inez and I just didn't like that QuickTemplates are sort of an object at the top and a template at the bottom. :)
[21:17:50] <MaxSem> I see no benefit over our existing templates. and it's PHP-only
[21:17:54] <TimStarling> because I don't want to waste OwynD's time writing RFCs and doing benchmarks if it doesn't have any chance of actually being used
[21:18:08] <csteipp> Iirc, Trevor seemed interested in it at the arch summit
[21:18:13] <sumanah> TrevorParscal: ^
[21:18:31] <OwynD> he was interested in the fact that it sort of inverts the normal data flow in a template system.
[21:18:42] <OwynD> that is, the template asks for data
[21:19:03] <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-02-21 - the logs from our last RFC discussion of HTML templates stuff include some thoughts from Trevor
[21:19:10] <OwynD> and then a controller supplies that data (with the appropriate sub-template)
[21:19:25] <gwicke> I also like the pull aspect, but it is not completely unique to nirvana
[21:19:50] <TimStarling> the template engine I wrote back in like 2006 was pull based
[21:19:53] <gwicke> calling lambdas from the model is similar
[21:20:39] <TimStarling> but it had registered callbacks rather than actually having the template call PHP code
[21:20:45] <TimStarling> kind of lisp-like
[21:20:55] <gwicke> we were thinking about async pull, similar to parsoid
[21:21:32] <TimStarling> have you looked at HHVM's threading model at all?
[21:21:46] <sumanah> MaxSem: when you say "our existing templates" what specifically do you mean?
[21:21:56] <MaxSem> QuickTemplate
[21:22:01] <gwicke> I haven't, but that's fairly orthogonal to how you hook calls to it up and manage async returns
[21:22:03] <OwynD> existing templates are QuickTemplate and the HTML builder object
[21:22:19] <TimStarling> threads are completely isolated in terms of variables etc., like PHP's thread extension
[21:22:43] <gwicke> whether stuff is processed on a thread or a different machine does not really matter much
[21:22:44] <OwynD> we didn't like the HTML thing either, although gwike's proposal has the safety features that are part of that XML/HTML object approach
[21:23:15] <TimStarling> and it has string message passing, "pagelets" and RPC
[21:25:23] <sumanah> TimStarling: ok, so do you perhaps want us to ask about Nirvana interest among WMf engineers, mark that as an action item (prerequisite to any others), and move on to structured logging?
[21:25:55] <TimStarling> yes, ok
[21:26:07] * sumanah presumes there is probably face-to-face conversation happening at WMF office right now including OwynD & gwicke as well :-)�
[21:26:23] <OwynD> there's a bit of side chat. i will type in some comments.
[21:26:24] <sumanah> #action sumanah ask about Nirvana interest among WMf engineers (prerequisite to any other action items on this topic)
[21:26:25] <gwicke> yeah, sorry; he is sitting right next to me
[21:26:34] <aude> perhaps there are some aspects of it that we like
[21:26:47] <TimStarling> someone has to really want nirvana specifically if we are going to use that
[21:26:49] <gwicke> we were just talking about async processing
[21:27:10] <TimStarling> which I think is unlikely, although I haven't talked to Trevor
[21:27:28] <TimStarling> I think more likely is some kind of template engine recommended by WMF and integrated into the core
[21:27:31] <OwynD> so, i looked at doing async rendering, and the benefit wasn't worth the complexity of doing it in pure PHP, but HHVM makes that more feasible.
[21:27:32] * sumanah waits a minute for last comments on Nirvana/MVC/templates before changing topic. bd808 you're up in a min�
[21:27:42] <TimStarling> and perhaps that could be integrated with the non-template bits of nirvana
[21:27:48] <gwicke> basically my outcome from looking at async stuff was that the template interface does not need to change; the main change would be in the interface for lambdas and bindings
[21:27:56] <gwicke> they'd receive an extra cb parameter
[21:28:10] <sumanah> #info Knockoff is our Knockout workalike https://github.com/gwicke/knockoff
[21:28:23] <sumanah> #info in terms of security model, <OwynD> nirvana is basically QuickTemplates++ yes
[21:28:31] <gwicke> sync lambdas could just ignore that and directly return a non-void result
[21:29:45] <OwynD> so, i was just happy to have people take a look at and criticize Nirvana in the context of what the future needs of Mediawiki are...
[21:29:47] <sumanah> #info discussion of HHVM & async pull
[21:29:59] <OwynD> it's sufficient for what Wikia is doing, but we're not very ambitious in that department. :)
[21:30:00] <sumanah> OwynD: shall we continue this discussion on the mailing list then?
[21:30:16] <sumanah> OwynD: perhaps in the wikitech-l thread I started about what HTML templates are and what our choices are?
[21:30:17] <gwicke> (the async bit is basically a rough description of parsoid token processing)
[21:30:21] <OwynD> sure, i'll respond on that thread.
[21:30:39] <sumanah> #action Owyn will respond to http://www.gossamer-threads.com/lists/wiki/wikitech/442532
[21:30:49] <sumanah> ok!
[21:30:50] <sumanah> #topic Structured logging proposal
[21:30:50] <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging
[21:30:50] <sumanah> #info This is our second discussion of this, following up on https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2013-12-04
[21:31:05] <sumanah> (thanks OwynD!)
[21:31:13] <sumanah> #link https://gerrit.wikimedia.org/r/#/c/112699/ strawman implementation in Gerrit
[21:31:21] <sumanah> #info According to Bryan on March 14th http://www.gossamer-threads.com/lists/wiki/wikitech/441656 "At this point the most controversial aspect of my proposed implementation seems to be importing third-party libraries into mw-core and/or the use of composer to manage that activity. I would welcome discussion of alternatives or consensus that this is a reasonable approach for the immediate future that should be revisited if and when a better i
[21:31:21] <sumanah> dea is found for the general problem."
[21:31:35] <sumanah> er, "if and when a better idea is found for the general problem."
[21:31:44] <bd808> sumanah: You did all the work for me. :)
[21:31:45] <sumanah> #info according to Bryan today: "I just want Tim to say he'll +2 the patch :)"
[21:31:53] <sumanah> #info <bd808> But yes, I need to find the way that we can use the external libraries in core or be told that's not how we do things here.
[21:32:06] <sumanah> bd808: Happy to, when I can!
[21:32:50] <sumanah> csteipp: is Brion anywhere nearby? :/
[21:33:08] <bd808> So my first question is has anyone looked at the proposed interface and implementation and not commented in gerrit?
[21:33:09] <csteipp> sumanah: No, haven't seen him today
[21:33:28] * bd808 is hoping to find closet supporters�
[21:33:41] <TimStarling> who is saying you can't have external libraries?
[21:34:10] <ebernhardson> the comments on the patch, basically
[21:34:33] <gwicke> I haven't looked yet, but we have recently done something very similar in parsoid
[21:34:37] <bd808> Several people seemed to think that putting libraries in mw-core.git was gross
[21:34:53] <bd808> I tried to do it in as nice a way as I could
[21:35:04] <gwicke> the possibly interesting bit we have is hierarchical log levels, so we are doing stuff like warning/api
[21:35:10] <OwynD> wikia uses a simple logging framework based on monolog, which i believe is what this proposal is...
[21:35:12] <gwicke> backends can subscribe to those by regexp
[21:35:26] <bd808> There is a new libs/ directory that uses composer to manage the files so we can upgrade in the future
[21:35:46] <TimStarling> bd808: who specifically?
[21:36:14] <bd808> parent5446 and mwjames
[21:36:34] <sumanah> from a skim of https://gerrit.wikimedia.org/r/#/c/112699/ it looks like Jeroen had objections but they were resolved
[21:36:49] <gwicke> there is a side discussion on how wiki is also using monolog and logstash here
[21:36:50] <bd808> Jeroen had some concerns initially but I think … yes sumanah beat me to it
[21:36:53] <gwicke> *wikia
[21:36:58] <TimStarling> well, tyler removed his -1, but is still recommending a git submodule, right?
[21:37:29] <bd808> TimStarling: I think that's correct, yes
[21:38:05] <ebernhardson> dd
[21:38:09] <ebernhardson> tus
[21:38:15] <bd808> I really hate submodules, but I wouldn't make my dislike of them a blocker if that's the right way forward
[21:38:36] <sumanah> ebernhardson: wrong window? :-)
[21:38:52] * YuviPanda blames ebernhardson's cat.�
[21:38:53] <ebernhardson> sumanah: :( didn't disable trackpad
[21:38:54] <TimStarling> how do you upgrade the libs with composer exactly?
[21:39:29] <MaxSem> I too think that submodules in core are bad because it kills the existing "ready to work after 1 git pull" model
[21:40:00] <bd808> You would change the composer.json and then run `composer update` and it would pull the new version into your local working copy
[21:40:18] <TimStarling> I would rather not have any submodules in a production branch, except submodules which point to our own repos
[21:40:19] <bd808> Then it would be git add and commit to push as a gerrit patch
[21:40:23] <TimStarling> otherwise there are security implications
[21:40:54] <bd808> I used a similar model in building the scholarships application
[21:41:06] <TimStarling> even if you specify a version, you're still giving everyone with force push access to the remote repository the ability to own the server
[21:42:57] <OwynD> one alternative to git submodules is subtree merging.
[21:43:09] <gwicke> http://git-scm.com/book/ch6-7.html
[21:43:19] <OwynD> one of our developers was just looking at that as a way of tracking the VE repo from inside the wikia repo
[21:44:08] <OwynD> basically you pull the remote repo into a branch, and then check that branch out into a directory.
[21:44:09] <sumanah> hey ori. log so far: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140319.txt (currently talking about structured logging)
[21:44:11] <bd808> Is there anything in particular that gains you over just having the files in the same repo?
[21:44:34] <OwynD> well, the original repo is actually remote.
[21:44:47] <gwicke> it looks like the ability to pull in upstream changes is preserved
[21:44:48] <ori> sumanah: thanks very much for that
[21:44:56] <bd808> ah, sure
[21:44:57] <OwynD> yeah, exactly
[21:45:48] <bd808> My management with composer should allow the same (importing new upstream). It worked well in schaolarships.
[21:46:59] <bd808> I think I just got a good endorsement on the patch. :)
[21:47:29] <bd808> Are there other questions people have other than the library management aspets?
[21:47:34] <bd808> *aspects
[21:47:44] <gwicke> bd808, in parsoid we were able to support tracing with the prefix matching mechanism I mentioned earlier
[21:48:06] <gwicke> do you think that could be useful in PHP too?
[21:48:07] <bd808> gwicke: hierarchal loggers?
[21:48:35] <gwicke> that would require you to log to a specific logger at the source, wouldn't it?
[21:48:44] <OwynD> that RFC does a good job of summarizing the current state of logging, but then it contains this:
[21:48:45] <OwynD> FIXME: Propose a PHP API for logging and a migration plan for existing wfErrorLog() calls
[21:49:02] <gwicke> with the prefix matching you can start listening to an arbitrary regexp at any time
[21:49:27] <gwicke> per backend
[21:49:30] <bd808> OwynD: Yes. I need to update the wiki doc. The patch at https://gerrit.wikimedia.org/r/#/c/112699/ is the real stuff
[21:49:35] <OwynD> ok, cool
[21:50:01] <bd808> gwicke: So you are doing that with log levels?
[21:50:08] <OwynD> wikia is basically using the PSR functions (info,debug,warning etc) and API ($message, $context) with monolog as the backend
[21:50:10] <gwicke> yes, stuff like warning/api
[21:50:13] * bd808 tries to find the scrollback�
[21:50:42] <OwynD> it works for us, so that would be a +1 from our perspective
[21:50:45] <gwicke> that's matched by a warning backend, but you can also subscribe to (trace|warning)/api
[21:50:46] <bd808> Isn't that the same as warning level on the api channel?
[21:51:13] <gwicke> the level does not let you drill down to an area
[21:51:23] <gwicke> a flat level, that is
[21:51:53] <gwicke> our tracing output is very verbose, so tracing everything is not feasible
[21:52:04] <bd808> But channels do if they are used well, and your level suffix needs the same care
[21:52:04] <gwicke> so we always narrow it down to some area of the code
[21:52:20] <sumanah> #action update the onwiki RfC with details from https://gerrit.wikimedia.org/r/#/c/112699/
[21:53:17] <gwicke> bd808, so channels are what you log to in the code?
[21:53:25] <bd808> I'm not quickly seeing the difference between "show me level (trace|warning)/api" and "show me trace and warning messages on the api channel"
[21:53:43] <bd808> gwicke: Yes. A channel == a named logger
[21:53:50] <gwicke> we have warning/api/foo as well
[21:53:59] <gwicke> you can be arbitrarily precise
[21:54:48] <gwicke> handy if you just want to log timeout errors from api template expansions for example
[21:55:07] <bd808> That bit would be nice. monolog doesn't support logger hierarchies out of the box
[21:55:20] <gwicke> we implemented that in our internal interface only
[21:55:23] <sumanah> ok, we're running out of time -- any particular #info to note or next actions or #agreed decisions?
[21:55:25] <gwicke> monolog would just be the backend
[21:55:38] <gwicke> it doesn't need to know about the filtering
[21:55:41] <bd808> You could add other metadata that would surface the same thing via logstash searches
[21:56:06] <gwicke> you can't trace everything all the time
[21:56:37] <gwicke> besides generating a ton of output it also takes up a good amount of cpu time, at least in parsoid
[21:57:00] <bd808> gwicke: Ah. I get you point more now. You're interested in fine grained runtime control
[21:57:12] <gwicke> yup
[21:58:03] <bd808> That would be interesting but I think out of the scope of the initial implementation here
[21:58:27] <gwicke> if you start with a string log level for now then you can add that later
[21:58:36] <gwicke> if you use magic constants that would be harder
[21:58:41] <bd808> Mediawiki doesn't have a stellar real-time configuration management layer yet across the cluster
[21:59:08] <bd808> I'd really like to stick to the PSR-3 interface as closely as possible
[21:59:34] <bd808> That's the PHP "standard" for logging
[21:59:56] <gwicke> we'd probably need a separate trace system then
[22:00:14] <gwicke> or rather, a separate way to filter on verbose trace output
[22:00:14] <TimStarling> "not stellar": hehe, that's very generous
[22:00:27] <OwynD> we've had good luck putting all the logs in elastic search and just querying it
[22:00:50] <OwynD> so, you can probably just punt on that as well, keep the simple interface and just process/filter them offline
[22:01:06] <bd808> That's working pretty well for us too. Now I just want richer data to feed to logstash
[22:01:25] <bd808> And an easier way to plug in a new transport
[22:01:49] <bd808> So what are my TODOs?
[22:02:04] <bd808> Update the wiki page
[22:02:15] <bd808> Anything else?
[22:02:26] <TimStarling> get someone to +2 the change?
[22:02:38] <bd808> :)
[22:02:43] <bd808> Cool
[22:02:59] <sumanah> #action bd808 get someone to +2 his changeset
[22:03:09] <sumanah> #info <OwynD> we've had good luck putting all the logs in elastic search and just querying it
[22:03:18] <sumanah> #info bd808 "interested in fine grained runtime control"
[22:03:52] <sumanah> I don't know what else to take from this conversation that others might also want to know
[22:03:59] <TimStarling> ok, I've got to go to the next meeting
[22:04:03] <sumanah> #endmeeting