Talk:Requests for comment/HTML templating library

Ideas about stuff to maybe consider

Handlebars

I [NRuiz (WMF)] want to propose Handlebars as a choice. I have used for years with great ease. Going through your requirements:

• Both a PHP and JS implementation

Handlebars is a superset of mustache so a PHP mustache implementation will have similar syntax to a client side implementation. I would not get to hang up in sharing templates server and client side, though. But if wanted templates in both instances differ little so having Handlebars client side and Mustache server side would work. To be fair I think we would find Mustache's lack of control ifs very hard to work with server side NRuiz (WMF) (talk) 22:45, 20 January 2014 (UTC)

I like handlebars. Which PHP implementation do you recommend, handlebars.php or lightncandy? -- S Page (WMF) (talk) 19:36, 23 January 2014 (UTC)

• if/else. array handling/detect an empty array

Handlebars syntax has these and several other handy loop constructs. See: http://handlebarsjs.com/ NRuiz (WMF) (talk) 22:45, 20 January 2014 (UTC)

Works with strings/empty strings while Mustache only works with boolean JGirault (WMF) (talk) 01:26, 9 December 2015 (UTC)

• Secure by default

Handlebars escapes by default, you need special syntax to tell it not to escape.NRuiz (WMF) (talk) 22:45, 20 January 2014 (UTC)

• Attribute sanitization (no XSS in href,)

You can easily write a Handlebars helper that creates links that looks like the following:

  <a id="photo_uploader_link" href="{{link "Page" "index" page_key=getPhoto.uploaderId }}">

Parameter parsing can be taylored to our needs. Handlebars helpers are really extensible]

This makes manual escaping slightly more convenient, but does not make it automatic. This means that it is easy to forget escaping. It will be necessary to manually audit all templates for proper escaping for the given DOM context. -- Gabriel Wicke (GWicke) (talk) 00:06, 22 January 2014 (UTC)

• Extensible enough to accommodate MediaWiki i18n system

Easy enough to provide a handlebars helper that can extract client side tokens (with runtime replacemants) from whatever localization system.

I have used handlebars with a javascript gettext implementation (http://jsgettext.berlios.de/) before but this localization system would be even better: http://slexaxton.github.io/Jed/

A handlebars helper for localization will look like:

 Handlebars.registerHelper('translate', function(domain, token) {
        var args = Array.prototype.slice.call(arguments);
         //remove options {} that handlebars passes at the end everytime
        args.pop();
        var t = I18N.translate.apply(I18N,args);
        return new Handlebars.SafeString(t);
 });

You would use it like the following in the template engine:

 
  <someHTMLTag>
 {{{translate "PostRegister" "%(some_id)He leído y acepto las %(read_link_start_c)1$s%(href_value)2$s%(link_target)3$sCondiciones de la Portabilidad%(link_end_c)4$s." "<a href='" portinConditionsLink "' target='_blank'>" "</a>"}}}
  </someHTMLTag>

Handlebars helpers are easy to use and give great flexibility. NRuiz (WMF) (talk) 22:45, 20 January 2014 (UTC)

• No huge client-side footprint

Handlebars templates can be precompiled in node easily (this is crucial for performance).

Which reduces the complexity of working with partials and the footprint of templates. As they pass from html to a big function in javascript that just concatenates strings.NRuiz (WMF) (talk) 22:48, 20 January 2014 (UTC)

The lightncandy handlebars implementation in PHP supports compilation, it claims "10~30 times faster than https://github.com/XaminProject/handlebars.php". Also, after compiling you can call LightnCandy::getJsonSchema() to "Know required data structure from your templates. Verify input data, or find out missing variables with any jsonSchema validator". Intriguing... -- S Page (WMF) (talk) 20:48, 23 January 2014 (UTC)

Going to have to throw my support behind Handlebars for performance reasons, as given the alternative it seems fastest for page load time reasons. A small footprint is also a major plus, as I'm a fan of anything that doesn't slow down server speed considerably. Arcane21 (talk) 11:08, 28 August 2014 (UTC)

• Dumb (the less logic the better). Some template languages, for example jinja2 have a concept of filters where computation happens in the template itself. In my opinion this is unnecessary and can lead to complicated unreadable templates. Data should be preprocessed in PHP before being passed to a template.

Much agree. Template engine should only deal with links and localization, rest should be flat data.NRuiz (WMF) (talk) 22:48, 20 January 2014 (UTC)

• Readable - templates should be easy to grasp for someone with basic HTML knowledge. Ideally they are just plain HTML with attributes, so that they can be displayed and edited with HTML tools (think VE). This can enable server-side pre-expansion followed by client-side updates.

Handlebars templates are simply HTML until compiled. NRuiz (WMF) (talk) 22:45, 20 January 2014 (UTC)

Handlebars uses curly braces, which are not HTML. The scope of curly brace blocks also does not need to align with DOM structures. -- Gabriel Wicke (GWicke) (talk) 00:09, 22 January 2014 (UTC)

• Handlebars helpers

They are great at handling small tasks, like formatting tasks.

A helper like {{#eq a b}} is very useful (otherwise in Mustache we need to set a boolean for each of these conditions).

Have to be introduced in moderation. Templates should contain as less logic as possible. JGirault (WMF) (talk) 01:26, 9 December 2015 (UTC)

Now that we've decided to use handlebars, the RFC should be amended to mention that fact. There are big improvements over Mustache, that devs should be aware of when designing templates. The feature most painfully lacking in Mustache, IMO, is the ability to address variables in the outer context from within the body of a loop. Adamw (talk) 18:09, 10 April 2015 (UTC)

Existing PHP templating in core

You might want to compare templates' PHP functioning to existing PHP solutions in Mediawiki core, like QuickTemplate. AndrewRussellGreen (talk) 02:52, 19 December 2013 (UTC)

knockout.js

I get to know that library from Gabriel Wicke works. I do not develop for MediaWiki in recent years, so my voice does not weight a lot. However I used knockout.js in non-MediaWiki projects and appreciated it's small-size, speed, power and unobtrusive design. "data-bind" attributes are really great thing, because even when template is not expanded, no double curly braces are displayed. All logic is hidden in data-bind attributes. Also html tag can become a template language construct. Take a look how tag is a html tag and foreach loop at the same time: http://knockoutjs.com/documentation/foreach-binding.html . It makes templates really clean and very error-prone. I think that great library is underestimated - even Google CDN does not serve it, unfortunately. Next version of knockout.js will allow to pass data-bind text as {{}} similar to AngularJS. However AngularJS is probably impossible to port to PHP, while for knockout.js that is very well possible. QuestPC (talk) 10:09, 28 August 2014 (UTC)

I18n

I was very happy about mustache when I tried it out in one MediaWiki extension. For frontend it is pretty easy to include i18n support by using data-i18n attributes and jquery.i18n (though message loading is still not in core (nor is template loading yet)), but haven't figured how to do i18n in PHP. --Nikerabbit (talk) 12:17, 27 December 2013 (UTC)

Beyond templating

I have this feeling that AngularJS like stuff with controller bindings to the DOM etc is going to be next years thing. Have we considered/looked into this and what it means for templating ? I think it is too early for that right now perhaps, as far as I know there are no winners yet when it comes to MVW + template bindings as a combination. I just wanted to throw it out there and make sure that it was on people's radar before we pick a template engine. TheDJ (talk) 13:51, 4 January 2014 (UTC)

+1 on raising this aspect. Dynamic forms with client-side validation and help are another example where something like AngularJS might be helpful. -- Gabriel Wicke (GWicke) (talk) 19:59, 9 January 2014 (UTC)

Sorry but I disagree, templating itself is a problem that should not be coupled with a client side framework, several client side frameworks can work with any kind of templating engine. There is much benefit from being able to render client side without having to use per se a client side framework which might be too much of a big step.NRuiz (WMF) (talk) 22:17, 20 January 2014 (UTC)

I agree that we don't necessarily need reactive updates, so in that sense AngularJS is doing more than we need. It is possible to not use that functionality though.

With very simple string-based libraries like handlebars we'd end up writing our own framework. Security is an issue at best as a string-based library has no knowledge about the DOM and thus cannot guarantee proper escaping of attributes. This leads to more time spent on security reviews and a higher potential for security bugs. To me the main things simple libraries like mustache / handlebars have going for it are a) the ability to template non-HTML content, and b) simplicity and client-side size. -- Gabriel Wicke (GWicke) (talk) 22:13, 21 January 2014 (UTC)

Other aspects to consider

Some aspects that I am missing somewhat in this RFC:

• form handling and -validation and its interaction with the template library

I think we should decouple the two problems (rendering and form parsing).The basic use case for templating is just rendering data that gets to the client as JSON, quite different from parsing user input in a form NRuiz (WMF)

• systematic user content sanitization in DOM context

user sanitization (i.e. escaping) is done by Handlebars for example by default. (talk) 22:16, 20 January 2014 (UTC)

In the longer term, there could be overlap with content HTML templating as planned in Parsoid. Some extra and possibly differing requirements there are:

• Visual editing: should support visual editing (so no magic syntax in templates, only HTML, simplicity)
• Encourage data abstraction: access to JSON data sources and data-generating code modules instead of templated data in page content
• Performance:
  • scoped variable definitions for efficient re-rendering of dynamic parts
  • integration with caching (dependency tracking, purges, incremental fragment updates)
• Security: no infinite loops (only iteration), no unsanitized content injection

Overall it is not clear that the same solution can work well for HTML templating in both content and UI. There is no reason for this RFC to block on the Parsoid work, but it might be useful to keep content templating in mind. -- Gabriel Wicke (GWicke) (talk) 23:10, 7 January 2014 (UTC)

Feedback and next steps

See the 21 Feb 2014 meeting summary for steps that are currently underway. The PHP port is not done yet. As for Knockout, see Talk:Requests for comment/HTML templating library/Knockout - Tassembly for discussion of current work. Gabriel Wicke and others have been discussing the Flow use case with other Features engineers, and are tweaking the implementation in consultation with users. Sharihareswara (WMF) (talk) 22:08, 10 March 2014 (UTC)

port the JS IR compiler / runtime to PHP

This was one of the action items (the main one, repeated three times): was it done? --Nemo 09:22, 22 April 2014 (UTC)

Yes, see https://github.com/mattofak/knockoff and Matt's announcement on wikitech-l. -- Gabriel Wicke (GWicke) (talk) 21:17, 22 April 2014 (UTC)

IRC discussion 11 June

From Architecture meetings/RFC review 2014-06-11:

• LINK: https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-06-11 (sumanah, 21:00:12)
• HTML templating library (sumanah, 21:00:36)
  • LINK: https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library (sumanah, 21:00:44)
  • LINK: http://lists.wikimedia.org/pipermail/wikitech-l/2014-June/076861.html Gabriel Wicke sent an update to wikitech-l recently (sumanah, 21:00:55)
  • LINK: https://www.mediawiki.org/wiki/HTML_content_templating - Anyone have initial thoughts on this? (sumanah, 21:01:48)
  • ACTION: jdlrobson to ask Juliusz, Maryana, Arthur, or similar to schedule time to give KnockOff / TAssembly a spin (sumanah, 21:06:40)
  • LINK: http://lists.wikimedia.org/pipermail/wikitech-l/2014-June/076846.html spagewmf talking about MobileFrontend's & Flow's experiences (sumanah, 21:10:02)
  • <gwicke> to set expectations(re HTML content templating): if we have a prototype before Q3 I'll be positively surprised (sumanah, 21:12:55)

Please see Architecture meetings/RFC review 2014-06-11 for the full log. Sharihareswara (WMF) (talk) 19:53, 12 June 2014 (UTC)

Translation

Could someone please review the RfC for accuracy? Is the information up-to-date? I'd like to translate it to other languages to get international feedback, but I was told that the RfC may need a review by a pair of critical eyes first. Please also do not close it. Thanks! --Gryllida 13:43, 28 August 2014 (UTC)

Some thoughts from a senior skin developer

Given that I've written my fair share of MediaWiki skins and extensions which use the QuickTemplate class in some way, I thought I'd share my thoughts on this RfC here, as whatever the outcome is, it's likely going to affect me.

First of all, this whole RfC seems rather Wikimedia-centric to me. The Existing implementations in MediaWiki extensions section lists a bunch of extensions basically developed by and for the WMF. Fair enough, for sure, but there are other extensions developed by (major) third parties, such as ShoutWiki, Wikia and wikiHow, and the decisions made here are likely to have an impact on the developers of these organizations, as well as the tools built by them.

Secondly, no matter what the outcome is, QuickTemplate must not go away anytime soon. Upgrades are pesky enough as-is and nobody can seriously even imagine a situation in the MediaWiki land where (skin/extension) code written several years ago runs on a modern MediaWiki. Whereas earlier this year I fiddled around with phpBB and had very little problems in applying a bunch of core patches from 2007 (!) against the current stable release of phpBB. Impressive, for sure, but not going to happen around here. MediaWiki developers seem to love breaking stuff for the sake of breaking it (whether it's labeled as "refactoring", "security fix" or something else), so what we're gonna need is a clear migration path and lots of accurate, up-to-date documentation for third-party users.

Now then, let's look at some of the obvious issues with this RfC's justification.

1. "Better code readability, editability, and portability" sounds nice, but the truth is that it's still nothing but a bunch of words on a paper (or a wiki page, as the case is here). The most readable code, in my opinion, is whatever comes closest to the actual output. QuickTemplate, when used properly, is pretty damn awesome at this — the templates are rather readable, even by newer developers, and mostly easy to understand. Of course the tradeoff is that sanitizing input and such is left to the developer, but I strongly believe that we shouldn't automate everything.
2. "Separate code from markup" — this is, or more accurately should've been, a no-brainer. Yet we not only have people who write <div class="whatever"> as Html::openElement( 'div', array( 'class' => 'whatever' ) ), but the latter is actually our recommended practise (!), which no doubt leads to secure code — which just happens to be a fucking mess to wade through.
3. "Avoid jQuery spaghetti" — could you give some examples of the mentioned spaghetti code? I'm not actually against the general idea, I'd just like to see what kind of real-world use cases we'd have for this.
4. "Avoid loading multiple JS libraries from different extensions" — about time, too; no need for each extension to reinvent the wheel when we can have the wheel in core. :)
5. "Make it easier to build complex and good looking admin interfaces" — are we trying to solve social issues with technical means once again? Sure, technical choices do have some kind of an impact on the final product, but at the end of the day, whether something looks good or not is largely dependent on the developer(s)/team working on it, as well as how familiar they are or aren't with design principles. Think of it this way: escaping everything by default without a question might sound good, but what if there's a valid reason to display the & character as-is on a page title instead of having it displayed as &amp;?

Finally a few notes on third parties and their HTML templating:

1. ShoutWiki uses QuickTemplate on their skins and some of their extensions
2. Wikia used to use EasyTemplate, a QuickTemplate-like class which didn't require the actual HTML templates to be wrapped in a wrapper class that extends the base class, but they recently moved to Mustache if I recall correctly
3. wikiHow uses a slightly modified version of Wikia's EasyTemplate, but some of the more recent extensions also use QuickTemplate

Personally I'd like to see QuickTemplate modified to support JS templating, but that might be close to impossible or at least too much for anyone to seriously consider. But no matter what the outcome is, QuickTemplate must not be removed anytime soon for it is widely used. --Jack Phoenix _(Contact) 17:27, 28 August 2014 (UTC)

en.wn dialog tools

From my perspective, the thing that matters most to me is whether or not it'll break the dialog tools I've been developing for en.wn for the past several years. Whose function is to allow interactive wiki pages to be cleanly constructed using wiki markup, and does it using jQuery under the hood (both for the page elements themselves, for actions to be performed by buttons, and for the sessionStorage that allows data to pass from page to page and from page to action). Tbh, I can't work out, from this RfC, the most basic things about what it allows one to do and what it might interfere with. (And yes, I realize it's extremely difficult, when you understand something inside-out, to explain it in a way that will be crystal clear to someone who's entirely outsdie; I've seen that problem from both sides.) --Pi zero (talk) 11:06, 29 August 2014 (UTC)

I'm pretty sure it won't affect you. This RFC is about how MediaWiki internally generates the html of a page. From a user prespective I don't really expect anything to change. Bawolff (talk) 16:39, 29 August 2014 (UTC)

Okay (hopefully :-). Thanks. --Pi zero (talk) 16:47, 29 August 2014 (UTC)

I added a note to the RfC (Requests for comment/HTML templating library) explaining how this is mostly unrelated to MediaWiki templating in wikitext. You don't want to accidentally send wikitext within the page through the templating processor that generated the page; doing it intentionally is, umm, interesting :-) -- SPage (WMF) (talk) 21:16, 1 October 2014 (UTC)