MediaWiki

Developer Wishlist/2017/Backend

< Developer Wishlist
Developer Wishlist 2017
Backend

11 proposals, 76 editors, 129 votes

The voting phase has concluded. Thanks for participating!

You can view the results or discuss how to follow up.


Make Monolog the default debug processing layer and deprecate wfDebug* and LegacyLogger

Tracked in Phabricator
Task T155552

Problem

The PSR3 logging interface has been introduced in MediaWiki to support structured logging , but no coordinated effort has been made to deprecate the use of wfDebug(), wfDebugLog(), wfLogDBError(), and wfErrorLog(). Several bugs are open in the #mediawiki-debug-logger project about the lack of parity between debug log usability on the Wikimedia Foundation production cluster and a typical development environment or external deployment of MediaWiki that are directly related to bd808 taking the structured logging project to a point where it is useful for the WMF but not pushing that usability further for other MediaWiki deployments.

Who would benefit

  • MediaWiki site operators who want better insight into their operational issues
  • MediaWiki developers who don't want to think about choosing between two largely compatible but very different debug logging layers

Proposed solution

  • Replace all usage of wfDebug* in MediaWiki core with direct PSR3 usage.
  • Add Monolog as a core dependency and the default debug logging solution.
  • Make configuring Monolog easier by making helpers in the MediaWiki\Logger\Monolog namespace.
  • Remove MediaWiki\Logger\LegacyLogger from core. (It could be made a library if there are people who really love it and want to keep maintaining a homegrown debug log formatting and routing layer.)
  • Deprecate wfDebug(), wfDebugLog(), wfLogDBError(), and wfErrorLog().

See also


Endorsements (T155552)

Support (T155552)

  1. BDavis (WMF) (talk) 17:28, 6 February 2017 (UTC)[reply]
  2. Greg (WMF) (talk) 17:55, 6 February 2017 (UTC)[reply]
  3. EBernhardson (WMF) (talk) 17:55, 6 February 2017 (UTC)[reply]
  4. Tgr (WMF) (talk) 08:54, 8 February 2017 (UTC)[reply]
Tracked in Phabricator
Task T74815

Please add a Lua function similar to mw.wikibase.getEntityObject for getting an entity by site link (title), just like Special:ItemByTitle or the wbgetentities API module with titles parameter.

On cswiki articles translated from other language wikis are marked by Translated template (with source wiki, article and revision parameters filled in). The template could detect (using Module:Wikidata), whether the article is connected with that source article and categorize it if not. But if that source article is connected with a different article, it is false positive there (could be categorized too, but in a different maintenance category). I can not single out these false positives, because I can not find out (using Lua function in Module:Wikidata) the Q-id for a given wiki:page.

On enwiki in Module:Wikidata talk page is another request from czar: QID lookup from enwp article title.

Endorsements (T74815)

Support (T74815)

  1. Info-farmer (talk) 05:22, 6 February 2017 (UTC)[reply]
  2. Dvorapa (talk) 09:42, 6 February 2017 (UTC)[reply]
  3. Deryck C.Meta 11:36, 7 February 2017 (UTC)[reply]
  4. Samuele2002 (talk) 23:32, 7 February 2017 (UTC)[reply]
  5. Jklamo (talk) 08:29, 8 February 2017 (UTC)[reply]
  6. Tpt (talk) 09:15, 8 February 2017 (UTC)[reply]
  7. Candalua (talk) 08:56, 9 February 2017 (UTC)[reply]
  8. Smalyshev (WMF) (talk) 02:28, 10 February 2017 (UTC)[reply]
  9. Aubrey (talk) 18:03, 10 February 2017 (UTC)[reply]
  10. 47.222.203.135 18:10, 13 February 2017 (UTC)[reply]
  11. B20180 (talk) 13:46, 14 February 2017 (UTC)[reply]
  12. Accurimbono (talk) 15:03, 14 February 2017 (UTC)[reply]
  13. Nemo 18:47, 14 February 2017 (UTC)[reply]

Allow excluding soft redirected categories on Special:UnusedCategories

Tracked in Phabricator
Task T96041

Special:UnusedCategories is flooded by tons of soft-redirected categories. But purpose of this special page was to show real unused categories, not all unused ”aliases” aka redirecting categories. It's necessary to define a special magic word which can be placed in ”Template:Category redirect” and will prevent categories containing it from showing up in Special:UnusedCategories. Such a thing is really necessary, because some wikipedias have over 15,000 redirected categories.

A similar technology is used for Special:DisambiguationPages where are listed all pages which contains the tag __DISAMBIG__.

Endorsements (T96041)

Support (T96041)

  1. ThurnerRupert (talk) 12:41, 6 February 2017 (UTC)[reply]
  2. Swpb (talk) 15:32, 6 February 2017 (UTC)[reply]
  3. Info-Screen (talk) 05:45, 7 February 2017 (UTC)[reply]
  4. Sunfyre (talk) 07:31, 7 February 2017 (UTC)[reply]
  5. Yamaha5 (talk) 09:28, 7 February 2017 (UTC)[reply]
  6. Deryck C.Meta 11:36, 7 February 2017 (UTC)[reply]
  7. Enterprisey (talk) 20:32, 8 February 2017 (UTC)[reply]
  8. Headbomb (talk) 20:58, 8 February 2017 (UTC)[reply]
  9. Calexit (talk) 22:21, 9 February 2017 (UTC)[reply]
  10. Smalyshev (WMF) (talk) 02:27, 10 February 2017 (UTC)[reply]
  11. — Mr. Stradivarius ♪ talk ♪ 08:46, 11 February 2017 (UTC)[reply]
  12. Helder 17:44, 12 February 2017 (UTC)[reply]
  13. FASTILY 08:46, 13 February 2017 (UTC)[reply]
  14. Jjjjjjdddddd (talk) 09:31, 13 February 2017 (UTC)[reply]
  15. James Martindale (talk) 17:02, 13 February 2017 (UTC)[reply]

ApiQueryImageInfo is crufty, needs rewrite

Tracked in Phabricator
Task T89971

The code is a mess, the limit semantics make no sense, and we have several other options that don't really fit non-images.

The best thing to do here is probably to just write a prop=fileinfo module from scratch so we don't have to worry about backwards compatibility, and then deprecate prop=imageinfo.

Current plans:

  • Right now, iilimit specifies the max number of revisions to return per file, which is inconsistent with the rest of the API and isn't particularly sane. For fileinfo, filimits will limit the number of file-info-objects returned per result, and a separate "fioldversions" property (default 0, values integers or 'all') will specify the max number of revisions to be returned per file.
  • fistart/fiend may result in the info for the current revision not being returned.
  • iiprops has three different metadata properties. There really should be only one, and if possible it should be key-value pairs rather than a list of objects with key and value properties.
  • Figure out something sane to replace iiurlwidth/iiurlheight/iiurlparam. Maybe multi-valued fiparams?
  • prop=stashimageinfo is very odd, it's a prop module but doesn't use any titles. It would make sense to me for prop=fileinfo to have a fifilekeys parameter instead of having a whole separate module for this.
  • prop=videoinfo really isn't needed either. Instead we should make it possible for extensions to add additional info to the fileinfo response.

Endorsements (T89971)

Support (T89971)

  1. —TheDJ (Not WMF) (talk • contribs) 09:16, 6 February 2017 (UTC)[reply]
  2. FASTILY 08:46, 13 February 2017 (UTC)[reply]
  3. Ijon (talk) 00:41, 14 February 2017 (UTC)[reply]
  4. Sage (Wiki Ed) (talk) 17:45, 14 February 2017 (UTC)[reply]
  5. Quiddity (WMF) (talk) 19:55, 14 February 2017 (UTC)[reply]

Add global logging context

Tracked in Phabricator
Task T142313

Certain kinds of information would be useful to have in the log context, but not possible or convenient to add manually. When such information is available from the log processing code, we use a Monolog processor to add it (e.g. IP, URL) but that's not always possible. For example, logging the canonical special page name would be handy but that's controller-level information and the log processing code has no access to the controller (and if it is logging an exception, the controller might be in some unexpected state).

The straightforward solution is to have some globally available temporary storage which application code can write into and log processing code can read from. (log4j calls this a diagnostic context.) 

20:28 < bd808> One feature that monolog is missing that I loved from log4j is a global diagnostic context. That's basically a thread local dict that any code can grab and stick key=value data into that then ends up in all log events
...
20:31 < bd808> at $DAYJOB-1 we tagged logs with lots of things as we found out about them in the app stack. customer id, databases used, services called, etc
...
20:37 < bd808> and we could expose it via LoggerFactory
20:39 < bd808> log4j (and some academic papers I can't find right now) call this thing a "diagnostic context" -- https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/MDC.html
20:41 < bd808> http://c2.com/cgi/wiki?PatternsForLoggingDiagnosticMessages

Endorsements (T142313)

  1. This would make it possible to flag errors which happen in the middle of something important (such as login or user rename). In the past, the lack of that capability has led us to detect some critical errors way slower than we should have. Tgr (WMF) (talk) 09:00, 8 February 2017 (UTC)[reply]

Support (T142313)

  1. BDavis (WMF) (talk) 17:28, 6 February 2017 (UTC)[reply]
  2. EBernhardson (WMF) (talk) 17:56, 6 February 2017 (UTC)[reply]
  3. Nikerabbit (talk) 08:07, 7 February 2017 (UTC)[reply]
  4. Tgr (WMF) (talk) 08:56, 8 February 2017 (UTC)[reply]
  5. Edward Chernenko (talk) 22:26, 9 February 2017 (UTC)[reply]
  6. Greg (WMF) (talk) 22:39, 13 February 2017 (UTC)[reply]
  7. Nemo 18:48, 14 February 2017 (UTC)[reply]
  8. Quiddity (WMF) (talk) 19:56, 14 February 2017 (UTC)[reply]

API's list=recentchanges should have rcrelated parameter (provide Special:RelatedChanges/Special:RecentChangesLinked functionality via API)

Tracked in Phabricator
Task T17552

... to implement something akin to Special:RelatedChanges.

Endorsements (T17552)

Support (T17552)

  1. Jdforrester (WMF) (talk) 16:23, 6 February 2017 (UTC)[reply]

Kill wgLegacyEncoding

Tracked in Phabricator
Task T128149

Since version 1.5, MediaWiki stores all data in Unicode. Before that, the encoding was configurable; $wgLegacyEncoding allows current MediaWiki to work with old non-Unicode database values. This is one of our oldest pieces of technical debt. Retire it and automatically convert database rows on upgrade instead.

Endorsements (T128149)

Support (T128149)

  1. ThurnerRupert (talk) 12:07, 6 February 2017 (UTC)[reply]
  2. ·addshore· talk to me! 12:18, 6 February 2017 (UTC)[reply]
  3. Amir E. Aharoni (talk) 14:03, 6 February 2017 (UTC)[reply]
  4. Jdforrester (WMF) (talk) 16:23, 6 February 2017 (UTC)[reply]
  5. EBernhardson (WMF) (talk) 17:54, 6 February 2017 (UTC)[reply]
  6. Krinkle (talk) 19:43, 6 February 2017 (UTC)[reply]
  7. Santhosh.thottingal (talk) 03:37, 7 February 2017 (UTC)[reply]
  8. Info-Screen (talk) 05:43, 7 February 2017 (UTC)[reply]
  9. Sunfyre (talk) 07:29, 7 February 2017 (UTC)[reply]
  10. Nikerabbit (talk) 08:05, 7 February 2017 (UTC)[reply]
  11. Yamaha5 (talk) 09:27, 7 February 2017 (UTC)[reply]
  12. Samuele2002 (talk) 23:30, 7 February 2017 (UTC)[reply]
  13. FASTILY 08:42, 13 February 2017 (UTC)[reply]
  14. BrentLaabs (talk) 01:12, 14 February 2017 (UTC)[reply]
  15. Quiddity (WMF) (talk) 19:57, 14 February 2017 (UTC)[reply]

Structured data side channel for wikitext

Tracked in Phabricator
Task T156876

The problem of passing structured data from wikitext to external applications comes up in a wide variety of contexts, and a garden of ugly workarounds has grown around it, usually consisting of encoding the data in the HTML rendered from wikitext in some way, then external applications parsing it out and restoring the structure. Examples include #CommonsMetadata, the various services (#mcs, all kinds of Tool Labs tools) exposing mainpage/featured content (article/picture of the day, anniversaries, in the news etc), article maintenance / warning templates, infoboxes, using Wiktionary for word translation.

Eventually these issues should be handled by separating wikitext and structured data (e.g. with T107595: [RFC] Multi-Content Revisions) but that's a huge project and will take a while. A quick win that would be possible right now and would make the life of developers mining structured from wikitext (and editors maintaining the wikitext) would be to create a side channel where wikitext code can output structured data (with a dedicated parserfunction Lua method), in a simple hierarchic key-value format. The data could exposed by the parser and the parse API, and eventually morph into a virtual MCR slot.

Endorsements (T156876)

Support (T156876)

  1. Daniel Mietchen (talk) 22:31, 6 February 2017 (UTC)[reply]
  2. Calexit (talk) 22:18, 9 February 2017 (UTC)[reply]
  3. Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 18:29, 13 February 2017 (UTC)[reply]
  4. Ijon (talk) 00:42, 14 February 2017 (UTC)[reply]

Showcase how the separation of concerns should work between MediaWiki API and web

Tracked in Phabricator
Task T156872

MediaWiki API modules and special pages contain lots of business logic, often duplicated between the two in similar-but-not-quite-identical ways. The business logic in these pages also tends to be inaccessible internally (so MediaWiki code that wants to access the functionality does horrible things like instantiating a SpecialPage object or making FauxRequest calls to the API). Everyone agrees the current situation sucks; no one seems to be sure how exactly the right way would look like, so newly written code does not necessarily end up in better shape.

We should pick some special pages and API modules (probably two of each since the answer will look very differently for something that does paged queries and for everything else), refactor them and turn them into a showcase that can be used as a guidance for future work.

Endorsements (T156872)

Support (T156872)

  1. This, that and the other (talk) 07:54, 6 February 2017 (UTC)[reply]
  2. ·addshore· talk to me! 12:18, 6 February 2017 (UTC)[reply]
  3. BDavis (WMF) (talk) 17:27, 6 February 2017 (UTC)[reply]
  4. EBernhardson (WMF) (talk) 17:55, 6 February 2017 (UTC)[reply]
  5. —Mainframe98 talk 18:01, 6 February 2017 (UTC)[reply]
  6. Jdlrobson (talk) 01:03, 7 February 2017 (UTC)[reply]
  7. MHolloway (WMF) (talk) 22:12, 7 February 2017 (UTC)[reply]
  8. André Costa (WMSE) (talk) 15:28, 8 February 2017 (UTC)[reply]
  9. Smalyshev (WMF) (talk) 02:29, 10 February 2017 (UTC)[reply]
  10. BSitzmann (WMF) (talk) 22:53, 11 February 2017 (UTC)[reply]
  11. FASTILY 08:44, 13 February 2017 (UTC)[reply]
  12. 47.222.203.135 18:10, 13 February 2017 (UTC)[reply]
  13. Ljonka (talk) 11:06, 14 February 2017 (UTC)[reply]
  14. Quiddity (WMF) (talk) 19:59, 14 February 2017 (UTC)[reply]

Add a maintenance script for complete cache reset

Tracked in Phabricator
Task T156695

Problem

There seems to be no way to reset all caches programmatically. One example is the ResourceLoader minification cache. In extension development and when updating mediawiki installations, I ran into issues with improper cache invalidation. A complete manual reset of all caches helped, yet a script would be very helpful.

Who would benefit

Developers and MediaWiki maintainers

Proposed solution

Create a maintenance script which invalidates all caches. Preferably with the option to selectively invalidate some types of caches, e.g. ResourceLoader minification.

Endorsements (T156695)

  1. This is annoying. I often find myself doing TRUNCATE TABLE objectcache; (since I use CACHE_DB). Sometimes I only want to get rid of some part of the cache (like only ResourceLoader caches) and the TRUNCATE is a nuclear option, but other alternatives are very tedious. This, that and the other (talk) 07:56, 6 February 2017 (UTC)[reply]

Support (T156695)

  1. Shizhao (talk) 07:00, 6 February 2017 (UTC)[reply]
  2. —TheDJ (Not WMF) (talk • contribs) 09:15, 6 February 2017 (UTC)[reply]
  3. —Mainframe98 talk 09:24, 6 February 2017 (UTC)[reply]
  4. Consulnico (talk) 09:37, 6 February 2017 (UTC)[reply]
  5. ThurnerRupert (talk) 12:10, 6 February 2017 (UTC)[reply]
  6. Jack Phoenix (Contact) 23:10, 6 February 2017 (UTC)[reply]
  7. [[kgh]] (talk) 23:15, 6 February 2017 (UTC)[reply]
  8. Nikerabbit (talk) 08:06, 7 February 2017 (UTC)[reply]
  9. Samuele2002 (talk) 23:31, 7 February 2017 (UTC)[reply]
  10. Ricordisamoa 11:21, 8 February 2017 (UTC)[reply]
  11. Enterprisey (talk) 20:31, 8 February 2017 (UTC)[reply]
  12. RichardHeigl (talk) 23:29, 9 February 2017 (UTC)[reply]
  13. Smalyshev (WMF) (talk) 02:29, 10 February 2017 (UTC)[reply]
  14. BSitzmann (WMF) (talk) 22:53, 11 February 2017 (UTC)[reply]
  15. Ciencia Al Poder (talk) 11:54, 12 February 2017 (UTC)[reply]
  16. FASTILY 08:44, 13 February 2017 (UTC)[reply]
  17. Mglaser (talk) 10:21, 13 February 2017 (UTC)[reply]
  18. Sebastian Berlin (WMSE) (talk) 10:24, 13 February 2017 (UTC)[reply]
  19. Osnard (talk) 11:38, 13 February 2017 (UTC)[reply]
  20. 47.222.203.135 18:10, 13 February 2017 (UTC)[reply]
  21. Framawiki (talk) 17:59, 14 February 2017 (UTC)[reply]
  22. Related: Developer_Wishlist/2017/Documentation#Complete documentation about different types of caching for extension developers. Nemo 18:52, 14 February 2017 (UTC)[reply]
  23. Daniel Renfro 20:41, 14 February 2017 (UTC)[reply]
  24. Dave Braunschweig (talk) 21:53, 14 February 2017 (UTC)[reply]

Add syntax highlighting to wiki diff of source code pages

Tracked in Phabricator
Task T156048

Problem

When we editing pages containing Lua, JavaScript or CSS code, we can't easily see what has changed due to a lack of syntax highlighting. Many times that needs a a lot of mental concentration.

Who would benefit

All Lua or JavaScript coders.

Proposed solution

When we edit Lua or JavaScript code, highlight differences like in Gerrit ( - + parts of line). Do this when we explicitly ask differences, but also as option in any change from the previous version.

Description of the task

The way used in gerrit to display changes in source code seems better than some others. In code panels, for old and new codes, gerrit colorises them in red and green for each changed character.

Then we could use it for other source codes: javascript for gadgets and user scripts, Lua modules, Lisp for templates.

Some chooses to select

guerrit uses 2 ways, which mode use? In the mixing way, gerrit mixes the old and the new code lines, identify them with - and + signs, and hightlights them with red and green colors for each character. In the columns way, gerrit put the old code in left and the new code in right column, and hightlights them with red and green colors for each character.

This styles also interfere with the usual hightlight of each code.

Where to use these display ways? Perhaps also in wikitext for wikicontent? Perhaps also when we display a revision-diff = "Difference between revisions"? Even in VisualEditor?

When to use these display ways? Each time when we edit a script or a module or a template and then clic on "show changes". Also when we clic "Preview" to see the effect on a test page? Begin to offer these ways for codes panels, and later extend them to other places?

When to activate these ways and modes? When the user chooses these ways for each code panel? When the user chooses them in his preferences?


Endorsements (T156048)

Support (T156048)

  1. David1010 (talk) 06:43, 6 February 2017 (UTC)[reply]
  2. Shizhao (talk) 07:00, 6 February 2017 (UTC)[reply]
  3. This, that and the other (talk) 07:56, 6 February 2017 (UTC)[reply]
  4. Consulnico (talk) 09:37, 6 February 2017 (UTC)[reply]
  5. Frettie (talk) 10:06, 6 February 2017 (UTC)[reply]
  6. Daniel Mietchen (talk) 22:33, 6 February 2017 (UTC)[reply]
  7. — MusikAnimal talk 22:45, 6 February 2017 (UTC)[reply]
  8. [[kgh]] (talk) 23:16, 6 February 2017 (UTC)[reply]
  9. Sunfyre (talk) 07:30, 7 February 2017 (UTC)[reply]
  10. Yamaha5 (talk) 09:27, 7 February 2017 (UTC)[reply]
  11. Deryck C.Meta 11:36, 7 February 2017 (UTC)[reply]
  12. Chico Venancio (talk) 13:25, 7 February 2017 (UTC)[reply]
  13. Matěj Suchánek (talk) 13:51, 7 February 2017 (UTC)[reply]
  14. Enterprisey (talk) 20:32, 8 February 2017 (UTC)[reply]
  15. NKohli (WMF) (talk) 12:48, 10 February 2017 (UTC)[reply]
  16. — Mr. Stradivarius ♪ talk ♪ 08:49, 11 February 2017 (UTC)[reply]
  17. Deniz (talk) 22:51, 11 February 2017 (UTC)[reply]
  18. Helder 17:44, 12 February 2017 (UTC)[reply]
  19. FASTILY 08:45, 13 February 2017 (UTC)[reply]
  20. James Martindale (talk) 17:01, 13 February 2017 (UTC)[reply]
  21. Strong support. 47.222.203.135 18:10, 13 February 2017 (UTC)[reply]
  22. TerraCodes (talk) 21:05, 13 February 2017 (UTC)[reply]
  23. PaleAqua (talk) 02:00, 14 February 2017 (UTC)[reply]
  24. Framawiki (talk) 17:57, 14 February 2017 (UTC)[reply]
  25. Dave Braunschweig (talk) 21:52, 14 February 2017 (UTC)[reply]
Retrieved from "https://www.mediawiki.org/w/index.php?title=Developer_Wishlist/2017/Backend&oldid=3410185"