User:JDrewniak (WMF)/notes/Search widgets at Wikimedia

This page contains technical research on the possible improvements to the search on desktop completed by JDrewniak (WMF)

Search widgets at Wikimedia

edit

There are currently (As of October 2019) 6 different search typeahead implementations at the Wikimedia Foundation.

  1. searchSuggest
  2. OOUI TitleWidget & TitleOptionWidget
  3. MobileFrontend SearchOverlay
  4. www.wikipedia.org search widget
  5. Wikidata typeahead for item statements
  6. Wikidata typeahead for site search

How do they differ?

edit
Screenshot Name Lives in Used by Depends on Uses API
 
searchSuggest MediaWiki Core

searchSuggest.js

  • Vector
  • Timeless
  • Monobook
  • Cologne Blue
  • Modern
  • jQuery
  • mw.api
opensearch
 
TitleOptionWidget MediaWiki Core

mw.widgets.TitleWidget.js

  • ContentTranslation
  • VisualEditor
  • Special:Search
  • jQuery
  • mw.api
  • OOUI
prefixsearch
 
SearchOverlay MobileFrontend

SearchOverlay.js, uses

WatchstarPageList.js for

results template and

SearchGateway.js for API request


  • MobileFrontend & Minerva
  • jQuery
  • mw.api
  • Mustache
configurable via

$wgMFSearchGenerator

(prefixsearch on Wikipedia)

 
WMTypeAhead wikimedia portals

wm-typeahead.js

  • www.wikipedia.org page
  • mw.html
prefixsearch
 
Wikidata Typeahead Wikidata extension
  • wikidata.org
jQuery UI prefixsearch
 
Wikidata site-search typeahead Wikidata extension
  • wikidata.org
jQuery UI prefixsearch

All widgets contain the same basic functionality: Producing a list of search suggestions when typing a query into a search input. On top of that:

  • MobileFrontend search widget adds a watchstar beside each suggestion, but it's the only one that lacks support for keyboard navigation.
  • SearchSuggest is the oldest and most bare-bones, as it doesn’t include thumbnails in the suggestions.
  • The OOUI-based TitleOptionWidget is versatile in that it has the option to render with or without the thumbnail.
  • The portal implementation is the smallest and most self-contained, requiring only 200 lines of JS (including comments).

The major differences between the implementations are in the underlying frameworks used to build them. All widgets except the portal depend on jQuery and mw.Api. The mobile widget depends on the Mustache templating library, and the OOUI widget depends on OOUI.

Factors to consider

edit

Performance

edit

These search widgets, on both mobile and desktop, are loaded on every page. The idea of replacing the old SearchSuggest widget has been around for a while (see https://phabricator.wikimedia.org/T125725) and a new specialized OOUI widget was even created https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/255436/, but the major blocker for deploying the OOUI based widget was the need to load OOUI on every page (~100kb).

On mobile, the biggest dependency outside of jQuery is the Mustache templating library. However, Mustache is only an 8kb library and is used for many other widgets on mobile, such as the main menu.

Design

edit

Having 4 different implementations of the same features inevitably leads to design inconsistency. Those concerns have been expressed in this task: https://phabricator.wikimedia.org/T153417.

Instrumentation

edit

There exists 3 search widget related eventLogging schemas: Schema:Search and Schema:MobileWebSearch (which is based on Schema:Search) and Schema:SearchSatisfaction. Schema:Search is the oldest of the three and has been marked deprecated. Instrumentation code for Schema:MobileWebSearch still exists in the MobileFrontend code base in MobileWebSearchLogger.js and is still functional (though buggy). Schema:MobileWebSearch is therefore tied to the mobile search widget.

Schema:SearchSatisfaction is used by the Search Platform team measure the efficacy of search algorithms. Its methodology is described in this research document. It is currently active and uses data collected from the SearchSuggest widget on desktop. The searchSuggest widget emits events with mw.track that are registered with the SearchSatisfaction schema. Since this schema is active, any new or revised desktop search widget should remain compatible with the searchSatisfaction schema and emit the same events.

The feasibility of porting the mobile implementation to desktop

edit

Because of the underlying framework dependencies, neither implementation can be ported from Vector to Minerva & vice versa without incurring a performance penalty. Porting the OOUI widget to mobile would mean loading the OOUI library on every page (~100kb) and porting the mobile implementation to desktop would mean loading Mustache on every page (~8kb). If the Mustache dependency could be dropped from the mobile implementation, porting it over to desktop could be feasible. We would however, still have to add keyboard navigation support to the mobile widget as well as compatibility with the searchSatisfaction schema.

The feasibility of creating a new widget

edit

The search suggestion feature is well suited for newer Javascript frameworks that rely on unidirectional data-flow and explicit state management. A search widget is fairly self-contained in that it doesn't interact or depend on other UI elements on page. With a React-like framework, the core behaviour of a search typeahead can be modelled as 1.) responding to input events 2.) Making a network request 3.) Updating a template automatically based on the network response.

This behaviour can now be achieved much easily than in the past. Using a Codepen with existing HTML/CSS from the design task above, I was able to create a typeahead widget in a few hours using the Preact framework: https://codepen.io/j4n/pen/PooqWQLq