File:Second BM25 AB Test Analysis.pdf

Go to page
next page →
next page →
next page →

Original file(1,275 × 1,650 pixels, file size: 144 KB, MIME type: application/pdf, 17 pages)


English: In order to assess the efficacy of BM25 in space-less language, Discovery’s Search team has decided to conduct a second A/B test in Chinese, Japanese and Thai Wikipedias. We observed that the test group that used per-field query builder with incoming links and pageviews as query-independent factors had a much better Zero Result Rate but slightly worse PaulScores, large decrease in clickthrough rate, and fewer users clicked on the first result first, which indicates that we are showing test group users worse results. However, longer dwell-time and fewer query reformulations show that test group users might actually like the results they are getting better than the control group in that respect. We recommend deploying BM25 for all wikis but not reindexing projects in space-less languages for now.
Source Own work
Author CXie (WMF)


I, the copyright holder of this work, hereby publish it under the following license:
w:en:Creative Commons
attribution share alike
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.
You are free:
  • to share – to copy, distribute and transmit the work
  • to remix – to adapt the work
Under the following conditions:
  • attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • share alike – If you remix, transform, or build upon the material, you must distribute your contributions under the same or compatible license as the original.


Add a one-line explanation of what this file represents

Items portrayed in this file


6 January 2017

File history

Click on a date/time to view the file as it appeared at that time.

current01:27, 7 January 2017Thumbnail for version as of 01:27, 7 January 20171,275 × 1,650, 17 pages (144 KB)CXie (WMF)User created page with UploadWizard

There are no pages that use this file.