Extension:Cargo/Performance testing

Discourse DB test

A PHP script was created (you can see its full contents here) that attempts to run equivalent queries on matching data sets using Cargo and Semantic MediaWiki. It was run on the political commentary wiki Discourse DB, a site run by Yaron Koren, the author of Cargo. During the time of the test, the site stored most of its information in both SMW and Cargo. The site was running SMW 2.2 and Cargo 0.8. (SMW has since been removed from the site, which means that this particular test is unfortunately not reproducible.)

Query URLs were created for both Cargo and SMW that get, in CSV format, the earliest 100 opinion items on the site, sorted by date in ascending order, that were published in The Wall Street Journal. For each item, the query got its name, author(s) and publication date. The Semantic MediaWiki URL (no longer working) was here, and the Cargo URL is here.

The script fetched each URL 10 times at a time, and alternated between the two URLs, to get average retrieval times.

After several runs of the script, the average run time for the Semantic MediaWiki query was determined to be 1.02 seconds, while the average run time for the Cargo query was determined to be 0.69 seconds. For this particular data set, then, Cargo's querying was approximately 50% faster than Semantic MediaWiki's.

Other tests

A similar test was run at NASA in early 2018. According to NASA engineer Kris Field, the test showed Cargo querying to run around 30% faster than Semantic MediaWiki's.

Though not a test, the wiki farm Gamepedia, in 2018 and 2019, migrated all its wikis that use Semantic MediaWiki to use Cargo instead, due to slowness and sometimes server crashes caused by SMW queries. Cargo seems to be working much better on their systems, for the same data.

Anyone should feel free to run a similar test themselves, on any data that happens to be stored in both Cargo and SMW. Whatever results you get, please let Yaron know by email (yaron57gmail.com), or write the results on the talk page.