MediaWiki

Wikiscan

Wikiscan is a web-based tool that computes and displays historical statistics about a wiki site, originally for all Wikimedia Foundation (WMF) hosted wiki sites. Its About page states: "Wikiscan is not linked to the Wikimedia Foundation, the statistics published here are not official."[1]

Manual on MediaWiki Tools
Wikiscan
Release status: beta
Implementation Database
Description Edit statistics for a wiki
Author(s) (Akerontalk)
Latest version 0.9.2 (3-4-2021)
MediaWiki 1.34+
PHP 7.2
License GPL v2
Download

GitHub:

Note:
No localisation updates are
provided by translatewiki.net .

Example Multi site:
https://wikiscan.org/,
single site:
https://stats.wikidex.net/

The Wikiscan application is standalone and written in PHP. It was created in 2011 on the initiative of a voluntary user for WMF wikis, and is located at https://wikiscan.org. Most of the software development was done on a voluntary basis. Although independent of the Wikimedia Foundation, Wikiscan was part subsidized by Wikimedia Foundation and the Wikimedia France chapter association, the latter of which is currently providing server hosting for Wikiscan.

The software code was originally available on the Wikiscan.org website for download. However, it is no longer available for download. In 2021, User:Ciencia Al Poder became interested in this tool for another wiki project, and was able to download a copy of the source code before it was removed; he documented some parts of it, and did a general review to fix some issues so it is able to work on independent wiki sites.

The Wikiscan tool generates statistics by querying public data from the wiki site's database, and stored on its own database. Data displayed on the tool is not updated in real time. The statistics data must be generated on a regular basis, using cron or similar.

Download

edit

Wikiscan source code can be downloaded from GitHub, at: https://github.com/ciencia/wikiscan/

Requisites

edit

This tool currently needs to be installed on it's own domain or subdomain. It does not support installing on a subdirectory other than the web root.

  • Webserver
  • PHP 5.5 or newer, tested on PHP 7.3
    • PHP modules : mbstring, curl, gd, mysqlnd, memcached
  • MySQL
    • It uses its own database and also needs access to the wiki's database
  • Memcached
    • Used for caching results

Installation

edit
  1. Extract Wikiscan to a directory (example /var/www/wikiscan/)
  2. Create a new Mysql database (example wikiscan_stats)
  3. Import ctrl/stats.sql to create the tables.
  4. Create or configure a Mysql user with SELECT,INSERT,UPDATE,DELETE rights to the wikiscan database (example: wikiscan_user)
  5. Create or configure a Mysql user with SELECT rights to the wiki's database (it can be the same previously created user)
  6. Go to config/ directory
  7. Copy db_conf.sample.php to db_conf.php
  8. Edit db_conf.php
    1. Configure "db" with your Mediawiki database
    2. Configure "dbs" with the new stats database
  9. Copy local_conf.sample.php to local_conf.php
  10. Edit local_conf.php to configure local paths and wiki urls
  11. Make the directories wpstats/ cache/ and img/* writable

The site should work at this stage, but without any statistics yet.

Setting up rewrite rules

edit

This tool generates some links that expect an existing set of rewrite rules for them to work.

apache

edit

You can see code in the .htaccess file.

nginx

edit

With alternate paths in Spanish: 

map $request_uri $wikiscan_qs {
        default "";
        "~^/\?" $args;
        "~^/(?:hours|horas)/?$" "menu=live&list=pages";
        "~^/(?:hours|horas)/(\d+)/?$" "menu=live&date=$1&list=pages&sort=weight";
        "~^/(?:hours|horas)/(\d+)/([^&/]+)/?$" "menu=live&date=$1&list=$2&sort=weight";
        "~^/(?:grid|matriz)/?$" "menu=grid";
        "~^/(?:grid|matriz)/(\d+)/?$" "menu=grid&date=$1";
        "~^/(?:calendar|calendario)$" "menu=dates&list=pages";
        "~^/(?:date|fecha)/(\d+)/?$" "menu=dates&date=$1&list=pages&sort=weight";
        "~^/(?:date|fecha)/(\d+)/([^&/]+)/?$" "menu=dates&date=$1&list=$2&sort=weight";
        "~^/(?:users|usuarios)$" "menu=userstats";
        "~^/(?:users|usuarios)\?graphs_details=1$" "menu=userstats&graphs_details=1";
        "~^/(?:user|usuario)/?$" "menu=userstats";
        "~^/(?:user|usuario)/([^&]+)$" "menu=userstats&user=$1&usort=date";
        "~^/(?:user|usuario)/.+$" "menu=userstats&user=_";
        "~^/(?:userlist|listausuarios)/([^&]+)$" "menu=userstats&userlist=$1";
        "~^/ip/?$" "menu=userstats_ip";
        "~^/ip/([^&]+)$" "menu=userstats_ip&user=$1";
        "~^/ip/.+$" "menu=userstats_ip&user=_";
        "~^/pages_vues/?$" "menu=pageview";
        "~^/pages_vues/(\d+)/(.*)$" "menu=pageview&year=$1&ns=$2";
        "~^/pages_vues/(\d+)$" "menu=pageview&year=$1";
        "~^/pages_vues/.+$" "menu=pageview&year=_";
        "~^/ranges/?$" "menu=ranges";
        "~^/range/([^&]+)$" "menu=ranges&range=$1";
        "~^/range/.+$" "menu=ranges&range=_";
        "~^/Derniers_utilisateurs_actifs" "menu=active_users";
}

server {
        listen 80;
        #listen 443 ssl;
        server_name stats.mywikisite.com;
        root /home/www/wikiscan;

        disable_symlinks on;

        access_log /var/log/nginx/wikiscan-access.log;
        error_log /var/log/nginx/wikiscan-error.log;

        location = / {
                try_files $uri @wikiscan;
        }

        location / {
            if ($wikiscan_qs = "") {
                return 404;
            }
            try_files $uri @wikiscan;
        }

        location /img {
                try_files $uri =404;
        }
        location /libs {
                try_files $uri =404;
        }
        location /wikiscan {
                try_files $uri =404;
        }

        location = /gimg.php {
                fastcgi_pass unix:/run/php/php-fpm-wikiscan.sock;
                include /etc/nginx/fastcgi_params;
                fastcgi_param HTTPS "on";
                fastcgi_param SCRIPT_FILENAME $document_root/gimg.php;
                fastcgi_param QUERY_STRING $query_string;
        }

        location @wikiscan {
                fastcgi_pass unix:/run/php/php-fpm-wikiscan.sock;
                include /etc/nginx/fastcgi_params;
                fastcgi_param HTTPS "on";
                fastcgi_param SCRIPT_FILENAME $document_root/index.php;
                fastcgi_param QUERY_STRING $wikiscan_qs;
        }
}

Configuration parameters

edit

Edit the conf/local_conf.php file, adding or updating keys of the $local_conf variable:

Key Description
root_path Local filesystem path of this wikiscan installation. Example: /home/www/wikiscan
logo URL of a small Wiki logo (80px tall recommended)
icon URL to a favicon image
wikilink_icon URL for a small icon to use as links to the wiki (defaults to WMF logo)
google_analytics Google analytics account (leave empty if you don't want to track users)
memcache_host Memcached host. Example: 127.0.0.1
memcache_port Memcached port. Example: 11211
site_language Language of the wiki. See Supported languages. This may determine some paths of the wikiscan site. See urlpath-menu-* language messages. Example: en.
interface_language Wikiscan's interface language. See Supported languages. Example: en.
menus_enabled List of menus enabled. Leave it to false to not restrict any. Implemented as security measure since there are some menus not fully developed and not reviewed. Example: ['home','live','grid','dates','userstats','userstats_ip']
mw_api API url, used to retrieve the list of namespaces, or members of a category to get users from. Example: https://en.wikipedia.org/w/api.php
link_page URL to article path (page titles will be appended to this URL). Example: https://en.wikipedia.org/wiki/
wiki Local wiki configuration (when not multi site). Array with keys:
  • url: URL to the wiki's homepage
  • site_host: Wiki site name
groups Local/custom group names/abbreviations. Use it if you have defined custom groups or you've localized existing group names.

Example: 

'groups' => [
    'rollback'=>[
        'name'=>'Reversor',
        'abbr'=>'R',
    ],
],
view_about_link Display about link. Defaults to true
userlist_enabled Disable search for userlist. Defaults to false. It allows to search for user lists in the users section, by typing a page that contains a list of user links, or a category page that contains user pages.
robots_policy Robots policy (for search engines). Example: noindex,nofollow
hits_available Do we have hits (pageviews) information available?. Defaults to true.
stats_join_revision_actor_temp use the revision_actor_temp table when building stats (need to set it to true for pre-1.35 wikis)
stats_join_revision_comment_temp use the revision_comment_temp table when building stats (need to set it to true for pre-1.35 wikis)
log_sql Log sql queries to the query_log table of dbg. Set it to false to not log queries, specially if you're not using a multi wiki site
live_hours Array with the available hours in the last hours (live) section. Example: [24, 48].

Multi wiki site

edit

The Wikiscan tool was created originally for use on a single wiki site; it was subsequently expanded to serve all WMF wiki sites under the wikiscan.org domain. In the multi wiki setup option, each wiki site has its own Wikiscan database, along with an additional database containing data to coordinate all wiki sites. Under this multi wiki setup, this tool makes heavy use of undocumented and WMF-specific database views, and also it has connections with Wikidata.

This multi wiki behavior is controlled by the 'multi' configuration parameter, that defaults to false.

Supported languages

edit

Currently supported interface languages:

  • English (en)
  • French / Français (fr)
  • Spanish / Español (es)

To add more languages, copy or add a file in include/languages/ with the desired ISO language code, translating from an existing one.

Known issues

edit

Populating statistics for the first time

edit

To update all daily stats and user stats, execute : 

php ctrl/run.php fullupdate
php ctrl/run.php fullupdate_months

To update last hours, execute : 

php ctrl/run.php update_live

To update grid, execute : 

php ctrl/run.php update_sum

Update statistics periodically

edit

To update statistics regularly you can set up cron jobs to run several scripts:

  • ctrl/run.php update_live: Updates section hours and grid. Run it several times a day. It's fast.
  • ctrl/run.php update_sum: Updates calendar and users section. Run it at least once a day.
  • ctrl/run.php update_misc: Updates userstats graphs. Run it at least once a week.
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikiscan&oldid=6067689"