Manual:Stats
The Stats library (added in MW 1.41) attempts to better define the interface for generating metrics in MediaWiki.
Global Configuration
editThe global configuration is in MainConfigSchema.php
and can be overridden in LocalSettings.php
.
$wgStatsFormat
is the output format all metrics will be rendered to.- Default
null
which disables metrics rendering. - Supported options are
statsd
,dogstatsd
,null
- See
Wikimedia\Stats\OutputFormats::SUPPORTED_FORMATS
for an up-to-date list of options.
- Default
$wgStatsTarget
is the URI the metrics will be forwarded to. E.g.udp://127.0.0.1:8125
- Default
null
which disables sending metrics.
- Default
$wgStatsPrefix
is the prefix which will be applied to all generated metrics. Required. DefaultMediaWiki
.
Metric Types
editThe supported metric types are:
- CounterMetric
- An only-ever-incrementing counter.
- Great for tracking rates.
- Implements
increment()
andincrementBy($number)
- GaugeMetric
- A settable value.
- Implements
set($number)
- TimingMetric
- Observes timing data.
- When the backend is configured to do so, histograms of timing metrics are generated.
- Implements
observe($number)
,start()
, andstop()
- Can call
start()
many times; call tostop()
can throw ifstart()
not called first.
Requesting a Metric
editUse a getter to get a metric from the StatsFactory
.
getCounter($name)
getGauge($name)
getTiming($name)
A Simple Example
editLet's create a counter for tracking each time a function is called:
// Get the StatsFactory from MediaWikiServices
$statsFactory = MediaWikiServices::getInstance()->getStatsFactory();
// Make a CounterMetric
$myCounter = $statsFactory->getCounter('function_calls')
->setLabel('function_name', 'my_function');
// increment the counter providing the namespace and the name of the function call
$myCounter->increment(); // StatsD output: "MediaWiki.function_calls.my_function 1"
StatsD Metric Namespace
editGenerated StatsD metrics follow a predictable pattern:
$output = implode( '.', [ $wgStatsPrefix, $component, $name, $label_key_1, $label_key_2, ...etc ] );
For example:
// assuming $wgStatsFormat = 'statsd' and $wgStatsPrefix = 'mediawiki'
$namespace = 'Draft';
MediaWikiServices::getInstance()->getStatsFactory()
->getCounter( 'function_calls' )
->setLabel( 'name', 'my_function' )
->setLabel( 'namespace', $namespace )
->increment();
would create a metric namespace:
mediawiki.function_calls.my_function.Draft
Features
editStatsFactory->withComponent()
editReturns a new StatsFactory instance with a component field appended to the globally-configured prefix. Intended for components and extensions, this feature isolates metrics generated by the component to their own namespace.
For example, a metric bar
without a component would look like:
mediawiki.bar
Adding a component foo
would make the same metric look like:
mediawiki.foo.bar
MetricInterface->copyToStatsdAt()
editWhen StatsFactory is configured with an IBufferingStatsdDataFactory instance, metrics will be copied to the legacy interface at the provided namespace. Intended to ease the transition to metrics generated by this library. For example:
// assuming:
// $wgStatsFormat = 'statsd'
// $wgStatsPrefix = 'mediawiki'
// $wgStatsTarget = 'udp://new_statsd_server:8125
// $wgStatsdServer = 'old_statsd_server:8125'
// $wgStatsdMetricPrefix = 'mediawiki'
MediaWikiServices::getInstance()->getStatsFactory()
->getCounter( 'function_calls' )
->setLabel( 'name', 'my_function' )
->setLabel( 'namespace', 'Drafts' )
->copyToStatsdAt( 'legacy_namespace.my_function.calls' )
->increment();
would send to the new_statsd_server
:
mediawiki.function_calls.my_function.Drafts
and to the old_statsd_server
mediawiki.legacy_namespace.my_function.calls
MetricInterface->setSampleRate()
editConfigures the metric to emit a subset of samples recorded. Takes a float: 0.0 (sends 0% of samples) to 1.0 (sends 100% of samples). Note: sample rate must be configured prior to recording any samples otherwise an IllegalOperationException will be thrown. This can be encountered inadvertently because metrics pulled from cache may have samples already recorded.
Notes
editCardinality
editHigh cardinality metrics present challenges for service operators and consumers of timeseries data. It is recommended to avoid using unbound values in labels or names.
Examples of high-cardinality data include:
- IDs and UUIDs
- Usernames
- IP Addresses
- User agents
- Page titles
Recommendations
editLabels
editFor StatsD output, declaration of label order matters. Take care to declare labels in the order you would like them to appear. Label setting order does not matter when the Metric instance is pulled from cache.
Metrics
editThe Observability team recommends following the guidance published by the Prometheus project. TL;DR: A metric:
- should not use string interpolation to set the metric name
- should not have label keys repeated in the metric name
- must measure a single unit (i.e. do not mix seconds with milliseconds, or seconds with bytes, etc.)
- should use base units (e.g. seconds, bytes, meters, etc.)
- should have a suffix describing the unit in plural form. (i.e. _seconds, _bytes, _total, etc.)
- must always have consistent label keys across all measurements.
- should represent the same logical thing being measured across all label dimensions - sum() or avg() across all label dimensions should be meaningful.
Patterns
editRecursion
editUse of TimingMetric->start()|stop()
helpers is unsupported in cases of recursion. When this is unavoidable, track the time separately:
$startTime = microtime( true );
# <do work>
$statsFactory->getTiming('foo')
->observe( ( microtime( true ) - $startTime ) * 1000 ); # expects ms
See also: phabricator:T368073
mw.track()
editSupport for metrics generated by mw.track()
needs implementation. See: phabricator:T355837
PerDbNameStatsdDataFactory
editSee: phabricator:T359436. In short, the prefixing data factory pattern was not re-implemented. The recommended path forward is for metrics users to fetch the needed datapoint and set the label with setLabel()
.
Developers
editLocal Testing in Docker
editAdd statsd-exporter service to docker-compose.override.yml
:
services:
statsd-exporter:
ports:
- "9112:9112"
image: docker.io/prom/statsd-exporter:v0.22.2
command: "--web.listen-address=:9112"
Configure LocalSettings.php
:
# statsd-exporter target config
$wgStatsFormat = 'dogstatsd';
$wgStatsTarget = 'udp://statsd-exporter:9125';
Scrape the metrics endpoint:
$ watch 'curl -s localhost:9112/metrics| grep mediawiki_'
Note that the production statsd-exporter configuration may differ from the default set and make the metrics render differently, especially timing metrics. Please refer to the production statsd-exporter configuration to get the most accurate Prometheus metrics representation.