User:ASarabadani (WMF)/Database for developers toolkit/How-to/Monitor

Depending on what is the change, you might want to look at different things:

Graphs edit

  • MySQL aggregated dashboard in Grafana is good for checking the general db load. Make sure you check max lag per shard, if you triggered a lot of writes.
  • MySQL dashboard is good for checking health status a certain database host but this is rare and you should not need this.

Logs edit

  • Database errors dashboard in logstash gives you a good overview of queries that errored.
  • Slow queries dashboard in logstash gives a flexible view of queries that might cause issues for database.
    • The current minimum is 3 seconds for read queries and 1 second for write queries.
      • You can increase the minimum by adding a filter on "actualSeconds" value
    • They are grouped by normalized query, you can remove or zoom to have only one group.
  • DBPerfromance channel can be used to get all types of performance-related queries. For example, making connection to primary database in a GET request.
  • If you want to check what queries have been made in a reproducible request. Enable WikimediaDebug, make those requests with "Verbose logging" enabled and then check mwdebug dashboard in logstash and filter for "DBQuery" channel.