Beta Cluster/2014-15-Q3

Measures of Success

edit

From the WMF RelEng Jan 2015 quarterly review:

  • Green nightly builds - T426
    • Set up multiversion on Beta Cluster - T67127
    • Make Beta Cluster stable over night (no puppet updates?)
    • Select a subset of known-useful browser tests
    • Run those browser tests against the nightly build
    • Publicly report metrics (graphs) of success over time
  • Stable uptime metrics
    • Pick a set of useful data sources of uptime
    • Ensure we're recording those
    • Make a dashboard of them
    • Define alert levels and enable alerts
  • By-team test history
    • (this is enabled by a gooder Beta Cluster)
    • Group build test history by engineering team
    • Report on those to the teams and engineering wide
    • Make a dashboard of these as well (to get a quick overview of overall test health)

Notes

edit
edit