Wikimedia Release Engineering Team/SpiderPig/Decision log

Date Category Decision Why? Comments
2024-08-27 Technical/Scope Limiting initial project scope to remove the queue and pre/post deployment claim/release. And all logged in users will be able to view deploys in progress. This limits the scope of development to only a web-based version of scap backport and that's it. Also, this may be hard to get right by guessing how users will use the tool. Winnowing scope/YAGNI. Decided in 1-on-1 with Tyler Cipriani + Ahmon Dancy
2024-09-05 Technical/Implementation Decided to have the web front-end communicate via API calls do a persistent jobrunner daemon. These two processes will communicate via a job queue/database. Allows web API calls to remain quick and light-weight while providing persistent monitoring over long-running processes. Decided in "SpiderPig Struggle" meeting.
2024-09-13 Technical/Scope Acknowledging that should the persistent jobrunner daemon process fail during a deployment, in process jobs will not be recoverable. The jobs themselves are a part of a separate process group and will outlive the daemon process failure (but will likely fail once they attempt to write to the closed file descriptors that were owned by the jobrunner). Solving this problem expands scope of the jobrunner for little benefit. A robust jobrunner should not fail often. This is a cost-benefit trade-off. Context on scap!433 + followup conversations.
2024-09-26 Technical/Dependencies Attempt to use Codex for frontend code, pairing with Eric Gardner from design systems for a two week effort Provides us with a familiar, Official™ look and feel to our tools. Design systems is available for initial support. Decided in "SpiderPig Struggle" meeting.
2024-10-17 Technical/Scope Scap backport refactoring (e.g., T362987) can wait until after the initial alpha version of SpiderPig is complete. We would still like to do this, but later, possibly Q3 or Q4. Two reasons:
  • This is the current behavior of scap backport on the command line and it's working OK.
  • The interaction interface of the spiderpig-api is sufficient for the portion of the refactor meant to provide actionable feedback to deployers.
Decided in "SpiderPig Struggle" meeting.
2024-10-28 Technical/Dependencies Use fastapi to implement the SpiderPig apiserver.
  • It is similar to Flask, so easy to learn and use
  • Has built-in handling of SQLAlchemy data models, which we're already using for the jobrunner database.
  • Good docs
  • Deployable via uvicorn, which it includes as a dependency
Decided during prototyping.