Developer Satisfaction Survey/2024/Cloud Services
📖 Developer Satisfaction 2024 Report
☁️ Cloud Services edit
tl;dr
- Which tools – Compared to 2022, the number of respondents working on on-wiki gadgets more than doubled (39%, up from 14%).
- Which wikis – The same as 2022, the majority of respondents' tools were build to work with Wikipedia, Wikidata, Wikimedia Commons, or all Wikimedia projects.
- Toolforge – The same as 2022, the majority of Toolforge users use Python 3, NodeJS, or PHP; the majority use Git; and the majority have almost all of their Toolforge work done locally (as opposed to remotely on Toolforge).
- Cloud VPS – The same as 2022, the most common reason given for using Cloud VPS was hosting one or more tools or other public services; and the majority said they do not use NFS to access the same files across different servers.
- Services experience – The same as 2022, more than two-thirds of respondents agreed that services provided by Cloud Services have high uptime, that it is easy to have code run on Toolforge or Cloud VPS.
# Cloud services use
We asked survey takers, “Do you use Toolforge or Cloud VPS to run or develop tools?”
Of those who answered the question:
- the majority (51.8%) said they use Toolforge (either exclusively or with Cloud VPS)
- 30.6% said they use neither
- 20.2% said they use Cloud VPS (either exclusively or with Toolforge)
# Previous cloud services use
For survey takers who said that they use neither Toolforge nor Cloud VPS, we asked, “Have you previously used Toolforge and/or Cloud VPS?” Almost two thirds (63.5%) said they had never either.
# Which tools
For survey takers who said that they use Toolforge and/or Cloud VPS, we asked, “What kind of tools do you work on?” Respondents were able to select multiple tools. Of the 108 who responded, a majority (73.1%) said web apps, and a majority (63.0%) said bots.
# Which wikis
For survey takers who said that they use Toolforge and/or Cloud VPS, we asked, “Which Wikimedia projects are your tools built to work with?” Respondents were able to select multiple tools. Of the 108 who responded, the majority (74.4%) said their tools were built to work with Wikipedia; while 43.6% said Wikidata, 38.5% said Wikimedia Commons, and 38.5% said all wiki projects.
# How many tools developed
For survey takers who said that they use Toolforge and/or Cloud VPS, we asked, “How many tools have you developed on Toolforge or Cloud VPS?” Of the 108 who responded, a majority (77.8%) said they have developed 1-5 tools.
# How many tools maintained
For survey takers who said that they use Toolforge and/or Cloud VPS, we also asked, “How many tools do you actively maintain on Toolforge or Cloud VPS?” Of the 108 who responded, a majority (78.7%) said they actively maintain 1-5 tools.
# Hours spent on tool maintenance
For survey takers who said that they use Toolforge and/or Cloud VPS, we also asked, “In the past year, on average, about how many hours per week did you spent developing or maintaining tools on Toolforge or Cloud VPS?” Of the 106 who responded, a majority (62.3%) said they spent about 1-5 hours per week.
# Storage/caching
For survey takers who said that they use Toolforge and/or Cloud VPS, we also asked, “What storage/caching services do you use when running your tools?” Respondents could select more than one service.
Of the 108 who responded, while there was no majority response, the two most common responses were MySQL/Maria DB (40.7%) and no storage/caching service (38.9%).
☁️🔨 Cloud Services: Toolforge edit
# Toolforge tenure
For survey takers who said that they use Toolforge, we asked, “How many years have you used Toolforge?” Of the 88 who responded, the majority (89.8%) had at least two years of experience using Toolforge. The most common response was 2-3 years (35.2% of respondents).
For survey takers who said that they use Toolforge, we asked, “Which source control mechanism do you use to manage your tools’ source code?” Of the 88 who responded, the majority (92.0%) said they use Git.
# Programming languages
For survey takers who said that they use Toolforge, we asked, “Which programming languages do you use on Toolforge?” Respondents could select more than one language.
Of the 88 who responded:
- the majority (63.6%) said they use Python 3
- 46.6% said they use PHP
- 33.0% said they use NodeJS
- smaller percentages said they use Java, Perl, Python2, Mono/.NET/C+, Ruby, or something else
# Local development
For survey takers who said that they use Toolforge, we asked, “When you develop a tool, how much of your work developing code to run on Toolforge is done locally on your machine (as opposed to remotely on Toolforge)?” Of the 88 who responded, the majority (62.5%) said almost all of the work is done on their local machine.
☁️🦄 Cloud Services: Cloud VPS edit
# Cloud VPS tenure
For survey takers who said that they use Cloud VPS, we asked, “About how many years have you used Cloud VPS?” Of the 50 who responded, there were a wide variety of responses; but the majority (76%) said they have used Cloud VPS for less than eight years.
# Cloud VPS use purpose
For survey takers who said that they use Cloud VPS, we asked, “What do you use Cloud VPS for?” Respondents could select more than one use case. Of the 50 who responded, there were a wide variety of responses; but the majority (74%) said they use Cloud VPS for hosting one or more tools or other public services.
# Cloud VPS NFS use
For survey takers who said that they use Cloud VPS, we asked, “Do you use NFS to access the same files across different servers?” Of the 50 who responded, there were a wide variety of responses; but the majority (60%) said they do not.
☁️✍️ Cloud Services: Documentation and Services edit
# Documentation experience
We asked survey takers how much they agreed with the following statement: “Cloud Services documentation is clear.”
- 55% said they agree
- 23% said they neither agree nor disagree
- 16% said they disagree
- 6% were unsure
We asked survey takers how much they agreed with the following statement: “Cloud Services documentation is easy to find.”
- 66% said they agree
- 13% said they neither agree nor disagree
- 15% said they disagree
- 6% were unsure
We asked survey takers how much they agreed with the following statement: “Cloud Services documentation is up-to-date.”
- 51% said they agree
- 21% said they neither agree nor disagree
- 17% said they disagree
- 11% were unsure
# Services experience
We asked survey takers how much they agreed with the following statement: “Services provided by Wikimedia Cloud Services, including Toolforge and Cloud VPS, have high uptime.”
- 83% said they agree
- 7% said they neither agree nor disagree
- 5% said they disagree
- 5% were unsure
We asked survey takers how much they agreed with the following statement: “It is easy to have code run on Toolforge or Cloud VPS.”
- 67% said they agree
- 19% said they neither agree nor disagree
- 12% said they disagree
- 2% were unsure
We asked survey takers how much they agreed with the following statement: “I feel that I am supported by the Cloud Services team when I contact them via cloud@lists.wikimedia.org, the #wikimedia-cloud IRC channel, or Phabricator.”
- 66% said they agree
- 14% said they neither agree nor disagree
- 16% said they disagree
- 4% were unsure
We asked survey takers how much they agreed with the following statement: “I receive useful information via the “cloud-announce” and/or “cloud” mailing lists.”
- 51% said they agree
- 25% were unsure
- 13% said they neither agree nor disagree
- 11% said they disagree
☁️👪 Cloud Services: Community and Support edit
# Mailing list subscriptions
We asked survey takers, “Do you subscribe to the Cloud mailing list?” and “Do you subscribe the Cloud-Announce mailing list?”
The majority of respondents (56.1%) subscribe to one or both of the mailing lists. More specifically:
- 51.4% subscribe to the Cloud-Announce mailing list
- 40.2% subscribe to the Cloud mailing list
# Discussion channels
We asked survey takers, “When discussing WMCS issues with other users, what channels do you use?” Respondents could select more than one channel. Of the 108 respondents who answered, a majority said they use Phabricator (54.6%) as well as IRC (50.9%).
We asked survey takers, “When seeking help from WMCS staff, which support channels do you use.” Respondents could select more than one channel. Of the 108 respondents who answered, again a majority said they use Phabricator (73.1%) as well as IRC (62.0%).
# Feedback
We asked survey takers, "If the Wikimedia Foundation could improve one thing in Toolforge in the next year, what should that be?" Some themes emerged from the answers.
Observability and Logging
(36% - 18 out 49 responses)
Dashboard to control each tool's work. If launched by cron, when they start running, how long and a spetial dash for failed jobs and the faillure error messages
Provide a recommended logging solution for tools so that we can easily stop writing logs to NFS home directories.
From my own experience, memory limiting (when processes are OOMKilled) is hard to detect and hard to tell what the cause is.
Build-service
(20% - 10 out 49 responses)
Build-service is seeing more adoption and users want expansion in what it offers.
The Build Service is very promising and I'm looking forward to seeing more work in that area (particularly T334587 "[builds-api] Add triggering support"). It cleanly handles all of my use cases, even the most complicated ones. Aside from this, a Cloud Services Team-supported central place to get tool uptime alerts would be helpful. Having to rely on external services to get tool uptime alerts is difficult and could lead to failure, as depending on the service, they may not have the same uptime or dedicated staff support that Toolforge or Cloud Services gets.
Communication
Inclusion of "meanwhile on Toolforge"-esque items in the Tech News newsletter so as to update non-frequent (non-incrowd) users of the general goings-on, instead of only reporting major stuff...
The following themes emerged when we asked "Is there anything you have tried but not been able to do in Toolforge?"
Logging
(10% - 3 out of 29 responses)
Logging was the most mentioned.
I've always wanted to have a way to push/access/view structured logs, since all of my tools use some form of structured logging. It would allow me to shift production debugging from a terminal to a browser, where I can more easily navigate through a heap of logs when one of my tools break down. How this would actually work in practice, I have no clue, as I don't know enough about OpenSearch to know if this is feasible or how access control would work.
BYOC - Bring Your Own Container
Using custom docker images
When we asked, "If the Wikimedia Foundation could improve one thing in Cloud VPS in the next year, what should that be?", the following themes emerged.
Monitoring
(18% - 3 out of 17 responses)
Statistics/alerts/metrics for my projects
Quota Management
(12% - 2 out of 17)
Make it easier to increase/decrease temporarily team/project quotas.
When we asked "If you have any other comments about WMCS, please share them with us here", some users are appreciative of the services WMCS provides
Thank you, the service has made it possible to do work that would have been much more difficult on other cloud providers!
Others were not too happy with the communication around the just-ended grid migration process
I'm considering moving off WMCS following the Grid shutdown, not because the deprecation and migration itself but because the communication surrounding it...
Some users want a faster process on abandoned-tool policy (i.e., taking over abandoned projects)
I think it should be much easier to take over projects. The current orphan policy is too slow and requires too much effort from those that want to fix things and too little responsibility on the missing maintainers.