As it turned out, Wikilabs is crap. Some sysop simply disabled CatScan without taking in mind, that in the German WP this tool is an integrated part of the category system and most categories even cannot be used since the German WP is using a faceted classification system, for instance it has very very big categories for the gender but then all person categories don't distinguish the gender. As an example, there is no w:en:Category:German female singers but there is only w:de:Category:Musiker (Deutschland) and w:de:Category:Frau and you need CatScan for intersecting the results. Thus CatScan is linked from any category description page. Thank you, dear unknown wikilabs sysop, for disabling the work with the German WP cat system. And don't tell me about resources: it was promised that wikilabs can do what the toolserver did. The result is: Wikilabs can't manage this. Wikilabs is crap. --Matthiasb (talk) 18:14, 11 October 2013 (UTC), very displeased
Talk:Wikimedia Labs
There was scheduled maintenance that took longer than expected. We've never promised that labs would never have maintenance periods where it would be down. We've promised that if a failure of some variety occurs there will be resources there to fix the issue. Toolserver has had unplanned outages that have lasted weeks at a time, so I think we're doing quite a bit better from that regard.
The weeks long outages of the toolserver happened years ago, before the toolserver capacity was doubled, correct me, when? 2007? Now it seems that 1) obviously there was (and perhaps still is) no skill at wikilabs how many resources are needed for the tools previously working on the toolserver and 2) a lack of knowledge about the importance of the several tools. Especially for a German WP user Catscan is the most important tool and in case of insufficiencies that tool should be among the latest to disable and again between the first to enable. Indeed all the bot stuff and other gimmicks should have lower priorities than tools which are part of the normal WP interface, as for example sulinfo (@ quentinv57) and contributions (@ luxo) – these are both still at the TS – or the stimmberechtigung tool (or other language specific "election" tools), then the category tools as above mentioned catscan_rewrite (@ magnus) or catgraph (@ dapete, still @ TS). If this is not possible or not appreciated I propose to put those tools permanently back on the toolserver.
If you take a look through the Technical pump archives, you'll notice a pretty normal pattern of the Toolserver being down for days or weeks at a time this year, here's a thread discussing it being down for two weeks in May. Check out most of the archives for the tech pump over the last couple years and you'll notice frequent day long outages or week long outages. Of course, this thread isn't a Toolserver bash fest. Toolserver needed hardware and it needed proper staffing.
Tool labs has had relatively few unplanned outages and this particular case was scheduled maintenance that took 11 hours longer than planned due to the amount of data that's being generated inside of the project. If you are interested in scheduled outages because it impacts your work, feel free to subscribe to labs-l for announcements of maintenance.
Neither Toolserver nor Wikimedia Labs are scoped for fully production workloads and there's an expectation in either infrastructure that tools and bots may be unavailable for extended periods of time on occasion.
Regarding your proposal on order of disabling tools: all tools were disabled at the same time because the underlying filesystem used for them was taken down for maintenance. The underlying filesystem was moved to another system to ensure better long-term stability.
Wait, hold on. Are you talking about this catscan tool? It seems it isn't working right now, but the labs maintenance was completed quite a while ago. If the tool isn't working you should contact the tool's maintainer (Magnus) and tell him it's broken.
It would have been massively more helpful if you would have stated the tool was broken rather than going on a rant...
Hi Matthiasb, are you subscribed to mail:labs-l? That's the list where you get all updates concerning Wikimedia Labs and Tool Labs. On Oct 8th, there was this thread about CatScan started: http://lists.wikimedia.org/pipermail/labs-l/2013-October/001726.html. It was not "just" disabled by an unknown admin but Coren informed the list and is in discussion about CatScan with Magnus to help debugging. There is nothing wrong with this, on the contrary, it's helpful.
And to both of you (and everybody else): To be honest, I think the comparisons between Toolserver and Tool Labs don't help at all, they just polarize the discussion and are not constructive. This discussion already is difficult enough. By keeping to compare directly, I'm afraid you may end up hurting the people who are more deeply involved in one of the two projects. If you ask me, we should stop this comparison and move forward to get the best out of the new infrastructure we got.
I'm afraid you may end up hurting the people who are more deeply involved in one of the two projects. -> Well, that damage already was done by the way the WMF decided and communicated that they will eventually take out the toolserver and force users onto labs, just another instance of how far nowadays the WMF is estranged from the Wikipedias' and other projects' communities and the smaller, and the more exotic a project seems to be, the lesser developped is the communication between them and the WMF – but that's a totally different nonetheless important issue.
I am happy to hear that Magnus Manske has been informed of the problem with his tool and is hopefully successful resolving the situation.
The page says,
'You can create a Labs account to do MediaWiki development, tools, or analytics by simply filling out the account creation form.'
But when I submit that account creation form, I get the error message mentioned in this note from the page bottom:
'Note:
- The error message "There was either an authentication database error or you are not allowed to update your external account." generally indicates an invalid shell account name was used (see bug 16524).'
For shell account name I tried 'roger', 'rogehc' and 'rogervoy' but none worked; all produced the error indicated in the above note.
How can I create an account? I do not have shell access nor obviously any shell account name. Is that a prerequisite?
It worked this evening however.
Yes, there was a problem caused by an LDAP upgrade which has since been fixed.
For those who wanted the goals, feature justification, & process notes that Ryan took out when he (reasonably) streamlined the page, see this version.
Hi, For the moment, I'm testing my extension with phpunit and tomorrow with selenium I have to install a special software on the server. So I created a container for that in openvz. Now, the extension is also in Gerrit. My extension has it the conditions to create an new instance (only during test) in Wikimedia Labs ? Can I use a container built with OpenVz in Wikimedia Labs ? Thanks
We only support KVM virtual machines at the moment. You can start working in Labs during development phases as well. You don't need to wait until you've finished it. You'll need to request an account and a project.
The project is in Gerrit : mediawiki/extensions/LinkedWiki and I have my account in gerrit and Wikimedia Lab. What next ? Thanks
Can you drop into the #wikimedia-labs channel on freenode (IRC) so that we can discuss this in more real-time?
I have an instance set up, but as I recall it was not possible to create an "instance" without being part of a "project", and adding me to a project required manual intervention.
This is the intention. By default you have no access. You must request a project, or ask to be added to a project by a current member. Also, you must be in the sysadmin role to create/delete instances. Someone already in the sysadmin role of a project can add you to the role.
The concept is for Labs to be a community, and for the community to maintain most actions.
Could Ryan, CT, or Andrew please add a rough Labs timeline either to the roadmap on this page or to the general engineering roadmap page? That'll make it easier for me to help volunteers understand when they'll, for example, be able to get a turnkey MediaWiki development environment. Thanks!
Is there a timeline for when bots may be able to run on the cluster? I ask because one of our most active anti-vandalism bots is down because the volunteer was unable to afford hosting. And there's been a request to see if the bot could be run on the Tool Labs. Thanks.
Hi, we are now in process of setting up this part of labs, does the operator have access to wikimedia svn? It's possible to run a bot there but with a lot of limitations
I'd also like to know how feasible it would be for me to move CorenSearchBot there. It's currently running on my own infrastructure, and I have no problem with doing so indefinitely, but since that bot is being relied on by the project for some critical copyright work, it'd probably be best if it lived somewhere someone could take over if I could no longer manage it.
That said, it's not lightweight; at peak hours it can eat up a whole IA64 core's worth of processing power. Coren 15:59, 6 December 2011 (UTC)
if you could step by in wikimedia-labs it would be cool (freenode)
@Petrb, I just wanted to thank you very kindly for all your work here and especially for your help with the bots. I've made an extra donation to the WMF in your honor. Thanks very much.
Thank you very much for support :-) actually I am a volunteer not a wmf employee, but I will try to do best in order to get it working soon
It states We'd like to have an environment where anyone can eventually become root, even on our production cluster. Ummmm. I guessing/hoping there are all sorts of safeguards in place and that root access to the prodcution cluster would be only for highly trusted users that have a specific need for root. I was just hoping for a bit of clarification on that. Other than that, I think this sounds like a great idea and think it could be a huge benefit to the community. Thanks.
This post was posted by Peachey88 (Flood), but signed as Hydroxonium.
Well, just from having a labs account, it's possible to make root level changes in the production cluster. We have our puppet configuration in a public repository, and having an account allows contributors to push changes to the repository. When the changes are code reviewed, they can be deployed to production.
Otherwise, though, we already have volunteers with root access on our production cluster. Our beginnings were founded in volunteers being root, and I'd like to continue that tradition. Over time, by contributing to the labs environment, contributors should be able to gain privileges. I believe that it should be possible for well trusted contributors to be given higher privileges, including privileges in the production cluster.
There are a number of safeguards in the labs environment to safely give out higher levels of permissions over time.
Thanks for the reply, Ryan. I also must thank you, and everybody else at the WMF, for all the support you've given us. I think this will be really helpful for our bot operators and the volunteers that write all those useful tools.
I knew we had a few volunteers with root, I just didn't realize that things were that open. I guess it makes sense since we're a volunteer run organization. Anyhoo, thanks for the clarification. Best regards. - Hydroxonium (T•C•V) 02:25, 6 December 2011 (UTC)