Topic on Project:Support desk/Flow

I am trying to add pages to various new categories on my Wiki, however it does not seem to be working at all. At the bottom of any given pages, say "Computer A" I want to be in category "Desktops"

So I add [[Category:Desktops]] to the bottom and go to Category:Desktops and nothing shows up.

I have another problem where "Computer B" Shows up in a category, despite no [[Category:XX]] being defined.

What am I missing?

Example: I want these three pages in http://wiki.ssdcougars.tv/index.php?title=Category:SETV_Machines

http://wiki.ssdcougars.tv/index.php?title=Advanced_Edit_Tower

http://wiki.ssdcougars.tv/index.php?title=Archive/Owncloud_Server

http://wiki.ssdcougars.tv/index.php?title=Wowza_Server

but no matter what I do, Category: SETV Machines is empty!

Well, apparently this is not an automatic, or at least an immediate process. I fixed it by going to my wiki/maintenance directory and running rebuildall.php from terminal.

Is there a way to make this process automatic? Why is this not an immediate process? To update category pages when a link has been added to a page?

Updates are handled by the job queue, make sure you have it set up correctly.

OK, so It wasn't entirely solved. I checked the job queue, set up a cron job, and manually ran runJobs.php but there are still several pages of mine with a template that has a category attached, and only 2 of the 10 articles I need in that category are in that category, and ShowJobQueue shows no jobs queued.

I have tried, with one article, like over 10 times to add a transcluded template with a category to an article, and no matter what I change, how many times I save it, the category does not show up on the page.

But it shows up on two other pages that have the template, but not the other ones I want to edit?

How can I ensure that when I edit an article, the changes are immediately propagated? If this is not a viable process, then I am scrapping MediaWiki in favor of something more practical and immediate.

You can also try to set up $wgRunJobsAsync to false in case running jobs asynchronously is a problem on your setup.

Monitor the job table to see if page requests are picking up jobs from there.

Hi i have the same problem how did you fixed your issue ?

I have the same problem. Running rebuildall.php doesn't help, running populateCategories.php doesn't help, turning off async jobs doesn't help, running runJobs.php manually until all jobs are out of the queue doesn't help. For me, it affects pages in the File namespace only, but it affects all of those pages.

(MW 1.27.1, PHP 7.0.8, MySQL 5.7.16)

I have solved this myself using a combination of the touch script in pywikibot and a loop that forces the job queue to run while the bot works.

./pwb.py touch -start:! -ns:File -pt:0

while true; do php ./runJobs.php --quiet; sleep 5; do php ./showJobs.php

I'm also having this problem, and it's compounded by pages that appeared in categories disappearing after a week or so. The bot-and-runJobs solution by 71.36.99.196 repopulates categories, as does manually editing and saving missing articles, but articles still disappear within a few days. MW 1.27.5, PHP 7.0.32, MySQL 5.7.24. I have runJobs running every 20 minutes on a cron job.

This is most likely caused by one of your extensions, that causes an error when populating categories when jobs are being run.

Creating custom log groups for "error" and "exception" as described in Manual:How to debug#Creating custom log groups would help determining if an error is happening when inserting data to those tables.

I've enabled logging and have reviewed the logs over the last several days. No extensions are reporting errors.

The logs do report hitting the database limit for maximum concurrent connections. This doesn't align with usage; MySQL max connections are the default 151, we never have more than 4 active editors working simultaneously, about 99.9% of the time our active connection count is <10, and these spikes don't correlate to editor activity. Increasing MySQL's max connections doesn't help; MediaWiki always consumes every available connection during these spikes.

It seems to occur consistently within a few seconds of anything involving PHP session management, including account logins and signups and external PHP session cleaning. We cache sessions to the DB with $wgSessionCacheType = CACHE_DB;.

We'll investigate alternative session caching options in the long term, and shorter session lengths to mitigate in the short term.

Thanks for your help!

It isn't sessions. We've moved them out of the DB to memcached and are still seeing inexplicable spikes, and confirmed these spikes are causing the catalog listing updates to fail.

When you say 4 active editors working simultaneously, did you take into account possible visitors of your site? If your wiki is exposed to the internet, it may be hit by webcrawlers like google, bing, or bad bots that analyze your website looking for ways to insert spam. Some of them may crawl your site at a high rates, possibly causing that connection spike

We aggressively cache rendered content for visitors not logged in because of the conditions you describe, and we do not open registration to the public. As expected, these database spikes do not correlate with visitor or crawler activity, and also occur on a test environment not exposed to the internet.

MediaWiki is periodically spawning dozens of database connections that promptly go to sleep for 20+ seconds. These pile up into a rolling thundering herd of 100-120 persistent connections; individual connections last 20-300 seconds each, but the thundering herd lasts anywhere from 1-10 minutes at a time due to new long-lived connections and are not correlated to visitor or editor activity. Moving sessions out of the DB prevents us from hitting the connections cap, which appears to resolve the category update issue.

Because the actual query and execution times are all still short (typically < 1 second), they don't appear in MySQL's slow queries log.

I've collected three days' worth of MySQL query logs (via slow query logs set to 0-second tolerance so we get timestamps for query durations) in order to determine which queries are doing this.

Do you have a clue where are those queries coming from?

The culprit appeared to be the use of $wgForeignFileRepos with ForeignAPIRepo to fetch images from a separate commons wiki.

In some cases (apparently most often around searches), API requests for an image on the commons wiki which would be immediately followed by database query for the image on the local wiki, back and forth in rapid succession. Combined with live search suggestions in the search bar, this could generate piles of API requests (4-8 per keystroke) whenever a user searched for something in the File: namespace, which on our wikis occurs pretty frequently.

Disabling $wgForeignFileRepos altogether has so far eliminated these spikes.

Uh, I also experienced that slam of api requests when pointing my test wiki (mirror of production wiki) to the production wiki for images... But not (only) in image search, but any page load that contains images.

Not great! I've added notes about these apparent caveats to Manual:$wgLocalFileRepo and Manual talk:$wgLocalFileRepo. Glancing at the bug tracker there are apparently changes to this feature in 1.31, which we are still quite far from moving toward, so I'm not confident that I can file a useful bug report.

So how exactly do you disable $wgForeignFileRepos?

Remove all config related to $wgForeignFileRepos and $wgUseInstantCommons from your LocalSettings.php file

PFWOz here coming back a year later to note that disabling wgLocalFileRepo resolved the DB spikes, but we have never resolved the core issue here of categories never populating correctly, and 1.31 did not resolve it either.

Our categories are partially useless because they almost never fully repopulate, and I'm stuck running constant bot touch jobs and running the runJobs maintenance script every minute just to prevent categories from completely depopulating. Even then I still have to go in from time to time with a logged-in user and save a null edit to get an image to populate into the category. We're almost always reporting 0 jobs in the queue with showJobs but also still have constant reports of incomplete categories. (And again, this only seems to affect our File namespace and no others.)

We're just resigned to this never working. It's extremely frustrating because image categorization could, and should, be one of our most useful features. Instead it's broken across the site.

This is tracked on phab:T26575