Topic on Talk:Quarry/Flow

I try to check some database queries most days, for example quarry:query/2934 which lists inappropriate redirects on English Wikipedia. Lately, however, the queries have returned outdated results. For example, today, query 2934 returned the following pages:

• https://en.wikipedia.org/w/index.php?curid=50269921
• https://en.wikipedia.org/w/index.php?curid=50268324
• https://en.wikipedia.org/w/index.php?curid=50266585
• https://en.wikipedia.org/w/index.php?curid=49764691
• https://en.wikipedia.org/w/index.php?curid=50267795
• https://en.wikipedia.org/w/index.php?curid=49763842
• https://en.wikipedia.org/w/index.php?curid=50267797
• https://en.wikipedia.org/w/index.php?curid=50288389
• https://en.wikipedia.org/w/index.php?curid=50287650
• https://en.wikipedia.org/w/index.php?curid=50269728

All of the pages have been deleted, but the query is not supposed to return deleted pages. Guessing that someone might have deleted the pages after the database report was generated but before I had opened the pages, I clicked on 'submit query' a second time, but I still got the same result set. Is there a lag in the database so that changes on Wikipedia do not immediately show up in the copy of the database which Quarry uses?

Yep, same here. Judging by Bernsteinbot results, queries are looking at a version of the database from 03:31, 24 April 2016 (UTC).

Now seems to be OK again, it's caught up with itself.

I retried with quarry:query/2934. Still 10 results. 8 of them appeared in the result set earlier today and should not be in the result set as the pages have been deleted.

Yes, it now seems to be stuck around 1700 UTC yesterday having briefly been working OK. If you have a query that adds articles frequently, you may want to look at rev_timestamp to see what the latest one is. I don't know if this is related to the new Texas data centre perhaps? https://wikitech.wikimedia.org/wiki/Switch_Datacenter#Schedule_for_Q3_FY2015-2016_rollout

quarry:query/2934 still returns 10 pages, partially the same ones as yesterday. Many of those should already have disappeared from the result set before it was generated yesterday, so the database appears to be partially stuck at some point well before 17:00 yesterday.

The same problem: quarry:query/2208 (still returns 1 page deleted few weeks ago) and quarry:query/2211 (still returns 1 page created few weeks ago).

The odd page showing up in queries long after it was deleted may be a separate issue to the general lag that Stefan and I were seeing a week ago. I get it sometimes - for instance 8938 has several at the moment created by User:Mandawariya's. I think this is related to the category problems I reported 2 months ago where some categories enter a kind of limbo until the database is purged or they get a WP:NULLEDIT. If they get deleted before the db can be purged or they get a null edit, then they stick around on queries and it seems there's no way to get rid of them other than by creating and deleting them again. That's my guess, anyway.

You can't recreate a page_id, so your suggesion to create and re-delete won't work. If you undelete a previously deleted page, or recreate it, then it will get a new page_id which is different from the old page_id.

quarry:query/8403 now reports that there are 193,737 files on English Wikipedia without templates, although there normally are fewer than ten of those. Something seems to be wrong with the database.