Open main menu

Extension talk:Semantic Drilldown

Database PostgreSQL: Error 42601 Syntax-ErrorEdit

I run a SMW (2.4.1) MW (1.27.1) with PHP (7.0.7) and PostgresSQL (9.4.9)

As long as I set only one filter in my drilldowninfo-tag everything on Special:BrowseData is fine. But as soon as a second (or third) filter the 42601 ERROR occur: 42601 ERROR: syntax error at or near "using" LINE 2: ...AS id, (IF(o_blob IS NULL, o_hash, CONVERT(o_blob using utf8... My guess. PostgreSQL hast no CONVERT()-Funktion. I also posted this for a while at Phabricator

Backtrace:

#0 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/db/DatabasePostgres.php(448): DatabaseBase->reportQueryError('ERROR: syntax ...', '42601', '\tCREATE TEMPORA...', 'DatabaseBase::q...', false)

  1. 1 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/db/Database.php(901): DatabasePostgres->reportQueryError('ERROR: syntax ...', '42601', '\tCREATE TEMPORA...', 'DatabaseBase::q...', false)
  2. 2 /storage/srv/www/htdocs/mediawiki-1.27.1/extensions/SemanticDrilldown/includes/SD_Filter.php(427): DatabaseBase->query('\tCREATE TEMPORA...')
  3. 3 /storage/srv/www/htdocs/mediawiki-1.27.1/extensions/SemanticDrilldown/specials/SD_BrowseData.php(970): SDFilter->createTempTable()
  4. 4 /storage/srv/www/htdocs/mediawiki-1.27.1/extensions/SemanticDrilldown/specials/SD_BrowseData.php(1187): SDBrowseDataPage->printUnappliedFilterLine(Object(SDFilter), '/developer/inde...')
  5. 5 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/specialpage/QueryPage.php(619): SDBrowseDataPage->getPageHeader()
  6. 6 /storage/srv/www/htdocs/mediawiki-1.27.1/extensions/SemanticDrilldown/specials/SD_BrowseData.php(127): QueryPage->execute('Digitalisat')
  7. 7 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/specialpage/SpecialPage.php(479): SDBrowseData->execute('Digitalisat')
  8. 8 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/specialpage/SpecialPageFactory.php(576): SpecialPage->run('Digitalisat')
  9. 9 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/MediaWiki.php(282): SpecialPageFactory::executePath(Object(Title), Object(RequestContext))
  10. 10 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/MediaWiki.php(745): MediaWiki->performRequest()
  11. 11 /storage/srv/www/htdocs/mediawiki-1.27.1/includes/MediaWiki.php(519): MediaWiki->main()
  12. 12 /storage/srv/ww

w/htdocs/mediawiki-1.27.1/index.php(43): MediaWiki->run()

  1. 13 {main}

Any suggestion or solutions? Thanks a lot!

What's the full SQL call, do you know? Yaron Koren (talk) 14:18, 19 January 2017 (UTC)
CREATE TEMPORARY TABLE semantic_drilldown_filter_values AS SELECT s_id AS id, (IF(o_blob IS NULL, o_hash, CONVERT(o_blob using utf8))) AS value FROM "smw_di_blob" JOIN "smw_object_ids" p_ids ON "smw_di_blob".p_id = p_ids.smw_id WHERE p_ids.smw_title = 'Sachbereich' --M art in (talk) 14:27, 19 January 2017 (UTC)
On Referata there is a similar bug - but more than one filters are possible there --M art in (talk) 14:53, 19 January 2017 (UTC)
Thanks for pointing out that bug on Referata, though it's a different bug. Yes, the issue does seem to be CONVERT() with PostgreSQL - that function exists, but it has different syntax and (I think) behavior. I just checked in what I think are fixes for both bugs - if you get the latest code, hopefully it should work now. Yaron Koren (talk) 04:35, 20 January 2017 (UTC)
Thanks for the fast fix. The Problem with CONVERT was solved but now there is stll a database error 42883 function if (boolean, text bytea) does not exist

Seems there are still some problems with PostgreSQL. The full SQL-query is this:

CREATE TEMPORARY TABLE semantic_drilldown_filter_values AS SELECT s_id AS id, (IF(o_blob IS NULL, o_hash, o_blob)) AS value FROM "smw_di_blob" JOIN "smw_object_ids" p_ids ON "smw_di_blob".p_id = p_ids.smw_id WHERE p_ids.smw_title = 'Sachbereich' Could you please have a look again? --M art in (talk) 08:23, 23 January 2017 (UTC)

Ah, I didn't realize that IF() was a problem too... I just checked in another fix, so hopefully now it works. Yaron Koren (talk) 18:04, 24 January 2017 (UTC)
Thank you very much for the fix! Works great now with postgresSQL. I really appreciate your patience with stupied users like me and your time you have to spend therefore. --M art in (talk) 08:44, 30 January 2017 (UTC)
Great. Sorry about the problems! Yaron Koren (talk) 22:12, 30 January 2017 (UTC)

Free text searchEdit

Hi. Is there a way to add a free-text search box pointed only to the filtered pages? I am developing a smw about Supreme Court orders fron my country, by now I have about 80k documents. Filters are great to narrow down the search, but a search box would be very helpfull. Greetings.

No, unfortunately. If you want this feature, you may have to install the Cargo extension (which I also wrote) - it is an alternative to Semantic MediaWiki that contains a drilldown feature, closely modeled after Semantic Drilldown, but which also supports free-text search. Yaron Koren (talk) 15:28, 25 January 2017 (UTC)
Thanks a lot. When you say alternative to smw, does it mean that both extensions cannot work together?
They can run on the same wiki, although each one stores data in its own location, and they can't query each other's data. Yaron Koren (talk) 17:03, 25 January 2017 (UTC)

Filtering Data type propertyEdit

I have a property wich is Date type. It works fine but, when I set $sdgMinValuesForComboBox=1; the filter doesnt give any result. I would thank any idea..

That's strange. What if you set it to 2? Yaron Koren (talk)
No matter the value, the moment ComboBox is activated for date filter, it doesnt work anymore.
Without ComboBox, when I click on a date, the follow line is generated: ...&Fecha_de_publicacion=febrero_2010 (and works)
With ComboBox: ...&_search_Fecha_de_publicacion%5B0%5D=febrero+2010 (and it doesnt work)
Well, that's a bug in the extension. I guess I never tried setting that variable to a value low enought that it would kick in for dates, so I never saw the issue. Out of curiosity, why do you want a combo box to appear, and not individual links, even if there are only a few values? Yaron Koren (talk) 19:43, 25 January 2017 (UTC)
Date ComboBox creates a group for each month, I have documents from 84 different months. So I would need to set up a very high value for $sdgMinValuesForComboBox. Also there is another property (key word) that has more that a hundred values. So, a ComboBox is more functional for me.
There may be 84 months, but only up to 12 months are shown at a time in the drilldown interface. Yaron Koren (talk) 20:37, 25 January 2017 (UTC)
You are right. I just set up $sdgMinValuesForComboBox=13. I'll let you know how it worked. Thanks.
It took a while but the date filter is already showing only 12 groups. Thanks.

PerformanceEdit

Hi. I have a Category with 42k pages already loaded and it will grow fast. By now, it takes one minute to load the drilldown page. I wolud appreciate any suggestion to keep a good performance. Thanks.

I'm surprised that it's taking that long to load - it may be one specific filter that is causing the problem. You could try removing the filters one at a time, to try to isolate the problem. In any case, another option is to install and use the Cargo extension, which provides its own very similar drilldown interface - either instead of or in addition to SMW+SD. Cargo has a more straightforward database structure, so I'm guessing that it's drilldown would be faster for your data. Yaron Koren (talk) 14:57, 7 February 2017 (UTC)

New release?Edit

I have seen that several fixes especially concerning the querying for dates were authored since the last 2.0.2 release. Thus it will be cool to have a new version. --[[kgh]] (talk) 16:42, 1 May 2017 (UTC)

It would be immensely useful to have a Composer version so that it can be integrated with the rest of the Semantic MediaWiki suite of tools. This is the last major extension from Extension:Semantic Bundle that still has to be installed manually without Composer.

"Special:Filters" and "Special:Create filters" missing after upgradeEdit

Hi, I've upgraded my MW installation:

FROM
MediaWiki           1.19.23
PHP                 5.3.3 (apache2handler)
MySQL               5.1.73
Semantic MediaWiki  1.7.0.2
Semantic Drilldown  1.1
TO
MediaWiki           1.23.15
PHP                 5.3.3 (apache2handler)
MySQL               5.1.73
Semantic MediaWiki  2.4.1
Semantic Drilldown  2.0.2

The wiki behaves ok, all existing filters are working as expected. However, both filter-related special pages seem to be gone. I've already run the SMW "Data repair and upgrade" multiple times but to no avail. Can this be a configuration setting issue in LocalSettings.php or some missing dependency? Any pointers welcome.

TIA --Nakohdo (talk) 15:07, 16 June 2017 (UTC)

Yes, this is not a bug - in version 2.0, those two special pages were removed, because filters are now defined in category pages, as opposed to in their own separate pages. (The "Filter:" pages are still supported, but they will stop being supported at some point, so I'd suggest converting everything over to use #drilldown_info.) Yaron Koren (talk) 15:31, 16 June 2017 (UTC)
Many thanks for this information, Yaron. This helps a lot. Nakohdo (talk) 13:04, 19 June 2017 (UTC)

#drilldownlink: Syntax for "filters" settingEdit

Hi,

I don't quite understand the correct syntax for the "filters" parameter of #drilldownlink. The documentation says:

#drilldownlink
filters - the set of filters to apply, in the format "a=b&c=d&..."

Whereas the syntax for #drilldowninfo looks different:

#drilldowninfo

{{#drilldowninfo:filters=Author (property=Was written by), ...

Could you perhaps provide a working example for #drilldownlink with filters set? Many thanks in advance.
--Nakohdo (talk) 09:34, 30 June 2017 (UTC)

Here's one example. Yaron Koren (talk) 14:03, 30 June 2017 (UTC)
Many thanks for the quick reply. However, that's an example for #drilldowninfo where the documentation is quite clear. I'm struggling with #drilldownlink. Nakohdo (talk) 14:19, 30 June 2017 (UTC)
Oh, oops. I can't think of any public examples of #drilldownlink, but for that earlier example you could have a parameter like "filters=Language=English&Data type=Education". Yaron Koren (talk) 18:46, 30 June 2017 (UTC)

Is the drilldown info cached somewhere?Edit

I am running into an issue I can't remember experiencing with Drilldown. No matter what I change in the #drilldowninfo tag in my category, the Special:BrowseData page still shows an earlier version of the drilldown filters. It is behaving as if the drilldown filter info was cached somewhere. --Lalquier (talk) 12:30, 12 March 2018 (UTC)

I don't think so... I can't think of why that would be happening for you. Yaron Koren (talk) 13:20, 12 March 2018 (UTC)
That's very odd. I was hoping a full rebuildData would clear things up but it didn't. For some reason, the filters displayed on the Special:BrowseData page are not the same filters that are defined in the category page.I will let you know if I figure this one out. --Lalquier (talk) 10:15, 14 March 2018 (UTC)
Update - I even tried to delete the call to #drilldowninfo in my category page and I can still see the Special:BrowseData page for that category, so it is really cached somewhere. I tried restarting the web server, forcing an exclusion to the mediawiki cache for that page, clearing my browser cache with no result. Very odd. --Lalquier (talk) 11:23, 15 March 2018 (UTC)
Update 2 - so it took a few days but the cache was eventually cleared on its own and the changes were applied. I still don't know which cache is causing such a delay between mediawiki, apache and the browser but at least I know it sorts itself out eventually. --Lalquier (talk) 01:22, 16 March 2018 (UTC)

Properties set by a parser hook cannot be filteredEdit

Property filtering works When setting {{#set::Has type=Text}} or [[Has type::Text]], but it doesn't when we set the property types through an customized parser hook. IE: {{#someparserhook: type=Text}}. Any idea why?

No idea. When you use the parser hook/function, does the data get set correctly? Does it show up in query results, for instance? Yaron Koren (talk) 00:42, 26 March 2018 (UTC)
A quick update after some further digging. Data gets set correctly, and the culprit is not the parser function but a query. If on a property page we put an ask-query like {{#ask: [[somerelationto::ThePropertyPage]]}}, filtering on that property ceases to works. Remove the query from the property page, and it still doesn't work until on the Special:SMWAdmin page we run the 'Schedule disposal' task. The filtering works again, until we put back the ask-query on the property page. I had wanted to show this on the SMW sandbox, but SemanticDrilldown is not installed there. --Remco de Boer 12:53, 27 March 2018 (UTC)

DB Unexpected ErrorEdit

I run a SMW (2.5.8) MW (1.31.0) with PHP (7.0.31)

When going to https://dicoado.org/dico/Sp%C3%A9cial:BrowseData I get this error :

[W5gpIHkNzHsXxe7F9uBXeAAAAJA] /dico/Sp%C3%A9cial:BrowseData/Classe_grammaticale Wikimedia\Rdbms\DBUnexpectedError from line 3769 of /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/includes/libs/rdbms/database/Database.php: TemporaryTableManager::queryWithAutoCommit: Expected mass commit of all peer transactions (DBO_TRX set).

Backtrace:

#0 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/extensions/SemanticDrilldown/includes/TemporaryTableManager.php(31): Wikimedia\Rdbms\Database->commit(string)
#1 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/extensions/SemanticDrilldown/specials/SD_BrowseData.php(230): TemporaryTableManager->queryWithAutoCommit(string, string)
#2 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/extensions/SemanticDrilldown/specials/SD_BrowseData.php(1150): SDBrowseDataPage->createTempTable(string, NULL, array, array)
#3 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/includes/specialpage/QueryPage.php(642): SDBrowseDataPage->getPageHeader()
#4 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/extensions/SemanticDrilldown/specials/SD_BrowseData.php(129): QueryPage->execute(string)
#5 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/includes/specialpage/SpecialPage.php(522): SDBrowseData->execute(string)
#6 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/includes/specialpage/SpecialPageFactory.php(568): SpecialPage->run(string)
#7 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/includes/MediaWiki.php(288): SpecialPageFactory::executePath(Title, RequestContext)
#8 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/includes/MediaWiki.php(861): MediaWiki->performRequest()
#9 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/includes/MediaWiki.php(524): MediaWiki->main()
#10 /home/clients/0486a60c95a26f96e318d8715f689dd7/web/wiki/index.php(42): MediaWiki->run()
#11 {main}

I tried a rebuild data. Even completely deleted all SMW data and also ran the MW update.php maintenance script. I really don't know what the cause of that problem is. Any hint ? DSwissK (talk) 20:47, 11 September 2018 (UTC)

I concur, tried MariaDB and Mysql 5.7.32, same issue
It's the same thing here with MW1.30.0 SMW 3.0.0 SD 2.0.2 Carchaias (talk) 09:44, 19 October 2018 (UTC)
Same with me: MW 1.31.1, MariaDB 10.2.15, SMW 3.0.0, SD 2.0.2

Compatibility with SMW 3?Edit

Hi Yaron, is current version of Semantic Drilldown compatible with SMW3? I tried to upgrade an older wiki from SMW2/Drilldown 1.4 to MW1.31 and SMW3/Drilldown latest master via git and it does not appear to work.

Return to "Semantic Drilldown" page.