Extension talk:Cargo/Archive January to May 2022

[ANSWERED] MySQL/MariaDB database user privileges required by Cargo

I know that the normal thing to do is GRANT ALL PRIVILEGES, but I'm thinking of reducing that and would like to know which privileges are required by Cargo for (a) the main database and (b) its own Cargo database. See Topic:Wnht96uupemgrsef. Thanks. Jonathan3 (talk) 11:55, 5 January 2022 (UTC)

For the main DB, no special privileges. For the Cargo DB, probably just the ones you listed (SELECT, INSERT, UPDATE, LOCK TABLES, DELETE), plus CREATE TABLE. Yaron Koren (talk) 21:16, 5 January 2022 (UTC)

Thanks. I will try this out eventually. I guess that nobody bothers with this and database rights have never been a concern. Jonathan3 (talk) 10:39, 3 February 2022 (UTC)

Datatype "Monolingual text"

Hello,

Does an equivalent to SMW Datatype "Monolingual text" exist with Cargo ?

No... would it be useful? Yaron Koren (talk) 15:14, 10 January 2022 (UTC)

The need is to change the field label like in This is a property of type [[Has type::Text]].

Language labels: [[Has preferred property label::Role@en]] [[Has preferred property label::Rolle@de]]

Where is that "Has preferred property label" property used? Yaron Koren (talk) 20:25, 11 January 2022 (UTC)

There is a description at "www.semantic-mediawiki.org/wiki/Help:Special_property_Has_preferred_property_label#Example"

I've seen that page before, but I don't really understand it - it doesn't seem to explain what the actual effect on the display, or user interface, is. Where in the interface is the "preferred property label" used? Yaron Koren (talk) 18:38, 13 January 2022 (UTC)

Hello, I found an exampla at"https ://en.demo.bluespice.com/wiki/Property:IMS_Standard_specification?action=edit" 2001:861:4745:7590:B92C:C726:9A2D:6AF5 05:07, 3 July 2022 (UTC)

That is indeed an example of "Has preferred property label", but again, where are the values used? Yaron Koren (talk) 15:07, 5 July 2022 (UTC)

Table created, nothing stored, no matter via calling template or directly calling parser function

The table "Level" holds no data, however, the multiple pages has already called templates that store data. Directly calling #cargo_store also does not work. I tried re-creating the table, or deleting and creating the table again, neither of which can fix. -- SolidBlock (talk) 09:28, 12 January 2022 (UTC)

Where is the actual #cargo_store call? The one right in that template is commented out. Yaron Koren (talk) 14:47, 12 January 2022 (UTC)

I undid the comment in that template. Besides, this page manually calls #cargo_store however the data “Level” is still empty. SolidBlock (talk) 03:35, 13 January 2022 (UTC)

After uncommenting #cargo_store, I think you just need to recreate the table - you can do that from Special:CargoTables. Yaron Koren (talk) 15:31, 13 January 2022 (UTC)

However, unfortunately, no matter how many times I recreate the table, the issue is not fixed. And pages storing that data has no data shown to be stored in the "?action=pagevalues" page. -- SolidBlock (talk) 08:27, 14 January 2022 (UTC)

Are you sure that #cargo_store is actually getting called? It's inside a rather complicated #ifeq statement, and I'm not sure if it ever gets called. Yaron Koren (talk) 13:43, 14 January 2022 (UTC)

Yes. It works normally before I initially recreate the table. Besides, I manually called #cargo_store in my sandbox page. It does not work, while another table works normally. SolidBlock (talk) 06:22, 15 January 2022 (UTC)

Same problem but more information here. When running runJobs.php it outputs things like:

2022-04-28 08:01:08 cargoPopulateTable dbTableName=Tests replaceOldRows= namespace=0 title= requestId=ec6edb748f3def8f77d2adaf (id=1177,timestamp=20220428074453) STARTING

2022-04-28 08:01:08 cargoPopulateTable dbTableName=Tests replaceOldRows= namespace=0 title= requestId=ec6edb748f3def8f77d2adaf (id=1177,timestamp=20220428074453) t=0 error=InvalidArgumentException: The given PageIdentity does not represent a proper page

...

When running Cargo maintenance script cargoRecreateData.php it outputs:

PHP Notice: Undefined property: stdClass::$page_namespace in /opt/mediawiki/includes/Title.php on line 3158

PHP Notice: Undefined property: stdClass::$page_title in /opt/mediawiki/includes/Title.php on line 3158

... Artsanzz (talk) 08:19, 28 April 2022 (UTC)

What version of Cargo are you running? Yaron Koren (talk) 13:46, 28 April 2022 (UTC)

branch REL1_37. Afterwards I changed to version 3.2 (master branch), then cargoRecreateData.php works fine, table successfully created; but runJobs.php still complains (don't know if this is still a problem, or should I just manually delete these broken jobs?) Artsanzz (talk) 14:44, 28 April 2022 (UTC)

Great. What errors are you seeing with runJobs.php? Yaron Koren (talk) 15:29, 28 April 2022 (UTC)

Same as above. You know what, I have just tried to manually clear the SQL job table and performed some recreating from web interface, and now everything goes fine without errors. Artsanzz (talk) 16:03, 28 April 2022 (UTC)

Great! Yes, the REL branches should always be avoided for Cargo. Yaron Koren (talk) 16:55, 28 April 2022 (UTC)

Could the reason be the exception of the replace table? Zes M Young (talk) 12:06, 19 January 2022 (UTC)

I deleted the cargo table again. However, in the template page, it said "replacement table is generated". However, when clicking the link to the page of replacement table, it saids that the table is not found. The MediaWiki version is 1.37.2 and the Cargo version is 3.2 (f811d9b). SolidBlock (talk) 01:58, 12 May 2022 (UTC)

More issues seem to encounter. See this page. SolidBlock (talk) 05:14, 20 June 2022 (UTC)

and another issue related to this

The following Lua throws MW internal exceptions, which began in recent days:

mw.ext.cargo.query('Level','Level.firstcameversion,Level.stars,Level.name_zhs')

SolidBlock (talk) 14:42, 21 January 2022 (UTC)

Maybe that's related. What are the exceptions? Yaron Koren (talk) 16:56, 21 January 2022 (UTC)

[993445ad01d110ae8272c772] 2022-01-22 14:07:00: Fatal exception of type "TypeError" -- SolidBlock (talk) 14:07, 22 January 2022 (UTC)

Could you please add $wgShowExceptionDetails = true; to your LocalSettings.php file? That will hopefully lead to a more useful error message. Yaron Koren (talk) 16:04, 24 January 2022 (UTC)

I have no permission to do that and have filed a task on miraheze phab. You may refer to it. SolidBlock (talk) 10:10, 25 January 2022 (UTC)

I'm not sure whether it is related. However, if I query other Cargo tables, including tables that work normally and tables that do no exist at all, an internal exception (not Lua exception) is thrown. Maybe that is some issues within the database?

=mw.ext.cargo.query('Level','Level.firstcameversion,Level.stars,Level.name_zhs')
[1f02547a699e28f815fd8b8f] Caught exception of type TypeError
mw.ext.cargo.query('Lev3el','Level.firstcameversion,Level.stars,Level.name_zhs')
[7b8dd1c87baefebc7452dbb0] Caught exception of type TypeError
=mw.ext.cargo.query('fffLevel','Level.firstcameversion,Level.stars,Level.name_zhs')
[2348b74fd7e685852918fc97] Caught exception of type TypeError
=mw.ext.cargo.query('Level')
[1fef44d640dfdc5483b47678] Caught exception of type TypeError
=mw.ext.cargo.query('Sandbox')
[48b50fee204846eb5de0b82c] Caught exception of type TypeError
=mw.ext.cargo.query('Version')
[54a0b10f8ab6ad9225be0020] Caught exception of type TypeError
=mw.ext.cargo.query('Version', 'Version.name')
[cf88d330f2037cd29ad95d89] Caught exception of type TypeError
=mw.ext.cargo.query
function
=mw.ext.cargo.query()
[83c8fd0afa3e275cc80d186c] Caught exception of type TypeError

—SolidBlock (talk) 02:05, 12 May 2022 (UTC)

Sort by date in Special:Drilldown

Is it possible to do this? I've tried using |_drilldownTabs=Table (format=table;fields=...;order by=Date) to the template but to no avail. Any ideas? Thanks. Jonathan3 (talk) 11:59, 13 January 2022 (UTC)

Just to clarify, I know the table is sortable - I just wondered whether there is a way to set a default sort column. Thanks. Jonathan3 (talk) 15:55, 13 January 2022 (UTC)

Exhibit display format supported?

Creating a query with display format exhibit results in a table of rows of headers only, though with proper itemid labels (i.e. pulled from the data). I am pulling from one table, one field only, with cargo data format = page. Display and source code contains none of the query data, though the export/toolbox widget button contains the data as expected. Multiple "Error: Missing property ID at position 1" js errors.

Is the Exhibit display format still maintained, or do I have something missing in my query?

{{#cargo_query:
tables=Classes
|fields=Classes.Name
|limit=20
|offset=0
|format=exhibit
|view=tabular
|topunit=millisecond
}}

--JStallings29 (talk) 20:30, 16 January 2022 (UTC)

That works fine for me - what versions of MediaWiki and Cargo are you running? Yaron Koren (talk) 01:36, 17 January 2022 (UTC)

MW 1.37.1, Cargo 3.0 JStallings29 (talk) 01:56, 21 January 2022 (UTC)

Duplicate rows again, but this time only in Special:Drilldown

I have a List (,) of String. Most pages have one value for that field, but some have two.

On the drilldown page, when I select two values as filters, any page with those two values is listed twice. The drilldown page works correctly in other respects and Special:CargoTables doesn't show any duplicate rows.

This is on Cargo 3.0 (07f465d) and MW 1.35.5. I've not noticed the same thing on a MW 1.34.4 wiki and Cargo 3.0 (55f3f72). Jonathan3 (talk) 12:19, 17 January 2022 (UTC)

If it helps, a page which matches three selected values on one filter field is displayed three times (ie on three rows). I guess this would apply to any number. Jonathan3 (talk) 10:37, 3 February 2022 (UTC)

I've downloaded the latest code today and it's still happening. Maybe it's something simple that I've missed. Jonathan3 (talk) 22:59, 12 February 2022 (UTC)

Query File column without HTML

Hi, first of all, this extension is awesome and makes our life really easy.

I have a column with the Type File which I query for a single File. Is it now possible to get the Image without the HTML surrounding's because I'd only need the filename, like it is stored? I would really like to keep the type of the column on file because of the nice view in the table view.

Thank's for getting back to me.

EDIT: I have found a solution. If I use format=template and just output the value, it work's fine. But if there is another, better solution, I'd be happy to hear from it . --DesignerThan (talk) 23:49, 30 January 2022 (UTC)

That's great! One semi-hack you can do is replace FieldName in the "fields=" parameter with CONCAT(FieldName) - then it will be interpreted as type String and not File. Yaron Koren (talk) 15:55, 31 January 2022 (UTC)

Display Format: Dynamic Table with hidden fields

Hi, I have encountered another thing, I don't know how to deal with it. Just wanted to use a query with format set to dynamic table where I want to use hidden fields. The thing now is, that it seem's so that only the last column can be set to hidden... I have this query here on a public wiki.

If someone could help me with it, that would be great. --DesignerThan (talk) 02:05, 3 February 2022 (UTC)

Sorry for the delay. I just tried "hidden fields=", and it works fine for me. What specifically are you trying to do, that isn't working? Yaron Koren (talk) 02:30, 18 February 2022 (UTC)

Sorry for my delay too. I just have looked into the discussion page again and seen your answer. I am basically just testing some stuff to see what is possible, because I'll need to create an index page later on.

I just did test it again on the linked page, you can have a look at the code there. But it is still the same. Hidden fields is still just working for the last column. I have added the Skill Level column and that is still showing up and there is no toggle link being generated. --DesignerThan (talk) 17:01, 22 March 2022 (UTC)

Sorry for writting again. Do you have ideas on that topic? Or do you need further information? Would be really thankfull about some help. DesignerThan (talk) 18:11, 3 April 2022 (UTC)

Scribunto : Empty fields missing in mw.ext.cargo.query results table

Hello, I am upgrading my installation from Mediawiki/Cargo 1.35.1/2.7.1 to 1.37.1/3.1.

I am using scribunto for a Cargo query. If I display the resulting table, I see that in the old version, the table has the same number of elements as fields in the Cargo query, whereas in the new version, fields with an empty value do not appear in the result table. Is there a way to get the same behaviour as before?

Best, --Ludovic Strappazon (talk) 16:39, 7 February 2022 (UTC)

Display Format: Dynamic Table with column widths

Hello, thanks for the new features in 3.1, in special "column widths" for dynamic tables. I just updated to 3.1 (b186d58) 08:12, 9. Feb. 2022. I always get an error when I try to set a value for the first column other than empty (i.e. before the first comma). The error seems to come of the library. Tested it on different tables with different number of columns.

DataTables warning: table id=DataTables_Table_0 - Requested unknown parameter '9' for row 0. For more information about this error, please see http://datatables.net/tn/4

And a second "error" or strange behavior I noticed: This setting for column widths acts on every cargo_query on a page and not only the one, where I added that parameter.

Brabi81 (talk) 07:38, 11 February 2022 (UTC)

Sorry about these problems; I believe they're both fixed now, in the latest Cargo code. Yaron Koren (talk) 14:40, 22 February 2022 (UTC)

Great! Just updated to the most recent version and everything I tested right now works without errors. Thank you. Brabi81 (talk) 11:58, 23 February 2022 (UTC)

Is Cargo right for this use case?

I'm hoping to use Cargo, but it's not clear to me from the docs I've read here whether or not it's appropriate for my use case. Specifically, at least in some cases, I do not want the typical user to be able to insert new records into the DB (or update or delete existing ones). From the docs, it seems like if I install Cargo and define Cargo table declarations within the wiki, then anyone with permission to edit or create a wiki page can just put cargo_attach and cargo_store calls into some random page, and voila, new database records.

Am I misunderstanding? If not, is there some way to prevent this, short of not letting users edit wiki pages, making the underlying DB read only, or things of that nature? For example, maybe there are $wg settings to define related permissions? I see there's one for create/recreate table, but I didn't see one for create/update records. -Rwv37 (talk) 04:06, 16 February 2022 (UTC)

I'm not sure I understand the question. Let's say that there's a template, "Person", which stores to a Cargo table called "People". If a regular user creates a new page that calls that "Person" template, should that page's data not be stored in the Cargo table? If so, when should that data get added to the table? Yaron Koren (talk) 04:28, 16 February 2022 (UTC)

Essentially, at least in some cases, what I want Cargo for is to pull data out of a database and easily use it in a wiki. Normal users would be able to use things like cargo_query to do whatever they want with the data, but they would not be able to use things like cargo_store to do whatever they want to the data. -Rwv37 (talk) 04:34, 16 February 2022 (UTC)

And to more specifically answer your actual questions: No, the typical user should not be able to insert data into the database, no matter whether through Cargo or otherwise. As for what should get data added to the table: Doesn't really matter, as long as only privileged users can do it. If there's a way to accomplish that in Cargo without allowing unprivileged users to do it too, great. But if not, regular direct DB access through SQL or whatever. -Rwv37 (talk) 04:39, 16 February 2022 (UTC)

Sounds like External Data Extension [1] is what you are looking for. I use it in addition to Cargo to pull of data of external MariaDB and display it. Of course on the MariaDB is restricted access and only reads are possible with External Data. Brabi81 (talk) 06:13, 16 February 2022 (UTC)

Thanks! I'll check it out. -Rwv37 (talk) 06:18, 16 February 2022 (UTC)

Field type "Wikitext string" missing?

There should be field type "Wikitext string", see [2].

I am using latest 3.1 version & Page Forms and it seems that no such type exists. Is it bug or feature? :-) And if feature, what is an alternative for short text in wikiformat?

--Radouch (talk) 16:40, 19 February 2022 (UTC)

I only ever use Text and String (maybe I should use their wikitext alternatives). Their contents are displayed properly (e.g. bold text) in content pages/queries, but displayed as raw wikitext (e.g. '''bold text''') on the Special:CargoTables page. Jonathan3 (talk) 00:17, 20 February 2022 (UTC)

The "Wikitext string" type has existed for a long time, but it wasn't showing up in Page Forms' Special:CreateTemplate and Special:CreateClass (I assume that's what you are referring to) due to a bug in Cargo - I just checked in a fix for it. Yaron Koren (talk) 04:36, 21 February 2022 (UTC)

Use MATCHES in query and template format

When I use the table format, it shows a relevant extract.

When I use the template format, the _fullText=Search results field displays the WHOLE page!

Is there something I need to do to fix this? Thanks. Jonathan3 (talk) 22:38, 24 February 2022 (UTC)

Display internal Cargo errors when invoking #cargo_store?

There are some bugs lurking somewhere in my chain of templates that eventually lead to a call to #cargo_store. However, these are always very tedious to debug because the errors don't seem to be displayed anywhere and are silently swallowed; the only symptom being that the row declared in an article isn't being added to the database.

I seem to vaguely recall that back when I was first setting up Cargo on our wiki, I could run rebuilds of cargo databases from the command line using cargoRecreateData.php and would get errors there that were helpful for debugging, but by now rebuilding the tables takes hours and that's not really an option.

1- is there a way to display the errors that show up on command line from #cargo_store in the wiki UI anywhere? Turning on the following flags didn't include anything useful (lots and lots of junk output, but nothing *useful*): wgShowSQLErrors, wgDebugDumpSql, wgDebugToolbar, wgShowDebug, wgShowExceptionDetails, wgShowSQLErrors

2- Failing that, is there any php script that can be ran on the command line that recreates a single article's data, so that any associated errors can easily be seen?

Thanks! Teltura01 (talk) 19:10, 5 March 2022 (UTC)

Why aren't my calls to {{#if}} working anymore?

I'm working on a query template that will allow entries to any one of 7 fields. In the past, I had this working with a {{#if}} parser call inside the cargo_query call. Now all I get is DB query errors. Here's what I'm doing:

{{#cargo_query:
tables=Repertoire
|fields=_pageName=RepertoirePage, Level, _pageID
|where= 
{{#if:
  {{{LevelLow|}}}
  |Level >= '{{{LevelLow|}}}' AND
}}
{{#if:
   {{{LevelHi|}}}
   |Level <= '{{{LevelHi|}}}' AND
}}
{{#if:
   {{{Tonic|}}}
   |Tonic= '{{{Tonic|}}}' AND
}}
{{#if:
   {{{Mode|}}}
   |Mode= '{{{Mode|}}}' AND
}}
{{#if:
   {{{Tempo|}}}
   |Tempo= '{{{Tempo|}}}' AND
}}
{{#if:
   {{{Period|}}}
   |Period = '{{{Period|}}}' AND
}}
{{#if:
   {{{Mood|}}}
   |Mood= '{{{Mood|}}}' AND
}}
_pageID > 0
|format=dynamic table
}}

The if parser was only including the where clause if the user had entered something into that field in the query form. The final -pageID>0 was a hack to close any trailing ANDs. This worked like a charm before (and I had to look this solution up on your Talk Archive from 2018 - and it was my solution!) Any reason why the parser if isn't working inside of cargo-query now? May214 (talk) 20:49, 17 March 2022 (UTC)

What are the DB error messages you're getting? Yaron Koren (talk) 21:43, 17 March 2022 (UTC)

Getting this: [YjPdaeMDSgi5VaCf-l8MpwABDQE] 2022-03-18 01:16:25: Fatal exception of type "Wikimedia\Rdbms\DBQueryError"

I've set $wgDebugDumpSql = true; in LocalSettings, but I'm not sure where to find the log of the SQL errors. May214 (talk) 01:29, 18 March 2022 (UTC)

Please add "$wgShowExceptionDetails = true;" and "$wgShowSQLErrors = true;" to LocalSettings.php. Yaron Koren (talk) 03:06, 18 March 2022 (UTC)

Thank you

Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '{#if:\x0A 4\x0A |Level >= '4' AND\x0A}}\x0A{{#if:\x0A 6\x0A |Level <= '6' ' at line 1 (localhost) Function: CargoSQLQuery::run Query: SELECT `mwhi_cargo__Repertoire`.`_pageID` AS `cargo_backlink_page_id_Repertoire`,`_pageName` AS `RepertoirePage`,`Level` AS `Level`,`_pageID` AS `_pageID` FROM `mwhi_cargo__Repertoire` WHERE {{#if: 4 |Level >= '4' AND }} {{#if: 6 |Level <= '6' AND }} {{#if: |Tonic= '' AND }} {{#if: |Mode= '' AND }} {{#if: |Tempo= '' AND }} {{#if: |Period = '' AND }} {{#if: |Mood= '' AND }} _pageID > 0 ORDER BY `mwhi_cargo__Repertoire`.`_pageID`,`_pageName`,`Level`,`_pageID` LIMIT 100 Backtrace: from /home4/pianoiki/public_html/wiki/includes/libs/rdbms/database/Database.php(1809) #0 /home4/pianoiki/public_html/wiki/includes/libs/rdbms/database/Database.php(1793): Wikimedia\Rdbms\Database->getQueryException(string, integer, string, string) #1 /home4/pianoiki/public_html/wiki/includes/libs/rdbms/database/Database.php(1768): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string) #2 /home4/pianoiki/public_html/wiki/includes/libs/rdbms/database/Database.php(1327): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean) #3 /home4/pianoiki/public_html/wiki/includes/libs/rdbms/database/Database.php(2012): Wikimedia\Rdbms\Database->query(string, string, integer) #4 /home4/pianoiki/public_html/wiki/extensions/Cargo/includes/CargoSQLQuery.php(1588): Wikimedia\Rdbms\Database->select(array, array, string, string, array, NULL) #5 /home4/pianoiki/public_html/wiki/extensions/Cargo/includes/parserfunctions/CargoQuery.php(102): CargoSQLQuery->run() #6 /home4/pianoiki/public_html/wiki/includes/parser/Parser.php(3407): CargoQuery::run(Parser, string, string, string, string) #7 /home4/pianoiki/public_html/wiki/includes/parser/Parser.php(3092): Parser->callParserFunction(PPTemplateFrame_Hash, string, array) #8 /home4/pianoiki/public_html/wiki/includes/parser/PPFrame_Hash.php(273): Parser->braceSubstitution(array, PPTemplateFrame_Hash) #9 /home4/pianoiki/public_html/wiki/includes/parser/Parser.php(3281): PPFrame_Hash->expand(PPNode_Hash_Tree) #10 /home4/pianoiki/public_html/wiki/includes/parser/PPFrame_Hash.php(273): Parser->braceSubstitution(array, PPFrame_Hash) #11 /home4/pianoiki/public_html/wiki/includes/parser/Parser.php(2930): PPFrame_Hash->expand(PPNode_Hash_Tree, integer) #12 /home4/pianoiki/public_html/wiki/includes/parser/Parser.php(1598): Parser->replaceVariables(string) #13 /home4/pianoiki/public_html/wiki/includes/parser/Parser.php(656): Parser->internalParse(string) #14 /home4/pianoiki/public_html/wiki/extensions/PageForms/specials/PF_RunQuery.php(110): Parser->parse(string, Title, ParserOptions, boolean, boolean) #15 /home4/pianoiki/public_html/wiki/extensions/PageForms/specials/PF_RunQuery.php(28): PFRunQuery->printPage(string, boolean) #16 /home4/pianoiki/public_html/wiki/includes/specialpage/SpecialPage.php(647): PFRunQuery->execute(string) #17 /home4/pianoiki/public_html/wiki/includes/specialpage/SpecialPageFactory.php(1366): SpecialPage->run(string) #18 /home4/pianoiki/public_html/wiki/includes/MediaWiki.php(314): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext) #19 /home4/pianoiki/public_html/wiki/includes/MediaWiki.php(930): MediaWiki->performRequest() #20 /home4/pianoiki/public_html/wiki/includes/MediaWiki.php(564): MediaWiki->main() #21 /home4/pianoiki/public_html/wiki/index.php(53): MediaWiki->run() #22 /home4/pianoiki/public_html/wiki/index.php(46): wfIndexMain() #23 {main}

This is the error log after entering a 4 for LevelLow and a 6 for LevelHi. Looks like template passed the #if parser to the query without parsing. Is there something I could wrap around the mutliple #ifs to get them to parse first before passing it on to the query? May214 (talk) 13:18, 18 March 2022 (UTC)

That's strange - I have no idea why those #if calls are not being parsed before #cargo_query is. What changed in your setup, do you know? Yaron Koren (talk) 13:12, 21 March 2022 (UTC)

The last time I had this working was in 2018. I took a new job and the project went off my radar, and when I returned to it, it had been completely spam-vandalized - I couldn't even roll back changes. I'm bringing the project back starting earlier this month. New version of Mediawiki? Cargo updates?

I tried the #if statements outside of the Cargo Query, and it looks like #if isn't parsing anywhere. (See here: [3] Has #if been deprecated? May214 (talk) 14:40, 21 March 2022 (UTC)

So, apparently if you want parser functions to work, you have to enable the ParserFunction extension. Who knew!?!

I had scanned the extensions included with my MediaWiki install, and the ParserFunctions folder was there, and I had assumed it was active. I checked my localsettings.php, and it wasn't. I have since enabled, and the #if is parsing beautifully. My apologies for the time suck this caused. My bad.... May214 (talk) 16:19, 21 March 2022 (UTC)

That would certainly explain the problem. :) I'm glad you figured it out. Yaron Koren (talk) 01:16, 22 March 2022 (UTC)

Creation error with huge data tables

I'm trying to create a cargo table with the following definition:

{{#cargo_declare:_table=StandardRevisions |Std_INT=String|Rev_INT=Integer|Amd_INT=String|Edn=Float|INTER=String|DoA_INT=Date |Std_ARE=String|Rev_ARE=Integer|Amd_ARE=String|ARE=String|DoA_ARE=Date |Std_ARG=String|Rev_ARG=Integer|Amd_ARG=String|ARG=String|DoA_ARG=Date |Std_AUS=String|Rev_AUS=Integer|Amd_AUS=String|AUS=String|DoA_AUS=Date |Std_BRA=String|Rev_BRA=Integer|Amd_BRA=String|BRA=String|DoA_BRA=Date |Std_CAN=String|Rev_CAN=Integer|Amd_CAN=String|CAN=String|DoA_CAN=Date |Std_CHE=String|Rev_CHE=Integer|Amd_CHE=String|CHE=String|DoA_CHE=Date |Std_CHN=String|Rev_CHN=Integer|Amd_CHN=String|CHN=String|DoA_CHN=Date |Std_DEU=String|Rev_DEU=Integer|Amd_DEU=String|DEU=String|DoA_DEU=Date |Std_EUR=String|Rev_EUR=Integer|Amd_EUR=String|EUR=String|DoA_EUR=Date |Std_GBR=String|Rev_GBR=Integer|Amd_GBR=String|GBR=String|DoA_GBR=Date |Std_IND=String|Rev_IND=Integer|Amd_IND=String|IND=String|DoA_IND=Date |Std_ISR=String|Rev_ISR=Integer|Amd_ISR=String|ISR=String|DoA_ISR=Date |Std_JPN=String|Rev_JPN=Integer|Amd_JPN=String|JPN=String|DoA_JPN=Date |Std_KOR=String|Rev_KOR=Integer|Amd_KOR=String|KOR=String|DoA_KOR=Date |Std_MEX=String|Rev_MEX=Integer|Amd_MEX=String|MEX=String|DoA_MEX=Date |Std_RUS=String|Rev_RUS=Integer|Amd_RUS=String|RUS=String|DoA_RUS=Date |Std_SAU=String|Rev_SAU=Integer|Amd_SAU=String|SAU=String|DoA_SAU=Date |Std_SGP=String|Rev_SGP=Integer|Amd_SGP=String|SGP=String|DoA_SGP=Date |Std_USA=String|Rev_USA=Integer|Amd_USA=String|USA=String|DoA_USA=Date |Std_THA=String|Rev_THA=Integer|Amd_THA=String|THA=String|DoA_THA=Date |Std_TWN=String|Rev_TWN=Integer|Amd_TWN=String|TWN=String|DoA_TWN=Date |Std_ZAF=String|Rev_ZAF=Integer|Amd_ZAF=String|ZAF=String|DoA_ZAF=Date }}

However, table creation always fails. When I delete half of the declaration above my system succeeds with creating the table. I'm running a mediawiki 1.37.1 installation on an Ubuntu 20.04 server and running Cargo 3.1. Holger Most (talk) 22:57, 18 March 2022 (UTC)

That is a large table indeed! From a data structure perspective, it may be better to change this into a table (and template) that only has six fields: one for the country (if that's what it is), and the other five for the five values being stored for each country or such. That would give you more flexibility, if you wanted to change either the set of countries, or the set of data being stored for each one. If, however, you want to keep the current structure, my guess is that you can do it by changing the MySQL settings to allow for more storage per table row. Another option is to change the field declarations from something like "Std_INT=String" to "Std_INT=String (size=10)", assuming there can be some limit set on them, to decrease the amount of storage for each String field. Yaron Koren (talk) 13:22, 21 March 2022 (UTC)

Thanks Yaron, for the feedback. You are right that it would be a better idea to store the country data in separate tables. The funny thing is, that I was able to create the table above on hte same server, however, with mediawiki 1.35.3 and Cargo 3.0 Holger Most (talk) 08:35, 28 March 2022 (UTC)

cargoRecreateData.php switch in replacement

Hi Yaron, first of all, thanks for the amazing extension and your support. So I have a table that has about 150000 entries and I had an issue where I wanted to switch in the replacement in the web interface and the wiki was showing some error's at first and then, that the DB is offline. So I restarted it and then it worked again. Do you think it is possible that was just to much for the DB? Are there any limitations in size of tables? I don't want to miss use the extension. So now I did recreate the table again on my development wiki with the cargoRecreateData.php-Script and the replacement option activated. Is there a way to switch in the table with this script, or just with the web interface?
Thanks for you help. DesignerThan (talk) 17:17, 22 March 2022 (UTC)

A table of that size shouldn't cause problems, I think... I don't know why your DB went offline. Hopefully it won't happen if you try it again. There's no way yet to do a switch from the command line, I don't think, unfortunately. Yaron Koren (talk) 19:42, 25 March 2022 (UTC)

CargoQuery help: Why does it work "sometimes"

Working on a cargo query that works "most of the time." I'm wondering if someone can help me figure out what is going on with the few cases where it doesn't work.

Relevant Tables: Repertoire (individual pieces of music) and Books (containing those pieces). I have my Books table set with a repertoire field holding a list of pages from category Repertoire. I would like my Repertoire template page to display the list of books in which that repertoire can be found. On the Repertoire template page I've added this cargo query.

{{#cargo_query:
|tables=Books=A, Repertoire=B
|join on = A.Repertoire HOLDS B._pageName
|fields=A._pageName
|where = B._pageName = '{{PAGENAME}}'
|format=ul}}

I set up some test books, and it works beautifully....71% of the time. In the first test case, I had Book A, Book B, and Book C. I chose 7 repertoire and assigned them (A) (AB) (ABC) (AC) (B) (BC) (C). Five of the Seven worked perfectly, showing one, two or (three) links to the books that contained them. Two of them showed "no results" in the query area (ABC) and (B).

I tried again with Books E, F, and G - all different repertoire. Again, five of seven worked beautifully except for (EF) and (F).

I'm okay with complete failure, or mostly failure because then I know I have something conceptually wrong. I'm not sure what to do with almost 75% success. I'm thinking the fail has to be somewhere in the pagename. Is there a way to get around that using pageID?

Here's the Cargo Table for Books [4]

The offenders are the Mozart Menuetto [5] (ABC), the Bach, JS, Aria [6] (B), Bartok Village girls [7](EF), and Gretchaninoff Woodland Glade [8](F)

A couple that work nicely: Clementi [9] (EFG), and Heller Etude [10] (AC)

Any ideas? May214 (talk) 16:06, 25 March 2022 (UTC)

That's very strange - I don't see anything unusual in the page names. Yes, you could try it with _pageID and the PAGEID variable. Another option is to simplify the query, to something like:

{{#cargo_query:
|tables=Books
|fields=_pageName
|where = Repertoire HOLDS '{{PAGENAME}}'
|format=ul}}

You might need something more complex from the query, though. Yaron Koren (talk) 19:48, 25 March 2022 (UTC)

Perfect. This query is functioning perfectly. Thanks for the more elegant and efficient query!!! May214 (talk) 13:54, 28 March 2022 (UTC)

"_pageID" in GROUP BY clause makes grouping of identical columns with different _pageIDs impossible

Since Cargo V3 "_pageID" must appear in the GROUP BY clause. However, this makes it impossible to group by columns that have the same values but different _pageIDs. GROUP BY is thus significantly restricted! Many old query codes are no longer functional.

_pageID« muss in der GROUP-BY-Klausel erscheinen oder in einer Aggregatfunktion verwendet werden
LINE 1: SELECT /* CargoSQLQuery::run */ "gi"."_pageID" AS "cargo_b...

--Barpfotenbaer (talk) 12:37, 1 April 2022 (UTC)

I don't think that is true. Do you have "_pageID" as one of the fields in the "fields=" parameter? If so, that is the cause of the problem. Yaron Koren (talk) 13:16, 1 April 2022 (UTC)

Only one single field parameter is set

The _pageID isn't referred anywhere. This string is completely missing in my code. I reduced the former complex query to these very simple lines (only identifiers were changed a little in this posting):

{{#cargo_query:
tables      = GNDINDEX_table = gi
|fields     = gi.GNDINDEX_PROPERTY_gnd_name = Schlagwort
|where      = gi.GNDINDEX_PROPERTY_gnd_type = 'Person'
|group by   = gi.GNDINDEX_PROPERTY_gnd_name
|format     = ol
}}

that's the answer:

Error 42803: FEHLER:  Spalte »gi._pageID« muss in der GROUP-BY-Klausel erscheinen oder in einer Aggregatfunktion verwendet werden
LINE 1: SELECT /* CargoSQLQuery::run  */  "gi"."_pageID" AS "cargo_b...
^

Barpfotenbaer (talk) 13:38, 1 April 2022 (UTC)

Okay, I did a web search on "muss in der GROUP-BY-Klausel" and found that this is a (German-language) PostgreSQL error message. So, you are probably using PostgreSQL. That makes it hard for me to test, unfortunately. I have no idea why this is happening. What exact Cargo version are you using? Yaron Koren (talk) 14:00, 1 April 2022 (UTC)

Cargo 3.1 (0f06913) 08:42, 25. Feb. 2022

MediaWiki 1.37.1

PHP 8.0.14 (apache2handler)

PostgreSQL 13.4

Perhaps it is interesting for you to know, that the problem was introduced with Version 3

Thanks for your help!

Barpfotenbaer (talk) 19:06, 1 April 2022 (UTC)

I think the issue is caused somewhere in file includes/parserfunctions/CargoQuery.php line 79-101. The behaviour is described there not as bug, but as feature :-)

// the query not always would include _pageID.

// Let's add _pageID entry for each table, with namespace to prevent collision with the fields in query

// Then, not modify the main query, just clone everything.

// (we can't just replace the $fieldsStr, because it can affect $havingStr part).

// Also remove limit so we'll include all potential results

Barpfotenbaer (talk) 19:45, 1 April 2022 (UTC)

Perhaps includes/parserfunctions/CargoQuery.php line 95 (V3.1) has something to do with it? I modified

$newFieldsStr = "$table._pageID=$fieldFullName, " . $newFieldsStr;

to

$newFieldsStr = "$table.\"_pageID\"=$fieldFullName, " . $newFieldsStr;

and the Error changed to something like:

A field named "gi."_pageID" was not found for any of the specified database tables.

However field with name "_pageID" definitely does exist! Barpfotenbaer (talk) 13:15, 11 April 2022 (UTC)

Sorry about all these problems! I checked in what I think is a fix for all these problems with cargo_backlinks. If you can, please try out the latest code and let me know if you still see any problems. Yaron Koren (talk) 23:21, 13 April 2022 (UTC)

Thanks a lot!!! That seems to have solved my problem completely! 👌😃👍 Barpfotenbaer (talk) 07:39, 19 April 2022 (UTC)

changelog?

Is there an official changelog for Cargo? I haven't updated in awhile and was wondering what goodies I can find. Frybread (talk) 23:41, 3 April 2022 (UTC)

Yes, see Version history. Yaron Koren (talk) 15:19, 4 April 2022 (UTC)

Latest Cargo Download and slow updating of tables

Hello!

As of last week (during the EMW Con) I downloaded the most recent Cargo to get the _pageData table access. I am noticing that when I try to update current table data (I have a bad habit of deleting the tables and creating the tables again) the tables are not accessing all the rows. I tried to cargoRecreateData.php to no avail.

Any ideas?

Thanks for the great EMW Convention!

Margaret 199.247.33.33 17:55, 11 April 2022 (UTC)

I turned off VisualEditor and now I can recreate cargo tables through commandline. This was my visual editor localsettings entry

wfLoadExtension( 'VisualEditor' );
wfLoadExtension( 'Parsoid', 'vendor/wikimedia/parsoid/extension.json' );
// Optional: Set VisualEditor as the default for anonymous users
// otherwise they will have to switch to VE
$wgDefaultUserOptions['visualeditor-editor'] = "visualeditor";
$wgVirtualRestConfig['modules']['parsoid']['url'] = 'http://localhost/rest.php';

That's unexpected. By "slow updating", do you mean that the full table does not get created, or that the update actually runs slowly, or both? And what exactly happened when you called cargoRecreateData.php? Yaron Koren (talk) 02:35, 12 April 2022 (UTC)

When VisualEditor was enabled and I ran cargoRecreateData.php there was no processing. Once visual editor was disabled the cargoRecreateData.php script ran AND the runJobs que holding any cargo scripts ran.

As for the speed of the cargo tables rows appearing (or missing rows), I believe that had something to do with using 'string' instead of 'wikitext' for my longer fields. Now I have a complete tables showing in the GUI because of the String to Wikitext field swap. 199.247.33.3 18:36, 12 April 2022 (UTC)

Scrollbar when table on Drilldown page is too wide

Currently, when there are too many columns in the table, the scrollbar appears at the bottom of the table. It would be useful if it could appear at the top as well. Would this be possible? Jonathan3 (talk) 08:32, 13 April 2022 (UTC)

I don't know of any way to do that. Yaron Koren (talk) 23:24, 13 April 2022 (UTC)

Thanks. Is this any help? https://stackoverflow.com/questions/3934271/horizontal-scrollbar-on-top-and-bottom-of-table Jonathan3 (talk) 08:40, 14 April 2022 (UTC)

CargoQuery page (navigated from "more" results on a query)

Hi, I have a simple query similar to:

{{#cargo_query:

|tables=Events

|fields=Name, COUNT(*)

|where=1=1

AND important=1

|group by=Name

|offset=100

|limit=100

|order by=COUNT(*) DESC

}}

When I press the "more results" text, I get to a new page "Special:CargoQuery" with my query inlines in the get params.

The page I see showing the results number 101-200 and it's ok, but when I press "view 100 previous results" the new page showing a query without the "order by" param and it shows the results in random (I guess str compare) order.

This is the desired behaviour? Thanks! Koshob (talk) 11:59, 30 April 2022 (UTC)

Any idea? :) User:Yaron Koren User:Jonathan3 Koshob (talk) 20:16, 9 May 2022 (UTC)

Sorry about that! This was a real bug, which has probably been around for a long time. I just checked in a fix for it. Yaron Koren (talk) 14:13, 10 May 2022 (UTC)

Thank you! Koshob (talk) 05:48, 11 May 2022 (UTC)

Controlling the visualization of ol format

Hi, I have a query that uses a template in order to show the results (inject some css classes to each row). I want to show the result_row_id for each of them (to be similar to ol - show an ordered results but with customize css).

What I can think of is a named arg that will be passed to the templates named "cargo_row_id" or similar, there is any way to achieve that? Thanms Koshob (talk) 12:17, 30 April 2022 (UTC)

I've not tried it but could you use ROW_NUMBER?

Or use the nth child stuff in css? Jonathan3 (talk) 12:42, 30 April 2022 (UTC)

Database query error - Error 1213: Deadlock found when trying to get lock

Hello,

I randomly get this message while browsing my wiki (1.37.2):

[68cbd0c31d5440ce51d3dc2a] /index.php/113 Wikimedia\Rdbms\DBQueryError: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading or after adding a new extension?

Please see https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Upgrading and https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:How_to_debug for more information.

Error 1213: Deadlock found when trying to get lock; try restarting transaction (localhost)

Function: CargoBackLinks::removeBackLinks

Query: DELETE FROM `cargo_backlinks` WHERE cbl_query_page_id = 549

Backtrace:

from /var/www/mediawiki-1.37.2/includes/libs/rdbms/database/Database.php(1809)

#0 /var/www/mediawiki-1.37.2/includes/libs/rdbms/database/Database.php(1793): Wikimedia\Rdbms\Database->getQueryException()

#1 /var/www/mediawiki-1.37.2/includes/libs/rdbms/database/Database.php(1768): Wikimedia\Rdbms\Database->getQueryExceptionAndLog()

#2 /var/www/mediawiki-1.37.2/includes/libs/rdbms/database/Database.php(1327): Wikimedia\Rdbms\Database->reportQueryError()

#3 /var/www/mediawiki-1.37.2/includes/libs/rdbms/database/Database.php(3655): Wikimedia\Rdbms\Database->query()

#4 /var/www/mediawiki-1.37.2/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->delete()

#5 /var/www/mediawiki-1.37.2/includes/libs/rdbms/database/DBConnRef.php(535): Wikimedia\Rdbms\DBConnRef->__call()

#6 /var/www/mediawiki-1.37.2/extensions/Cargo/includes/CargoBackLinks.php(31): Wikimedia\Rdbms\DBConnRef->delete()

#7 /var/www/mediawiki-1.37.2/extensions/Cargo/includes/CargoBackLinks.php(44): CargoBackLinks::removeBackLinks()

#8 /var/www/mediawiki-1.37.2/extensions/Cargo/includes/parserfunctions/CargoQuery.php(176): CargoBackLinks::setBackLinks()

#9 /var/www/mediawiki-1.37.2/includes/parser/Parser.php(3407): CargoQuery::run()

#10 /var/www/mediawiki-1.37.2/includes/parser/Parser.php(3092): Parser->callParserFunction()

#11 /var/www/mediawiki-1.37.2/includes/parser/PPFrame_Hash.php(273): Parser->braceSubstitution()

#12 /var/www/mediawiki-1.37.2/includes/parser/Parser.php(3281): PPFrame_Hash->expand()

#13 /var/www/mediawiki-1.37.2/includes/parser/PPFrame_Hash.php(273): Parser->braceSubstitution()

#14 /var/www/mediawiki-1.37.2/includes/parser/Parser.php(2930): PPFrame_Hash->expand()

#15 /var/www/mediawiki-1.37.2/includes/parser/Parser.php(1598): Parser->replaceVariables()

#16 /var/www/mediawiki-1.37.2/includes/parser/Parser.php(656): Parser->internalParse()

#17 /var/www/mediawiki-1.37.2/includes/content/WikitextContent.php(327): Parser->parse()

#18 /var/www/mediawiki-1.37.2/includes/content/AbstractContent.php(548): WikitextContent->fillParserOutput()

#19 /var/www/mediawiki-1.37.2/includes/Revision/RenderedRevision.php(263): AbstractContent->getParserOutput()

#20 /var/www/mediawiki-1.37.2/includes/Revision/RenderedRevision.php(235): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached()

#21 /var/www/mediawiki-1.37.2/includes/Revision/RevisionRenderer.php(217): MediaWiki\Revision\RenderedRevision->getSlotParserOutput()

#22 /var/www/mediawiki-1.37.2/includes/Revision/RevisionRenderer.php(154): MediaWiki\Revision\RevisionRenderer->combineSlotOutput()

#23 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}()

#24 /var/www/mediawiki-1.37.2/includes/Revision/RenderedRevision.php(197): call_user_func()

#25 /var/www/mediawiki-1.37.2/includes/poolcounter/PoolWorkArticleView.php(137): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()

#26 /var/www/mediawiki-1.37.2/includes/poolcounter/PoolCounterWork.php(162): PoolWorkArticleView->doWork()

#27 /var/www/mediawiki-1.37.2/includes/page/ParserOutputAccess.php(281): PoolCounterWork->execute()

#28 /var/www/mediawiki-1.37.2/includes/page/Article.php(691): MediaWiki\Page\ParserOutputAccess->getParserOutput()

#29 /var/www/mediawiki-1.37.2/includes/page/Article.php(506): Article->generateContentOutput()

#30 /var/www/mediawiki-1.37.2/includes/actions/ViewAction.php(80): Article->view()

#31 /var/www/mediawiki-1.37.2/includes/MediaWiki.php(543): ViewAction->show()

#32 /var/www/mediawiki-1.37.2/includes/MediaWiki.php(320): MediaWiki->performAction()

#33 /var/www/mediawiki-1.37.2/includes/MediaWiki.php(930): MediaWiki->performRequest()

#34 /var/www/mediawiki-1.37.2/includes/MediaWiki.php(564): MediaWiki->main()

#35 /var/www/mediawiki-1.37.2/index.php(53): MediaWiki->run()

#36 /var/www/mediawiki-1.37.2/index.php(46): wfIndexMain()

#37 {main} Carloposo (talk) 09:22, 2 May 2022 (UTC)

Upgraded from Cargo 3.1 to 3.2 today and still experiencing the same issue :( Carloposo (talk) 14:55, 4 May 2022 (UTC)

Field not found when using MAX()/MIN()/AVG() with Chinese field name

Hello, I have created a table with {{#cargo_declare:_table=表格|数字=Integer|number=Integer}}, and I failed to query {{#cargo_query:tables=表格|fields=MAX(`数字`)|format=table}}. It told me Error: No field named "`数字`" found for any of the specified database tables. But it works well with fields=COUNT(`数字`),SUM(`数字`) . Is it recommended to use Chinese field name?

I'm using MediaWiki 1.37.1 and Cargo 3.2 (b46d012). SkyFish624 (talk) 17:09, 4 May 2022 (UTC)

Using non-ASCII characters in table and field names should work in theory, but in practice I am not surprised that there are some bugs. Yaron Koren (talk) 13:28, 5 May 2022 (UTC)

Relational database in Cargo

Hi

I'm new to Mediawiki & Cargo but have been writing relational database apps for 30 odd years, and I seem to be at the bottom of a steep learning curve as to how best to store data.

I want to store data about manufacturers such as the year they started, their web site etc. All fairly simple stuff, but each manufacture can have one or more physical address which I also want to store. With a relational database I'd just have two tables, Manufacture and Address with a one to many relationship.

How would I do such a thing with MediaWiki/Cargo ? I had though that I could have a page for the manufacture and a page per address e.g Manufacturer Page: Sony Corp, Address Page #1: Sony New York..... Address Page #2: Sony London..... Address Page #3: Sony Paris.... but that only takes one user to create an address page called just London, rather than Sony London and then the whole thing becomes a mess which would need lots of moderation.

What's the best way to do something like this please (or should I be asking somewhere else) ? 91.85.121.33 16:48, 24 May 2022 (UTC)

It sounds like your concern is not really with the data structure but with data entry. In that case, I would suggest you look into the Page Forms extension, if you haven't already. Yaron Koren (talk) 13:23, 25 May 2022 (UTC)

Module Requires Binser

Hi. I am hoping this is the right place.

I am trying to replicate a Infobox for a sports team on my wiki. I have discovered they use Cargo to store the information for the specific call. I have set everything up as instructed. But I notice that a few of the scripts require 'binser' module. I have researched and found binser and LuaRocks, but I have no idea (and seemingly no guidance online) on how to install this into my profile, the sites I am using the modules from don't have a Module:binser on them either.

Any help would be more than appreciated. 2601:285:303:DE60:F0D2:6FB5:3C86:611B 17:23, 25 May 2022 (UTC)

I've never heard of the "Binser" module, but I assume you can just copy their page "Module:Binser" to your wiki. Yaron Koren (talk) 13:45, 26 May 2022 (UTC)