Taylor 49

FYI: Lua normally can use up to 200 local variables per function definition

The edit summary of your recent edit to Extension:Scribunto/Lua reference manual is somewhat inaccurate or deceptive. The summary contains: pretty absurd with max 60 upvalues only :-( (FYI: BASIC dialects from 1970-ties supported typically up to 128 variables) and though the number of upvalues is limited by default to 60 (controlled by LUAI_MAXUPVALUES in luaconf.h) the number of local variables is limited to a different and typically larger number that defaults to 200 (controlled by LUAI_MAXVARS in luaconf.h). The upvalue limit is only for closures, a concept first implemented in 1970. I seriously doubt any dialects of BASIC from that time frame supported closures (which requires support of first-class functions a concept unlikely to be in any BASIC implementations since I believe BASIC only got subroutines with SBASIC in 1976) or any upvalues (much less 60) and 200 variables is considerably larger than the "typical" 128 variable limit of "1970-ties" era BASIC dialects you referred to. —Uzume (talk) 02:55, 3 April 2024 (UTC)

@User:Uzume: Hello and thank you for this clarification. The limit of max 200 local variables per function is not documented anywhere (within Extension:Scribunto/Lua reference manual) either, and I had not yet succeeded to violate it. Taylor 49 (talk) 13:54, 3 April 2024 (UTC)

@Taylor 49: If you are running into so many variables (even in the upvalue case) you should probably organize/restructure them into a table or set of tables just for modularity and cohesiveness. Each upvalue reference is actually sort of expensive (even global variable references are technically cheaper though they are discouraged as they lead to bad practice and maintenance issues later; in Lua global variables are stuffed into a global table so they are table references). Organizing them into a single structure means you have only a single actual upvalue where one can just refer to one of its individual members/fields and you can further reduce the upvalue reference cost by creating a local and copying the upvalue just once and then using the local within the function (and yes, one can even name it the same with slightly confusing variable shadowing code like local xyzzy=xyzzy -- define local xyzzy in terms of the upvalue xyzzy which will now be "shadowed" and directly inaccessible while within the current scope/function). Another much cheaper alternative is to just pass the values as function arguments. During the function definition these arguments can be given parameter names which are basically just implicit local variables assignments. One can also directly "slice" the function arguments using ... vararg notation (select() can be useful here) since all functions in Lua (like C) are technically variadic. The ... vararg reference is technically not a variable itself (although there is also a deprecated global variable named arg) and it can also be used during module initialization (since they are treated as a chunk/function). This can be useful for determining if a module is being invoked as the main module (vs. being required as a library package). —Uzume (talk) 20:22, 3 April 2024 (UTC)

@User:Uzume: Hello and thank you again. YES I have been using tables to circumvent this problem. But what's the benefit with shadowing? I've just tested, and additional accesses to same upvalue within same function do NOT seem to increase the expensive upvalue counter further. The other discovery is that sometimes a violation is detected already when saving the module, but sometimes only at run time. Taylor 49 (talk) 17:46, 4 April 2024 (UTC)

@Taylor 49: I am not sure what you mean by "expensive upvalue counter". I think you may be conflating things. Manual:$wgExpensiveParserFunctionLimit is a MediaWiki arbitrary limited counter to contain certain feature usage deemed "expensive" (several parser functions including Scribunto can contribute to this). What I meant by upvalue accesses are expensive is that they require more work/resources for Lua to perform. Thankfully the MediaWiki "expensive" counter is not incremented for every little thing (although some of the things it is incremented for sometimes seem questionable in light of other means to accomplish the same or similar things). I am sure you can imagine creating an arbitrary long long loop computing something (perhaps foolishly, i.e., just for testing). Such a loop is expensive, but it does not ping the MediaWiki "expensive" counter. That said if you make the loop consume enough resources, the Scribunto extension will shut it down with an error. Upvalue accesses will contribute towards resource usage (as does every variable access but local and global accesses are usually considerably less). Lua is a dynamic language meaning most errors are only discovered at runtime, however, there is a lexer step that "compiles" module script to an internal virtual machine representation. This can catch obvious lexicographical errors like misspelled keywords, etc. That probably accounts for the detecting errors at save time (I believe Scribunto also has some warnings it can generate in the editor before saving too). —Uzume (talk) 18:29, 4 April 2024 (UTC)

@User:Uzume: I indeed meant the expensive upvalue counter that aborts the module and triggers an error when a (new?) upvalue is accessed and it would reach 61. Additional accesses to same upvalue within same function do NOT seem to increase it (they probably waste the time limited to 10 seconds, but that's another story). wikt:id:Modul:TESTTEST. But what's the benefit with shadowing then? Taylor 49 (talk) 18:56, 4 April 2024 (UTC)

@Taylor 49: Oh, my mistake. I would not call that an "expensive" counter (as it is never labelled such anywhere I know of). That is just a resource limitation imposed by Lua itself. One can access the same upvalue as many times as there are resources to continue processing such, however, accessing more than the limit (default of 60) unique upvalues trips the limit and thus the error. Shadowing just hides an upvalue with a local. If one defines the local in terms of the upvalue then future references cannot impose any upvalue access cost as only the local is accessible. —Uzume (talk) 20:03, 4 April 2024 (UTC)

@User:Uzume: So if I understand it right, shadowing will never help with the 60-upvalue-limit. It will just improve the performace (and save some other resources such a time and memory) a bit if the upvalue is accessed many times in one function. The impact of non-unique upvalue uses on the expensive upvalue counter will be very same ie none whether I do shadow or not. Taylor 49 (talk) 20:10, 4 April 2024 (UTC)

@Taylor 49: Yes, that is correct. Doing such is more of an optimization. Since you are probably not running into performance issues yet (except in contrived silly testcases) it is not really required. So knowing about such things is basically a learning exercise. When you do hit performance issues, upvalue references are not often a large part of the performance issue but it depends on the code and is good to know. —Uzume (talk) 20:20, 4 April 2024 (UTC)