Manual talk:Pywikibot

About this board

Archives 


Please use one of the communication channels listed on Manual:Pywikibot/Communication rather than using this discussion board. There is very little traffic here, so it may take a while before you get a response.

Replace script and fix option

12
Summary by Speravir

The cause for the error messages was on user side: I mistakenly added a comma at a place where it does not belong.

Speravir (talkcontribs)

@Xqt, I do not know whether this is a bug or an issue on my side, therefore here and not in Phabricator:

I’ve added this to user-fixes.py:

fixes['ampCode'] = {
    'generator': [
        r'-ns:0',
        r'-search:insource:"%26" insource:/%26/',
    ],
    'regex': False,
    'msg': {
        '_default': 'Bot-Änderung: überflüss. Kodierung für [[Und-Zeichen]] ersetzt, vgl. [[Spezial:Diff/233908956|Anfrage]]',
    },
    'replacements': [
        (r' %26 ', ' & '),
    ]
}

and then started Pywikibot with this (Windows) command line (user is pre-configured):

pwb replace -simulate -lang:de -fix:ampCode -log:fix-ampCode.log

This resulted in this error message:

2023-05-24 00:40:34            http.py,  123 in              flush: VERBOSE  Traceback (most recent call last):
  File "C:\Programs\Netz\pywikibot\pwb.py", line 39, in <module>
    sys.exit(main())
             ^^^^^^
  File "C:\Programs\Netz\pywikibot\pwb.py", line 35, in main
    runpy.run_path(str(path), run_name='__main__')
  File "<frozen runpy>", line 291, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 514, in <module>
    main()
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 498, in main
    if not execute():
           ^^^^^^^^^
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 485, in execute
    run_python_file(filename, script_args, module)
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 147, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
  File "C:\Programs\Netz\pywikibot\scripts\replace.py", line 1113, in <module>
    main()
  File "C:\Programs\Netz\pywikibot\scripts\replace.py", line 1008, in main
    if not fix['replacements']:
           ~~~^^^^^^^^^^^^^^^^
TypeError: tuple indices must be integers or slices, not str
2023-05-24 00:40:34            http.py,  132 in              flush: CRITICAL Exiting due to uncaught exception TypeError: tuple indices must be integers or slices, not str

Used Pywikibot version is:

Release version: 8.1.2
setuptools version: 65.5.0
mwparserfromhell version: 0.6.3
wikitextparser version: 0.51.1
requests version: 2.28.1

With explicit command line options the task worked then, the call was:

pwb replace -simulate -lang:de -ns:0 -search:"insource:\"%26\" insource:/ %26 /" -summary:"Bot-Änderung: überflüss. Kodierung für [[Und-Zeichen]] ersetzt, vgl. [[Spezial:Diff/233908956|Anfrage]]" -log:fix-ampCode.log " %26 " " & "
Sakura emad (talkcontribs)

CHATGPT generated Answer:

Hi there! It seems that there is an issue with the replacements field in your code snippet. The error message indicates that "tuple indices must be integers or slices, not str." To fix this, please replace the parentheses with square brackets in the replacements field to define a list of tuples. Here's the corrected code snippet:

fixes['ampCode'] = {
    'generator': [
        r'-ns:0',
        r'-search:insource:"%26" insource:/%26/',
    ],
    'regex': False,
    'msg': {
        '_default': 'Bot-Änderung: überflüss. Kodierung für [[Und-Zeichen]] ersetzt, vgl. [[Spezial:Diff/233908956|Anfrage]]',
    },
    'replacements': [
        [r' %26 ', ' & '],
    ]
}

Speravir (talkcontribs)

No, that is not the reason:

  1. Inspite I wondered about this change (see next item), I tested it and the error message did not disappear.
  2. Though it could be outdated on Manual:Pywikibot/user-fixes.py, the syntax with parentheses is still visible there.
  3. Also, I actually had copied older entries and adjusted them to new task. Back in the past all had worked fine.
Sakura emad (talkcontribs)

: ) New Answer:

If correcting the replacements field did not resolve the issue, here are a few additional suggestions to investigate further:

Check for any recent changes or updates in the Pywikibot library or related dependencies that could be causing compatibility issues. It's possible that there might be a newer version or a specific configuration that is causing the error.

Double-check the command line arguments and options being used to start Pywikibot. Ensure that all necessary arguments are provided correctly, and there are no missing or incorrect options in the command.

Verify if you have the latest version of Pywikibot installed. While you mentioned version 8.1.2, it's worth checking if there are any newer releases available that might include bug fixes or improvements related to the issue you are facing.

I recommend reaching out to the Pywikibot community or forums for assistance. They have extensive experience with the library and might be able to provide specific guidance or insights based on similar issues they have encountered.

Xqt (talkcontribs)

Could you please add the following lines in front of line 1008 in replace.py:

from pprint import pprint
pprint(fix)

Seems that the replace.py script found an evaluated fixes entry different from that given above. The fix itself is read as tuple instead a dict somehow.  @xqt 04:24, 31 May 2023 (UTC)

Speravir (talkcontribs)

With how much indentation? I did it this way:

if isinstance(fix['msg'], str): # line 1007
    from pprint import pprint
    pprint(fix)
    set_summary = i18n.twtranslate(site, str(fix['msg']))

The error message remains.

I also have deleted the whole pywikibot (without my user scripts and config, but unintentionally the logs) and then re-extracted the zip file and then temporarily uninstalled mwparserfromhell, because I noticed the subdirectory in pywikibot. No success at all.

Side question: Can I without any risk add -simulate as generator option into user-fixes.py?

Xqt (talkcontribs)

> With how much indentation? I did it this way:

Seems you have updated the script in meantime. What you should do is the following: Replace the following code

       if not fix['replacements']:
           pywikibot.warning(f'No replacements defined for fix {fix_name!r}')
           continue

with

       from pprint import pprint
       print(type(fix))
       pprint(fix)
       if not fix['replacements']:
           pywikibot.warning(f'No replacements defined for fix {fix_name!r}')
           continue

The reason is that fix seems not to be a dict but a tuple. Maybe you have added a comma at the end of the ampCode fix in your fixes.py.  @xqt 11:57, 2 June 2023 (UTC)

Speravir (talkcontribs)

> “Maybe you have added a comma at the end of the ampCode fix in your fixes.py.“

Oh. My. God. This is sooo embarrassing! Auf Deutsch: Ich möchte im Boden versinken!

Yes, this was my simple, but that extensive mistake.

On one hand I am glad not having opened a Phabricator task, but on the other hand (again) so much stress and effort for such a tiny cause. Nethertheless thank you, Xqt, for having the right idea!

Just for the record and for comparison: With the addition of the print code this will be displayed in the terminal (but not the log) as first lines after the command input:

  • in case of defunct user-fix.py with spurious comma (obviously changed code for test case):
<class 'tuple'>                                                                                                      
({'generator': ['-ns:0', '-search:insource:"eszett"'],                                                               
  'msg': {'_default': 'Test Pywikibot, Replace-Skript'},                                                            
  'regex': False,                                                                                                    
  'replacements': [('eszett', 'ß')]},)
  • in correct working state without this comma:
<class 'dict'>                                                                                                       
{'generator': ['-ns:0', '-search:insource:"eszett"'],                                                                
 'msg': {'_default': 'Test Pywikibot, Replace-Skript'},                                                             
 'regex': False,                                                                                                     
 'replacements': [('eszett', 'ß')]}
Xqt (talkcontribs)

> the print code this will be displayed in the terminal (but not the log)

Yes this is intentional. print and pprint does not go throught the ui wrapper but directly to the terminal. I usually use them for debugging. Schönes Wochenende wünsch' ich.  @xqt 09:51, 3 June 2023 (UTC)

Xqt (talkcontribs)

> Side question: Can I without any risk add -simulate as generator option into user-fixes.py?

This does not work because generator options are only taken as pagegenerators options. It may contains generators and filters, see generator options and filter options. -simulate is a global option and must be given with the replace.py script or the pwb wrapper script.  @xqt 11:13, 2 June 2023 (UTC)
Speravir (talkcontribs)

Thanks, I will have a look.

Reply to "Replace script and fix option"

How to upload images to commons via command line?

12
TiagoLubiana (talkcontribs)

I am trying to do this for ages, but there is a lot of old information and misdirections.


I just want to upload a file via command line. Can someone here point me to a clear reference resource where I can really learn how to do it?

sorry about the angry tone, lots of ours of failure.

~~~~

Dvorapa (talkcontribs)
TiagoLubiana (talkcontribs)

In the end it all worked by:


Installing pywikibot via pip

Adding user-config.py file to the same folder as upload.py (and the directory of the files to upload too)

Running the uploads script


I was not able to specify some things, though, like wikidata items depicted, or choose the proper license, or adding a legend. Any tips on that?

Thanks!

Akbarali (talkcontribs)

Can anyone prepare the video tutorial of uploading photos using command line (python) into Wikimedia Commons

This post was hidden by Dvorapa (history)
Dvorapa (talkcontribs)

Not everything is supported by Pywikibot currently, see T223820. You can update upload.py script or write your own script using this brief manual, this basic template and this api module if you need specific feature not yet supported by Pywikibot. You could also propose a patch to Pywikibot if you manage to fix any of these.

Hbf878 (talkcontribs)

There is a parameter that allows for adding a file description string; this will become the wiki code of the Commons page of the file that you uploaded. By the way, the best description of parameters is to be found in upload.py itself, in my opinion. It's more clear than the documentations online.

python3 pwb.py upload "path/to/some_file.png" -summary:"uploaded with pywikibot" "source code string of the file page" -ignorewarn -always -keep

So in the source code string, you can include everything that you usually find on a Commons page (in wiki syntax), like filedesc with the information template, the license-header with the license template of your choice and categories. --Hbf878 (talk) 14:56, 21 April 2020 (UTC)

TiagoLubiana (talkcontribs)

Okay, thank you for the answers!

I will take a look and try to figure a way of doing it.

Reply to "How to upload images to commons via command line?"
SSethi (WMF) (talkcontribs)

Hello, we often get requests from smaller wiki communities part of the Small Wiki Toolkits initiative to learn how to create, deploy, run, and manage bots. Most of the requests are from novice programmers. As we do not have enough mentors in our technical community to address these needs, I am wondering if folks here might be interested in running a few sessions/workshops and helping develop a curriculum on the topic in the coming months. Thoughts?

TimBorgNetzWerk (talkcontribs)

Hello User:SSethi (WMF),

in the process of getting into pwb, I'd do something similar of said curriculum either way. If there is still an interest in such a guide, please let me know.

Best regards

Tim from BorgNetzWerk

Reply to "Bots workshops"

No module named pywikibot

15
KlosseBot (talkcontribs)

CMD writing no module named pywikibot when i start login.py

Xqt (talkcontribs)

Looks like your cmd has opened the "../core/scripts" path. You should start from parent "core" path and use the pwb.py wrapper script:

pwb.py login

2A00:1028:9192:FA72:F903:A911:3814:C073 (talkcontribs)

And when i write python pwb.py login it said no module named request.

Xqt (talkcontribs)

You have to install request module as a side package first. Read the message and do as proposed. The message is:

Python module 'requests' is required.

Try running 'pip install requests'

do it with the command

pip install requests

188.23.125.244 (talkcontribs)

I executed "pip install requests", it did so, but the same error doesn't go away. I tried both Python 3.6 and 2.7, it's the same with both.

Framawiki (talkcontribs)

Hello, can you type python pwb.py version and post here what you obtain ?

188.23.125.244 (talkcontribs)

This doesn't even work for me, as it gets an error in line 163 of pwb.py

Xqt (talkcontribs)

Did you installed requests for both python releases?

88.117.63.15 (talkcontribs)

I installed 2.7 after trying 3.6. I eventually got things to work after installing pip seperately again and then installing requests via "pip install requests" and restarting. I may have repeated this some times and can't pinpoint what got things to work, but i eventually got it to work.

KlosseBot (talkcontribs)

'pip install requests' doesnt work

Xqt (talkcontribs)

What is your python version? I propose using 2.7.9 or higher or 3.4 or higher release because the en:python package installer (pip) is shipped with these versions. Otherwise you have to install pip first.

KlosseBot (talkcontribs)

I have 3.4 but still it isnt pip in it. How do i install it?

Xqt (talkcontribs)

Maybe this could help.

JoKalliauer (talkcontribs)

I solved the issue using pip install pywikibot ~~~~

Xqt (talkcontribs)

This is always good enough if you have your own scripts based on pywikibot. If you want to use scripts shipped with pywikibot you have to clone/checkout the repository (either git or svn) or you can download the nightly from https://pywikibot.org. There are only some maintenance scripts which you can use if pywikibot is installed as a site-package like you did it. Refer https://doc.wikimedia.org/pywikibot/stable/utilities/scripts.html for the description.

Reply to "No module named pywikibot"

Cannot edit protected namespaces

2
Backlitt (talkcontribs)

I'm running 1.35 and use $wgNamespaceProtection to restrict edit access to several namespaces, including the mainspace. ($wgNamespaceProtection[NS_MAIN] = ['main-edit'])

When I try to make an edit with pywikibot (add_text for example), the bot logs in and reads the page properly, but I'm told that the bot lacks permissions to edit in the "Page" namespace". I've made sure that both the bot and the associated user have the ['main-edit'] = true permission.

When I remove that configuration, the bot edits just fine. It edits other namespaces that don't have the special permission structure.

Is there anything I can do to get around this, other than constantly protecting/unprotecting the namespace.

Legoktm (talkcontribs)

How is your bot authenticating? If you're using BotPasswords or OAuth, you need to make sure the "main-edit" permission is in an appropriate grant (see Manual:$wgGrantPermissions) so the bot session actually has that permission.

txt file and broken links do not get reported on the talk page with weblinkchecker

2
SSethi (WMF) (talkcontribs)

I'm trying the Manual:Pywikibot/weblinkchecker.py to obtain broken links on the page "Aeroflot" on test.wikipedia.org. It does generate a "deadlinks-wikipedia-test.dat" file. I am looking for a ".txt" file, but that isn't generated by re-running the command a second time as per instructions given in the manual. Also, I've modified the settings in "user-config.py" to set "report_dead_links_on_talk" to true to report the dead links on the talk page. That also doesn't seem to be working. Any leads?

Dvorapa (talkcontribs)

After how long did you re-run the script and what was your command (did you set -day)? Usually with most parameters left in defaults you have to run the script and then re-run the script between one and two weeks later. Only then it will generate the second txt file and talk pages memos as expected.

Reply to "txt file and broken links do not get reported on the talk page with weblinkchecker"

Examples of bots and tools that rely on the Pywikibot framework

3
SSethi (WMF) (talkcontribs)

I'm looking for examples of active, helpful, and popular bots and tools that rely on the Pywikibot framework that we can highlight in an upcoming workshop on the topic as part of the "Small wiki toolkits" initiative. If you could point me to some examples, that would be super useful. Thank you!

Pppery (talkcontribs)
Reply to "Examples of bots and tools that rely on the Pywikibot framework"

Count sections of a page

3
ElBe (talkcontribs)

Hello. How I can count the number of sections on a page? --ElBe (talk) 18:37, 10 February 2022 (UTC)

Dvorapa (talkcontribs)
ElBe (talkcontribs)

Ok, thanks. I will try it. --ElBe (talk) 05:23, 11 February 2022 (UTC)

Reply to "Count sections of a page"

Check if a page exists

5
ElBe (talkcontribs)

If I use

if not pywikibot.Page.exists('Page'):
    #Do something

I get the error

[...]    
    title = self._link.canonical_title()
AttributeError: 'str' object has no attribute '_link'
CRITICAL: Exiting due to uncaught exception <class 'AttributeError'>

What I have to do? --ElBe (talk) 20:31, 20 January 2022 (UTC)

178.209.138.12 (talkcontribs)

I think you should use it like this:

if not pywikibot.Page('Page').exists():

ElBe (talkcontribs)

Thank you! --ElBe (talk) 05:58, 21 January 2022 (UTC)

ElBe (talkcontribs)

It works if I use

[...]
page = pywikibot.Page(site, 'Page')

if not pywikibot.Page(page).exists():
   #Do something

Thank you! --ElBe (talk) 06:09, 21 January 2022 (UTC)

Xqt (talkcontribs)

Hi ElBe, this is not a proper idea but it works by a side effect:

exists()

is a method of a Page object, more precisely of a BasePage object. You have to create the object first to use it:

 page = pywikibot.Page(site, 'Page')
 if not page.exists(): pass  # or do something sensefull

Your example works because

 pywikibot.Page(page)

clones the original page and creates a new Page object. This functionality to implemented to upcast objects e.g.

 fp = pywikibot.FilePage(page)

which creates a FilePage object and provides further methods. But keep in mind that page must be in File: namespace here.

Refer: https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.page.html?highlight=exists#pywikibot.page.BasePage.exists

Reply to "Check if a page exists"

Creating new pages using add_text.py

5
Leyo (talkcontribs)

I wanted to create a bunch of pages using add_text.py, but I get the error message

WARNING: Page [[de:xyz]] does not exist on wikipedia.de.

I there any possibility to suppress this error message and to create the pages anyway?

Xqt (talkcontribs)

Currently not because AddTextBot is derived from ExistingPageBot which skips existing pages. Maybe a -create option should be implemented?

This post was hidden by Leyo (history)
Return to "Pywikibot" page.