Manual talk:Pywikibot

About this board

Archives 


Please use one of the communication channels listed on Manual:Pywikibot/Communication rather than using this discussion board. There is very little traffic here, so it may take a while before you get a response.

Cannot edit protected namespaces

2
Backlitt (talkcontribs)

I'm running 1.35 and use $wgNamespaceProtection to restrict edit access to several namespaces, including the mainspace. ($wgNamespaceProtection[NS_MAIN] = ['main-edit'])

When I try to make an edit with pywikibot (add_text for example), the bot logs in and reads the page properly, but I'm told that the bot lacks permissions to edit in the "Page" namespace". I've made sure that both the bot and the associated user have the ['main-edit'] = true permission.

When I remove that configuration, the bot edits just fine. It edits other namespaces that don't have the special permission structure.

Is there anything I can do to get around this, other than constantly protecting/unprotecting the namespace.

Legoktm (talkcontribs)

How is your bot authenticating? If you're using BotPasswords or OAuth, you need to make sure the "main-edit" permission is in an appropriate grant (see Manual:$wgGrantPermissions) so the bot session actually has that permission.

txt file and broken links do not get reported on the talk page with weblinkchecker

2
SSethi (WMF) (talkcontribs)

I'm trying the Manual:Pywikibot/weblinkchecker.py to obtain broken links on the page "Aeroflot" on test.wikipedia.org. It does generate a "deadlinks-wikipedia-test.dat" file. I am looking for a ".txt" file, but that isn't generated by re-running the command a second time as per instructions given in the manual. Also, I've modified the settings in "user-config.py" to set "report_dead_links_on_talk" to true to report the dead links on the talk page. That also doesn't seem to be working. Any leads?

Dvorapa (talkcontribs)

After how long did you re-run the script and what was your command (did you set -day)? Usually with most parameters left in defaults you have to run the script and then re-run the script between one and two weeks later. Only then it will generate the second txt file and talk pages memos as expected.

Reply to "txt file and broken links do not get reported on the talk page with weblinkchecker"

Examples of bots and tools that rely on the Pywikibot framework

3
SSethi (WMF) (talkcontribs)

I'm looking for examples of active, helpful, and popular bots and tools that rely on the Pywikibot framework that we can highlight in an upcoming workshop on the topic as part of the "Small wiki toolkits" initiative. If you could point me to some examples, that would be super useful. Thank you!

Pppery (talkcontribs)
Reply to "Examples of bots and tools that rely on the Pywikibot framework"
ElBe (talkcontribs)

Hello. How I can count the number of sections on a page? --ElBe (talk) 18:37, 10 February 2022 (UTC)

Dvorapa (talkcontribs)
ElBe (talkcontribs)

Ok, thanks. I will try it. --ElBe (talk) 05:23, 11 February 2022 (UTC)

Reply to "Count sections of a page"
ElBe (talkcontribs)

If I use

if not pywikibot.Page.exists('Page'):
    #Do something

I get the error

[...]    
    title = self._link.canonical_title()
AttributeError: 'str' object has no attribute '_link'
CRITICAL: Exiting due to uncaught exception <class 'AttributeError'>

What I have to do? --ElBe (talk) 20:31, 20 January 2022 (UTC)

178.209.138.12 (talkcontribs)

I think you should use it like this:

if not pywikibot.Page('Page').exists():

ElBe (talkcontribs)

Thank you! --ElBe (talk) 05:58, 21 January 2022 (UTC)

ElBe (talkcontribs)

It works if I use

[...]
page = pywikibot.Page(site, 'Page')

if not pywikibot.Page(page).exists():
   #Do something

Thank you! --ElBe (talk) 06:09, 21 January 2022 (UTC)

Xqt (talkcontribs)

Hi ElBe, this is not a proper idea but it works by a side effect:

exists()

is a method of a Page object, more precisely of a BasePage object. You have to create the object first to use it:

 page = pywikibot.Page(site, 'Page')
 if not page.exists(): pass  # or do something sensefull

Your example works because

 pywikibot.Page(page)

clones the original page and creates a new Page object. This functionality to implemented to upcast objects e.g.

 fp = pywikibot.FilePage(page)

which creates a FilePage object and provides further methods. But keep in mind that page must be in File: namespace here.

Refer: https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.page.html?highlight=exists#pywikibot.page.BasePage.exists

Reply to "Check if a page exists"

Creating new pages using add_text.py

5
Leyo (talkcontribs)

I wanted to create a bunch of pages using add_text.py, but I get the error message

WARNING: Page [[de:xyz]] does not exist on wikipedia.de.

I there any possibility to suppress this error message and to create the pages anyway?

Xqt (talkcontribs)

Currently not because AddTextBot is derived from ExistingPageBot which skips existing pages. Maybe a -create option should be implemented?

This post was hidden by Leyo (history)
SSethi (WMF) (talkcontribs)

Hello, we often get requests from smaller wiki communities part of the Small Wiki Toolkits initiative to learn how to create, deploy, run, and manage bots. Most of the requests are from novice programmers. As we do not have enough mentors in our technical community to address these needs, I am wondering if folks here might be interested in running a few sessions/workshops and helping develop a curriculum on the topic in the coming months. Thoughts?

Reply to "Bots workshops"

Alternative Implementation of Pywikibot

4
Hogü-456 (talkcontribs)

I am interested in programming and there especially in a kind of programming that is nearly related to natural language. I am writing scripts in the programming language R with a SQL-Integration. I created a script for extracting out of a sentence with a defined structure the Variables. This can be used to write a Program in a scripting language with own defined structured sentences that is nearly related to the natural language and then convert it into the source code in a more detailed programming language. You can find the script in the following folder in my PAWS-Profile. The script from where it starts is called Structuredtexttranspiler 20211021.R .Do you think that defining sentences and mapping them to source code in Python can be useful for Pywikibot to make it for more people possible to use the framework. I dont have deeper understanding of Python but can help setting up sentences with a defined structure when you tell me how the Python equivalent looks like.

Xqt (talkcontribs)

Currently I don't yet know what to mean by that. Perhaps there could be an operator interface which parses the more or less natural speeking "commands" to get the expected results. Or parsing such commands and translate it into SQL commands to retrieve data via SQL queries or pass them to our sparql interface.

Hogü-456 (talkcontribs)

I think that the current way how to use Pywikibot requires an understanding of programming and maybe that what I linked there is an possibilty to reduce the required knowledge about programming to make it possible for more people to use programming. The idea is to make what you say in the second sentence. Creating an operator interface which parses the more or less natural speeking "commands" to get the expected results.

Dvorapa (talkcontribs)

Pywikibot can be operated without any programming knowledge, that's what Pywikibot existing scripts are for. You call the script together with its parameters. But this is a pretty old approach, not that modern. R-like languages are a modern approach, for sure. But creating an R interface to Pywikibot's Page, Site, Family, Pagegenerator, Bot ... objects is quite a lot of work. Basically you would have to be pretty good at Python and have pretty good understanding of how Pywikibot works to make this happen. First I would recommend you to explore the internals of Pywikibot, get some basic idea how it works and what are its key parts, then perhaps you would find more concrete way R lang can help the project. Also I should mention there was a Pywikibot competitor written in R, you should be able to find it being mentioned on Mediawiki.org's or English Wikipedia's Bot Help pages.

Reply to "Alternative Implementation of Pywikibot"
Tigraan (talkcontribs)

There was a discussion in April on the mailing list about having a deprecation policy based on the one for the MediaWiki interface. Did this go anywhere?

I am asking because I have been bitten by a breaking change - specifically, this removal of APISite.getuserinfo() broke my bot on Toolforge. I am not saying the removal was unreasonable (2+ month after the "deprecated" decorator was added sounds reasonable) but there is no documentation of expectations anywhere I could find.

(My bot breaking is 95% my fault, since I did not notice the breakage in timely fashion, nor did I regularly review the Pywikibot code for deprecation tags, so even a one-year deprecation period would not have helped me much, but still.)

Xqt (talkcontribs)

The deprecation warning was shown every time you are running your bot. There is a warning in our changelog for the stable release [1]. With 6.4 Pywikibot follows semantic versioning. All deprecated code will be removed with version 7.0.0 (comming at the end of this month I guess) and new deprecations are kept until 8.0 is deployed (probably until end of 2022). Please also have a look at the current deprecations [3], the code changes comming with 7.0 [4] (more to come here!) and pay attention to the deprecation warnings of your scripts.

[1] https://doc.wikimedia.org/pywikibot/master/changelog.html#id6 [2] https://www.mediawiki.org/wiki/Manual:Pywikibot/Development/Guidelines#Deprecation_Policy [3] https://doc.wikimedia.org/pywikibot/stable/changelog.html#deprecations [4] https://doc.wikimedia.org/pywikibot/master/changelog.html#current-release-changes

Reply to "Deprecation policy"

How to do a string replacement?

6
Summary by Marbot

Solution: The text must match exactly, including whitespace etc.

Marbot (talkcontribs)

I would like to replace {{MyFirstTemplate|field1=MyText|field2=MyText}} with {{MySecondTemplate|field3=MyNewText}}. According to the docu this should be the line:

python3 pwb.py replace -page:My_Page "{{MyFirstTemplate|field1=MyText|field2=MyText}}" "{{MySecondTemplate|field3=MyNewText}}" -summary:Awesome_replacement

However, nothing is replaced. The script tells that nothing to replace was found. I am a bit fuzzy what to enter. Any help appreciated.

Marbot (talkcontribs)

I guess it has somehting to do with the pipes but adding a \ in front of them does not help either.

Matěj Suchánek (talkcontribs)

Check for spelling mistakes and that you are connecting to the correct wiki. Note that the text must match exactly, including whitespace etc. Otherwise you will need to make a regex (and use -regex). The regex will then need the slashes to escape the pipes and other characters.

Marbot (talkcontribs)

Thanks for your response. This contained the decisive hint: Check the whitespace! I had {{MyFirstTemplate|field1=My Text|field2=My Text}} but wanted to replace {{MyFirstTemplate|field1=My_Text|field2=My_Text}}. The other paramteres understandably require _ though this does not which is understandable, too. Thanks a lot for helping me out of my misery! Much apprechiated.

Xqt (talkcontribs)

Works for me as expected:

C:\pwb\GIT\core>pwb replace -page:MyPage "{{MyFirstTemplate|field1=MyText|field2=MyText}}" "{{MySecondTemplate|field3=MyNewText}}" -site:wikipedia:test -simulate
 
The summary message for the command line replacements will be something like: 
Bot: Automated text replacement  (-{{MyFirstTemplate|field1=MyText|field2=MyText}} +{{MySecondTemplate|field3=MyNewText}}) 
Press Enter to use this automatic message, or enter a description of the changes your bot will make: Retrieving 1 pages from wikipedia:test.

>>> MyPage <<< @@ -6 +6 @@ - {{MyFirstTemplate|field1=MyText|field2=MyText}} + {{MySecondTemplate|field3=MyNewText}}

Do you want to accept these changes? ([y]es, [N]o, [e]dit original, edit [l]atest, open in [b]rowser, [m]ore context, [a]ll, [q]uit):

Note: There is no -regex option activated and the source string must match the text, see https://test.wikipedia.org/w/index.php?title=MyPage&action=edit

Marbot (talkcontribs)

Thank you for confirming! I know what I did wrong. See my answer to Matěj!

Return to "Pywikibot" page.