User:DWalden (WMF)/Permissions Testing

Introduction

This presents a strategy I often use when testing features that give users access to data which may be sensitive.

For example, information about who made a particular edit or performed a particular logged action.

The basic idea is to compare the information the feature you are testing gives the user access to with what information they can find out through existing means.

For example, the feature should not give the user information about a revision which they cannot see in the revision history of a page.

You also want to enumerate all the variables which affect whether or not a user has permission to see a particular bit of data and test all the combinations of these variables.

For example, revisions and logged actions can have various visibility settings which might mean a user is not allowed to see some or all of the information about it.

Enumerating variables

The relevant variables will depend on the data your feature gives access to. I give examples for revisions and logged actions.

Revisions

Visibility of revision (see Help:RevisionDelete and Manual:RevisionDelete)

• Is the comment hidden?
• Is the revision's wikitext hidden?
• Is the editor hidden?
• Is it suppressed (meaning it is visible only to users with suppressor rights)? Bear in mind that this option only works if you also hide one or more of the above options.

Editor's suppressed status

Is the user who made the edit suppressed or not?

Editor type

Is the user who made the edit an anonymous (IP), temporary or named (logged in) user?

Your user's rights

Is there a specific right associated with using this feature and does the user have it?

Your user's blocked status

Should a user be able to see this information if they are blocked or suppressed?

You can then test each of the combinations of these such as in the below table, which shows only a small sample of the combinations:

Comment Hidden	Revision Text Hidden	Editor Hidden	Revision Suppressed	Editor Suppressed	Editor Type	User rights	User blocked
yes	no	no	no	no	anon	yes	no
no	yes	no	no	no	anon	yes	no
yes	yes	no	no	no	named	no	yes

Here is a real-life example I used to test phab:T324603: User:DWalden_(WMF)/T324603#Permissions_and_Validation.

You can also generate them directly by querying the database, which will also tell you which revisions to test. For example, User:DWalden_(WMF)/IPInfo/Permission_Testing_Data#Revisions.

Logged actions

TODO

Generating data

You will often find that your test environment won't have revisions or logged actions which have all the combinations of relevant variables.

Here is a script to help you generate them.

1. Follow the install instructions for pywikibot: Manual:Pywikibot/Installation.
2. Copy the above file (generate_combination_visibility.py) into the pywikibot "scripts" directory.
3. Find some revision IDs or log item IDs you want to change the visibility of. Ideally, you should find 15 of them. In the next step, we assume the revisions are numbered from 1 to 15.
4. Run: python3 pwb.py generate_combination_visibility -t revision -i {1..15} -lang:en -family:dockerwiki

Comparing

Revisions

There are lots of places to see revision information on a wiki, which you can compare to the information your feature is returning. For example, you can look at a pages revision history or Special:Diff.

If you want to do it programmatically, you can use prop=revisions. For example, to look up revision 123456 you can go to https://www.mediawiki.org/w/api.php?action=query&prop=revisions&revids=123456.

Example of a script doing this (except using the list=allrevisions).

Here is another one using prop=revisions.

Logged actions

TODO