User:DWalden (WMF)/Copy paste diffs

Test strategy for CommTech's Copy and paste from diffs project.

This strategy is non-exhaustive and not final. As we learn more about the software/project, we should update this document to reflect that.

Rationale and high-level requirements for this project can be found on meta:Community_Tech/Copy_paste_diffs.

Instructions

From article history page

Find an article
Click "View History"
- If necessary, change the radio buttons to choose which edit revisions to compare
Click "Compare selected revisions"

While editing a page

Find an article
Edit it with the source editor
Change some text
Click "Show changes" at the bottom of the page

Oracles

(i.e. how we recognise potential bugs)

Initially, most of the changes are only going to be on beta. Therefore, we can compare beta's behaviour against https://test.wikipedia.org (or any production wiki).
Actions like selecting text, copying, keyboard shortcuts, etc. all have fairly well defined behaviours and people expect them to behave in a particular way.
The browser console will display JavaScript exceptions.

Test Ideas

How do different types of edit (add, remove, move text) appear on the diff page?
Test that keyboard shortcuts that relate to selecting and copying text work (you can find lists of keyboard shortcuts on the internet, such as here and here).
- It appears these might behave slightly differently in different browsers (e.g. shift+arrows on Firefox vs. Chrome/Safari).
Test on different user interface languages (especially right-to-left).
Test on different skins.
Test on different devices and browsers (What about touch devices?)
Test with different user preferences enabled/disabled (by going to Special:Preferences). For example:
- "Do not show page content below diffs"
- "Edit pages on double click"
- Need to enumerate all of these
Test the diff page with javascript disabled on your browser (probably low priority though).

Instrumentation

Using browser devtools you should be able to access JavaScript objects/methods that might give you more insight into what the browser is currently selecting:

Open Questions

Will it work with the Revision Slider?
Should/will these changes affect the diff you see when reviewing your changes in Visual Editor? (link)
What about Visual Diff?
Does it work with Live Preview?

Tools

Script to apply random edits to wikitext https://gitlab.wikimedia.org/dwalden/misc/-/blob/main/tokenise_edit.py