User:DWalden (WMF)/Thumbor migration

Project documentation edit

For Phabricator board, team members involved, etc. see 2022_Thumbor_updates.

The Phabricator board is: https://phabricator.wikimedia.org/tag/thumbor_migration/

Feature documentation edit

Thumbor allows users to see resized versions of media hosted on a wiki (e.g. Commons). For example, for thumbnails.

It can be used for any media including images, videos, PDFs, etc.

Test documentation edit

Where to test it edit

Thumbor is available on the beta and production environments. However, to test the latest version you need to install it locally.

How to install locally edit

git clone https://gerrit.wikimedia.org/r/operations/software/thumbor-plugins/
cd thumbor-plugins

Create a thumbor.conf in the same directory. Here is the one I currently use phab:P37284.

If you want to run it in docker (recommended):

blubber .pipeline/blubber.yaml dev > Dockerfile
make build

Otherwise, if you want to run it on "bare metal":

python3 -m pip install -r requirements.txt
python3 setup.py build
python3 setup.py install --user
thumbor --port 8800 --conf=thumbor.conf -a wikimedia_thumbor.app.App

Thumbor should then be running on port 8800.

To check it is running, visit: http://localhost:8800/thumbor/unsafe/320x/https://upload.wikimedia.beta.wmflabs.org/wikipedia/en/1/1b/CuteGoat.jpg

(See also README.)

Coverage criteria edit

Types of media (PNG, JPG, OGV, webm, 3D, PDF, DjVU, SVG, etc.)

Oracles edit

Previous versions of thumbor (e.g. those installed on beta or production)

Risks edit

Quality criteria edit

Techniques edit

Blink testing
Automatic comparison (e.g. using imagemagick's compare or GIMP)

Test data edit

Any media (including video, pdf) on Commons can be passed to Thumbor. There are lots of places to find images.

https://commons.wikimedia.beta.wmflabs.org/wiki/Special:NewFiles (allows you to filter by types of media, e.g. vector, bitmap, video)
https://commons.wikimedia.beta.wmflabs.org/wiki/Special:MIMESearch (allows you to filter by mime type)
allimages API (see script)

Test tools and scripts edit

Script to compare an instance of thumbor running locally with thumbor on beta: https://gitlab.wikimedia.org/dwalden/thumbor-testing/-/blob/main/comparison_prototype.sh
- You should change line 31 with output from the below script
Script to find lots of media for testing: https://gitlab.wikimedia.org/dwalden/thumbor-testing/-/blob/main/allimages.py

Logs and debugging edit

To see more detailed logs on the command line, run with the -d -l debug flags, like:

thumbor -d -l debug --port 8800 --conf=thumbor.conf -a wikimedia_thumbor.app.App

Current testing status edit

Test strategy edit

Compare the thumbnails generated by the latest version of thumbor with the older version running on beta.