User:DWalden (WMF)/Thumbor migration
Project documentation edit
For Phabricator board, team members involved, etc. see 2022_Thumbor_updates.
The Phabricator board is: https://phabricator.wikimedia.org/tag/thumbor_migration/
Feature documentation edit
Thumbor allows users to see resized versions of media hosted on a wiki (e.g. Commons). For example, for thumbnails.
It can be used for any media including images, videos, PDFs, etc.
Test documentation edit
Where to test it edit
Thumbor is available on the beta and production environments. However, to test the latest version you need to install it locally.
How to install locally edit
git clone https://gerrit.wikimedia.org/r/operations/software/thumbor-plugins/
cd thumbor-plugins
Create a thumbor.conf
in the same directory. Here is the one I currently use phab:P37284.
If you want to run it in docker (recommended):
blubber .pipeline/blubber.yaml dev > Dockerfile
make build
Otherwise, if you want to run it on "bare metal":
python3 -m pip install -r requirements.txt
python3 setup.py build
python3 setup.py install --user
thumbor --port 8800 --conf=thumbor.conf -a wikimedia_thumbor.app.App
Thumbor should then be running on port 8800.
To check it is running, visit: http://localhost:8800/thumbor/unsafe/320x/https://upload.wikimedia.beta.wmflabs.org/wikipedia/en/1/1b/CuteGoat.jpg
(See also README.)
Coverage criteria edit
- Types of media (PNG, JPG, OGV, webm, 3D, PDF, DjVU, SVG, etc.)
Oracles edit
- Previous versions of thumbor (e.g. those installed on beta or production)
Risks edit
Quality criteria edit
Techniques edit
- Blink testing
- Automatic comparison (e.g. using imagemagick's compare or GIMP)
Test data edit
Any media (including video, pdf) on Commons can be passed to Thumbor. There are lots of places to find images.
- https://commons.wikimedia.beta.wmflabs.org/wiki/Special:NewFiles (allows you to filter by types of media, e.g. vector, bitmap, video)
- https://commons.wikimedia.beta.wmflabs.org/wiki/Special:MIMESearch (allows you to filter by mime type)
- allimages API (see script)
Test tools and scripts edit
- Script to compare an instance of thumbor running locally with thumbor on beta: https://gitlab.wikimedia.org/dwalden/thumbor-testing/-/blob/main/comparison_prototype.sh
- You should change line 31 with output from the below script
- Script to find lots of media for testing: https://gitlab.wikimedia.org/dwalden/thumbor-testing/-/blob/main/allimages.py
Logs and debugging edit
To see more detailed logs on the command line, run with the -d -l debug
flags, like:
thumbor -d -l debug --port 8800 --conf=thumbor.conf -a wikimedia_thumbor.app.App
Current testing status edit
Test strategy edit
Compare the thumbnails generated by the latest version of thumbor with the older version running on beta.
You can do this with blink testing or using the scripts described above.