Release status: experimental
|Implementation||other (invalid type)|
|Description||Service for detecting possible child exploitation content|
|Author(s)||Wikimedia Core Platform & Trust and Safety Teams (Peter.ovchyntalk)|
|Compatibility policy||release branches|
|License||GNU General Public License 2.0 or later|
|Translate the MediaModeration extension if it is available at translatewiki.net|
|Check usage and version matrix.|
|Issues||Open tasks · Report a bug|
The MediaModeration extension for detecting possible child exploitation content.
The purpose of the extension is to improve the Foundation’s existing workflows for child protection content.
Prior to deployment of this extension, when the Foundation receives a report of images that depict child sexual abuse, the images are deleted from the projects and reported to law enforcement according to legal requirements. This setup requires volunteers, who unlike staff have no professional training or mental health support, to initially deal with this very emotionally taxing content.
This extension aims to protect the community from being exposed to such content in nearly all cases and get it off the platform a lot faster. It would check images against a database of hashed, known images of child sexual abuse and notify Foundation staff of hash matches to allow Foundation staff to remove the images and report their existence to law enforcement.
This extension does not automatically remove any content without human review by Foundation staff.
MediaModeration provides the following:
- Check uploaded image against PhotoDNA
- Send email to pre-configured recipients if suspicious content found
- Allows sending 160x160 thumbnails instead of a full image
Before installation, the PhotoDNA subscription key should be obtained on Microsoft cloud portal.
- Download and place the file(s) in a directory called
After installed, the extension should be configured and PhotoDNA subscription key should be provided in order to work properly.
extension.json and fill the
confing section with actial values:
|MediaModerationPhotoDNAUrl||https://api.microsoftmoderator.com/photodna/v1.0/Match||Url to photoDNA service endpoint|
|MediaModerationPhotoDNASubscriptionKey||subscription-key||Key for access to photoDNA service endpoint, obtained on microsoft cloud portal|
|MediaModerationRecipientList||[ "email@example.com" , "firstname.lastname@example.org" ]||List of emails to be notified when suspicious content is detected|
|MediaModerationFromemail@example.com||Email from which notification are sent|
|MediaModerationCheckOnUpload||false||Indicates whether files should be checked on upload. If false, checking will only be done by the ModerateExistingFiles.php maintenance script.|