Developing security patches

The following is a proposed workflow for managing security/embargoed patches for Wikimedia code.

Reporting, creating and deploying security patches

Initial steps

Do not post security reports or related patches to gerrit, github or any other PUBLIC developer tools, websites, mailing lists, IRC channels, etc. prior to any responsible disclosure of the underlying security issue.

1. Upon discovery of any security issues, please review the information and steps within Wikimedia's Reporting Security Bugs documentation.
2. If you have opted to create a Phabricator security report (as opposed to emailing a report to security@wikimedia.org), you will first need to create a Phabricator account to do so if you do not have one. Then submit a secure report via this form. Submit as much detailed information as possible following the guidelines within the aforementioned Reporting Security Bugs documentation.
3. Develop your patch locally (preferably on a temporary branch) and test it within MediaWiki-Vagrant, MediaWiki-Docker or an an-hoc local development environment. It is recommended to not use a public-facing tool such as patchdemo.
4. Create a patch file within your working directory by running git format-patch HEAD^ --stdout > Txxxxx.patch (where "Txxxxx" is the Phabricator task id).
5. Upload the patch by attaching it to the relevant Phabricator task as a secure file attachment. Coordinate with other developers to review your patch by reaching out to them via email, IRC, etc. and then subscribing them to the security task. Add the Phabricator #Patch-For-Review project to the task.

For code which canonically exists within gerrit.wikimedia.org, please follow the steps below

1. Once again, please do not upload the patch publicly to gerrit for review at this time, unless approved to do so by a member of the Security Team. Backports can happen via gerrit once any relevant security patches have been deployed to Wikimedia production OR when relevant parties have been sufficiently alerted to the issue. For any clarifications on this policy, please contact security-help@wikimedia.org.

For code which canonically exists at gitlab.wikimedia.org, please follow the steps below

Assuming a security patch has been created and reviewed within Phabricator, as per the initial steps mentioned above, please proceed with the following Gitlab-specific steps.

See GitLab/Workflows/Security patches

Other systems

1. For code which canonically exists within github, please follow the steps within Wikimedia's Reporting Security Bugs documentation and do not create any public pull requests.
2. If the relevant code repositories exist canonically within another versioning system or repository, please contact security-help@wikimedia.org for additional guidance.

Deploying security patches to production

1. Once the security issue has been privately discussed within Phabricator and all relevant security patches have been tested, reviewed and approved, they should be uploaded to the current deployment server.
   • Check that the patch applies with git apply --check /path/to/patchfile on the relevant deployment branch(es).
   • Apply the patch with git am /path/to/patchfile to the relevant deployment branch(es).

   Note that some config files are made public via noc.wikimedia.org; don't put anything non-public within those.

2. Deploy as usual but use --no-log-message to prevent the automatic logging from revealing too much information regarding affected files or components. Log the deployment manually by saying "!log Deployed patch for Txxxxx" in #wikimedia-operations ^connect.
3. Ensure the security patch will be applied to future deployment branches:
   • The .patch file should be stored on the deployment host under /srv/patches/<branch>/, for whichever branches have the security patch applied.
     • Patches to MediaWiki core should be stored in /srv/patches/<branch>/core/
     • Patches to extensions should be stored under /srv/patches/<branch>/extensions/ExtensionName.
       • Filenames should be prefixed with a 2-digit number to indicate the order in which patches should be applied in the repo. Your file should be prefixed with '01-' if it is the first patch in the directory, or the next highest number if other patches already exist.
       • Patch files should be git-committed to the local repository within /srv/patches/.
   • Add a note or link to a log entry to the relevant Phabricator task noting that the security patch has been deployed.
   • If the Security Team isn't already aware of this particular security issue, be sure to inform them that you deployed the patch to prevent duplicate effort.
4. Work with the Security Team to make sure the vulnerability is resolved and that your patch makes it into the next security release.
   1. If there is no PII or other sensitive data within the associated Phabricator task, it can now be made public. If you are unsure how to do this or if the task should be made public, please consult a member of the Security Team.
   2. Perform any necessary backports within the relevant versioning systems (gerrit, GitLab, etc.) to supported release branches assuming the patch is relevant for and applies to previous versions. As an extra step, once the branch is backported to master, it can be removed from /srv/patches/ as it will no longer apply to future production release branches. This will make Release Engineering's life a little easier.
   3. Request a CVE, if appropriate. Note the CVE ID within the task's title, description and within any related security release tasks.

Potential deployment problem 1: Submodule security patches not committed

Sometimes you may find a security patch for an extension that has not been committed:

[you@deploy1001 php-1.999.0-wmf.1 (wmf/1.999.0-wmf.1 * u)]$ git status
On branch wmf/1.999.0-wmf.1
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   extensions/InsecurityExtension (new commits)

Submodules changed but not updated:

* extensions/InsecurityExtension fffffff...eeeeeee (1):
  > SECURITY: Make the Insecurity extension secure

no changes added to commit (use "git add" and/or "git commit -a")

This is normal! The reason for this state is that subsequent updates to the submodules require human intervention to fix merge/rebase conflicts.