Security/Application Security Pipeline

PurposeEdit

This document provides guidance on how to implement security into the CI/CD pipeline, leveraging both GitLab's integrated tools and custom tools provided and developed by the Security Team.

In an effort to improve application security testing, our goal has been to “shift left” to remove more vulnerabilities earlier. The idea is to empower developers to find and fix vulnerabilities earlier in the software development lifecycle, when changes are less costly and more timely.

With security embedded into the development workflow, developers can get feedback on the security of their code as they are working, they can remediate in real time, and free up the security team’s time to focus on monitoring issues, assessing risk, and solving vulnerabilities that can’t be fixed by the developer. By continuously testing even small, incremental code changes, an avalanche of work is avoided at the end of the SDLC.

Note: The Security Team strongly recommends including security scanning tools into either migrated or new repository pipelines. These features have to be triggered both for new Merge Requests and for Continuous Development/Delivery.

Application Security Automation MethodologiesEdit

Templates Provided by the Security TeamEdit

The Security Team strongly suggests to use the relevant templates located at this GitLab repository. In order to include these templates, please see the documentation.

Custom TemplatesEdit

Even though the Security Team strongly advises against the use of custom templates, there is the possibility to implement your own custom ones.

GitLab Default TemplatesEdit

For all of the remaining Languages/Frameworks not supported by the Security Team, there is the possibility to use the GitLab default one. It features automatic language detection which works even for mixed-language projects. If any supported language is detected in the project source code it automatically runs the appropriate SAST analyzers.

Even though the results may not be the greatest, it could still provide some value. Here is the list of GitLab supported languages.

How to configure GitLab default SAST templatesEdit

To enable and configure SAST with default settings:

  1. On the top bar, select Menu > Projects and find your project.
  2. On the left sidebar, select Security & Compliance > Configuration.
  3. In the SAST section, select Enable via MR.
  4. Review the draft MR that enables SAST with the default recommended settings in the .gitlab-ci.yml file.
  5. Merge the MR to enable SAST. You should see SAST jobs run in that MR’s pipeline.

SCA ConfigurationEdit

Npm AuditEdit


Name of the template: npm-audit-nodejs-ci.yml

Description: The audit command submits a description of the dependencies configured in your project to your default registry and asks for a report of known vulnerabilities. If any vulnerabilities are found, then the impact and appropriate remediation will be calculated. If the fix argument is provided, then remediations will be applied to the package tree.

How to include it: change your .gitlab-ci.yml accordingly:

# set appsec-related template variables
variables:
    WM_APPSEC_NPM_IMAGE: "nodejs12-devel:0.0.1"

# security ci templates
include:
    # NPM audit template
    - https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/npm-audit/npm-audit-nodejs-ci.yml

Environmental variables: These tools are triggered for each and every new push or merge request. It is also possible to trigger them manually by visiting the CI/CD section on GitLab.

  • WM_APPSEC_NPM_IMAGE:the suggested image is: nodejs12-devel:0.0.1
  • WM_APPSEC_NPM_AUDIT_OPTIONS: Use this variable to specify custom flags. Default ones: --moderate=true --skip-dev=true --report=true

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. NPM Audit can also output in json.

NancyEdit


Deprecated

This tool is currently deprecated, the security team is currently working to provide an alternative.

Name of the template: golang-nancy-ci.yml

Description: It is a tool to check for vulnerabilities in your Golang dependencies, powered by Sonatype OSS Index. It currently works for projects that use dep or go mod for dependencies.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_GOLANG_IMAGE: "bullseye:20220220"
    WM_APPSEC_GOLANG_NANCY_OPTIONS: ""

include:
- https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/golang-nancy/golang-nancy-ci.yml

Environmental variables:

  • WM_APPSEC_GOLANG_IMAGE:the suggested image is: bullseye:20220220
  • WM_APPSEC_GOLANG_NANCY_OPTIONS: Use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. Multiple different output formats are supported such as: json, json-pretty, text and csv.

AuditJSEdit


Deprecated

This tool is currently deprecated, the security team is currently working to provide an alternative.

Name of the template: auditjs-nodejs-ci.yml

Description: Audits JavaScript projects using the OSS Index v3 REST API to identify known vulnerabilities and outdated package versions.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_NPM_IMAGE: "nodejs12-devel:0.0.1"


include:
    - https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/npm-auditjs/auditjs-nodejs-ci.yml

Environmental variables:

  • WM_APPSEC_NPM_IMAGE:the suggested image is: nodejs12-devel:0.0.1
  • WM_APPSEC_AUDITJS_OPTIONS: use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. AuditJS can output directly as json or as xml specifically formatted for JUnit test cases.

Npm OutdatedEdit


Name of the template: npm-outdated-nodejs-ci.yml

Description: This tool will check the registry to see if any (or, specific) installed packages are currently outdated.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_NPM_IMAGE: "nodejs12-devel:0.0.1"

include:
- https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/npm-outdated/npm-outdated-nodejs-ci.yml

Environmental variables:

  • WM_APPSEC_NPM_IMAGE:the suggested image is: nodejs12-devel:0.0.1
  • WM_APPSEC_NPM_OUTDATED_OPTIONS: use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. NPM Outdated can also output in json.

Php composer outdatedEdit


Name of the template: php-composer-outdated-ci.yml

Description: Composer Outdated is a sub-function of composer that checks for outdated dependencies.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_PHP_IMAGE: "releng/composer-php72:0.5.0-s6"

include:
    - https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/php-composer-outdated/php-composer-outdated-ci.yml

Environmental variables:

  • WM_APPSEC_PHP_IMAGE: the suggested image is: releng/composer-php72:0.5.0-s6
  • WM_APPSEC_PHP_COMPOSER_OUTDATED_OPTIONS: Use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. Composer Outdated can also output in json.

Php security checkerEdit


Name of the template: php-security-checker-ci.yml

Description: PHP Security Checker is a command line tool that checks if your PHP application depends on PHP packages with known security vulnerabilities. It uses the Security Advisories Database behind the scenes.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_PHP_IMAGE: "releng/composer-php72:0.5.0-s6"

include:
    - https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/php-security-checker/php-security-checker-ci.yml

Environmental variables:

  • WM_APPSEC_PHP_IMAGE:the suggested image is: releng/composer-php72:0.5.0-s6
  • WM_APPSEC_PHP_SECURITY_CHECKER_OPTIONS: Use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. Php Security Checker can also output in ansi, markdown, json, junit, and yaml.

Safety dbEdit


Deprecated

This tool is currently deprecated, the security team is currently working to provide an alternative.

Name of the template: python-safety-db-ci.yml

Description: Safety DB is a database of known security vulnerabilities in Python packages. The data is made available by pyup.io and synced with this repository once per month. Most of the entries are found by filtering CVEs and changelogs for certain keywords and then manually reviewing them.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_PYTHON_IMAGE: "python3-build-bullseye:0.0.1-20220306"
    
include:
- https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/python-safety-db/python-safety-db-ci.yml

Environmental variables:

  • WM_APPSEC_PYTHON_IMAGE:the suggested image is: python3-build-bullseye:0.0.1-20220306
  • WM_APPSEC_PYTHON_SAFETY_DB_OPTIONS: Use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output.

BanditEdit


Name of the template: python-bandit-ci.yml

Description: Bandit is a tool designed to find common security issues in Python code. To do this Bandit processes each file, builds an AST from it, and runs appropriate plugins against the AST nodes. Once Bandit has finished scanning all the files it generates a report.

Bandit was originally developed within the OpenStack Security Project and later rehomed to PyCQA.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_PYTHON_IMAGE: "python3-build-bullseye:0.0.1-20220306"

# security ci templates
include:
    # python bandit ci template
    - https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/python-bandit/python-bandit-ci.yml

Environmental variables:

  • WM_APPSEC_PYTHON_IMAGE:the suggested image is: python3-build-bullseye:0.0.1-20220306
  • WM_APPSEC_PYTHON_BANDIT_OPTIONS: Use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. Bandit can also output in json.

SAST ConfigurationEdit

Php Phan-taint-checkEdit


Name of the template: php-phan-taint-check-ci.yml

Description: Phan is a static analyzer for PHP that prefers to minimize false-positives. Phan attempts to prove incorrectness rather than correctness. The phan-taint-check plugin is a phan plugin which performs security-related static analysis against PHP codebases, with a specific adaptor for MediaWiki codebases.

How to include it: change your .gitlab-ci.yml accordingly:

# set appsec-related template variables
variables:
    WM_APPSEC_PHP_IMAGE: "releng/composer-php80:0.0.2-s8"

# security ci templates
include:
    - https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/php-phan-taint-check/php-phan-taint-check-ci.yml

Environmental variables:

  • WM_APPSEC_PHP_IMAGE:the suggested image is: releng/composer-php80:0.0.2-s8 (php8+ is required for a symphony dependency of phan)
  • WM_APPSEC_PPT_OPTIONS: Use this variable to specify custom cli flags.
  • WM_APPSEC_PPT_BRANCH: The branch of mediawiki to clone, if analyzing a MediaWiki extension or skin. Default is master. At some point, it would be nice to automate this to reliably use $CI_COMMIT_BRANCH.
  • WM_APPSEC_PPT_CODE_TYPE: The current supported MediaWiki types are extensions and skins. For any other type of php application, set this variable to an empty string or any other value than "extensions" or "skins".
  • WM_APPSEC_PPT_PROJECT_NAME: The project's git name or slug. Defaults to $CI_PROJECT_NAME.
  • WM_APPSEC_PPT_FULL_PHAN_OUTPUT: Specify whether you'd like to see all phan issues (prefixed with Phan) or just those from the phan-taint-check plugin (prefixed with SecurityCheck). Defaults to false

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output.

GosecEdit


Name of the template: golang-gosec-ci.yml

Description: Inspects source code for security problems by scanning the Go AST.

How to include it: change your .gitlab-ci.yml accordingly:

variables:
    WM_APPSEC_GOLANG_IMAGE: "bullseye:20220220"
    WM_APPSEC_GOLANG_GOSEC_OPTIONS: ""

include:
- https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/golang-gosec/golang-gosec-ci.yml

Environmental variables:

  • WM_APPSEC_PYTHON_IMAGE:the suggested image is: bullseye:20220220
  • WM_APPSEC_GOLANG_GOSEC_OPTIONS: Use this variable to specify custom flags.

Expected output: The Default output is the textual report of vulnerabilities in the Gitlab CI console output. Gosec also supports json, yaml, csv, sonarqube, JUnit XML, html and golint output formats.

SemgrepEdit


In order to use Semgrep, include it in your .gitlab-ci.yml like any other provided templates. You can configure which ruleset to run by changing the variable WM_APPSEC_SEMGREP_OPTIONS as showed here (you can chain multiple --config=r/*ruleset*):

variables:
    WM_APPSEC_PYTHON_IMAGE: "python3-build-bullseye:0.0.1"
    # A simple configuration for a PHP application:
    WM_APPSEC_SEMGREP_OPTIONS: "--config=r/php --metrics=off --time --exclude=vendor"

include:
    # semgrep template
    - https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/0.1.0/semgrep/semgrep-ci.yml

You can select one of the available docker image from https://docker-registry.wikimedia.org/.

Note: The Security Team discourages the use of --config=auto because metrics will be reported back to Semgrep.

Default RulesetsEdit

The semgrep-ci template contains the most relevant security-related ruleset picked by the security team:

  1. p/security-audit
  2. p/ci
  3. p/secrets
  4. p/supply-chain
  5. p/insecure-transport
Suggested RulesetsEdit

In order to keep the workload at the minimum, the security team has picked the most relevant security rulesets, however here are reported some other good generic and language-specific candidates that could be added:

Security GenericEdit
  1. p/owasp-top-ten
  2. p/r2c-ci
  3. p/jwt
  4. p/xss
JavaScriptEdit
  1. p/javascript
  2. p/typescript
  3. p/nodejsscan
  4. p/nodejs
  5. p/clientside-js
PHPEdit
  1. r/php
  2. p/phpcs-security-audit
PythonEdit
  1. p/python
  2. p/gitlab-bandit
GolangEdit
  1. p/golang
  2. p/semgrep-go-correctness
JavaEdit
  1. p/java
  2. p/test
Custom RulesetsEdit

Semgrep offers the chance to develop your own ruleset. The suggested way to do that would be by creating a new .yml file that contains your rules (either on your repository, or on the security/gitlab-ci-security-templates repository), and then referencing this .yml file from the .gitlab-ci.yml like this:

variables:
        SEMGREP_OPTIONS:
            --config=https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/raw/main/semgrep/rules/custom-rules.yml
Default Scanning behaviourEdit

Semgrep by default will only check the changed files from any Push and Merge Requests. In order to trigger a full scan, you must manually run the pipeline or schedule it.

Note: It is still possible to override this default behaviour by setting the variable WM_APPSEC_SEMGREP_ALL_FILES to "true" in your .gitlab-ci.yml:

WM_APPSEC_SEMGREP_ALL_FILES: "true"

ResultsEdit

Security automation is far from perfect. Even though our goal is to have straight-forward scan results and fully automated remediations, unfortunately that is not always possible. Results may be polluted by false positives, and remediations may appear unclear. When these scenarios occur, please get in touch with the Security Team to work on possible solutions.

Risk review and acceptanceEdit

Note: Repository owners are implicitly accepting the risk of the findings from various security tools run in Gitlab CI. It is thus their responsibility to fix issues or to accept the risk of these results. Please contact the Security Team to get help on how to address the issues.

How to check resultsEdit

In order to check the output of various security tools, please go to the Jobs section of your repository, under the CI/CD > Pipelines section of your repository. Make sure you are viewing the correct branch of a repository as well.

Based on the specific tool configuration, a job could never fail, or fail only when new critical/high/moderate vulnerabilities are discovered. For example, the composer outdated and npm outdated are set to allow failures within a given pipeline, implying they are more informational in nature and shouldn't block any merge or deployment. Whereas as semgrep and other tools would fail within a pipeline, with the expectation that a code maintainer would work through various reported issues and either resolve them or report them as false positives and possibly suppress future output with various mechanisms supported by an individual tool.

JSON ArtifactsEdit

Sometimes security tools emit JSON output in the form of a report. The JSON report file can be downloaded from the CI pipelines page, or the pipelines tab on merge requests by setting artifacts: paths to gl-sast-report.json. For more information see the documentation and [1]. This can be a more secure and customizable approach to security reports, but as nearly all Wikimedia repos, merge requests, etc. are or will be public, there isn't much advantage to this approach given current realities.

Example

For phpcs-security-tool add this to the gitlab-ci.yml file:

phpcs-security-audit-sast:
artifacts:
paths:
- gl-sast-report.json

ReferencesEdit

Known Guides and DocumentationEdit