MediaWiki

User:Aron Manning/RfC: Evaluate alternative Node package managers for improved package security

< User:Aron Manning
Tracked in Phabricator
Task T257068

Prepared by Demian aka. Aron. Initial message to wikitech-l.

This RFC (descoped from phab:T199004) is about evaluating alternative package managers in regards of:

  • security benefits
  • impact on development and deployment workflows
  • migration workload

The purpose is to test these PMs without changing or disrupting established workflows. One or both PMs can be introduced for testing by developers as an optional alternative in the codebase, without impacting production.
This testing phase might take months, involving compatibility updates to wikimedia projects and upstream packages.
If such test is successful, it can be considered to migrate production workflows from NPM to a chosen alternative.

The package managers in scope (details below, feel free to expand):


Motivation edit

The issue of package security has been concerning the Node community for years. Packages in the NPM repository are vetted by the community only after publication, leaving a chance for attackers to publish a malicious package version, which then gets deployed to developer boxes and possibly servers under the radar when npm update is run. Such malicious packages are detected only after some time, possibly reaching a number of users. This is a possibility to be prepared for, but constant fear would be unjustified: only a few cases happened in the years and those were quickly removed. The community is resilient and constructive, leaving little room for bad intent.

Nonetheless, even a short breach at a content provider of the size of Wikimedia could affect a great number of users, therefore it's appropriate to have prevention measures in place.

It's important to note that all public, open package repositories are subject to such malice, not just NPM (for ex. PyPI). The difficulty with NPM is that there are many small packages, resulting in a bigger dependency tree (phab:T199004#6045136) and significantly more packages to review than with PHP packages for ex.

A few examples of NPM incidents edit

Status quo edit

Currently Wikimedia is somewhat protected by pinning the version number of major libraries (the top-level dependencies of projects) and vetting those versions. The list is in libraryupgrader2. However, the version of dependencies of dependencies are not controlled with this solution. An existing attack vector is to gain control of the credentials of a minor library deep in the dependency tree and upload a new version with malicious code injected, masquerading behind the declared functionality, or to create one with future malicious intent and get some established libraries to use it before injecting the malicious code. Such code could be released to production with a train deployment if the 'package-lock.json' file on the master branch was updated by libraryupgrader2 or a developer after the release of the malicious version and before it gets removed from NPM. It's a short timeframe and a difficult attack to carry out with a small, but non-zero possibility.

Package managers edit

Without aiming for completeness, there are 2 recent solutions that target specifically the question of security:

Yarn 2 edit

  1. npm i -g yarn@berry
  2. Introductory article
  3. Yarn 2 was released in 2020. It is a recent, fundamentally different version from (Classic) Yarn 1 (offline mirror ), which made only minor improvements to npm.
  4. A central offline cache can be synced to individual nodes through a git repository, or presumably other means, thus guaranteeing complete control over outside source-code entering the Wikimedia ecosystem.
  5. With Yarn 2, package files are loaded directly from the offline cache (doc) , thereby eliminating the 'node_modules' folders, the duplicated packages and speeding up the install step. This breaks many packages assuming the presence of 'node_modules'.
  6. Packages using resolve 1.9.0 (published 2 years ago) are compatible (popular packages ).
  7. Some upstream packages and wikimedia projects need to be upgraded to use 'resolve'. In the meantime an adapter called 'pnpify' can be used to execute these. If 'pnpify' fails then the package can be 'unplugged' traditional 'node_modules' can be created with the nodeLinker: node-modules setting (migration ).
  8. In strict mode only direct dependencies are reachable by packages. This is the correct behavior, but NPM's flattening of node_modules allowed developers to look for indirect dependencies too. Those packages need to be upgraded or the pnpMode: loose setting set.
  9. Migration: CLI , Q&A
  10. Yarn supports workspaces, but uses a confusing vocabulary where Project > Worktree > Workspace. Our git repos (skins, extensions) would be called "workspace". The "core" repo would be the "project" or "worktree".
  11. Script names containing ':' are global, can be executed in any "workspace" of the "project". Scripts like 'svgmin' need to be defined only in the root package.json.
  12. '.yarn/cache' is stored only in the root.
  13. The classic version was created by Facebook, Exponent, Google, and Tilde (article), GitHub lists 511 contributors . The active developers now are not Facebook employees (ref: Q&A ). Being released just half a year ago, it's popularity is beneficial when it comes to its drawback: incompatibility .

Pnpm edit

  1. npm i -g pnpm
  2. Pnpm package store .
  3. Pnpm was released in 2017, after Classic Yarn.
  4. Pnpm deduplicates packages using symlinks, recreates node_modules, but only including direct (top-level) dependencies without flattening the dependency tree.
  5. Packages can load only direct dependencies. Other than that there aren't many complications and setup is easier: if a tool is missing a package, add it to 'devDependencies'.
  6. Supports workspaces with a simple vocabulary: Workspace > Project. Node packages are linked to the instance in the workspace root if the version constraints allow it. This is the case with libraryupgrader2 keeping dependency versions consistent along projects.
  7. Drawback of its workspace implementation: 'pnpm install' anywhere in the workspace attempts to create/fix the 'node_modules' folder of all projects in the workspace, even if the developer only needs to do that for one project. If installing a package fails in any of the repos, the install process fails for all repos, creating an unusable state. This might be a reason to not use workspaces, or find a way around this behavior.
  8. Pnpm was created by one person, it's sponsored by 2 companies , used by quite a few know companies and GitHub lists 61 contributors .
  9. An article briefly mentioning why Pnpm workspaces were simpler to set up than Yarn 2's.

npm edit

  • For completeness it should be mentioned that full control over package versions is achievable with plain old NPM by generating 'package-lock.json' from libraryupgrader2, or using a 'package.json' generated with the exact version of all dependencies, direct and indirect. If developers use `npm ci` instead of `npm i`, their installed version will match too.

Common edit

The configuration and lock files of all 3 PMs can be checked into the same repository without conflict. The PM to use on any box can be chosen independently, allowing the testing of an alternative PM while NPM is used in production. Depending on usability and personal preference, it's also an option for developers to chose their favored PM to use. The workspace management features and simple setup might be reasons to choose on PM over the other.

  1. 'package.json' is shared, the package versions installed are expected to be the same if run at the same time as NPM (same state of NPM repo).
  2. 'package-lock.json' is NOT shared, there are individual 'yarn.lock' and 'pnpm-lock.yaml' files, these need to be generated separately.
  3. Plugins often use 'peerDependencies' to refer to their main package (eg. eslint-plugin-*, stylelint-plugin-*). Both PMs are stricter than NPM and require that the main package is properly declared as dependencies besides the plugins. Some wikimedia 'package.json' files need to be updated to satisfy this constraint, most notably by adding 'eslint', 'stylelint' and their plugins. This strictness might be unwelcome at first, but quickly adapted.
  4. Grunt currently can't load plugins with either PM. Issue submitted upstream. Grunt has lost momentum in last years, so I'll be looking into submitting a PR.

Scope edit

This ticket is only concerned about the tool distributing already audited packages. Code auditing practices and tools are out of scope. It is assumed that libraryupgrader2 (or similar) would be used to track all packages in our dependency trees, down to the last leaf. If the matter of tracking vetted package versions need to be discussed, please open a separate ticket.

Questions edit

  1. Security:
    1. How much control it gives over package versions and code integrity?
    2. How to integrate with libraryupgrader2 ?
    3. How much delay is between a version is published on NPM -> delivered to 1) developers, 2) CI, 3) pre-deploy build?
    4. How much time a version is exposed to CI before being delivered to 1) developers, 2) pre-deploy build?
    5. How much time a version is exposed to developers before being used for pre-deploy build?
  2. Progressive transition:
    1. Is the PM usable in parallel with NPM to allow preparation without disrupting the established processes?
    2. What features require committing to changing those processes, what's the impact?
    3. What's the impact of changing PM on CI nodes, developer machines?
  3. Changes to package loading:
    1. How to "unplug" (Yarn terminology) packages incompatible with alternative package store (node_modules)?
    2. What packages need to be unplugged or upgraded to load dependencies (those using custom loaders, not require.resolve())?
  4. What steps are necessary to make wikimedia projects compatible with the PM, in detail:
    1. The install process works without warnings: $PM install
    2. Installed packages work (load) as expected.
    3. Scripts work as with NPM: $PM run <scriptname>
  5. What's the resource usage:
    1. Storage space required for package store (central), local cache (each CI node and developer box), installed packages (in workspace and each project).
    2. Network usage vs cache usage.
    3. Install and update time.
  6. Usability:
    1. What are the changes to developers?
    2. What are the common problems encountered by users?
    3. Is the developer experience improved?
  7. Workspaces:
    1. Is it suitable for our multi-repository setup?
    2. What benefits it gives for managing multiple projects that make up a MediaWiki developer instance?
Retrieved from "https://www.mediawiki.org/w/index.php?title=User:Aron_Manning/RfC:_Evaluate_alternative_Node_package_managers_for_improved_package_security&oldid=3953306"