Parsoid

This page is a translated version of the page Parsoid and the translation is 100% complete.
Other languages:
Deutsch • ‎English • ‎français • ‎polski • ‎日本語

Parsoid est une bibliothèque qui permet de convertir le wikitext en HTML et réciproquement. L'application originale est écrite en JavaScript (à l'aide de Node.js) et était opérationnelle sur le cluster Wikimedia de décembre 2012 à décembre 2019.
En 2019, Parsoid a été porté sur PHP et la version PHP a remplacé la version JS sur le cluster Wikimedia en décembre 2019. Parsoid est censé être intégré au noyau MediaWiki, dans le but de remplacer éventuellement l'analyseur natif actuel de MediaWiki.

Vue d'artiste de l'environnement d'exécution du wiki Parsoid HTML5 + RDFa

Parsoid (the PHP version) is natively bundled in MediaWiki 1.35, released in September 2020. For non-Wikimedia installations, Parsoid/JS will be supported until the end-of-life of MediaWiki 1.31 (LTS) in September 2021.

Détails techniques

Parsoid is an application which can translate back and forth between MediaWiki's wikitext syntax and an equivalent HTML/RDFa document model with enhanced support for automated processing and rich editing.

It has been under development by a team at the Wikimedia Foundation since 2012. It is currently used extensively by VisualEditor, Flow, Content Translation and other applications.

Parsoid is intended to provide flawless back-and-forth conversion, i.e. to avoid information loss and also prevent "dirty diffs".

On Wikimedia wikis, for several applications, Parsoid is currently proxied behind RESTBase, which stores the HTML translated by Parsoid. It is expected that RESTBase will eventually be replaced with a cache more tightly integrated with MediaWiki.

For more on the overall project, see this blog post from March 2013. To read about the HTML model being used, see MediaWiki DOM spec.

Parsoid was originally structured as a web service and written in JavaScript, making use of Node.js. A tech talk from February 2019 (slides) and blog post describes the porting process. The Parsoid extension API is currently under active development; a tech talk from August 2020 describes this work.

GitHub Repository: https://github.com/wikimedia/parsoid

Utilisation

Installation

In MediaWiki 1.35 LTS Parsoid/PHP is included in the bundle and loaded automatically by Visual Editor. No configuration necessary if used on a single server.

If you are a developer working on 1.36, explicitly loading Parsoid is required since August 24, 2020 (the auto-load hack was removed in 1.36-wmf.6). Add to LocalSettings.php:

wfLoadExtension( 'Parsoid', 'vendor/wikimedia/parsoid/extension.json' );

This is expected to change for the release of 1.36.

Développement

Development happens in the Parsoid Git repository. Code review happens in Gerrit. See Gerrit/Getting started to set up an account for yourself.

If you use the MediaWiki-Vagrant development environment using a virtual machine, you can simply add the role visualeditor to it and it will set up a working Parsoid along with Extension:VisualEditor. (This may have been broken by the switch to Parsoid/PHP: T258940)

Note that the most-recently released version of Parsoid is written in PHP, and installation of Parsoid/PHP is what is described below. This is what you should use if you are running MediaWiki 1.35 or later. Check Parsoid/JS if you are running the old version of Parsoid written in JavaScript, and used for MW 1.34 and earlier.

Lier une copie développeur (checkout) de Parsoid

In a standard MediaWiki installation, Parsoid code is bundled in two different ways: first, Parsoid is included from MediaWiki as a composer library, wikimedia/parsoid. This contains the main codebase, but does not contain the REST API used by VisualEditor and RESTBase. In order to enable the REST API, the extension code included in the Parsoid library can be loaded with a call to wfLoadExtension(...) in your LocalSettings.php.

For development purposes you usually want to use a git checkout of Parsoid, and not the version bundled in MediaWiki core as a composer library.

The following lines added to LocalSettings.php allow use of a git checkout of Parsoid (optionally), load the Parsoid REST API with wfLoadExtension (rather than using the version bundled in VisualEditor) and manually do the Parsoid configuration which is usually done by VisualEditor:

$PARSOID_INSTALL_DIR = 'vendor/wikimedia/parsoid'; # bundled copy
#$PARSOID_INSTALL_DIR = '/my/path/to/git/checkout/of/Parsoid';

// For developers: ensure Parsoid is executed from $PARSOID_INSTALL_DIR,
// (not the version included in mediawiki-core by default)
// Must occur *before* wfLoadExtension()
if ( $PARSOID_INSTALL_DIR !== 'vendor/wikimedia/parsoid' ) {
    AutoLoader::$psr4Namespaces += [
        // Keep this in sync with the "autoload" clause in
        // $PARSOID_INSTALL_DIR/composer.json
        'Wikimedia\\Parsoid\\' => "$PARSOID_INSTALL_DIR/src",
    ];
}

wfLoadExtension( 'Parsoid', "$PARSOID_INSTALL_DIR/extension.json" );

# Manually configure Parsoid
$wgVisualEditorParsoidAutoConfig = false;
$wgParsoidSettings = [
    'useSelser' => true,
    'rtTestMode' => false,
    'linting' => false,
];
$wgVirtualRestConfig['modules']['parsoid'] = [];

These lines are not necessary for most users of VisualEditor, who can use auto-configuration and the bundled Parsoid code included in MediaWiki 1.35 and VisualEditor, but they will be required for most developers.

If you're serving MediaWiki with Nginx, you'll need to also add something like this to your server conf:

location /rest.php/ {
    try_files $uri $uri/ /rest.php?$query_string;
}

To test proper configuration, visit {$wgScriptPath}/rest.php/{$domain}/v3/page/html/Main%20Page where $domain is the hostname in your $wgCanonicalServer. (Note that production WMF servers do not expose the Parsoid REST api to the external network.)

Exécuter les tests

To run all parser tests and mocha tests:

$ composer test

The parser tests have quite a few options now which can be listed using php bin/parserTests.php --help. If you have the environment variable MW_INSTALL_DIR pointing to a configured MediaWiki installation, you can run some additional tests with:

$ composer phan-integrated

Conversion de wikicode simple

You can convert simple wikitext snippets from the command line using the parse.php script in the bin/ directory:

echo 'Foo' | php bin/parse.php

The parse script has a lot of options. php bin/parse.php --help gives you information about this.

Debogage de Parsoid (pour les développeurs)

See Parsoid/Debugging for debugging tips.

Documents techniques

Liens pour les développeurs Parsoid

Liens pour le déploiement de Parsoid (sur les grappes Wikimedia)

Voir aussi

Liens externes

Contact

If you need help or have questions/feedback, you can contact us in #mediawiki-parsoid connect or the wikitext-l mailing list. If all that fails, you can also contact us by email at parsing-team at the wikimedia.org domain.