Extension:CirrusSearch/de

MediaWiki-Erweiterungen

CirrusSearch Freigabestatus: stabil
Einbindung	Suche, API , Hook
Beschreibung	Implementiert eine Suchfunktion in MediaWiki, die Elasticsearch benutzt.
Autor(en)	Nik Everett, Chad Horohoe, Erik Bernhardson
Letzte Version	Kontinuierliche Aktualisierungen
Kompatibilitätspolitik	Snapshots werden zusammen mit MediaWiki veröffentlicht. Der Master ist nicht abwärtskompatibel.
Composer	mediawiki/cirrussearch
Lizenz	GNU General Public License 2.0 oder neuer
Herunterladen	Erweiterung herunterladen Git [?]: Git master herunterladen Repository durchsuchen (Phabricator · GitHub) Ergänzungen und Änderungen Repository-Mitwirkende (GitHub) Code-Review README
Parameter $wgCirrusSearchDeduplicateInQuery $wgCirrusSearchLanguageWeight $wgCirrusSearchAutomationCIDRs $wgCirrusSearchUseIcuFolding $wgCirrusSearchStemmedWeight $wgCirrusSearchQueryStringMaxDeterminizedStates $wgCirrusSearchCrossClusterSearch $wgCirrusSearchExtraIndexSettings $wgCirrusSearchAutomationUserAgentRegex $wgCirrusSearchTalkNamespaceWeight $wgCirrusSearchPrefixWeights $wgCirrusSearchPrefixSearchRescoreProfile $wgCirrusSearchDisableUpdate $wgCirrusSearchActiveTest $wgCirrusSearchExtraFieldsInSearchResults $wgCirrusSearchMoreLikeThisMaxQueryTermsLimit $wgCirrusSearchUseIcuTokenizer $wgCirrusSearchCompletionBannedPageIds $wgCirrusSearchOptimizeIndexForExperimentalHighlighter $wgCirrusSearchRescoreProfiles $wgCirrusSearchPhraseRescoreBoost $wgCirrusSearchInterwikiProv $wgCirrusSearchDefaultCluster $wgCirrusSearchQueryStringMaxWildcards $wgCirrusSearchElasticQuirks $wgCirrusSearchMaxFileTextLength $wgCirrusSearchFallbackProfiles $wgCirrusSearchMoreLikeThisTTL $wgCirrusSearchAllowLeadingWildcard $wgCirrusSearchInterwikiPrefixOverrides $wgCirrusSearchMaintenanceTimeout $wgCirrusSearchReplicas $wgCirrusSearchPhraseSlop $wgCirrusSearchBoostOpening $wgCirrusSearchWriteBackoffExponent $wgCirrusSearchUserTesting $wgCirrusSearchShardCount $wgCirrusSearchUseCompletionSuggester $wgCirrusSearchPhraseSuggestReverseField $wgCirrusSearchFallbackProfile $wgCirrusSearchFragmentSize $wgCirrusSearchUnlinkedArticlesToUpdate $wgCirrusSearchCustomPageFields $wgCirrusSearchClientSideUpdateTimeout $wgCirrusSearchIgnoreOnWikiBoostTemplates $wgCirrusSearchRegexMaxDeterminizedStates $wgCirrusSearchInterwikiHTTPConnectTimeout $wgCirrusSearchExtraIndexes $wgCirrusSearchCategoryDepth $wgCirrusSearchMergeSettings $wgCirrusSearchClusters $wgCirrusSearchCrossProjectShowMultimedia $wgCirrusSearchBannedPlugins $wgCirrusSearchMoreLikeThisConfig $wgCirrusSearchClusterOverrides $wgCirrusSearchCrossProjectBlockScorerProfiles $wgCirrusSearchEnableIncomingLinkCounting $wgCirrusSearchNearMatchWeight $wgCirrusSearchWriteIsolateClusters $wgCirrusSearchIndexedRedirects $wgCirrusSearchIndexAllocation $wgCirrusSearchNumCrossProjectSearchResults $wgCirrusSearchLanguageDetectors $wgCirrusSearchUpdateShardTimeout $wgCirrusSearchEnableCrossProjectSearch $wgCirrusSearchFullTextQueryBuilderProfiles $wgCirrusSearchCompletionDefaultScore $wgCirrusSearchWriteClusters $wgCirrusSearchCompletionSuggesterHardLimit $wgCirrusSearchRecycleCompletionSuggesterIndex $wgCirrusSearchLogElasticRequests $wgCirrusSearchConnectionAttempts $wgCirrusSearchElasticaWritePartitionCounts $wgCirrusSearchWikiToNameMap $wgCirrusSearchMaxFullTextQueryLength $wgCirrusSearchLogElasticRequestsSecret $wgCirrusSearchEnableAltLanguage $wgCirrusSearchNaturalTitleSort $wgCirrusSearchDeduplicateInMemory $wgCirrusSearchEnableRegex $wgCirrusSearchClientSideSearchTimeout $wgCirrusSearchIndexFieldsToCleanup $wgCirrusSearchUseEventBusBridge $wgCirrusSearchDeduplicateAnalysis $wgCirrusSearchExtraBackendLatency $wgCirrusSearchNamespaceMappings $wgCirrusSearchWMFExtraFeatures $wgCirrusSearchPreferRecentUnspecifiedDecayPortion $wgCirrusSearchNamespaceResolutionMethod $wgCirrusSearchDocumentSizeLimiterProfiles $wgCirrusSearchSearchShardTimeout $wgCirrusSearchCategoryMax $wgCirrusSearchPrivateClusters $wgCirrusSearchSimilarityProfiles $wgCirrusSearchCategoryEndpoint $wgCirrusSearchRescoreFunctionChains $wgCirrusSearchPoolCounterKey $wgCirrusSearchCompletionProfiles $wgCirrusSearchMaxShardsPerNode $wgCirrusSearchWeights $wgCirrusSearchEnableArchive $wgCirrusSearchSimilarityProfile $wgCirrusSearchInterwikiThreshold $wgCirrusSearchIndexDeletes $wgCirrusSearchDocumentSizeLimiterProfile $wgCirrusSearchFiletypeAliases $wgCirrusSearchDevelOptions $wgCirrusSearchPrefixSearchStartsWithAnyWord $wgCirrusSearchUpdateConflictRetryCount $wgCirrusSearchInterwikiHTTPTimeout $wgCirrusSearchFetchConfigFromApi $wgCirrusSearchBoostTemplates $wgCirrusSearchExtraIndexBoostTemplates $wgCirrusSearchPrefixIds $wgCirrusSearchFullTextQueryBuilderProfile $wgCirrusSearchStripQuestionMarks $wgCirrusSearchMoreLikeThisFields $wgCirrusSearchIndexBaseName $wgCirrusSearchMasterTimeout $wgCirrusSearchSanityCheck $wgCirrusSearchTextcatConfig $wgCirrusSearchNamespaceWeights $wgCirrusSearchCrossProjectOrder $wgCirrusSearchTextcatModel $wgCirrusSearchRescoreProfile $wgCirrusSearchMoreAccurateScoringMode $wgCirrusSearchMaxPhraseTokens $wgCirrusSearchCrossProjectSearchBlockList $wgCirrusSearchCrossProjectProfiles $wgCirrusSearchLanguageToWikiMap $wgCirrusSearchMaxIncategoryOptions $wgCirrusSearchEnableEventBusWeightedTags $wgCirrusSearchWikimediaExtraPlugin $wgCirrusSearchCompletionSuggesterUseDefaultSort $wgCirrusSearchLinkedArticlesToUpdate $wgCirrusSearchCompletionSuggesterSubphrases $wgCirrusSearchPreferRecentDefaultHalfLife $wgCirrusSearchSlowSearch $wgCirrusSearchFunctionRescoreWindowSize $wgCirrusSearchEnablePhraseSuggest $wgCirrusSearchCompletionSettings $wgCirrusSearchUseExperimentalHighlighter $wgCirrusSearchFeedbackLink $wgCirrusSearchUpdateDelay $wgCirrusSearchRefreshInterval $wgCirrusSearchInterleaveConfig $wgCirrusSearchPhraseRescoreWindowSize $wgCirrusSearchMoreLikeThisAllowedFields $wgCirrusSearchPreferRecentDefaultDecayPortion $wgCirrusSearchClientSideConnectTimeout $wgCirrusSearchPhraseSuggestUseText $wgCirrusSearchPhraseSuggestProfiles $wgCirrusSearchDefaultNamespaceWeight $wgCirrusSearchInterwikiSources $wgCirrusSearchPhraseSuggestUseOpeningText $wgCirrusSearchICUNormalizationUnicodeSetFilter $wgCirrusSearchICUFoldingUnicodeSetFilter $wgCirrusSearchReplicaGroup
Verwendete Hooks APIAfterExecute APIQuerySiteInfoGeneralInfo APIQuerySiteInfoStatisticsInfo ApiBeforeMain ArticleRevisionVisibilitySet BeforeInitialize CirrusSearchAddQueryFeatures CirrusSearchAnalysisConfig CirrusSearchSimilarityConfig GetPreferences LinksUpdateComplete PageDelete PageDeleteComplete PageMoveComplete PageUndeleteComplete PrefixSearchExtractNamespace ResourceLoaderGetConfigVars SearchGetNearMatch SearchIndexFields ShowSearchHitTitle SoftwareInfo SpecialSearchResults SpecialSearchResultsAppend SpecialStatsAddExtra TitleMove UploadComplete UserGetDefaultOptions
Bereitgestellte Hooks CirrusSearchAddQueryFeatures CirrusSearchAnalysisConfig CirrusSearchBuildDocumentFinishBatch CirrusSearchBuildDocumentLinks CirrusSearchBuildDocumentParse CirrusSearchFulltextQueryBuilder CirrusSearchFulltextQueryBuilderComplete CirrusSearchMappingConfig CirrusSearchProfileService CirrusSearchRegisterFullTextQueryClassifiers CirrusSearchScoreBuilder
Quarterly downloads	263 (Ranked 19th)
Public wikis using	1,226 (Ranked 212nd)
Übersetze die CirrusSearch-Erweiterung, wenn sie auf translatewiki.net verfügbar ist
Vagrant-Rolle	cirrussearch
Probleme	Offene Aufgaben · Einen Fehler melden

Die CirrusSearch-Erweiterung implementiert eine Suchfunktion in MediaWiki, die Elasticsearch benutzt.

Elasticsearch is a standalone third-party software you must install as a requirement for this extension. It is a database system that provides search and indexing functionality, where the current text of your wiki pages gets indexed for faster and improved search results. The communication between MediaWiki and ElasticSearch is done through web services.

See also the help page on using this extension.

Ziele

• Keine native Abhängigkeiten, die die Installation erschweren würden

** Die einzigen Abhängigkeiten sind reine PHP-Mediawiki-Erweiterungen und Elasticsearch selbst

• Einen nahezu in Echtzeit aktualisierten Suchindex für Wikiseiten bereitstellen, der durch andere MediaWiki-Erweiterungen erweiterbar ist.

• Alle Abfrageoptionen bereitstellen, die MWSearch Usern lieferte und mehr

Abhängigkeiten

PHP und cURL

• Zusätzlich zu den Mediawiki-Standard-Anforderungen an PHP, muss PHP für CirrusSearch mit cURL-Support kompiliert worden sein.

Elasticsearch

• Man muss install Elasticsearch|Elasticsearch installieren.

Every version of ElasticSearch changes how web services work and causes compatibility problems. You must install the version of Elastic Search compatible with the version of MediaWiki you are currently using:

MediaWiki 1.39+ require Elasticsearch 7.10.2 (6.8.23+ is possible using a compatibility layer ). See this revision for compatibility information with earlier versions of MediaWiki.

Elasticsearch versions before 6.8 are incompatible with PHP 8+.

Zu beachten ist, dass zusätzlich eine Java-Installation, wie OpenJDK, benötigt wird. It's best to use the official Elasticsearch Docker image or a self-hosted version. A managed product like Amazon OpenSearch (formerly Amazon Elasticsearch) can work but may require additional configuration depending on its specifics. For example, Amazon OpenSearch only listens for Elasticsearch API requests over HTTPS on port 443 (i.e., it does not expose the default Elasticsearch port 9200), so a TLS-enabled proxy (e.g., Nginx) can enable CirrusSearch to communicate with an Amazon OpenSearch cluster.

Elastica

• Elastica ist eine PHP-Bibliothek um mit Elasticsearch zu reden. Elastica kann nach der Anleitung weiter unten installiert werden.

Andere

• Aufgrund der tatsächlichen Handhabung der Aufträge durch die CirrusSearch-Erweiterung ist es ratsam, Aufträge in redis aufzusetzen, um Nachrichten wie Notice: unserialize(): Error at offset 64870 of 65535 bytes in JobQueueDB.php und Unsupported operand types zu verhindern. Siehe task T157759.

Installation

Elastica

Obwohl unten beschrieben wird, dass man Composer nur ausführen soll soll, wenn man von Git installiert, kann es nötig sein, Composer trotzdem auszuführen, um alle PHP-Abhängigkeiten zu installieren.

• Die Erweiterung herunterladen und die Datei(en) in ein Verzeichnis namens Elastica im Ordner extensions/ ablegen.
  Entwickler und Code-Beitragende sollten stattdessen die Erweiterung von Git installieren, mit:cd extensions/
git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/Elastica
• Wird von git aus installiert, ist Composer auszuführen, um PHP-Abhängigkeiten zu installieren, indem composer install --no-dev im Erweiterungsverzeichnis ausgeführt wird. (Siehe task T173141 für mögliche Komplikationen.)
• Folgenden Code am Ende deiner LocalSettings.php -Datei einfügen:

  wfLoadExtension( 'Elastica' );
  

•  Erledigt – Navigiere zu Special:Version in deinem Wiki, um zu überprüfen, ob die Erweiterung erfolgreich installiert wurde.

CirrusSearch

• Die Erweiterung herunterladen und die Datei(en) in ein Verzeichnis namens CirrusSearch im Ordner extensions/ ablegen.
  Entwickler und Code-Beitragende sollten stattdessen die Erweiterung von Git installieren, mit:cd extensions/
git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/CirrusSearch
• Wird von git aus installiert, ist Composer auszuführen, um PHP-Abhängigkeiten zu installieren, indem composer install --no-dev im Erweiterungsverzeichnis ausgeführt wird. (Siehe task T173141 für mögliche Komplikationen.)
• Folgenden Code am Ende deiner LocalSettings.php -Datei einfügen:

  wfLoadExtension( 'CirrusSearch' );
  

• Now follow the setup instructions in the CirrusSearch README delivered with your extension i.e. $IP/extensions/CirrusSearch/README. Note that all info in it might not apply to your version of the extension, especially the version of Elasticsearch supported.
• Konfiguriere nach Bedarf.
•  Erledigt – Navigiere zu Special:Version in deinem Wiki, um zu überprüfen, ob die Erweiterung erfolgreich installiert wurde.

Enable regex queries

This is an optional step. You will need to install the search-extra plugin for this. Do so by following these steps:

1. execute the following command:

   /usr/share/elasticsearch/bin/elasticsearch-plugin/elasticsearch-plugin install org.wikimedia.search:extra:7.10.2-wmf12
   

2. add the following line to your LocalSettings.php file:

   $wgCirrusSearchWikimediaExtraPlugin[ 'regex' ] = [ 'build', 'use', 'max_inspect' => 10000 ];
   

3. restart Elasticsearch with the following command:

   systemctl restart elasticsearch
   

4. recreate the search index by executing the following commands:

   1. php path/to/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php --startOver
      

   2. php path/to/extensions/CirrusSearch/maintenance/ForceSearchIndex.php
      

Upgrade

Bitte folge die Upgrade-Anweisung in der CirrusSearch-UPGRADE-Datei.

Konfiguration

The configuration parameters of CirrusSearch are documented at the "settings.txt" file. See also documentation on CirrusSearch configuration profiles.

Elasticsearch will fail to index for CirrusSearch if one uses a database name for MySQL containing a capital character, e.g., "MyWikiDatabaseName." To mitigate this, CirrusSearch provides the $wgCirrusSearchIndexBaseName configuration parameter, which one needs to set, e.g., $wgCirrusSearchIndexBaseName = 'mywikidatabasename';.

Hooks

CirrusSearch extension defines a number of hooks that other extensions can make use of to extend the core schema and modify documents. The following hooks are available:

• CirrusSearchAnalysisConfig - allows to hook into the configuration for analysis
• CirrusSearchMappingConfig - allows configuration of the mapping of fields
• CirrusSearchBuildDocumentParse - allows extensions to modify ElasticSearch document produced from a page
• CirrusSearchBuildDocumentLinks - allows extensions to process incoming and outgoing links for the document
• CirrusSearchBuildDocumentFinishBatch - called when a batch of pages has been indexed
• CirrusSearchAddQueryFeatures - allows extensions to add query parser features
• CirrusSearchScoreBuilder - allows extensions to define rescore builder functions
• CirrusSearchProfileService - allows extension to declare various search components and configuration

API

CirrusSearch features can be used in API queries. Searching happens via the normal search API, action=query&list=search; you can use CirrusSearch-specific features, such as the morelike: special prefix to find pages related to Marie Curie and radium:

api.php?action=query&list=search&srsearch=morelike:Marie_Curie%7Cradium&srlimit=10&srprop=size&formatversion=2

Custom APIs and parameters are provided for querying CirrusSearch configuration and debug information:

• action=cirrusdump module: 2014?action=cirrusdump
• cirrusDumpQuery parameter to Special:Search or search API queries: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpQuery
• cirrusDumpResult parameter to Special:Search or search API queries: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpResult
• An additional parameter, cirrusExplain, can be passed with cirrusDumpResult to have the Lucene explanation of the score included with the result dump: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpResult&cirrusExplain It can also be used to get the explanation in a human-readable format, by giving it one of the values verbose, pretty or hot, such as: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpResult&cirrusExplain=pretty
• cirrus-config-dump, cirrus-settings-dump, cirrus-mapping-dump, cirrus-profiles-dump modules to obtain dump from the CirrusSearch setup: api.php?action=cirrus-config-dump&formatversion=2

Siehe auch

Allgemeine Links

• Usage help page - CirrusSearch usage documentation (needed after the install)
• Projektseite
• Info about Wikimedia Cirrus/Elastic setup
• Configuration help page - sets of tunable parameters that influence various aspects of the indexing
• Extension:WikiSearch - provides faceted search API for Semantic MediaWiki using ElasticSearch.
• Extension:AdvancedSearch - Enhances Special:Search by providing advanced parameters

Debugging

• How to determine that Cirrus is actually used as the search backend

Local development

Elastic Search service can be run with the Vagrant role (cirrussearch) and MediaWiki Vagrant.

For Docker, you can use a command like docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:6.8.2. Then follow the installation and configuration directions. If your web host is in a container, you'll want to make sure the above container is on the same network, and in the LocalSettings.php file, you will want to reference the elasticsearch as the hostname. This will not have the WMF plugins but can be sufficient for basic testing.