Extension:CirrusSearch

MediaWiki扩展手册

CirrusSearch 发行状态： 稳定版
实现	搜索, API , 函数钩
描述	使用Elasticsearch來实现搜索MediaWiki
作者	Nik Everett, Chad Horohoe, Erik Bernhardson
最新版本	持续更新
兼容性政策	快照跟随MediaWiki发布。 master分支不向后兼容。
Composer	mediawiki/cirrussearch
许可协议	GNU通用公眾授權條款2.0或更新版本
下載	下載扩展 Git [?]: 浏览存储库 (Phabricator · GitHub) Gerrit代码复核 Git提交日誌 下載原始碼(tarball壓縮) README
参数 $wgCirrusSearchDeduplicateInQuery $wgCirrusSearchLanguageWeight $wgCirrusSearchAutomationCIDRs $wgCirrusSearchUseIcuFolding $wgCirrusSearchStemmedWeight $wgCirrusSearchQueryStringMaxDeterminizedStates $wgCirrusSearchCrossClusterSearch $wgCirrusSearchExtraIndexSettings $wgCirrusSearchAutomationUserAgentRegex $wgCirrusSearchTalkNamespaceWeight $wgCirrusSearchPrefixWeights $wgCirrusSearchMustTrackTotalHits $wgCirrusSearchPrefixSearchRescoreProfile $wgCirrusSearchDisableUpdate $wgCirrusSearchActiveTest $wgCirrusSearchExtraFieldsInSearchResults $wgCirrusSearchMoreLikeThisMaxQueryTermsLimit $wgCirrusSearchUseIcuTokenizer $wgCirrusSearchCompletionBannedPageIds $wgCirrusSearchOptimizeIndexForExperimentalHighlighter $wgCirrusSearchRescoreProfiles $wgCirrusSearchPhraseRescoreBoost $wgCirrusSearchInterwikiProv $wgCirrusSearchDefaultCluster $wgCirrusSearchQueryStringMaxWildcards $wgCirrusSearchElasticQuirks $wgCirrusSearchMaxFileTextLength $wgCirrusSearchFallbackProfiles $wgCirrusSearchMoreLikeThisTTL $wgCirrusSearchAllowLeadingWildcard $wgCirrusSearchInterwikiPrefixOverrides $wgCirrusSearchMaintenanceTimeout $wgCirrusSearchReplicas $wgCirrusSearchPhraseSlop $wgCirrusSearchBoostOpening $wgCirrusSearchWriteBackoffExponent $wgCirrusSearchUserTesting $wgCirrusSearchShardCount $wgCirrusSearchUseCompletionSuggester $wgCirrusSearchPhraseSuggestReverseField $wgCirrusSearchFallbackProfile $wgCirrusSearchFragmentSize $wgCirrusSearchUnlinkedArticlesToUpdate $wgCirrusSearchCustomPageFields $wgCirrusSearchClientSideUpdateTimeout $wgCirrusSearchIgnoreOnWikiBoostTemplates $wgCirrusSearchRegexMaxDeterminizedStates $wgCirrusSearchInterwikiHTTPConnectTimeout $wgCirrusSearchExtraIndexes $wgCirrusSearchCategoryDepth $wgCirrusSearchMergeSettings $wgCirrusSearchClusters $wgCirrusSearchCrossProjectShowMultimedia $wgCirrusSearchBannedPlugins $wgCirrusSearchMoreLikeThisConfig $wgCirrusSearchClusterOverrides $wgCirrusSearchCrossProjectBlockScorerProfiles $wgCirrusSearchEnableIncomingLinkCounting $wgCirrusSearchNearMatchWeight $wgCirrusSearchReplicaGroup $wgCirrusSearchIndexedRedirects $wgCirrusSearchIndexAllocation $wgCirrusSearchNumCrossProjectSearchResults $wgCirrusSearchLanguageDetectors $wgCirrusSearchUpdateShardTimeout $wgCirrusSearchEnableCrossProjectSearch $wgCirrusSearchFullTextQueryBuilderProfiles $wgCirrusSearchCompletionDefaultScore $wgCirrusSearchWriteClusters $wgCirrusSearchCompletionSuggesterHardLimit $wgCirrusSearchRecycleCompletionSuggesterIndex $wgCirrusSearchLogElasticRequests $wgCirrusSearchConnectionAttempts $wgCirrusSearchWikiToNameMap $wgCirrusSearchMaxFullTextQueryLength $wgCirrusSearchLogElasticRequestsSecret $wgCirrusLanguageLanguageKeywordExtraFields $wgCirrusSearchLanguageToWikiMap $wgCirrusSearchNaturalTitleSort $wgCirrusSearchDeduplicateInMemory $wgCirrusSearchUseEventBusBridge $wgCirrusSearchEnableRegex $wgCirrusSearchClientSideSearchTimeout $wgCirrusSearchIndexFieldsToCleanup $wgCirrusSearchDeduplicateAnalysis $wgCirrusSearchNamespaceResolutionMethod $wgCirrusSearchExtraBackendLatency $wgCirrusSearchNamespaceMappings $wgCirrusSearchWMFExtraFeatures $wgCirrusSearchPreferRecentUnspecifiedDecayPortion $wgCirrusSearchIndexWeightedTagsPrefixMap $wgCirrusSearchDocumentSizeLimiterProfiles $wgCirrusSearchSearchShardTimeout $wgCirrusSearchWeightedTags $wgCirrusSearchPrivateClusters $wgCirrusSearchSimilarityProfiles $wgCirrusSearchCategoryMax $wgCirrusSearchCategoryEndpoint $wgCirrusSearchPoolCounterKey $wgCirrusSearchCompletionProfiles $wgCirrusSearchMaxShardsPerNode $wgCirrusSearchRescoreProfile $wgCirrusSearchRefreshInterval $wgCirrusSearchSimilarityProfile $wgCirrusSearchInterwikiThreshold $wgCirrusSearchEnableArchive $wgCirrusSearchIndexDeletes $wgCirrusSearchDocumentSizeLimiterProfile $wgCirrusSearchFiletypeAliases $wgCirrusSearchPrefixSearchStartsWithAnyWord $wgCirrusSearchUpdateConflictRetryCount $wgCirrusSearchInterwikiHTTPTimeout $wgCirrusSearchFetchConfigFromApi $wgCirrusSearchDevelOptions $wgCirrusSearchExtraIndexBoostTemplates $wgCirrusSearchBoostTemplates $wgCirrusSearchPrefixIds $wgCirrusSearchFullTextQueryBuilderProfile $wgCirrusSearchMoreLikeThisFields $wgCirrusSearchStripQuestionMarks $wgCirrusSearchIndexBaseName $wgCirrusSearchSanityCheck $wgCirrusSearchMasterTimeout $wgCirrusSearchNamespaceWeights $wgCirrusSearchCrossProjectOrder $wgCirrusSearchTextcatConfig $wgCirrusSearchTextcatModel $wgCirrusSearchMoreAccurateScoringMode $wgCirrusSearchMaxPhraseTokens $wgCirrusSearchCrossProjectSearchBlockList $wgCirrusSearchRescoreFunctionChains $wgCirrusSearchCrossProjectProfiles $wgCirrusSearchMaxIncategoryOptions $wgCirrusSearchEnableEventBusWeightedTags $wgCirrusSearchWikimediaExtraPlugin $wgCirrusSearchEnableAltLanguage $wgCirrusSearchLinkedArticlesToUpdate $wgCirrusSearchCompletionSuggesterUseDefaultSort $wgCirrusSearchPreferRecentDefaultHalfLife $wgCirrusSearchCompletionSuggesterSubphrases $wgCirrusSearchFunctionRescoreWindowSize $wgCirrusSearchEnablePhraseSuggest $wgCirrusSearchPhraseSuggestUseOpeningText $wgCirrusSearchUseExperimentalHighlighter $wgCirrusSearchCompletionSettings $wgCirrusSearchFeedbackLink $wgCirrusSearchUpdateDelay $wgCirrusSearchInterleaveConfig $wgCirrusSearchPhraseRescoreWindowSize $wgCirrusSearchDefaultNamespaceWeight $wgCirrusSearchMoreLikeThisAllowedFields $wgCirrusSearchClientSideConnectTimeout $wgCirrusSearchPhraseSuggestUseText $wgCirrusSearchPhraseSuggestProfiles $wgCirrusSearchPreferRecentDefaultDecayPortion $wgCirrusSearchInterwikiSources $wgCirrusSearchWeights $wgCirrusSearchICUNormalizationUnicodeSetFilter $wgCirrusSearchICUFoldingUnicodeSetFilter $wgCirrusSearchSlowSearch
使用的函数钩 APIAfterExecute APIQuerySiteInfoGeneralInfo APIQuerySiteInfoStatisticsInfo ApiBeforeMain ArticleRevisionVisibilitySet BeforeInitialize CirrusSearchAddQueryFeatures CirrusSearchAnalysisConfig CirrusSearchSimilarityConfig GetPreferences LinksUpdateComplete PageDelete PageDeleteComplete PageMoveComplete PageUndeleteComplete PrefixSearchExtractNamespace ResourceLoaderGetConfigVars SearchGetNearMatch SearchIndexFields ShowSearchHitTitle SoftwareInfo SpecialSearchResults SpecialSearchResultsAppend SpecialStatsAddExtra TitleMove UploadComplete UserGetDefaultOptions
提供的函数钩 CirrusSearchAddQueryFeatures CirrusSearchAnalysisConfig CirrusSearchBuildDocumentFinishBatch CirrusSearchBuildDocumentLinks CirrusSearchBuildDocumentParse CirrusSearchFulltextQueryBuilder CirrusSearchFulltextQueryBuilderComplete CirrusSearchMappingConfig CirrusSearchProfileService CirrusSearchRegisterFullTextQueryClassifiers CirrusSearchScoreBuilder
季度下載量	200 (Ranked 14th)
正在使用的公开wiki数	1,226 (Ranked 212nd)
前往translatewiki.net翻譯CirrusSearch扩展
Vagrant角色	cirrussearch
問題	开启的任务 · 报告错误

CirrusSearch扩展使用Elasticsearch实现搜索MediaWiki。

Elasticsearch是一個獨立的第三方軟體，您必須安裝才能使用此擴充功能。 它是一個提供搜尋和索引功能的資料庫系統，您目前的wiki頁面的文字會被編入索引，以獲得更快和更好的搜尋結果。 MediaWiki與Elasticsearch之間的溝通是透過網路服務完成的。

另請參閱此擴充功能的使用說明網頁。

目标

• 去除使该扩展难以安装的本地相依关系。 仅有的相依關係是纯PHP、MediaWiki扩展、和Elasticsearch本身。
• 为可由其他MediaWiki扩展去扩充的Wiki页面，提供一個接近实时的搜索索引。
• 提供MWSearch 为用户提供的所有查询选项等。

依存组件

PHP 和 cURL

除了MediaWiki对php的标准要求之外，CirrusSearch还要求PHP编译时支持cURL。

Elasticsearch

• 你必需安装Elasticsearch。

Elasticsearch 的每個版本都會改變網路服務的運作方式，並導致相容性問題。 您必須安裝與您目前使用的 MediaWiki 版本相容的 Elasticsearch 版本：

MediaWiki 1.39+需要Elasticsearch 7.10.2 (6.8.23+使用compatibility layer 可能是可以的)。 有關與MediaWiki早期版本的相容性資訊，請參閱此版本。

6.8之前的Elasticsearch版本與PHP 8+不相容。

请注意，还需要像OpenJDK这样的Java安装。 最好使用官方的Elasticsearch Docker镜像或自己托管的版本。 Amazon OpenSearch（前身為 Amazon Elasticsearch）這類受管理的產品可以運作，但可能需要額外的組態，視其具體情況而定。 例如，Amazon OpenSearch 只在port 443 上侦听通过 HTTPS 发送的 Elasticsearch API 请求（即不暴露默认的 Elasticsearch port 9200），因此启用TLS 的代理（如 Nginx）可以使CirrusSearch与Amazon OpenSearch的集群之間通信。

Elastica

• Elastica是一个可使得CirrusSearch向Elasticsearch對話的PHP库。按照下面的说明安装Elastica。

其他

• 由于CirrusSearch扩展实际处理作业，建议在Redis中设置作业以防止Notice: unserialize(): Error at offset 64870 of 65535 bytes in JobQueueDB.php之类的消息和Unsupported operand types之类的后续错误。 参见T157759。

安裝

Elastica

儘管下面的指示告訴你從git安裝時只需執行Composer即可，但可能仍是有必要發佈必需要安裝所有的PHP依存組件。

• 下载文件，并将解压后的Elastica文件夹移动到extensions/目录中。
  开发者和代码贡献人员应从Git安装扩展，输入：cd extensions/
git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/Elastica
• 只有從git安裝才运行Composer来安装PHP依赖，通过发行composer install --no-dev至扩展目录。 （参见T173141了解潜在问题。）
• 将下列代码放置在您的LocalSettings.php 的底部：

  wfLoadExtension( 'Elastica' );
  

• 完成 – 在您的wiki上导航至Special:Version，以验证已成功安装扩展。

CirrusSearch

• 下载文件，并将解压后的CirrusSearch文件夹移动到extensions/目录中。
  开发者和代码贡献人员应从Git安装扩展，输入：cd extensions/
git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/CirrusSearch
• 只有從git安裝才运行Composer来安装PHP依赖，通过发行composer install --no-dev至扩展目录。 （参见T173141了解潜在问题。）
• 将下列代码放置在您的LocalSettings.php 的底部：

  wfLoadExtension( 'CirrusSearch' );
  

• 現在請依照CirrusSearch README中的設定指示來進行設定。請注意，其中的所有資訊可能不適用於您的擴充功能的版本，尤其是有支援Elasticsearch的版本。
• 按照要求进行配置
• 完成 – 在您的wiki上导航至Special:Version，以验证已成功安装扩展。

啟用regex查詢

This is an optional step. You will need to install the search-extra plugin for this. Do so by following these steps:

1. execute the following command:

   /usr/share/elasticsearch/bin/elasticsearch-plugin/elasticsearch-plugin install org.wikimedia.search:extra:7.10.2-wmf12
   

2. add the following line to your LocalSettings.php file:

   $wgCirrusSearchWikimediaExtraPlugin[ 'regex' ] = [ 'build', 'use', 'max_inspect' => 10000 ];
   

3. restart Elasticsearch with the following command:

   systemctl restart elasticsearch
   

4. recreate the search index by executing the following commands:

   1. php path/to/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php --startOver
      

   2. php path/to/extensions/CirrusSearch/maintenance/ForceSearchIndex.php
      

升级

请遵循在CirrusSearch UPGRADE中的升级指南。

配置

CirrusSearch 的組態參數記錄在 "settings.txt" 檔案中。 另請參閱CirrusSearch組態概況的說明文件。

如果使用包含大寫字元的MySQL資料庫名稱，例如「MyWikiDatabaseName」，Elasticsearch 將無法為CirrusSearch建立索引。為了緩解這個問題，CirrusSearch提供了$wgCirrusSearchIndexBaseName組態參數，這需要某人去設定它，例如：$wgCirrusSearchIndexBaseName = 'mywikidatabasename';。

钩子

CirrusSearch擴充功能定義了許多钩子，它是其他的擴充功能可以利用這些钩子擴充核心模式並修改文件。 以下钩子可用：

• CirrusSearchAnalysisConfig - 可掛鉤到組態之中進行分析
• CirrusSearchMappingConfig - 可讓欄位的映射有組態設定
• CirrusSearchBuildDocumentParse - 可讓擴充功能修改來自頁面所產生的Elasticsearch文件
• CirrusSearchBuildDocumentLinks - 可讓擴充功能處理文件的傳入和傳出連結
• CirrusSearchBuildDocumentFinishBatch - 當一批頁面已被索引時會被呼叫
• CirrusSearchAddQueryFeatures - 可讓擴充功能去增加查詢解析器的各個功能
• CirrusSearchScoreBuilder - 可讓擴充功能去定義rescore builder的函式。
• CirrusSearchProfileService - 可讓擴充功能去宣告各種搜尋元件和設定組態

API

CirrusSearch 功能可用於 API 查詢。 Searching happens via the normal search API, action=query&list=search; you can use CirrusSearch-specific features, such as the morelike: special prefix to find pages related to Marie Curie and radium:

api.php?action=query&list=search&srsearch=morelike:Marie_Curie%7Cradium&srlimit=10&srprop=size&formatversion=2

Custom APIs and parameters are provided for querying CirrusSearch configuration and debug information:

• action=cirrusdump module: 2014?action=cirrusdump
• cirrusDumpQuery parameter to Special:Search or search API queries: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpQuery
• cirrusDumpResult parameter to Special:Search or search API queries: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpResult
• An additional parameter, cirrusExplain, can be passed with cirrusDumpResult to have the Lucene explanation of the score included with the result dump: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpResult&cirrusExplain It can also be used to get the explanation in a human-readable format, by giving it one of the values verbose, pretty or hot, such as: https://en.wikipedia.org/wiki/Special:Search/cat%20dog%20chicken?cirrusDumpResult&cirrusExplain=pretty
• cirrus-config-dump, cirrus-settings-dump, cirrus-mapping-dump, cirrus-profiles-dump modules to obtain dump from the CirrusSearch setup: api.php?action=cirrus-config-dump&formatversion=2

參見

General links

• Usage help page - CirrusSearch usage documentation (needed after the install)
• 项目页面
• Info about Wikimedia Cirrus/Elastic setup
• Configuration help page - sets of tunable parameters that influence various aspects of the indexing
• Extension:WikiSearch - provides faceted search API for Semantic MediaWiki using ElasticSearch.
• Extension:AdvancedSearch - Enhances Special:Search by providing advanced parameters
• Extension:SearchParserFunction - provides a search parser function that supports Cirrus Search.

调试

• How to determine that Cirrus is actually used as the search backend

Local development

Elasticsearch service can be run with the Vagrant role (cirrussearch) and MediaWiki Vagrant.

For Docker, you can use a command like docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:6.8.2. Then follow the installation and configuration directions. If your web host is in a container, you'll want to make sure the above container is on the same network, and in the LocalSettings.php file, you will want to reference the elasticsearch as the hostname. This will not have the WMF plugins but can be sufficient for basic testing.