Extension talk:SphinxSearch

Archives

Feature requests Search options and features Troubleshooting Prio the Liquid threads switch (May 2012) Resolved and deprecated issues

The talk page now uses liquid threads, all open messages prior to liquid threads have been converted into a talk subpage and can be found here.

It is recommended to use SphinxSearch 0.8.5 and a recent stable release of Sphinx 2.1. Please bear in mind that this extension only handles the communication between MediaWiki and Sphinx and any specific issues related to the search feature (character sets, ability to search with *, search categories, minimum length on search terms etc.) are handled in Sphinx (see sphinx.conf file) and those questions should be redirected to the Sphinx forum.

The development on this extension is done on a voluntary basis and while this forum provides a platform to share experiences and solutions, it is up to its community members to fill in suggestions.

Help

When seeking help and/or support, you might want to consider to mention your system environment (SphinxSearch Extension version, Sphinx version, MW version etc.) otherwise it might be difficult for people to make appropriate recommendations.

For Windows users and related issues see here, for Linux users and related issues see here, and some advice on how to configure a SQLite setup see here.

Previous page history was archived for backup purposes at Extension talk:SphinxSearch/LQT Archive 1 on 2015-07-10.

I am new to media wiki and for that matter to any web application development. I was exploring Mediawiki and have installed "MediaWiki 1.18.2" and have attempted to integrate Sphinx search. I have used "Sphinx 2.0.4-release".

The problem I am facing is that I am unable to search through the contents of the pages or the documents embedded in the MediWiki. It is only able to search the Title of the Pages. This is the same irrespective of the "GO" button or the "SEARCH" button. Whereas Through the command in the Sphinx Search, I am able to find all the text. Can someone please help me in debugging this.

The documentation is quite extensive (compared with other extensions), and it might be that you need to do the setup more than ones to fully understand the implication in how searchd, indexer and SphinxSearch is working together. If you follow the steps outlined in the documentation you should be able get the extension working but it will be hard for anyone to give any guidance as long their are no particular problems to solve.

For example did you do "Step 4 - Test Out Sphinx" to ensure that the infrastructure is setup correctly otherwise if this step fails something is wrong in your configuration and searching within MediaWiki will definitely not work as advertised.

In order to use Sphinx in MediaWiki you have register the extension, see "Step 9 - Extension Installation - Local Settings"

You should also consult the Extension:SphinxSearch/Troubleshooting page.

After migraging to MediaWiki 1.19.0 SphinxSearch 0.85 does not work on the Spechial:Search page anymore (no results). The sidebar Search works.

After logging the searchd with --console, you found a message "query error: no such filter attribute 'page_is_redirect'" in the console. I searched this text and found it in 'SphinxMWSearch.php'. I change the line

$cl->SetFilter( 'page_is_redirect', array( 0 ) );

to

// $cl->SetFilter( 'page_is_redirect', array( 0 ) );

and everything works (for me).

I also change the method 'userHighlightPrefs' to

	public static function userHighlightPrefs() {return array( 2, 75 ); }

because the the definition changed (and I got warning when I activated "$wgShowExceptionDetails" in LocalSettings.php).

I hope this helps others fixing the same problems.

Perhaps page_is_redirect is missing in the sphinx.conf? It did not use to be there by default, but in more recent versions you should see it in at least three places in your sphinx.conf, in both sql_query settings, and in one sql_attr_uint setting. After you change the config file, you need to reindex the wiki.

Hello Svemir,

thanx, that was the solution.

During our tests with MW 1.19 / SphinxSearch 0.8.5 we haven't seen any problems and therefore are able to run SphinxSearch 0.8.5 without any changes on MW 1.19.

I got tripped up by the use of 'spinxconf' instead of 'sphinxconf' as an option to SphinxSearch_setup.php. The following patch may avoid minor confusion for new users:

--- a/SphinxSearch/SphinxSearch_setup.php   2012-06-04 14:08:10.000000000 -0700
+++ b/SphinxSearch/SphinxSearch_setup.php   2012-06-05 12:59:57.289298038 -0700
@@ -24,7 +24,7 @@
        
        /* Override parameters setup becuase we do not need some of the default ones */
        protected function addDefaultParams() {
-               $this->addOption( 'spinxconf', 'Location of Sphinx configuration file', true, true );
+               $this->addOption( 'sphinxconf', 'Location of Sphinx configuration file', true, true );
                $this->addOption( 'indexer', 'Full path to Sphinx indexer if not in the path', false, true );
                $this->addOption( 'useindex', 'Sphinx index to use (defaults to wiki_main)', false, true );
                $this->addOption( 'maxwords', 'Maximum number of words to extract (defaults to 10000)', false, true 
@@ -36,7 +36,7 @@
                $max_words = intval( $this->getOption( 'maxwords', 10000 ) );
                $indexer = wfEscapeShellArg( $this->getOption( 'indexer', 'indexer' ) );
                $index = wfEscapeShellArg( $this->getOption( 'useindex', 'wiki_main' ) );
-               $conf = wfEscapeShellArg( $this->getOption( 'spinxconf' ) );
+               $conf = wfEscapeShellArg( $this->getOption( 'sphinxconf' ) );
 
                $cmd = "$indexer  --config $conf $index --buildstops sphinx.dic $max_words";
                $this->output( wfShellExec( $cmd, $retval ) );

Thank you for the fix. MW and extensions are moving from SVN to Git - I will apply and commit your change once all that is done.

It seems like its going it be a while before this extension is moved to Git. I made the change in SVN now (revision 115486)

I'm installing Sphinx-2.0.4 with the Mediawiki 1.19 with SphinxSearch extension version 0.8.5. I am running on Windows 2008 R2, Apache 2.4.2, MYSQL 5.5.24, and PHP5.4.3.

When I just added sphinx to my wiki and with I do a search for say "conventions" where I know I have pages I get back "Results 1 - 14 of 233" for Conventions but the "View (previous 20 | next 20) are in black and can't be selected. How do I fix this. Users need to get to the next batch of 20 pages. Why is it only showing 14 (1 -14 off 233)? This is for most searches I have had a couple searches the result in "Results 1 - 20 of 143" and here I do get previous / next links down to bottom that work. I really need help with this it seem pretty random...

I have never encountered an error like that one. Perhaps there are some errors in your logs that may provide a clue?

Posting removed - wrong location. I thought the above response was to my question which is actually below. My apologies. I've seen your suggestion there and will respond to it.

Those queries look OK to me. Do you get any errors when you run them directly?

Documentation has been updated not to suggest the confusing way to test things out.

Has anyone been able to successfully use this extension with Sphinx version 2.0.4? I receive the following error when "testing out sphinx" (step 4):

 index 'wiki_main': search error: .

The only explanation I can think of is that perhaps the way the config file sets up the index is no longer compatible with the newer version of Sphinx. Has anyone been able to successfully run Sphinx 2.0.4 with their wiki, or should I downgrade my installation to Sphinx 2.0.3? I appreciate any assistance.

Update: The SphinxSearch extension itself appears to be functional even though queries from the command line are not. Feel free to close/delete this support thread.

Actually we are using 2.0.4 in production with the latests SphinxSearch 0.8+ and we are using the exact same setup as with. (we haven't changed the configuration neither in case of 2.0.3/2.0.4). I'm not sure from where you get the message testing out sphinx, we have never seen this kind of message coming from Sphinx nor SphinxSearch. --MWJames (talk) 00:43, 6 May 2012 (UTC)

Oh no, I meant that I received the error in the code block in my original post when following the instructions under the "test out sphinx" step on the SphinxSearch article. The string "testing out sphinx" was never returned from either system.

As I wrote in my update though, SphinxSearch seems to work perfectly on the wiki itself (through the search daemon) even though it's broken from the command line. I have no idea what's causing this, but I guess it must be unique to my system configuration.

Thanks a lot for your help, and for your work on this excellent extension! :)

That's my experience too. The "search" command line function from 2.0.4 (built from source) and 2.0.5 (both built from source and installed from deb) gives "index 'wiki_main': search error: ." if any search terms at all are entered on the command line. However the "searchd" daemon and the MW extension which relies on that work. So the instructions to use "search" to test the installation currently lead to a dead end. Still, "search" is only a test utility, not meant for use once it's in production.

In step 8.2 of install you are supposed to: "Copy all remaining files of the extension (SphinxSearch.php, SphinxSearch_body.php, etc.) to your extensions/SphinxSearch directory." But the only files that come with the latest extension download are: LICENSE sphinx.conf SphinxSearch.i18n.php SphinxSearch_setup.php README SphinxMWSearch.php SphinxSearch.php

So where's SphinxSearch_body.php, let alone the rest of what's supposed to be in etc? Maximilianklein (talk) 21:20, 30 September 2012 (UTC)

This is simply an old piece of documentation - since version 0.8, the files have changed. I updated the list to the best of my knowledge. However, you don't need to worry about that - the "copy the rest of the files" part is the important bit anyway, whatever they are :-)

After I upgraded from a reasonably old version, one of my users noted that the "all words" checkbox disappeared. Is this functionality now contained entirely in the Boolean operators that are available?

Yes.

Hi folks,

I installed SphinxSearch in my companies internal wiki and we're very excited about it. However there's one problem. On some searches the search just redirects to the starting page of the wiki. The HTTP response is 301 moved permanently with the redirect address given as startpage of our wiki. This problem is reproducable on all browsers and pcs (except some browser seem to not be able to handle 301 correctly...).

Is this a known malfunction? Can anyone help me with that?

Thanks

Signpainter

What is different about those searches? If they are a full title match to an existing article in the wiki, it will try to redirect you to that article. You could test this by trying the same query on the actual search page (or making sure you use Search and not Go.)

Good news! It's not the SphinxSearch extension that's causing problems. After some more investigation I found the problem to be lying in another extension (AccessControl) that's installed. Everytime a protected page is part of the search result, AccessControl redirects the user to the login page. This redirect seems to be broken or to be not programmed properly.

Hi, I have installed SphinxSearch Extension (0.8.5) on MediaWiki 1.21.1 and Sphinx (2.2.4). The general engine works well, including "did you mean" and other nice features. But I cannot make to work the incategory, intitle and prefix search.

The sql_attr_multi = uint category from query; SELECT... is there.

What else can I check? Thank you for your help.

UPDATE. It is working indeed, I found it does not work if I use + or - in front of incategory. I.e. +incategory:"Mycategory" does not work, instead myword incategory:Mycategory works.

And this (almost) solves my problem. Thank you.

There is a phabricator task for this now: https://phabricator.wikimedia.org/T155306

I want to know what is the difference between ranking, sorting, matching modes and filtering in Sphinx and how can I do more than one thing at a time.

In documentation also, it is written you can do ranking and sorting simultaneously. How is it possible because if I rank then results will be arranged in some order and if I do sorting the results will be arranged in some different order.

If possible please explain using some example. I am using version 2.1.9.

Please don't refer me to Sphinx documentation. I have already read it.

Every answer is appreciated.

Thanks in advance.

Ranking controls how the weight of each match is calculated. Sorting will by default sort by that ranking, but you can use advanced sorting mode to sort by things other then the ranking (or by something else in addition to the ranking, so that two articles with the same @weight are further sorted by something you specify, such as page_title or old_id.)

Filtering will filter things out not to show at all, so in the above scenarios it happens first, and therefore you can filter "at the same time" with ranking and sorting.