Extension talk:SphinxSearch/archive
 | This page is an archive. Do not edit the contents of this page. Please direct any additional comments to the current talk page. |
2007
Major failures -- help!
Line #s 304 and 305 error out on SphinxSearch_body.php in version 1.8 of Mediawiki. Either i need to comment it out or upgrade to v1.11
Even after upgrading line # 171 errors with Fatal error: Call to undefined method SphinxClient::SetFilter() in C:\wamp\www\wiki\extensions\SphinxSearch_body.php on line 171
Commenting doesn't help since it gives me a "Fatal error on the DB" or something like that. I'm thinking i need to install some package, but don't know which one (something in PEAR?)
Help :(
--17 October 2007
- It looks like you are running this extension on Windows. To the best of my knowledge, this extension has not been tried as such yet. But, at least in theory, there is nothing that should prevent it from working. Of course, the big differences are all path related. So, let's first start by making sure your setup is correct. Were you successfully able to perform step 3, step 4, and step 5? Can you also please verify that step 7 was done correctly and you have the sphinxapi.php file in your C:\wamp\www\wiki\extensions\ directory? --Gri6507 12:20, 17 October 2007 (UTC)
- I kinda solved this; upgrading to v1.11 off course takes care of the lines 304 & 305. As for the setFilter method, apparently the rc1 of sphinxapi doesn't have this method. I'm trying to copy-paste the method into my API as first option and then will try and build the current non-production API. Will keep you informed --18 October 2007
- Thanks for looking into this. I am running my installation with MW 1.9.3 and I don't know which version Svemir (the other developer) is running. I will start a new section on the main page with information about known supported MW versions. As for sphinxapi.php being incorrect, I am assuming you are using v0.9.8rc1? Both Svemir and I based this extension on 0.9.7 (the latest stable release). We'll keep a keen eye on the Sphinx project to make sure that our extension will be completely compatible with future version of Sphinx.
- On a side note, I was wondering if you have implemented the windows equivalent of setting up the cron jobs to keep the indexes up to date. If you have, can you please add that information to the documentation? We would much appreciate it! --Gri6507 12:36, 18 October 2007 (UTC)
- I'm still stuck and couldn't get much progress. Apparently the line
- $sql = "SELECT old_text FROM ".$db->tableName('text')." WHERE old_id=".$docinfo['attrs']['old_id']; ends up with the value of $sql being Select old_text from 'text' where old_id=
- I'm not sure why in the first place text is in single quotes (looks like some bug to me) and why the old_id is not getting picked up. Searchd does show the hit coming to it, but it could failing because i'm using a hacked version of the API
- Also to answer Gri6507's question, i'm using 0.9.6 rc1 because that's the one that has the windows binaries. I don't have Visual Studio or VC++ to compile from the source code, so even my step #2 (using latest version and compiling) is at hold.
- Adding the Windows cron job shouldn't be too tough (my guess); but i'll try it and let you know -- ALl the above posts bought to you by the guy who had the so useful signature Help :( :)
- --19 October 2007
- According to Sphinx's website, http://www.sphinxsearch.com/downloads/sphinx-0.9.7-win32-release.zip is a windows release of 0.9.7. Is there any reason you are not using it? --Gri6507 11:42, 19 October 2007 (UTC)
- Doesn't seem to contain the sphinxapi.php -- that's the reason why i had to choose an older version; this should probably be posted on that developer's website saying the API is missing from the 0.9.7 windows release, but i'm too lazy...any helpers? :) --25 October 2007
- I have updated Step #1 and Step #7 with details of how to obtain the sphinxapi.php for Windows. It seems that the intent of the Win32 release binaries package is to only contain the binary EXEs. The PHP files are in either the source code or the API packages. --Gri6507 11:34, 25 October 2007 (UTC)
- Does it even work on Windows? Installation instruction (step 1) of the Sphinx site has this to say "At the moment, Windows version of Sphinx's searchd daemon is not intended to be used in production because it can only handle one client at a time." --22 November 2007
Windows --rotate workaround?
I want Sphinx to update our index very often. If I could, I would love for the index to be incrementally updated every time the db changes. Barring that, I'd install a task to run every 15 minutes or so. However, no matter how often I update the index, I need to take down the Sphinx daemon to do it (limitation on Windows). Can anyone suggest a workaround or modification to the code such that a search request, when the daemon isn't running, waits for it to respond and re-searches? I don't so much care about restarting the daemon. I do care about search appearing broken while the daemon is down. --Cedarrapidsboy 2 November 2007
- I am not sure if it is going to work, but here's what I'd try. Open the sphinxapi.php file. In function _Connect (), around line 136, there is a call to
if (!( $fp = @fsockopen ( $this->_host, $this->_port ) ) )</php>
- change that to
if (!( $fp = @fsockopen ( $this->_host, $this->_port, $errno, $errstr, 30 ) ) )</php></php>
- where the 30 is the timeout in seconds for establishing the connection. The basic idea is that if searchd is not running, no one will be listening on the other end of the socket until searchd comes back to life. This change should block MW from dieing during that brief period of time. Of course, it would be up to you to make sure that
- before running the indexer, you must stop searchd
- after running the indexer, you must restart searchd
- Let me know if that works :-) --Gri6507 22:22, 3 November 2007 (UTC)
- That looks promising. I'll give it a try. Since my post, I installed the search on a separate machine (Linux) and that works pretty good. But, this procedure may be what I need to reduce the number of servers in the equation. I'll come back with results. --Cedarrapidsboy 14:25, 5 November 2007 (UTC)
- UPDATE - the above code change didn't appear to have an effect. The search still timed-out to a blank page.
- Stop searchd
- Issue search request
- Start searchd (within 30 sec)
- --205.175.225.24 16:30, 5 November 2007 (UTC)
- UPDATE - the above code change didn't appear to have an effect. The search still timed-out to a blank page.
- Ok. I think I found the issue. According to PHP documentation, the fsockopen() function may not honor the timeout ("Note: Depending on the environment, the Unix domain or the optional connect timeout may not be available."). So, to work around that, change the following code in sphinxapi.php
if (!( $fp = @fsockopen ( $this->_host, $this->_port ) ) )
{
$this->_error = "connection to {$this->_host}:{$this->_port} failed";
return false;
}
- to
$connect_timeout = 30; # wait this long in seconds
$start_connect = time();
do {
$fp = @fsockopen ($this->_host, $this->_port, $errno, $errstr, $connect_timeout);
} while (!$fp && !sleep(1) && ((time()-$start_connect) < $connect_timeout));
if (!$fp)
{
$this->_error = "connection to {$this->_host}:{$this->_port} failed";
return false;
}
- This way, you can set the waiting period yourself via the use of $connect_timeout variable. I tested this on my machine and it seems to work as expected. Please post your results when you try it out. --Gri6507 23:05, 5 November 2007 (UTC)
- Unfortunately, same result. Blank page. I did the following:
- Kill searchd
- Issue search request
- Start searchd
- In this case, searchd was still running on a separate machine.
- --Cedarrapidsboy 20:20, 6 November 2007 (UTC)
- Unfortunately, same result. Blank page. I did the following:
- UPDATE!
- Here's a change to the code that works:
$connect_timeout = 5; # wait this long in seconds each loop iteration
$loop_timeout = 30; #wait this long in seconds for the entire loop
$start_connect = time();
do {
$fp = @fsockopen ($this->_host, $this->_port, $errno, $errstr, $connect_timeout);
} while (!$fp && !sleep(1) && ((time()-$start_connect) < $loop_timeout));
if (!$fp)
{
$this->_error = "connection to {$this->_host}:{$this->_port} failed";
return false;
}
- I added an additional timeout. Without it, a single connection was waiting for 30 seconds, just as long as the entire loop. The previous code never tried the connection again. This code *did* work for me using the testing steps above. --Cedarrapidsboy 20:30, 6 November 2007 (UTC)
- Glad to see that it's working for you! I will submit this as an improvement suggestion to the developers of Sphinx. --Gri6507 20:41, 6 November 2007 (UTC)
The solution developed on this talk page seems to be a part of the official sphinxapi.php file in 0.9.8 release :-) Svemir Brkic 14:37, 25 August 2008 (UTC)
Sphinx Search Terms Limit
It seems that the Sphinx search only accepts 10 search terms. Perhaps it is the same story for the built-in MW search? Any way to change that? Perhaps make it unlimited? Cedarrapidsboy 13:50, 5 November 2007 (UTC)
- I did not look at the code yet, but the limit seems to happen only in the sense of number of separate words and counts displayed on top of the search results. That lists only up to 10 words, but the eleventh word I used was also used to filter (and rank) the results. Svemir Brkic 14:31, 5 November 2007 (UTC)
- Ah... I can confirm that. I tested it, but the 11th and above search terms were not highlighted red, so didn't think they were included in the results. I'd still be interested in getting all search terms highlighted. Thanks for the reply! --Cedarrapidsboy 14:56, 5 November 2007 (UTC)
- This seems to be fixed in the latest version of sphinx (0.9.8-rc2) - according to change log: fixed highlighting (uses 256 words by default now instead of former 10) Svemir Brkic 16:05, 28 March 2008 (UTC)
- Ah... I can confirm that. I tested it, but the 11th and above search terms were not highlighted red, so didn't think they were included in the results. I'd still be interested in getting all search terms highlighted. Thanks for the reply! --Cedarrapidsboy 14:56, 5 November 2007 (UTC)
Everything is OK it seems, but no search-results
I've installed the Sphinx-extension on a 1.11 mediawiki with the most recent version of Sphinx. Everything seems to work OK, I installed the service (windows-server), tested the search on the command prompt, which gives results. When I go to special:searchSphinx, it displays OK.
The only thing is that nothing happens when I try to search something, it reloads and displays nothing. Do you have any idea what might be causing the problem? It also seems it cannot create a searchd.pid file & logfiles, although the search-indexes are created without a problem. --11 December 2007
- Please clarify "reloads and displays nothing". Do you get a blank screen? In that case, you would need to check your php error log for clues. Or you get the same thing as Mark describes below? Svemir Brkic 15:44, 5 January 2008 (UTC)
- Hi Svemir, Thanks for your response. I have the same problem as Mark. I discovered that if I disable the internal search, the sphinx special pages suddenly is gone (it also isn't visible in Special:Specialpages). If internalsearch isn't disabled, the sphinxsearch special page is visible, but it doesn't return results. (i've got the same config as mark, only the mediawikiversion is 1.11) 213.132.179.227 10:03, 7 January 2008 (UTC)
We have the same problem. We're on Sphinx 0.9.7 (Win32) and MediaWiki 1.9.7 installed on Win2003 server. After starting search daemon, I can run test.php from command line and get back results there, but searches entered from Sphinx Special Page in the wiki just bounce you back to the main page. The URL looks like the search took place, however. For example, a search on the term "SAM" bounces you back to the wiki's main page, but now this query string appears on the URL: "sphinxsearch=SAM&fulltext=Search&match_all=0&ns0=1" --Mark price 01:26, 14 December 2007 (UTC)
- Did you make any changes to you sphinx search configuration file - SphinxSearch.php? You could also try making Sphinx the default search, just to verify if the problem is in the search itself or the way your wiki handles the paths in the "special page only" case. Svemir Brkic 15:44, 5 January 2008 (UTC)
We had the same when we tried using Sphinx-0.9.8-svn-r1112 (Jan 28, 2008 snapshot). Getting the previous version (Sphinx 0.9.7) solved the problem for us.130.234.189.190 11:52, 30 January 2008 (UTC)
- I just tested it with 0.9.8-svn-r1112 on MW 1.11 on Linux and it works correctly. I will try it on Windows at some point as well. Would you please make sure you were using the correct version of sphinxapi.php? It needs to be copied from your sphinx download/api folder into the SphinxSearch extension folder each time you change the version of sphinx on your system. If you still have issues with 0.9.8, please post your sphinx.conf somewhere so we can take a look. Svemir Brkic 19:55, 2 February 2008 (UTC)
- I also tested it with 0.9.8-svn-r1112 on MW 1.11 on Linux, but mine is giving me the same error as the above users are experiencing. I am doing this on a shared hosting plan. I installed Sphinx to /home/myusername/local/sphinx. I can get everything to work on the command line via SSH, but not on the actual site over http -- it just returns me to the main page, despite the URL looking as though it should have performed the query. I have double-checked the version of sphinxapi.php and gone through all the installation instructions twice but I am dumbfounded. Also, the search daemon seems to stop running after about 5 or 10 minutes. The process will not stay alive any longer than that. I am not that great with Linux yet, but I am slowly becoming more familiar. I'd rather not bother installing an earlier version, so I would appreciate any suggestions. My sphinx.conf file appears below [UPDATE: removed to save space]. --Wikitonic 18:40, 13 February 2008 (UTC)
- There seem to be two separate issues here. One is related to how your wiki is setup in general regarding the URLs etc. - please send me a link if you do not mind. Another issue is with the search deamon being stopped. Perhaps your shared host does not allow user processes to run longer than certain amount of time, or use more than certain amount of memory, etc. You should probably ask - maybe they are willing to make an exception if they know exactly what are you doing. Svemir Brkic 00:59, 14 February 2008 (UTC)
- Ok, I will check with my webhost about user processes. In the meantime, you can find my wiki here. Unfortunately, it is locked to the general public right now since we haven't officially launched. Even viewing is disabled for anonymous users. I'll get in touch with you about letting you in, after I take care of the search daemon disruption issue. --Wikitonic 16:03, 14 February 2008 (UTC)
- Well, I meant to give an update sooner but I guess now is better than never. My shared hosting plan doesn't allow user processes, so until it does, I guess everything is a moot point. I have removed my sphinx conf file so as not to take up unnecessary space. Thanks for your help anyway, Svemir Brkic. -Wikitonic 21:18, 3 March 2008 (UTC)
We have the same problem. SphinxSearch 0.5.3 with sphinx 0.9.8-svn-r1112 on MediaWiki 1.11 on Windows2003 Server.Rroblem is MediaWiki 1.11 needs "title" argument in URL.Like "/mediawiki/index.php?title=XXXX:SphinxSearch&search=foo&....". I change SphinxSearch_body.php line 351 like this,and it goes well.
$wgOut->addHTML("<form action='$kiAction' method='GET'>
<input type='hidden' name='title' value='$titleObj'>
<input type='text' name='$searchField' maxlength='100' value='$SearchWord'>
--219.121.144.178 15:26, 13 March 2008 (UTC)
- Interesting. This made me think I did not notice the problem because I am using "short" links. However, even after changing to the default index.php?title=... configuration, the search still worked. Can anyone else confirm that this fixes the search in their configuration? Svemir Brkic 11:26, 15 March 2008 (UTC)
- This did not fix for me and I started getting parse errors. I changed the end like this:
$wgOut->addHTML("<form action='$kiAction' method='GET'>
<input type='hidden' name='title' value='$titleObj'>
<input type='text' name='$searchField' maxlength='100' value='$SearchWord'>");
- which cleared the parse errors but still just a blank page. 18:56, 12 May 2008 (UTC)
Issues discussed here are probably fixed in the version 0.6 (released today.) There was an issue with a ? vs. & in the URL, as well as using the correct special page title when internal search is disabled. Svemir Brkic 14:34, 25 August 2008 (UTC)
2008
Warnings on MW 1.11
Warning: Call-time pass-by-reference has been deprecated - argument passed by value; If you would like to pass it by reference, modify the declaration of [runtime function name](). If you would like to enable call-time pass-by-reference, you can set allow_call_time_pass_reference to true in your INI file. However, future versions may not support this any longer. in extensions/SphinxSearch_PersonalDict.php on line 75 (also lines 142 and 190) -71.217.0.96 08:18, 5 January 2008 (UTC)
- Thanks for the notice. I have fixed this in the CVS already. If you are using a public release, all you need to do is remove the ampersands from all calls to readPersonalDictionary. The method is already declared correctly (the ampersands should stay there.) Svemir Brkic 13:15, 2 February 2008 (UTC)
Showing X of Y documents in search results, but document links not showing
- "Displaying 0-0 of 0 matches for query specifi* retrieved in 0.000 sec with following stats: specifi* found 8 times in 8 documents".
- Your query indicates that you are using a "star" search, which is not enabled by default in sphinx (it is a new, almost undocumented feature.) Our default sphinx.conf does not enable it either. Maybe you are using a different config file for command line search? To enable it in the config file used by the extension, you need to add this to the main index section:
min_infix_len = 1 enable_star = 1
- After this, you need to do a full reindex and restart the searchd (--rotate alone is not enough when you change the config file.) Svemir Brkic 20:57, 2 February 2008 (UTC)
Installation issues
WARNING: key 'sql_group_column' is deprecated in /var/sphinx/sphinx.conf line 35; use 'sql_attr_uint' instead. WARNING: key 'sql_group_column' is deprecated in /var/sphinx/sphinx.conf line 36; use 'sql_attr_uint' instead.
solution: replace sql_group_column by sql_attr_uint --24 March 2008
- This depends on the version of Sphinx you are using. CVS version of the extension is already using sql_attr_uint, as it requires the latest version of Sphinx. Svemir Brkic 02:05, 6 April 2008 (UTC)
ERROR: unknown key name 'FROM' in /var/sphinx/sphinx.conf line 48 col 10. FATAL: failed to parse config file '/var/sphinx/sphinx.conf'.
solution: merge line 48 with 47 making sure to remove the change of line / (not sure why this error) --24 March 2008
- Weird. Maybe some invisible extra character sneaked in somehow. Anyway, I merged the lines in CVS, just in case. Svemir Brkic 02:05, 6 April 2008 (UTC)
Making title match appear first
Is there a way to do this? --25 March 2008
- Greater weight is already given to title matches. Do you need any title match (even a partial one) to appear before any content match (even an exact one?) Svemir Brkic 02:19, 26 March 2008 (UTC)
- YES. All direct title match should appear first. This is not the case. --207.96.208.130 22:12, 1 April 2008 (UTC)
Sphinx API has changed recently, and that probably caused title matches to show so far in the search results. SetWeights method has been deprecated and the method to use now is SetFieldWeights. The new method expects an associative array of index names and weights. Until we release the next version, here is what you should try:
- In SphinxSearch.php, change the line that sets $wgSphinxSearch_weights to:
$wgSphinxSearch_weights = array('old_text'=>1, 'page_title'=>100);
- In SphinxSearch_body.php, change the $cl->SetWeights call in wfSphinxSearch method to:
$cl->SetFieldWeights($wgSphinxSearch_weights);
This will not make all the title matches appear before all the text matches, but it will probably give much better results. Svemir Brkic 21:13, 5 April 2008 (UTC)
- Note: Above has been committed to CVS and released in 0.6beta3 package. Svemir Brkic 03:13, 10 May 2008 (UTC)
Did you mean?
I have SphinxSearch working on my site and it is fast! Now I am attempting to implement the "Did you mean?" function and get the following error on the search: " Uninitialized string offset: 0 in C:\path\to\SphinxSearch_spell.php on line 96 " line 96 is this:
if ($value[0] == "&") {
$correction = explode(" ",$value);
$word = $correction[1];
$suggstart = strpos($value, ":") + 2;
$suggestions = substr($value, $suggstart);
$suggestionarray = explode(", ", $suggestions);
$guess = $this->bestguess($word, $suggestionarray);
if (strtolower($word) != strtolower($guess)) {
$word_suggestions[$word] = $guess;
$this->suggestion_needed = true;
}
}
}
Can anyone help with this? Steve Goble14:06, 30 May 2008 (UTC)
- You seem to have your PHP error reporting level set too high. If you have to have it that way, change the line to:
if (substr($value, 0, 1) == "&") {
- P. S. There will probably be other places where you get a similar warning... Svemir Brkic 14:51, 30 May 2008 (UTC)
- P. P. S. Above tweak is now in the CVS and the latest 0.6 release. Svemir Brkic 15:10, 25 August 2008 (UTC)
You are the man! That fixed the error but "Did you mean" does not work. The search is conducted but returns 0 matches for a mispelling. Crap! One thing after another. Steve Goble 15:16, 30 May 2008 (UTC)
I got it working. I have ASPELL on my FCKEditor and trying to use that with CLI. I just enabled the php-pspell and that works good enough for me!! Thanks again, Svemir! BTW I am on a Windows 2008 box. Works great! Steve Goble 15:48, 30 May 2008 (UTC)
Problem with SphinxSearch_body?
I have installed Sphinx Search on a wiki (MW1.10), running on Ubuntu Hardy. When I go to the Special:SphinxSearch page, I get the following error...
Catchable fatal error: Object of class Title could not be converted to string in /var/www/wiki/extensions/SphinxSearch_body.php on line 456
Please advise. Eric --87.86.35.34 15:10, 11 July 2008 (UTC)
- What version of PHP are you using? Title class has a __toString method, which in PHP 5 makes conversion to string possible. If you are using PHP 4, try changing any place that does
echo $title;(or something like that) toecho $title->getPrefixedText();Svemir Brkic 15:29, 11 July 2008 (UTC)
root@wiki:/var/www/wiki/extensions# php --version PHP 5.2.4-2ubuntu5.1 with Suhosin-Patch 0.9.6.2 (cli) (built: May 9 2008 16:34:16) Copyright (c) 1997-2007 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
Eric --87.86.35.34 15:37, 11 July 2008 (UTC)
- In that case maybe your version of MW does not have the __toString method in Title.class? If so, above fix would still work. I think I will update extension code to use that anyway. Svemir Brkic 16:50, 11 July 2008 (UTC)
Thanks for your continued help Svemir. I have tried to implement your fix: As far as I can tell I changed any line that dereferences $title to a call to getPrefixedText(). I still get the error above though. The offending line 456 has no reference to class Title, and so far as I can tell neither do it's backlinks.
454 $wgOut->addHTML("<form action='$kiAction' method='GET'>
455 <input type='hidden' name='title' value='$titleObj'>
465 <input type='text' name='$searchField' maxlength='100' value='$SearchWord'>
457 <input type='submit' name='fulltext' value='" . wfMsg('sphinxSearchButton') ."'>");
Do you have any suggestions on what I can change further? Thanks again, Eric --87.86.35.34 16:04, 16 July 2008 (UTC)
- It seems to be the
value='$titleObj'part. Tryvalue='".$titleObj->getPrefixedText()."'216.52.121.66 16:58, 16 July 2008 (UTC)
- Problem solved! Good job mystery user! Thanks again Svemir - great job. Eric --87.86.35.34 08:55, 17 July 2008 (UTC)
Problems with search within page title / SOLVED
Hi, I installed the sphinxsearch on ubuntu and it is working very good. Thx for this extension! I had some problems with title search. Originally, the suggested query (default value in the installation package) was:
#sql_query = SELECT page_id, page_title, page_namespace, old_id, old_text \
# FROM mw_page, mw_revision, mw_text \
# WHERE rev_id=page_latest AND old_id=rev_text_id
The page_title content in the database looks like 'HOW_TO_edit_homepage'. There are underlines between each character sequence (mediawiki replaces blanks through underlines when new page is created). I changed the query (for initial and incremental index):
initial index:
sql_query = SELECT page_id, replace(page_title,'_',' ') as page_title, page_namespace, old_id, old_text \
FROM mw_page, mw_revision, mw_text \
WHERE rev_id=page_latest AND old_id=rev_text_id
incremental updates:
SELECT page_id, replace(page_title, '_',' ') as page_title, page_namespace, old_id, old_text FROM mw_page, mw_revision, mw_text WHERE rev_id=page_latest AND old_id=rev_text_id AND page_touched>=DATE_FORMAT(CURDATE(), '%Y%m%d070000')
Now it works. --Wikigeil 17:15, 21 January 2008 (UTC)
- I do not think this is necessary. Perhaps there was another issue with your setup and it got resolved while you were changing the queries. If you look at this line in the suggested spihinx.conf file:
charset_table = 0..9, A..Z->a..z, _-> , a..z, \
- Among other things, this instructs Sphinx to consider an underscore the same as a space. Perhaps you should try again with the original queries, as they would probably work faster. On the other hand, I might be misunderstanding what were you trying to fix, so please let me know if that is the case. Svemir Brkic 13:05, 2 February 2008 (UTC)
- Hi Svemir, thx for response. The charset_table property for main index is:
# charset definition and case folding rules "table" charset_table = 0..9, A..Z->a..z, _-> , a..z, \ U+C0->a, U+C1->a, U+C2->a, U+C3->a, U+C4->a, U+C5->a, U+C6->a, \ U+C7->c,U+E7->c, U+C8->e, U+C9->e, U+CA->e, U+CB->e, U+CC->i, \ U+CD->i, U+CE->i, U+CF->i, U+D0->d, U+D1->n, U+D2->o, U+D3->o, \ U+D4->o, U+D5->o, U+D6->o, U+D8->o, U+D9->u, U+DA->u, U+DB->u, \ U+DC->u, U+DD->y, U+DE->t, U+DF->s, \ U+E0->a, U+E1->a, U+E2->a, U+E3->a, U+E4->a, U+E5->a, U+E6->a, \ U+E7->c,U+E7->c, U+E8->e, U+E9->e, U+EA->e, U+EB->e, U+EC->i, \ U+ED->i, U+EE->i, U+EF->i, U+F0->d, U+F1->n, U+F2->o, U+F3->o, \ U+F4->o, U+F5->o, U+F6->o, U+F8->o, U+F9->u, U+FA->u, U+FB->u, \ U+FC->u, U+FD->y, U+FE->t, U+FF->s,
- It contains underscore too. --Wikigeil 17:34, 4 February 2008 (UTC)
- Yes, that is why I think the replace function in the query is not necessary. Everything works fine without it, with the original suggested queries. Svemir Brkic 04:21, 6 February 2008 (UTC)
- Having the underscore in the charset table makes it a regular character, so titles do not get indexed properly. If "_-> ," is removed, titles get indexed correctly and replace function is not needed anymore. Of course, if you do want to index words with underscores (maybe if your wiki contains lots of code examples with underscores in function names?) you should replace "_-> ," with just "_," and still use the replace function in the query. Svemir Brkic 15:19, 8 May 2008 (UTC)
How to search Chinese
I'm chinese wiki's admin, and I have configed sphinx.conf follow all the steps. Now I can search English words correctly but if I just search only chinese words like "注册"， I can get nothing? That means i must search mixed words like "sbc 注册"，so it can give my right results. Who can help me？ --Fzy 163 01:21, 15 May 2008 (UTC)
- Initial term check was too strict and it would not let Unicode-only strings through. I have changed it in the CVS, but you can see below how to fix it in the version you are currently using. Svemir Brkic 13:14, 16 June 2008 (UTC)
- thanks a lot, it get works ^_^ --23 June 2008
Blank search check
In function «wfSphinxSearch», please, replace line
if (!preg_match('/[\w\d]/', $term)) {
to line
if (!preg_match('/[\w\pL\d]/u', $term)) {
Because first variant ignore non-latin unicode quieries (for example russian search terms). --StasFomin 16:25, 11 June 2008 (UTC)
- I could not get this to work on my installation for some reason - \pL would not match any random Russian word I copy-pasted. Internally, PHP saw those words as \x... sequences, but they just would not match - maybe they were not fully valid UTF-8. However, I am not sure why we would go to such great lengths here anyway. I have changed it to:
if (trim($term) === '') {
- and it works fine now. I committed it to CVS and will make it a part of the next release. Thanks for pointing this out. Svemir Brkic 13:04, 16 June 2008 (UTC)
getActionURL did not preserve query terms
getActionURL did not preserve query terms when clicking on «page URL».
So query like "Foo+Bar" on page 2 transforms to query "Foo Bar".
You have to replace (at function getActionURL) line
$qry = $kiaction . "?$searchField={$term}&fulltext=".wfMsg('sphinxSearchButton')."&";
to something like this
$sterm=urlencode($term);
$qry = $kiaction . "?$searchField={$sterm}&fulltext=".wfMsg('sphinxSearchButton')."&";
--StasFomin 12:50, 10 November 2008 (UTC)
- Thanks! I guess nobody reported this until now because in some browsers it works anyway. Fixed and committed to CVS. Svemir Brkic 00:25, 11 November 2008 (UTC)
match all words as default
Not sure if this is something that is controlled by Sphinx or not, but it would be better for me if "match all words" was the default. Matching mode set to: SPH_MATCH_ALL - by default it is SPH_MATCH_EXTENDED, but does not work properly as http://www.sphinxsearch.com/doc.html#extended-syntax explains --1 April 2008
- You can set things such as $wgSphinxSearch_mode in LocalSettings. The reason we use SPH_MATCH_EXTENDED is to be able to modify the query internally and do "match all" vs. "match any" with a radio box instead of teaching users the Sphinx query syntax. Svemir Brkic 03:08, 6 April 2008 (UTC)
2009
Installation issues
Problem 1
Query failed: connection to localhost:3312 failed
I can search from the command line but it does not seem to work from the site. Any idea what i am missing??
Solution:
used 3306, i think it's ubuntu's default
make sure to run /usr/local/bin/searchd and it works! also 0.0.0.0 is used as should be by default (but is not) --24 March 2008
--213.220.226.220 15:25, 28 April 2009 (UTC)milan.m.masek@gmail.com
In case there is SELinux enabled on your server, check it´s configuration or disable it.
[root@levi ~]# getenforce [root@levi ~]# setenforce 0
Problem 2
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 775173422 bytes) in .../extensions/SphinxSearch/sphinxapi.php on line 311
Solution: This problem was cause by the solution to problem 2, that is using port 3306 which is used by mysql already...
see http://sphinxsearch.com/forum/view.html?id=1178 --24 March 2008
Namespaces for weight
Is it possible to set a weight to different namespace that way the checkboxes can be removed all together... --25 March 2008
- This is possible and I will look into it for the next release. It will not replace the need for checkboxes, as in some cases you want the users to specify which namespaces they are interested in. Svemir Brkic 02:17, 26 March 2008 (UTC)
Orphaned pages not indexed?
It seems SphinxSearch is unable to find orphaned pages in my wiki. Only things linked somehow (also indirectly) from the Main Page can be found. Is that something that can be configured away? I am using sphinx-0.9.9-rc1, SphinxSearch extension 0.6.1, MediaWiki 1.13.3. Thanks! — User:Trohlfing Feb 8, 2009
- SphinxSearch extension is not using wiki links in any way. It is running direct database queries, as specified in sphinx.conf. You need to check the namespace of those orphaned pages. Svemir Brkic 21:44, 8 February 2009 (UTC)
- There is a case in which it only finds linked-to pages. If a page title has spaces in it, these are stored as underscores in the MediaWiki database (in the "page" table). Sphinx doesn't know about this, so if you search for a page called "Sphinx search engine", by typing "Sphinx search engine" instead of "Sphinx_search_engine", you'll only see the links to that page -- so, if your page is orphaned (no pages reference your page title using spaces), you won't get the result. User:mphasak 23:30, 9 February 2009 (UTC)
- Check your sphinx.conf file. If it has "_-> ," in the charset_table, you should remove it and reindex. Svemir Brkic 02:34, 10 February 2009 (UTC)
- Perfect!! Thanks, Svemir. User:mphasak 19:25, 10 February 2009 (UTC)
Searching multiple namespaces as a default
I've had problems getting this working. The following has been specified in the LocalSettings.php with no luck:
$wgNamespacesToBeSearchedDefault = array(
NS_MAIN => true,
NS_HowTo => true,
Will SphinxSearch pick this up? I've tried applying this before and after the SphinxSearch extension loading lines in LocalSettings.php.
CaliVW78 06:07, 8 February 2009 (UTC)
- SphinxSearch will pick it up the same way the default MediaWiki search picks it up - for new and for anonymous users. Existing users already have their own default set of namespaces saved in their user preferences and have to change it manually. See Manual:$wgNamespacesToBeSearchedDefault for details (and how to update existing user preferences with a maintenance script.) Svemir Brkic 12:58, 8 February 2009 (UTC)
- Thanks for the reply, I clearly had missed some items when looking at the wgNamespacesToBeSearchedDefualt page earlier. I am however, still running into an issue where those who are not logged in, are not getting search results from the additional namespace that I am wanting. Below is the code I have in my LocalSettings.php, and the new name space I am wanting to be searched is "HowTo".
$wgNamespacesToBeSearchedDefault = array(
NS_MAIN => true,
NS_HowTo => true,
);
In addition to that, this is what what placed in the LocalSettings.php to create that namespace...
$wgExtraNamespaces =
array(
100 => "HowTo"
);
define('NS_HowTo', 100);
Thanks again for the help. CaliVW78 13:12, 17 February 2009 (UTC)
- Ok, here's how I got this to work. Adding a line to $wgNamespacesToBeSearchedDefault was clearly NOT working. Instead, I added the following to LocalSettings.php to define the default user option. "100" is the ID that was given to my newly created namespace that I wanted to have included in the search results. CaliVW78 09:16, 18 March 2009 (UTC)
$wgDefaultUserOptions = array(
'searchNs100' => 1,
);
- This $wgDefaultUserOptions worked with the search option but caused trouble with our SMW(Semantic Mediawiki). We had to rewrite the statement as of (Namespaces in SMW from 100-109 are reserved, so we started at 150 respectively) --MWJames 11:24, 25 June 2010 (UTC)
$wgDefaultUserOptions['searchNs154'] = 1;
Can someone show how to edit sphinx.conf correctly
I am having issues with my sphinx.conf. I have it mostly working, however when I attempt to index my wiki I get an error regarding my sql_query_pre = wiki_ entry in the sphinx.conf: "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'wiki_' at line 1". wiki_ is a standard MediaWiki table prefix so I am unsure why it throws an error. I found the other presets here. --137.71.23.54 5 August 2008
# pre-query, executed before the main fetch query sql_query_pre = SET NAMES utf8
- Now, I don't know what's in your sphinx.conf, but based on mine, sql_query_pre is not the place to put your database prefix. Take another look at step 2; you have to go through sphinx.conf and tack your prefix onto the table names (but not the row names!). It's annoying, but I eventually figured out that there are annoyances all over the place with MediaWiki if you use a prefix. My advice is to bite the bullet now and get rid of the prefix now if it's an option. —Emufarmers(T|C) 03:51, 6 August 2008 (UTC)
- Add the prefix to the names listed after FROM in all cases, typically three or four. Say your prefix is cm_ you use:
FROM cm_page, cm_revision, cm_text
- Works fine, no big deal. 76.243.138.215 20:39, 8 July 2009 (UTC)
- Add the prefix to the names listed after FROM in all cases, typically three or four. Say your prefix is cm_ you use:
Abysmal setup/configuration instructions
One moment it's talking about win32 builds, then next rc.local on crontabs. This REALLY needs seperating out into Unix and Windows setup and configuring.
- Some of the cleanup was done already, and will be completed for the next release. Svemir Brkic 19:46, 12 December 2009 (UTC)
Having trouble with the Sort By command
I'm trying to use the Sort By command so that I can see the latest postings on our wiki in the search , but I'm not sure of the syntax - what should it be? $wgSphinxSearch_sortby = "SPH_SORT_TIME_SEGMENTS, 'rev_timestamp'"; doesn't seem to work - I added rev_timestamp to the SQL query for the indexing, but no luck - can someone help me? --6 June 2008
- The way the code currently works, it always uses SPH_SORT_EXTENDED as the sort mode, and only uses $wgSphinxSearch_sortby as a second argument in the SetSortMode call. I will make this more flexible, but until then you can edit SphinxSearch_body.php and find this line:
$cl->SetSortMode(SPH_SORT_EXTENDED, $wgSphinxSearch_sortby);
- Set your $wgSphinxSearch_sortby to 'rev_timestamp' and change above line to:
$cl->SetSortMode(SPH_SORT_TIME_SEGMENTS, $wgSphinxSearch_sortby);
- Committed a change for next release: added $wgSphinxSearch_sortmode, and fixed support for $wgSphinxSearch_sortby. You will be able to set $wgSphinxSearch_sortmode = SPH_SORT_TIME_SEGMENTS and $wgSphinxSearch_sortby = 'rev_timestamp' in your LocalSettings to make this work. Svemir Brkic 20:38, 12 December 2009 (UTC)
Special:Search not recognized
I'm running SphinxSearch 0.6 with MediaWiki 1.12, and Special:SphinxSearch works great. However, when I set
$wgDisableInternalSearch = true; $wgDisableSearchUpdate = true; $wgSearchType = 'SphinxSearch';
Special:Search yields a No such special page error. —Emufarmers(T|C) 11:22, 12 June 2008 (UTC)
- Are your above lines followed by this:
if ( !function_exists( 'extAddSpecialPage' ) ) {
# Download from http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/ExtensionFunctions.php
require_once( dirname(dirname(__FILE__)) . '/ExtensionFunctions.php' );
}
extAddSpecialPage( dirname(__FILE__) . '/SphinxSearch_body.php', ($wgDisableInternalSearch ? 'Search' : 'SphinxSearch'), 'SphinxSearch' );
- Er, no, I hadn't seen that code before; I had ExtensionFunctions installed, which I assumed was sufficient. Adding those lines (and adjusting the paths) does seem to make things work, but I'm a bit confused about why they're necessary. —Emufarmers(T|C) 22:29, 16 June 2008 (UTC)
- require line is necessary to make sure extAddSpecialPage function is available. It is a function inside ExtensionFunctions.php file. Basically, it provides a backwards-compatible way of adding a special page. The conditional in extAddSpecialPage call specifies whether Sphinx replaces the default Search special page or is used as a stand-alone search page. Svemir Brkic 01:05, 17 June 2008 (UTC)
- Shouldn't this be added to Extension:SphinxSearch#Mode_Of_Operation then? --Patrick Nagel 09:40, 10 April 2009 (UTC)
- I think I know now what happened to Emufarmers and to me as well: we overread that the settings must be adjusted in SphinxSearch.php, and not (as usual with most extensions) added to LocalSettings.php. I put a note into Extension:SphinxSearch#Mode_Of_Operation - but maybe the author(s) of the SphinxSearch extension should consider changing to the usual way of letting the user change an extension's behaviour? It's easier to update the extensions that way, since the whole directory can just be replaced with a new version, all settings are in LocalSettings.php and don't get overwritten. --Patrick Nagel 10:13, 10 April 2009 (UTC)
- Thanks. Most settings can be changed the usual way as well, but we definitely need to do some cleanup for the next release. Svemir Brkic 04:26, 7 December 2009 (UTC)
Database error when running on MediaWiki 1.13.0
I get the following error:
A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:
(SQL query hidden)
from within function "SphinxSearch::wfSphinxSearch". MySQL returned error "1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1 (localhost)".
Searching from the command line DOES work, however. It only fails when I use the extension from within my Wiki.
Any ideas on how to fix this appreciated! --Zerbey 19:37, 11 September 2008 (UTC)
To display the hidden SQL statement place $wgShowSQLErrors = true; in your LocalSettings.php file. When you try the search again it will display something like:
SELECT old_text FROM `wikitext` WHERE old_id=
And this is a result of what is identified by Svemir Brkic in the $sql statement below found in the SphinxSearch_body.php file. warens 23:00, 18 November 2008 (UTC)
- The only query in that method is generated with this line:
$sql = "SELECT old_text FROM ".$db->tableName('text')." WHERE old_id=".$docinfo['attrs']['old_id'];
- Find the line and error_log or echo the query to see what might be wrong with it. Svemir Brkic 17:07, 12 September 2008 (UTC)
I have the same error in MW 1.11.0, if i enter a search string, which appears in the wiki ( like foo ). But when i enter something like dhfbvjhsfdbvjhsb, sphinx says, that 0 results ar found. On the commandline the search works very well an fast. --212.87.151.131 15:29, 16 September 2008 (UTC)
- Well here's the error:
[Wed Oct 08 10:16:19 2008] [error] [client 10.20.60.26] PHP Notice: Undefined variable: wgSphinxSuggestMode in /var/www/htdocs/wiki/extensions/SphinxSearch/SphinxSearch.php on line 85, referer: http://wiki.galaxy.invalid/wiki/index.php/Special:SphinxSearch [Wed Oct 08 10:16:19 2008] [error] [client 10.20.60.26] PHP Notice: Undefined variable: wgSphinxSuggestMode in /var/www/htdocs/wiki/extensions/SphinxSearch/SphinxSearch.php on line 89, referer: http://wiki.galaxy.invalid/wiki/index.php/Special:SphinxSearch [Wed Oct 08 10:16:20 2008] [error] [client 10.20.60.26] PHP Notice: Undefined index: old_id in /var/www/htdocs/wiki/extensions/SphinxSearch/SphinxSearch_body.php on line 347, referer: http://wiki.galaxy.invalid/wiki/index.php/Special:SphinxSearch
- Not sure were to go from here. Zerbey 14:20, 8 October 2008 (UTC)
Answer
The first two notices can be ignored, but I have fixed them in the CVS anyway. To fix them in your code, find this line in SphinxSearch.php:
#$wgSphinxSuggestMode = true;
Change it to:
$wgSphinxSuggestMode = false;
Unless, of course, you want to turn the suggestions on. The third notice indicates that something is wrong with your sphinx.conf file, or with the version of sphinx you are using. Recent versions of sphinx require this to be set in sphinx.conf:
sql_attr_uint = old_id
Older versions used sql_group_column instead. If your sphinx.conf has one of these, try the other one (or try upgrading sphinx itself.) Svemir Brkic 14:54, 8 October 2008 (UTC)
- I have the exact same problem, i have checked my sphinx.conf file and it already has the sql_attr_uint = old_id line already, replacing it doesnt seem to help either. Im using Sphinx 0.9.8.1. Would there be any other ways to correct this problem? --213.122.168.100 15:08, 27 November 2008 (UTC)
- If you are having exact same problem as above, with all of those warnings and error messages, you first need to download the latest version of the SphinxSearch extension. Also, make sure your index has been rebuilt and search deamon restarted. If none of that helps, perhaps post your sphinx.conf somewhere (without the db password...) and someone may be able to tell you where the problem is. Svemir 13:45, 4 December 2008 (UTC)
Sphinx.conf file below without passwords and comments. I apologise if this is the wrong place to post code. For the searchd settings near the bottom of the file, should the address be 127.0.0.1 if the search deamon is on a remote ubuntu server. Regardless though of the entry there, it still has the same error. Any help people can give would be gratefully received!!
source src_wiki_main
{
- type = mysql
- sql_host = 10.150.2.71
- sql_user = wikiuser
- sql_pass = password
- sql_db = wikidb
- sql_query_pre = SET NAMES utf8
- sql_query = SELECT page_id, page_title, page_namespace, old_id, old_text FROM page, revision, text WHERE rev_id=page_latest AND old_id=rev_text_id
- sql_attr_uint = page_namespace
- sql_attr_uint = old_id
- sql_query_info = SELECT page_title, page_namespace FROM page WHERE page_id=$id
}
source src_wiki_incremental : src_wiki_main
{
- sql_query = SELECT page_id, page_title, page_namespace, old_id, old_text FROM page, revision, text WHERE rev_id=page_latest AND old_id=rev_text_id AND page_touched>=DATE_FORMAT(CURDATE(), '%Y%m%d070000')
}
index wiki_main
{
- source = src_wiki_main
- path = /var/data/sphinx/wiki_main
- docinfo = extern
- morphology = stem_en
- min_word_len = 1
- charset_type = utf-8
- charset_table = 0..9, A..Z->a..z, a..z, \
U+C0->a, U+C1->a, U+C2->a, U+C3->a, U+C4->a, U+C5->a, U+C6->a, \ U+C7->c,U+E7->c, U+C8->e, U+C9->e, U+CA->e, U+CB->e, U+CC->i, \ U+CD->i, U+CE->i, U+CF->i, U+D0->d, U+D1->n, U+D2->o, U+D3->o, \ U+D4->o, U+D5->o, U+D6->o, U+D8->o, U+D9->u, U+DA->u, U+DB->u, \ U+DC->u, U+DD->y, U+DE->t, U+DF->s, \ U+E0->a, U+E1->a, U+E2->a, U+E3->a, U+E4->a, U+E5->a, U+E6->a, \ U+E7->c,U+E7->c, U+E8->e, U+E9->e, U+EA->e, U+EB->e, U+EC->i, \ U+ED->i, U+EE->i, U+EF->i, U+F0->d, U+F1->n, U+F2->o, U+F3->o, \ U+F4->o, U+F5->o, U+F6->o, U+F8->o, U+F9->u, U+FA->u, U+FB->u, \ U+FC->u, U+FD->y, U+FE->t, U+FF->s,
}
index wiki_incremental : wiki_main
{
- path = /var/data/sphinx/wiki_incremental
- source = src_wiki_incremental
}
indexer
{
- mem_limit = 64M
}
searchd
{
- address = 127.0.0.1
- port = 3312
- log = /var/log/sphinx/searchd.log
- query_log = /var/log/sphinx/query.log
- read_timeout = 5
- max_children = 30
- pid_file = /var/log/sphinx/searchd.pid
- max_matches = 1000
}
- --eof--
how to change the search page language
I want to translate the search page to chinese, I add some codes in sphinxsearch_body.php
$allMessages = array(
'en' => array(
'sphinxsearch' => 'Search Wiki Using Sphinx',
'sphinxSearchInNamespaces' => 'Search in namespaces:',
'sphinxSearchInCategories' => 'Search in categories:',
'sphinxResultPage' => 'Result Page: ',
'sphinxPreviousPage' => 'Previous',
'sphinxNextPage' => 'Next',
'sphinxSearchPreamble' => "Displaying %d-%d of %d matches for query %s retrieved in %0.3f sec with following stats:",
'sphinxSearchStats' => "* %s found %d times in %d documents",
'sphinxSearchButton' => 'Search',
'sphinxSearchEpilogue' => 'Additional database time was %0.3f sec.',
'sphinxSearchDidYouMean' => 'Did you mean',
'sphinxMatchAny' => 'match any word',
'sphinxMatchAll' => 'match all words',
'sphinxLoading' => 'Loading...'
)
'zh-cn' => array(
'sphinxsearch' => '使用斯芬克斯搜索工具',
'sphinxSearchInNamespaces' => '在名字空间内搜索:',
'sphinxSearchInCategories' => '在目录内搜索:',
'sphinxResultPage' => '结果页数: ',
'sphinxPreviousPage' => '前一页',
'sphinxNextPage' => '下一页',
'sphinxSearchPreamble' => "Displaying %d-%d of %d matches for query %s retrieved in %0.3f sec with following stats:",
'sphinxSearchStats' => "* %s 被发现 %d 次在 %d 个文件里",
'sphinxSearchButton' => '搜索',
'sphinxSearchEpilogue' => 'Additional database time was %0.3f sec.',
'sphinxSearchDidYouMean' => '你的意思是',
'sphinxMatchAny' => '匹配任何词',
'sphinxMatchAll' => '匹配全部词',
'sphinxLoading' => '导入...'
)
but it can't work. The page still has english word display, not chinese. can somebody help me? kgbkgbkgb 15:34, 5 December 2008 (UTC)
- Perhaps you already used the extension before adding those translations? I think in that case you probably need to translate the strings in your Special page -> System messages. Svemir Brkic 18:59, 17 January 2009 (UTC)
newworldencyclopedia not such a good example
Went to http://www.newworldencyclopedia.org as this was given as a great example of Sphinx in action.
Did a search for pople to see if it would correct this to people.
The result was somewhat uninspiring.
OK to ask did I mean pope. Not OK to tell me it was showing zero matches and then said it found pople 15 times in 9 documents without showing any of them?
My users would not be impressed I'm afraid.
There is no page titled "pople". You can create this page. Did you mean pope?
Displaying 0-0 of 0 matches for query pople retrieved in 0.031 sec with following stats:
* pople found 15 times in 9 documents
- So far nobody suggested another example. The problems at NWE will be corrected soon. Currently there is only one index, used by two different interfaces. One of the interfaces exposes additional namespaces. Sphinx always tells you the full number of matches in the index, even though some of them may be filtered out by namespace and other filters. Svemir Brkic 19:44, 15 April 2009 (UTC)
- This is mostly corrected now, though some more tweaks are needed. Note that you still need to select all available namespaces in order to see all the documents that the Sphinx stats mention. Svemir Brkic 00:08, 7 December 2009 (UTC)
- You could try wiki.service-now.com they use this in their instance of mediawiki.
- It looks good, but they do not use spelling suggestions nor categories, so it would not fully illustrate this extension. I will create a list of sites that use SphinxSearch and list it there. Svemir Brkic 04:16, 15 December 2009 (UTC)
Bug: MediaWiki variables (like {{CURRENTTIMESTAMP}}) get expanded on results page but not in search
see http://www.newworldencyclopedia.org/entry/Special:Search?search={{CURRENTTIMESTAMP}}&fulltext=Search
--Patrick Nagel 04:25, 27 July 2009 (UTC)
- Good catch :-) I fixed it in the CVS. If you need to fix this in your wiki before the next release, change the line 49 in SphinxSearch_body.php and add "nowiki" tags around %s. Svemir Brkic 11:49, 27 July 2009 (UTC)
MW API Bug
Causes a 500 when calling search from MW API (ie: api.php?action=query&list=search&srsearch=samplequery)
- Initial API support added and committed to CVS. Will be a part of the next release. Svemir Brkic 03:55, 14 December 2009 (UTC)
Tweaking the Search Excerpts?
Does anyone know how I might adjust the excerpts displayed with the search returns? I find 5 lines to be a bit too long, and would like to experiment with fewer. Thanks!
--John Thomson 03:13, 11 June 2009 (UTC)
- In SphinxSearch_body.php, line 343 or so, change "limit" => 400 to a smaller number. I guess we should make it configurable, but there it is for now. Svemir Brkic 13:22, 11 June 2009 (UTC)
- Ah! Many thanks, Svemir, that did the job nicely! (I went to 250, and am very pleased!)
--John Thomson 22:14, 11 June 2009 (UTC)
- Ah! Many thanks, Svemir, that did the job nicely! (I went to 250, and am very pleased!)
Searching multiple wikis
Make sure the $wgSphinxSearch_index variable is set to the main index in the localsettings or somewhere. Also, assuming you are using the main+delta scheme outlined on main page, you need to roll the two indexes into one. Something like (stripped paths for brevity-haha):
0 9,15,21 * * * indexer --quiet --config sphinx.conf wiki1_incremental wiki2_incremental --rotate && indexer --quiet --config sphinx.conf --merge wiki1_main wiki1_incremental && indexer --quiet --config sphinx.conf --merge wiki2_main wiki2_incremental
From the sphinx docs it says that merging the indexs does take time, but is normally faster than reindexing. --UnwashedMeme 22:41, 22 October 2009 (UTC)
- Playing around with it a bit more: it works better if you sleep for a few seconds(I'm using 5) between running the indexer on the 2 incremental indexes and then running the merge command. Otherwise the .new version of the index isn't there yet when the indexer sends searchd the sighup that causes it to rotate and you don't actually get the updated index until the next time cron rotates the indexes. I don't think that sleep is related to index size; i.e. a large db that takes longer to index would still only need to sleep long enough to give the kernel a chance to get things synced again? UnwashedMeme 19:28, 23 October 2009 (UTC)
Multiple Indexes on same searchd returning strange results
When using multiple indexes (for multiple wikis) running against the same searchd, we end up with a search matching against all indexes, but only displaying results for the configured index. This can cause several dozen pages of blank entries interspersed with a few links.
Results are even worse when sphinx is used for things other than mediawiki - there will be SQL syntax errors and search feature will be broken completely - when there are matches from other indices which do not have all the expected fields in the query result.
in SphinxSeach_body.php change
$res = $cl->Query($search_term, "*");
to
$res = $cl->Query($search_term, "index1 index2");
where index1 and index2 are names of indices you want to search.
there really should be a config variable - specifying which indices need to be searched.
- I just committed a change that will be included in the next release: $wgSphinxSearch_index_list lets you specify the list of indexes to search. You can have multiple main and delta indexes on the machine, and each wiki can define its own pair to search. You will also be able to combine several index files and assign different weight to each of them, using $wgSphinxSearch_index_weights array. Svemir Brkic 20:07, 12 December 2009 (UTC)
Query failed: index wiki_incremental,wiki_main: syntax error, on Sphinx 0.9.9/SphinxSearch 0.6.1
With Sphinx version 0.9.9 the following search term 'Vance/2009' created an error Query failed: index wiki_incremental,wiki_main: syntax error, unexpected '/', expecting $end near '/2009'. If you just search with 'Vance' then search results will return but not for 'Vance/2009'.
With Sphinx version 0.9.8 the search term 'Vance/2009' produces ...for query Vance/2009 retrieved in 0.049 sec... search results.
The system uses the following configuration LightTPD 1.4.22, MediaWiki 1.15.1, PHP 5.2.9-1 (cgi-fcgi), MySQL 5.0.77-community-nt, SphinxSearch (Version 0.6.1).
- Thanks for the catch. Happens to me as well. Seems like a change in sphinx internals. I will commit a fix for this for the next release, but until then, find the line in SphinxSearch_body.php that does something like this:
$res = $cl->Query($search_term, "*");
and replace it with
$res = $cl->Query(addcslashes($search_term, '/()[]"!'), "*");
--Svemir Brkic 18:35, 20 December 2009 (UTC)
I know this is somehow rather odd, but in case of "Vance/2009" the solution worked but when I change the search term to "Vance / 2009" the system produces an error message Query failed: index wiki_incremental,wiki_main: syntax error, unexpected '|' near '\/|2009'.
I have the same issue. When you search, is the option "Match Any Word" enabled, or "Match All Words" enabled? When I change from Match Any to Match All, it works for me. Also, This might occur because your configuration file is set to only allow searching of words >= 3 chars (see min_word_len in sphinx.conf for your wiki -- 3 should be the default -- which is where mine failed)
-- dkettle@userful.com, 16:51, 11 January 2011 (MST)
Query failed: syntax error, unexpected $end near --> ["]
The following search term Knowledge Economy" in SphinxSearch causes the following error --> Query failed: index wiki_incremental,wiki_main: syntax error, unexpected $end near where the same search with search --config c:\Sphinx\sphinx.conf Knowledge Economy" produces no error.
Searching with SphinxSearch the term "Knowledge Economy" produces no error.
The system uses LightTPD 1.4.22 Sphinx 0.9.9-release (r2117), SphinxSearch (Version 0.6.1), MediaWiki 1.15.1, PHP 5.2.9-1 (cgi-fcgi), MySQL 5.0.77-community-nt.
- I have updated the fix above to cover this case as well. Sphinx engine added more cool features in version 0.9.9, and their API is not fully backward-compatible. I hope to straighten it out for next release. Svemir Brkic 14:24, 24 December 2009 (UTC)
- It works, Thanks for the swift support.
Query failed: syntax error, unexpected $end near --> [&]
I found another , Query failed ... if you search with the term Lawyers & Doctors 2008, the search will produce a Query failed: index wiki_incremental,wiki_main: syntax error, unexpected '|' near '&|Doctors|2008'. When you search with Lawyers Doctors 2008 the system will produce results.
The system uses LightTPD 1.4.22 Sphinx 0.9.9-release (r2117), SphinxSearch (Version 0.6.1), MediaWiki 1.15.1, PHP 5.2.9-1 (cgi-fcgi), MySQL 5.0.77-community-nt.
- This only happens in "match any word" case. The fix is in a different line. Find something like:
preg_replace( '/[\s_\-]+/', '|', ...
- and replace with:
preg_replace( '/[\s_\-&]+/', '|', ...
Svemir Brkic 17:29, 28 December 2009 (UTC)
- Thanks, always appreciate the nice support.
Query failed: syntax error, unexpected $end near --> [!!]
The term いことがスグわかる!! produced the error Query failed: index wiki_incremental,wiki_main: syntax error, unexpected '-' near '!'. where with いことがスグわかる their is no problem for the search.
- I have updated the fix above (the one with addcslashes) to also escape the !. Please apply both fixes until next release (I am very close to publishing it, maybe next weekend...) Svemir Brkic 23:06, 10 January 2010 (UTC)
@page_title DOES NOT WORK
for some reason @old_text works but @page_title never returns any results... maybe it has something to do with the new releases of sphinx. --207.96.208.130 22:22, 1 April 2008 (UTC)
- New in 0.7: 'match titles only' checkbox on the search page. Svemir Brkic 01:06, 18 February 2010 (UTC)
127.0.0.1 instead of localhost
As mentioned elsewhere (on the talk page?) it is important to use IP address 127.0.0.1 instead of localhost in the sphinx.config files. Found this hint after lengthy trials. It should be mentioned more prominently! Only this change made my installation sucessful and start working with mediawiki (on Windows XP).
On the command line localhost works fine.
Thanks! Had the same problem with Suse Linux.
No Result Page
I'm running on Windows and MediaWiki 1.14. The daemon is started, and I can use the command line search, but when I try and search using the special page, nothing happens. The page simply appears to reload. There are no messages in Event Viewer, the query.log file shows my query terms, there are no messages in searchd.log. I have no idea why I'm not getting any result pages. Any able to give me a clue?
Also, the service appears to be listening since I was able to connect via telnet to localhost at 3312. I haven't changed any of the properties in the SphinxSearch.php page.
Running through the options in SphinxSearch.php I noticed that the $wgSphinxSearch_index value didn't match my Sphinx.conf. I've corrected that but still no search results. The URL in the address bar doesn't even change after hitting the search button.
- Is Sphinx set to be the default search? Whatever it is now, try the other option. Also make sure php.ini is set to display or log errors and watch for PHP errors or warnings. Svemir Brkic 02:36, 18 March 2009 (UTC)
- 207.126.248.6 23:54, 22 April 2010 (UTC) I ran into this issue too with windows 2008 server R2. I was able to resolve it by changing 'localhost' to '127.0.0.1' in SphinxSearch.php. See this thread for more info [1]
- --80.216.253.12 22:00, 21 July 2010 (UTC)Had the same issue with running an XAMPP setup on Windows Vista machine, and this resolved my problems as well.
- 207.126.248.6 23:54, 22 April 2010 (UTC) I ran into this issue too with windows 2008 server R2. I was able to resolve it by changing 'localhost' to '127.0.0.1' in SphinxSearch.php. See this thread for more info [1]
Query failed: no enabled local indexes to search
Hi,
Pls help! Sphinx Search Problem - Query failed: no enabled local indexes to search I am able to run indexer in console and do a search in console. i am able to see the results.
i have started the daemon and it is running. When i use Special: Sphinx Search: it says "Could not instantiate Sphinx client. " And then when search for a word : it says "Query failed: no enabled local indexes to search "
If anyone have an idea, what mistake i am doing, pls let me know.
Some details :
Mediawiki 1.15.3
XAMPP 1.7.2
Windows Xp SP2
Sphinx 0.9.9 (win32)
SphinxSearch 0.7.1
Thanks in Advance.
--Ramesh
I have this same problem, here is what I have: XAMPP: 1.5.4a
Mediawiki: 1.16.0
MySQL: 5.0.33
PHP 5.2.1
Sphinx 0.9.9 (win32)
SphinxSearch 0.7.2
Thanks,
--Nate
How to create new documents when Sphinx is enabled as unique search engine ?
Hello, usually to create a new document I put the title in the search box and execute a search. Obviously the doc doesn't exist and the wiki propose a red link to create it. If I enable the Sphinx search engine to replace the basic MediaWiki search ($wgSearchType = 'SphinxSearch';) also if the doc is not found I don't get any red link or other (fast) way to create it. I'm missing something ??? Thanks in advance. --FrancoR 11:43, 18 February 2011 (UTC)
How to use together with External Data
I am having the same problem as FrancoR. If I change the search to SphinxSearch, I no longer get the red link create missing articles when I click the Go button.
Also, I have set up the External Data extension and it is working quite well. I am using #store_external_table to save the table values as properties, but what do I change the Sphinx queries to in order to relate the wiki pages to the properties? Using the fruit example, if someone searches for "apple", and apple appears in a table on a page, I want to provide a search result that directs the user to the page that contains "apple" in a table.
Thanks,
--Will July 19, 2011
Enable Editing Personal Dictionary
To enable the editing of the personal dictionary within the wiki on version 0.7.1, edit SphinxSearch.php, on or about line 27, right after:
$wgAutoloadClasses['SphinxSearch'] = $dir . 'SphinxSearch_body.php';
$wgExtensionMessagesFiles['SphinxSearch'] = $dir . 'SphinxSearch.i18n.php';
$wgExtensionAliasesFiles['SphinxSearch'] = $dir . 'SphinxSearch.alias.php';
And add:
$wgAutoloadClasses['SphinxSearchPersonalDict'] = $dir . 'SphinxSearch_PersonalDict.php';
$wgSpecialPages['SphinxSearchPersonalDict'] = 'SphinxSearchPersonalDict';
The restricted page Special:SphinxSearchPersonalDict will now be available.
Version 0.7 released
New release of this extension has been uploaded to SourceForge. The most important change is that you are not supposed to edit SphinxSearch.php anymore. All your modifications (host, port, mode, etc.) go to LocalSettings.php AFTER you include SphinxSearch.php. The only exception is this line that has to happen before SphinxSearch.php is included (if you want Sphinx to completely replace default MW search)
$wgSearchType = 'SphinxSearch';
Also, if you use the "personal dictionary" option, that needs to be set before inclusion - as explained in SphinxSearch.php. Default port has been changed to the new official sphinx port (9312.) All the fixes discussed on the talk page have been included as well. Both pages will be updated soon, just making sure the breaking changes are listed immediately.
Moving to MediaWiki repository
SphinxSearch is moving from SourceForge to MediaWiki SVN repository. Links in the infobox already point to the latest code. Some bugs have been fixed and a dozen or so translations added to MW version already. Changelog will be updated once all the cleanup and tweaks are done. Svemir Brkic 22:48, 10 April 2010 (UTC)
Could not instantiate Sphinx client
I finally got Sphinx and the extension working (Linux Ubuntu 9.10, MediaWiki 15.3, SphinxSearch-trunk-r66651). But always when browsing to the "Special Page: Search wiki using Sphinx" I get: "Could not instantiate Sphinx client.". This thing is: Sphinx IS working. So why does this message appear?? ;-) And by the way: I'm really looking forward to the new version Svemir is talking about. Keep up the great work! --SmartK 10:48, 18 May 2010 (UTC)
- +1 here. I am looking to see if I can hack in a fix. I'll post here if I find something. Gomeztogo 00:01, 22 June 2010 (UTC)
- It appears that there is no code that handles a null $term passed to the special page. So when you load the page it runs all of the search code with a null value, and returns the instantiate error. For now I'm not gonna put in the exception handling as a new major version of the extension is expected soon, and this is really just a cosmetic issue.
- If you want to get rid of this in a bit of a hacky way, look in SphinxSearch_body.php near line 228, and modify to match:
}
# No error on blank search term
} elseif ( $cl == false and trim( $term ) == "") {
} else {
$wgOut->addWikiText( wfMsg( 'sphinxClientFailed' ) . "\n" );
}
PHP Warning: AutoLoader::require
I have SphinxSearch running on a WAMP setup with Mediawiki 1.15.0. The search daemon is running fine, the indexer is running fine, an we can execute searches from the command line. When we try to run it from the wiki, however, we get the following error:
PHP Warning: AutoLoader::require(\\leia\wiki/\\leia\wiki\extensions\SphinxSearch/SphinxSearch_body.php) [function.AutoLoader-require]: failed to open stream: No such file or directory in \\leia\wiki\includes\AutoLoader.php on line 582 PHP Fatal error: AutoLoader::require() [function.require]: Failed opening required '\\leia\wiki/\\leia\wiki\extensions\SphinxSearch/SphinxSearch_body.php' (include_path='\\leia\wiki;\\leia\wiki/includes;\\leia\wiki/languages;.;C:\php5\pear') in \\leia\wiki\includes\AutoLoader.php on line 582
Leia is the name of our server, and wiki is the shared path to the mediawiki install in the Apache/htdocs folder. the issue seems to be that the path is getting doubled up (\\leia\wiki/\\leia\wiki\), but I am not sure where that is coming from. If we remove the $IP from the path to the extension in LocalSettings.php, the error message changes to this:
PHP Warning: require_once(/extensions/SphinxSearch/SphinxSearch.php) [function.require-once]: failed to open stream: No such file or directory in \\leia\wiki\LocalSettings.php on line 30 PHP Fatal error: require_once() [function.require]: Failed opening required '/extensions/SphinxSearch/SphinxSearch.php' (include_path='\\leia\wiki;\\leia\wiki/includes;\\leia\wiki/languages;.;C:\php5\pear') in \\leia\wiki\LocalSettings.php on line 30
Can anyone offer any help on this?
- Do not remove $IP. This is probably caused by ExtensionFunctions. We are getting rid of that for the next release. In the meantime, you will need to play with this line in SphinxSearch.php:
extAddSpecialPage( dirname(__FILE__) . '/SphinxSearch_body.php', ...
- Try changing it to:
extAddSpecialPage( 'extensions/SphinxSearch/SphinxSearch_body.php', ...
- If that does not work, check the error message again and try to adjust accordingly. If nothing else helps, new release should be ready soon. Svemir Brkic 01:30, 17 February 2010 (UTC)
Special Pages broken
The SVN version 73342 of SphinxSearch.php has been extended to give choices whether to use SphinxSearch, SphinxMWSearch and not to replace Mediawiki Search. This breaks the Special:SpeciaPages list. The follwing error is returned: Fatal error: Class 'SphinxSearch' not found in [install path of mediawiki]\includes\SpecialPage.php on line 390
To fix this add the $wgAutoloadClasses['SphinxSearch'] expression just below the } else {, which should be line 40 (Rev. 73342, 19-Sep-2010):
30:if ( $wgSearchType == 'SphinxMWSearch' ) {
31: $wgAutoloadClasses['SphinxMWSearch'] = $dir . 'SphinxMWSearch.php';
32:} else {
33: if ( $wgSearchType == 'SphinxSearch' ) {
34: $wgAutoloadClasses['SphinxSearch'] = $dir . 'SphinxSearch_body.php';
35: $wgDisableInternalSearch = true;
36: $wgDisableSearchUpdate = true;
37: $wgSpecialPages['Search'] = 'SphinxSearch';
38: $wgDisableSearchUpdate = true;
39: } else {
--> $wgAutoloadClasses['SphinxSearch'] = $dir . 'SphinxSearch_body.php';
40: $wgExtensionAliasesFiles['SphinxSearch'] = $dir . 'SphinxSearch.alias.php';
41: $wgSpecialPages['SphinxSearch'] = 'SphinxSearch';
42: }
43:}
This error cannot be observed when replacing Wikimwdia's search with SphinxSearch.
- Thank you very much. Fixed in revision 73790 by moving the line just before that second if. Svemir Brkic 19:53, 26 September 2010 (UTC)
sphinxapi.php - Permission Denied
I've installed the extension 0.6.1 and SphinxSearch 0.9.8.1 using MediaWiki 1.14 on Centos 5.2. when i uncomment the line require_once( "$IP/extensions/SphinxSearch/SphinxSearch.php" ); in localsettings.php I get the following error:
Warning: require_once(/var/www/mediawiki/extensions/SphinxSearch/sphinxapi.php) [function.require-once]: failed to open stream: Permission denied in /var/www/mediawiki/extensions/SphinxSearch/SphinxSearch.php on line 20
Fatal error: require_once() [function.require]: Failed opening required '/var/www/mediawiki/extensions/SphinxSearch/sphinxapi.php' (include_path='.:/var/www/mediawiki:/var/www/mediawiki/includes:/var/www/mediawiki/languages') in /var/www/mediawiki/extensions/SphinxSearch/SphinxSearch.php on line 20
permissions and user is the same as all other extensions that work fine. When i comment out require_once ( dirname( __FILE__ ) . "/sphinxapi.php" ); in SphinxSearch.php the extension is loaded and listed in the Special pages, although it can't be used to search. Help is much appreciated.
- Are you sure permissions are exactly the same on sphinxapi.php file as on the other files in the same folder? There has to be some difference. 17:01, 12 May 2009 (UTC)
They are the same as noted below
-rwxr--r-- 1 root root 2217 May 4 08:09 ExtensionFunctions.php
-rwxr--r-- 1 root root 34191 May 11 13:54 sphinxapi.php
-rw-r--r-- 1 root root 14081 May 7 08:50 sphinx.conf.bak
-rwxr--r-- 1 root root 22472 Nov 10 18:59 SphinxSearch_body.php
-rwxr--r-- 1 root root 1758 Nov 10 19:00 SphinxSearch.js
-rwxr--r-- 1 root root 8616 Nov 10 2008 SphinxSearch_PersonalDict.php
-rwxr--r-- 1 root root 4222 May 11 13:59 SphinxSearch.php
-rw-r--r-- 1 root root 4287 May 11 13:11 SphinxSearch.php.bak
-rwxr--r-- 1 root root 5534 Nov 10 2008 SphinxSearch_spell.php
-rw-r--r-- 1 root root 1456 Apr 6 2008 spinner.gif
Thanks J
2010
<noexactmatch> in MW 1.16
Upgraded my wiki to 1.16 (in DEV), and now when I don't get an exact match, I see <noexactmatch> at the top of the search results, instead of seeing "There is no page titled "Foo Bar". You can create this page."
Any ideas? Thanks!
--John Thomson 21:17, 14 April 2010 (UTC)
- Thanks for the report. That message was removed from MW in 1.16. The fix for this will be implemented in the next release of the extension. Svemir Brkic 00:23, 15 April 2010 (UTC)
- Hi Svemir. Version 0.7.1, released in September of 2010, is still afflicted by this issue.
--John Thomson 04:33, 6 March 2011 (UTC)
- Hi Svemir. Version 0.7.1, released in September of 2010, is still afflicted by this issue.
Temp Fix Between Releases
In the meantime I fixed this in my installation by modifying SphinxSearch_body.php
$wgOut->addWikiText( wfMsg( 'noexactmatch', wfEscapeWikiText( $term ) ) );
needs to be changed to
$wgOut->addWikiText( wfMsg( 'searchmenu-new', wfEscapeWikiText( $term ) ) );
Gomeztogo 00:00, 22 June 2010 (UTC)
- Thank you, Gomeztogo! This did the trick for me! Much appreciated!
--John Thomson 04:33, 6 March 2011 (UTC)
- This still isn't fixed, but the temp fix doesn't work for me until after adding 'searchmenu-new' as an additional entry in the $messages array in SphinxSearch.i18.n.php:
'sphinxPspellError' => 'Could not invoke pspell extension.', 'searchmenu-new' => 'There is no article titled [[$1]]. You can [[$1|create this page]].'
modify "SphinxSearch_body.php" to work with user rights restriction extension
there are many user rights restriction extensions which use the "usercanread" hook to make some page can not be touched, such as "Extension:Restrict_access_by_category_and_group". how can i modify the SphinxSearch_body.php, to add somes codes like " if (!$t->userCanRead()) {}", to prevent some important messages don't listed on the search result page. 121.8.153.6 08:02, 26 July 2010 (UTC)
solved, but not perfectly
no matter what extensions of user rights you are using, most of them use the "userCan" hook. For example,on the page of "Extension:Restrict access by category and group" , click the "hooks show", you can see that this extension uses the "userCan" hook. If an extension uses "userCan" hook to carry out user rights restriction, we can make sphinxsearch do not show the infomation not permitted. I modified the codes, in the SphinSearch_body.php, find " function wfSphinxDisplayResults( $term, $res, $cl ) {",
+ if( !$title_obj->userCanRead() ) {
+ $wgOut->addWikiText("");
+ }else{
$wgOut->addWikiText( "* $wiki_title" );
# uncomment this line to see the weights etc. as HTML comments in the source of the page
# $wgOut->addHTML("");
strip_tags( $entry, '<span><br />' )
);
$wgOut->addHTML( "<div style='margin: 0.2em 1em 1em 1em;'>$entry</div>\n" );
+ }
}
}
}
}
these two modifies can make the information not permitted not be shown by the sphinxsearch, but still on the result page displys the result count and the total page number. if do more job, we can show that how many docs are found in xx time, how many docs are not shown due to the permission, and shows the exact numbers of pages.121.8.153.6 02:07, 27 July 2010 (UTC)
Another solution for 0.7.1
The above didn't fit on 0.7.1, so I developed a smaller patch for this version:
--- SphinxSearch_body.php.org 2010-09-15 03:23:57.000000000 +0200
+++ SphinxSearch_body.php 2011-01-04 10:00:31.751716172 +0100
@@ -494,7 +494,7 @@
);
if ( $page_content ) {
$title_obj = Title::newFromID( $doc );
- if ( is_object( $title_obj ) ) {
+ if ( is_object( $title_obj ) && $title_obj->userCanRead()) {
$wiki_title = $title_obj->getPrefixedText();
$wiki_path = $title_obj->getPrefixedDBkey();
$wgOut->addWikiText( "* <span style='font-size:110%;'>[[:$wiki_path|$wiki_title]]</span>" );
--Rrosenfeld 09:05, 4 January 2011 (UTC)
Project still alive?
I hope this project is still active. Many of the threads above mention a new build possibility in May of 2010, but the only build still available is from Feb.
This extension is great, but some missing features are holding me back from deployment. - Gomeztogo 01:06, 18 August 2010 (UTC)
- Apologies to everybody who has been waiting for the new release. I am about to start working on the extension again. --Svemir Brkic 02:06, 11 September 2010 (UTC)
undefined function wfLoadExtensionMessages with install on MediaWiki 1.10.1
I am attempting to install sphinx search extension 0.7 on MediaWiki 1.10.1, but I get the error: undefined function wfLoadExtensionMessages.
How can I resolve this without upgrading mediawiki?
- For older versions of MW you need to use older versions of the extension. They are available at SourceForge --Svemir Brkic 02:03, 11 September 2010 (UTC)
- One thing you could try, if you downloaded 0.7 from SourceForge, is to download the latest build instead. Svemir Brkic 00:23, 13 September 2010 (UTC)
runaway strikethrough in search results
Sometimes in the middle of my search results, in the middle of an excerpt, the text changes to strikethrough and this continued to the end of the page. Somehow this was caused by embedded hyphens in the excerpt. I fixed it by stripping hyphens out of the excerpt before it gets formatted, in SphinxSearch_body.php line 424, with this change (added "\-" to the characters):
$entry = preg_replace('/([\[\]\{\}\*\#\|\!\-]+|==+)/',
' ',
strip_tags($entry, '<span><br>')
I don't understand exactly what happened here, but most likely this is why. My page names are not standard mediawiki usage, but rather lower case words separated by hyphens, like .../wiki/my-page and .../wiki/another-page. I'm also using the semantic mediawiki extensions so there is additional metadata embedded in the wikitext.
The generated html where the strikethrough began looked something like this - the "<span>" tag gets split somehow and we unfortunately end up with an "<s>" tag (after ...issues-in-)
modernities-issues-in-<s><br>pan style='color:red'>culture-and
10 The
which if I put here as wikitext yields
modernities-issues-in-10 The
pan style='color:red'>culture-and
and oh no!! that continues until I close it with an "</s>" tag.
Thanks for a great extension! The results are SO much better than the default search! Initialcapsarestupid 18:18, 13 September 2010 (UTC)
"Table 'mywiki.documents' doesn't exist" issue
Hi,
I've installed Sphinx in Windows 2003, to index a Mediawiki page, under XAMPP server.
When I configure the Sphinx.conf.in file, and run the indexer.exe, I've this:
C:\sphinx\bin>indexer.exe --config c:\sphinx\sphinx.conf.in --all Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff using config file 'c:\sphinx\sphinx.conf.in'... indexing index 'test1'... ERROR: index 'test1': sql_query: Table 'mywiki.documents' doesn't exist (DSN=m ysql://root:***@127.0.0.1:3306/mywiki). total 0 docs, 0 bytes total 0.008 sec, 0 bytes/sec, 0.00 docs/sec indexing index 'test1stemmed'... ERROR: index 'test1stemmed': sql_query: Table 'mywiki.documents' doesn't exist (DSN=mysql://root:***@127.0.0.1:3306/mywiki). total 0 docs, 0 bytes total 0.001 sec, 0 bytes/sec, 0.00 docs/sec distributed index 'dist1' can not be directly indexed; skipping. total 0 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg total 0 writes, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
In phpmyadmin I can see that "documents" table doesn't exists.
What is the table taht I have to configure?
Thanks in advanced.
- Isn't sphinx.conf.in a sample configuration file that comes with sphinx? You sure you updated and are using the sphinx.conf file that came with the extension? - Lucas Billett 11:53, 12 October 2010 (UTC)
Remote host doesn't work
You have a line in SphinxSearch.php:
$wgSphinxSearch_host = 'localhost';
with no checking to see if this is overriding a user-configured variable, i.e. it needs - if (!isset($wgSphinxSearch_host)) - around it. I had to remove this line to get it to work. DimeCadmium 22:04, 19 June 2010 (UTC)
- Please use the latest version of the extension and set these variables after you include SphinxSearch.php Svemir Brkic 17:18, 13 November 2010 (UTC)
Won't return some obvious results
Hello. I've got Sphinx 0.9.9 working in MW 1.16.0 with the latest extension (i think). I just noticed today that some obvious should-be-hits aren't making the results (ie. search for 'floor' gets many instances but misses it in a title and an internal link that I know are there!). The closest similar errors I see look like one that won't return link hits and another where sphinx.conf was interpreting underscores wrong. Any other ideas as to what might cause this? I rebuilt the index twice! - Lucas Billett 21:27, 6 October 2010 (UTC)
- Ok. Weird. I tried setting wgSearchType = 'SphinxMWSearch' as mentioned above and the problem went away. Not sure why that would happen? I like the idea of the search output being packaged in MW. Nice work. However I get an 'undefined index:matches' error on line 189 of SphinxMWSearch.php when it doesn't find an exact match anywhere. I tried surrounding the culprit statement with an if(isset()) and it went away... Not sure what I might have broken though. - Lucas Billett 12:33, 7 October 2010 (UTC)
- Thanks for isset catch Lucas - it has been committed in revision 76659, along with some other tweaks - will post about that in a minute. Svemir Brkic 15:24, 14 November 2010 (UTC)
Bug fix
If you get this error when you search only on a stop word: Undefined index: words near line 446
Here is what I did to fix it:
#if ( is_array( $res["words"] ) ) {
if ( isset($res["words"]) && is_array( $res["words"] ) ) {
2011
Categories under "Search in categories" appear in all caps with recent MediaWiki
I've updated MediaWiki to 1.18alpha (current SVN trunk), and since then, SphinxSearch's category list under "Search in categories" looks ugly, since all categories are in all capital letters. There is no problem with functionality, it's just a cosmetic issue.
The following patch (changing nothing more than getting the category names from the page_title column instead of the cl_sortkey column) corrects the issue for me, but I don't know if it breaks SphinxSearch on older MediaWiki versions, please test:
Index: SphinxSearch_body.php
===================================================================
--- SphinxSearch_body.php (revision 83693)
+++ SphinxSearch_body.php (working copy)
@@ -101,7 +101,7 @@
array( 'ORDER BY' => 'cl_sortkey' )
);
while ( $x = $dbr->fetchObject ( $res ) ) {
- $categories[$x->cl_from] = $x->cl_sortkey;
+ $categories[$x->cl_from] = $x->page_title;
}
if ( $cache_key ) {
# cache query results for a day