Sphinx (search engine)
Developer(s) | Andrew Aksyonoff |
---|---|
Initial release | 2001 |
Stable release | 3.5.1[1]
/ 3 February 2023 |
Written in | GPLv2 until version 2 and commercial; proprietary since version 3 |
Website | sphinxsearch |
Sphinx is a
Overview
Sphinx can be used either as a stand-alone server or as a
SphinxAPI
If Sphinx is run as a stand-alone server, it is possible to use SphinxAPI to connect an application to it. Official implementations of the API are available for PHP, Java, Perl, Ruby and Python languages. Unofficial implementations for other languages, as well as various third party[3] plugins and modules are also available. Other data sources can be indexed via pipe in a custom XML format.[4]
SphinxQL
The Sphinx search daemon supports the MySQL binary network protocol and can be accessed with the regular MySQL API and/or clients. Sphinx supports a subset of SQL known as SphinxQL. It supports standard querying of all index types with SELECT, modifying RealTime indexes with INSERT, REPLACE, and DELETE, and more.
SphinxSE
Sphinx can also provide a special storage engine for MariaDB and MySQL databases. This allows those MySQL, MariaDB to communicate with Sphinx's searchd
to run queries and obtain results. Sphinx indices are treated like regular SQL tables. The SphinxSE storage engine is shipped with MariaDB.
Full-text fields and indexing
Sphinx is configured to examine a data set via its Indexer. The Indexer process creates a full-text index (a special
Attributes
Attributes are additional values associated with each document that can be used to perform additional filtering and sorting during search. Attributes are named. Attribute names are case insensitive. Attributes are not full-text indexed; they are stored in the index as is. Currently supported attribute types are:
- unsigned integers(1-bit to 32-bit wide);
- timestamps;
- floating point values (32-bit, IEEE 754 single precision);
- string ordinals (specially computed integers);
- strings
(since 1.10-beta);
- MVA, multi-value attributes (variable-length lists of 32-bit unsigned integers).
JSON attributes in Sphinx
Sphinx, like classic SQL
License
Up until version 3, Sphinx is
- GNU General Public License version 2or
- proprietary licensing is available for use-cases which are not within the terms of the GNU GPLv2.
Since version 3, Sphinx has become proprietary, with a promise to release its source code in the future[7]
Sphinx use examples
- Craigslist.org[8]
- Recruitment.aleph-graymatter.com[9]
- Tradebit.com[10]
- vBulletin.com[11]
- MediaWiki extension[12]
- Boardreader.com[13]
- OMBE.com[14]
- Limundo.com[14]
Feature list
- Batch and incremental (soft real-time) full-text indexing.
- Support for non-text attributes (sets, JSON).
- Direct indexing of SQL databases. Native support for ODBCconnectivity.
- XML document indexing support.
- Distributed searching support out-of-the-box.
- Integration via access APIs.
- SQL-like syntax support via MySQL protocol (since 0.9.9)
- Full-text searching syntax.
- Database-like result set processing.
- Relevance ranking utilizing additional factors besides standard BM25.
- Text processing support for tokenizingexceptions, and "blended characters" (dual-indexing as both a real character and a word separator).
- Supports UDF (since 2.0.1).
Performance and scalability
- Indexing speed of up to 10-15 MB/sec per core and HDD.
- Searching speed of over 500 queries/sec against 1,000,000 document/1.2 GB collection using a 2-core desktop system with 2 GB of RAM.[15]
- The biggest known installation using Sphinx, Boardreader.com, indexes 16 billion documents.[16]
- The busiest known installation, Craigslist, serves over 300,000,000 queries/day[16] and more than 50 billion page views/month.[17]
Fork
In 2017, key members of the original Sphinx team formed a fork of the project called Manticore.[18][19] The Manticore team has set itself the following goal: to deliver fast, stable and powerful free software for full text search. Manticore team keep it's fork open source, releasing it under GPLv2 license[20] as opposed to Original Sphinx search, which closing the source from the third version.
See also
References
- ^ "Feb 3, 2023. Sphinx 3.5.1 released". Retrieved 15 June 2023.
- ^ "AskMonty: About SphinxSE". kb.askmonty.org. Monty Program AB. Retrieved 16 August 2013.
- ^ "Sphinx Wiki: Third Party Tools". sphinxsearch.com. Sphinx Search Wiki. Retrieved 16 August 2013.
- ^ "xmlpipe2". sphinxsearch.com. Sphinx Search Documentation. Retrieved 16 August 2013.
- ^ "JSON Attributes in Sphinx 2.1.1". sphinxsearch.com. Sphinx Search Blog. 7 February 2013. Retrieved 16 August 2013.
- ^ "Full JSON Support in Trunk". sphinxsearch.com. Sphinx Search Blog. 8 August 2013. Retrieved 16 August 2013.
- ^ "Sphinx | Open Source Search Server".
- ^ "Sphinx at Craigslist". craigslist.org. Craigslist. Retrieved 17 August 2013.
- ^ "GM Recruitment". aleph-networks.com. Aleph-networks. Retrieved 1 October 2012.
- ^ "Lightning Fast PHP Site Search". tradebit.com. Tradebit. Retrieved 17 August 2013.
- ^ "Sphinx Search beta for Vbulletin 4.0". vbulletin.com. Vbulletin. Retrieved 17 August 2013.
- ^ "Sphinx Search Extension for MediaWiki". mediawiki.org. MediaWiki: Svemir Brkic, Paul Grinberg. Retrieved 17 August 2013.
- ^ "Powered by Sphinx Search: Boardreader". sphinxsearch.com. Sphinx Search. Retrieved 17 August 2013.
- ^ a b "Powered by Sphinx". sphinxsearch.com/.
- ^ "About Sphinx". sphinxsearch.com. Sphinx Search. Retrieved 16 August 2013.
- ^ a b "Powered by Sphinx". sphinxsearch.com. Sphinx Search. Retrieved 10 May 2015.
- ^ "Craigslist: Factsheet". craigslist.org. Craigslist. Archived from the original on 5 August 2012. Retrieved 16 August 2013.
- ^ "About Manticore Search". manticoresearch.com. Retrieved 24 April 2023.
- ^ "Manticore Search — форк Sphinx: отчёт за 3 года". Хабр (in Russian). Retrieved 24 April 2023.
- ^ "Сравниваем поисковики для сайта: Manticore и Sphinx". Журнал (in Russian). Retrieved 24 April 2023.
Further reading
- Aksyonoff, Andrew (2011). Introduction to Search with Sphinx: From installation to relevance tuning. O'Reilly Media. ISBN 978-0-596-80955-3.
- Ali, Abbas (2011). Sphinx Search Beginner's Guide. Birmingham, England: Packt Publishing. ISBN 978-1-84951-254-1.
- No more open-source (2017) https://sphinxsearch.com/blog/2017/07/24/sphinx-2017/
External links
- Official website
- SphinxSE in MariaDB KnowledgeBase
- https://manticoresearch.com/ - Manticore opensource fork site.