Vertical search
This article possibly contains original research. (September 2012) |
A vertical search engine is distinct from a general
In contrast to general web search engines, which attempt to index large portions of the World Wide Web using a web crawler, vertical search engines typically use a focused crawler which attempts to index only relevant web pages to a pre-defined topic or set of topics. Some vertical search sites focus on individual verticals, while other sites include multiple vertical searches within one search engine.
Benefits
Vertical search offers several potential benefits over general search engines:
- Greater precision due to limited scope,
- Leverage domain knowledge including ontologies,
- Support of specific unique user tasks.
Vertical search can be viewed as similar to
Domain-specific search
Domain-specific verticals focus on a specific topic. John Battelle describes this in his book The Search (2005):
Domain-specific search solutions focus on one area of knowledge, creating customized search experiences, that because of the domain's limited corpus and clear relationships between concepts, provide extremely relevant results for searchers.[3]
Any general search engine would be indexing all the pages and searches in a breadth-first manner to collect documents. The spidering in domain-specific search engines more efficiently searches a small subset of documents by focusing on a particular set. Spidering accomplished with a reinforcement-learning framework has been found to be three times more efficient than breadth-first search.[4]
DARPA's Memex program
In early 2014, the Defense Advanced Research Projects Agency (
In April 2015, it was announced parts of Memex would be open sourced.[9] Modules were available for download.[8]
References
- ^ Rao, Leena (5 March 2013). "Data-Driven Comparison Shopping Platform FindTheBest Raises $11M From New World, Kleiner Perkins And Others". TechCrunch. Archived from the original on 1 June 2013. Retrieved 27 May 2013.
- ^ HO, VICTORIA (11 May 2013). "Asian Price Comparison Site Save 22 Gets Angel Round Of "Mid Six Figures"". Archived from the original on 7 June 2013. Retrieved 27 May 2013.
- ^ Battelle, John (2005). The Search: How Google and its Rivals Rewrote the Rules of Business and Transformed Our Culture. New York: Portfolio.
- ^
McCallum, Andrew (1999). "A Machine Learning Approach to Building Domain-Specific Search Engines". IJCAI. 99: 662–667. CiteSeerX 10.1.1.88.3818.
- ^ a b "Memex Aims to Create a New Paradigm for Domain-Specific Search" (Press release). DARPA. February 9, 2014. Archived from the original on February 11, 2015. Retrieved February 11, 2015.
- ^ "Memex (Domain-Specific Search)". www.darpa.mil. Archived from the original on 2016-09-16. Retrieved 2016-09-21.
- ^ Kim Zetter (February 2, 2015). "Darpa Is Developing a Search Engine for the Dark Web". Wired. Archived from the original on June 29, 2023. Retrieved November 19, 2020.
- ^ a b "Memex (Domain-Specific Search)". DARPA. Archived from the original on June 10, 2015. Retrieved April 20, 2015.
- ^ Forbes (April 17, 2015). "Watch Out Google, DARPA Just Open Sourced All This Swish 'Dark Web' Search Tech". Forbes. Archived from the original on April 20, 2015. Retrieved April 20, 2015.