A specialized search engine is a great asset when hunting for a particular thing on the web10/8/2023 ![]() ![]() By replacing a batch-based indexing system with an indexing system based on incremental processing using Percolator, we process the same number of documents per day, while reducing the average age of documents in Google search results by 50%. ![]() An excerpt: We have built Percolator, a system for incrementally processing updates to a large data set, and deployed it to create the Google web search index. The Percolator paper is titled "Large-scale Incremental Processing Using Distributed Transactions and Notifications" ( PDF). Percolator is the database powering Caffeine, which is Google's new system to provide fresher search results by adding new documents and updates to documents to their search index in near real-time. The first paper is a USENIX 2010 paper describing Percolator. Googlers have published two papers recently at academic conferences detailing new specialized databases that are heavily used within Google.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |