Rockset Inc. Introduces Native Support for Hybrid Search: Text, Vector, and Metadata Filtering in One Query

Rockset Inc., a real-time analytics database platform, has announced native support for hybrid search incorporating text search, vector search, and metadata filtering into a single query. As artificial intelligence technology evolves, the systems that support data search and retrieval must keep pace to ensure that AI models have access to the data they need to process information. We have already seen a surge in applications that need access to both keyword search and vector search, as well as robust indexing and ranking mechanisms.

With the introduction of the new capabilities, Rockset is pioneering the next generation of search and AI applications. Users can now utilize Rockset hybrid search that combines text, vector, geospatial, and structured data to get the most relevant results. The rapid development of AI models, including OpenAI’s GPT-4, Meta’s Llama-3, Google’s Gemini, and Databricks’ DBRX has ushered in a new era of enhanced AI, where powerful data search and retrieval systems are crucial to their success.

While AI models are getting better at an astounding pace, they lack the ability to retain knowledge or have inherent memory capabilities. To overcome these limitations, developers integrate knowledge into AI models from multiple sources. However, multiple disparate systems mean risk of quality issues, lack of responsiveness, and lower performance. This is where Rockset’s hybrid search comes in. It simplifies the process of integrating various types of data searches for AI applications. Users can do a keyword search, perform metadata filtering, or call on a vector search, all at once through a single query.

AI model developers often have to incorporate ranking algorithms, indexes, and signals to improve relevance. With Rockset’s hybrid search, users can reindex vectors without disruption to live search applications. In addition, Rockset’s cloud-native database eliminates the need to download, install, or configure software packages. This makes it easier to manage installations, access data from anywhere, and scale easily based on demand.

The new release features a multi-tenant design for RAG applications, new ranking algorithms, including BM25 and reciprocal rank fusion (RRF), and a new search design that uses compressed bitmaps and covering indexes for enhanced performance at scale. “All search will soon be hybrid search,” said Venkat Venkataramani, co-founder and CEO of Rockset. “Similarity search has limitations around domain awareness and requires combining vector search results along with text search, geospatial search, and structured search to provide the necessary context. Support for hybrid search requires best-in-class indexing technology designed for fast retrieval. We continuously innovate on our Converged Indexing technology, and we’re thrilled to introduce text search and ranking algorithms for hybrid search.”

Venkat, who was a Datanami Person to Watch for 2022, founded Rockset in 2016 to meet the growing need for real-time analytics solutions capable of handling a variety of data. Prior to starting Rockset, Venkat spent 8 years with Facebook where he worked on building and scaling their online data systems. Last year, Rockset raised $44 million to power search, analytics, and AI applications. The total capital raised by Rockset has reached $105 million. As more organizations look to leverage the efficiency and performance of AI hybrid search, we can expect Venkat and his team at Rockset to be at the forefront of this transformation.