Rockset integrates real-time analytics platform with relational databases

Rockset today announced it has integrated its analytics database with both MySQL and PostgreSQL relational databases to enable organizations to run queries against structured data in real time.

Rather than having to shift data into a cloud data warehouse to run analytics, organizations can now offload analytics processing to a Rockset database running on the same platform, Rockset CEO Venkat Venkatramani told VentureBeat. The Rockset platform is based on Facebook-developed RocksDB, an open source log structured database engine based on a key/value store that has been extended to support SQL queries.

The approach enables organizations to offload queries to an indexing engine that can process sub-second queries while transactions continue to be processed using a relational database, Venkatramani added. The issue many organizations face today is that they already have extensive investments in open source relational databases. Neither MySQL nor PostreSQL are designed to process analytics at scale, which is one reason so many organizations have either adopted a NoSQL database or a data lake in the cloud. Replacing those databases with a proprietary relational database that can also process analytics in real time would be cost-prohibitive for many.

A fresh approach

Rockset is making a case for an alternative approach based on a Converged Index that can be employed to analyze structured relational data, as well as semi-structured, geographical, and time-series data in real time. Complex analytical queries can be scaled to include JOINS with other databases, data lakes, or event streams. All fields are entered into a converged index that includes an inverted index, a columnar index, and a row index.

In addition to integrations with open source relational databases, the company also provides connectors to MongoDB, DynamoDB, Kafka, Kinesis, Amazon Web Services (AWS), and Google Cloud Platform, among others.

As organizations collect data in real time, they increasingly also need to analyze it in real time, Venkatramani said. “Batch-based workloads are becoming real-time workloads,” he added.

Moving data into a data lake using a batch-oriented process only provides a means to process a larger amount of historical data, Venkatramani said. IT organizations may still have a need for a data lake, but real-time analytics are going to be at the heart of most digital business processes, Venkatramani noted.

Rockset earlier this year published the results of a Star Schema Benchmark test showing millisecond-latency query performance against the Star Schema Benchmark (SSB). The company claims it’s the only vendor to publish benchmarks showing it can execute queries up to 9.4 times faster than rivals while simultaneously ingesting 1 billion events a day with one second of data latency.

The company last fall raised an additional $40 million to grow its workforce and accelerate product development and research while bolstering its go-to-market efforts.

Future of real-time platforms

It’s not clear to what degree batch-oriented processes that have the dominated IT architecture for decades will give way to real-time platforms. Historically, the data organizations have applied analytics to is usually several hours to a day old because the underlying database has typically been updated overnight using a batch-oriented process. Today organizations want to be able to continuously apply analytics to, for example, clickstream data from social media feeds — in real time, as it’s being processed. Other use cases include supply chain logistics and delivery tracking systems, gaming leaderboards, fraud detection systems, health and fitness trackers, and ecommerce applications.

Of course, the days when organizations standardized on one database platform are now long over. The challenge now is weaving a polyglot set of databases together in a way that allows an organization to take advantage of the capabilities of multiple platforms optimized for varying classes of workloads.