One of the reasons we love hosting RoachFest, our global event series for data professionals, is that it brings together a community focused on operational resilience and performance at scale.
For the final RoachFest 2024 gathering in New York City, we were thrilled to welcome Matt Stephenson, who fondly remembers two of our co-founders, Spencer Kimball and Peter Mattis, from their days working together at Square. Today, Matt Stephenson is a Senior Principal Software Engineer at Starburst, the developers of the fastest and most scalable, open, hybrid data lakehouse.
Visit our on-demand hub to experience Matt’s talk and the rest of our RoachFest presentations from London, Menlo Park, and New York.
“How CockroachDB powers Starburst's multi-region cloud lakehouse platform”
In 2021, Starburst embarked on building Starburst Galaxy, a cloud-managed service for their innovative distributed data lakehouse platform. Starburst Galaxy is built on Trino (formerly Presto), an open-source distributed SQL query engine used by hundreds of organizations worldwide for big data analytics at massive scale to run queries close to the data – significantly enhancing performance. (Matt kicked off the talk with an homage to his dog, sporting a matching Trino hat).
Matt shared that it’s particularly important for Starburst to bring data as close as possible to their users so they don’t experience latency when accessing and analyzing their data. Ultimately, Starburst chose CockroachDB because of its high survivability and performance goals. They knew they needed a database that could:
Provide high availability, at scale, with no downtime for the end-user
Perform online schema changes without downtime and transparently scale to larger clusters
Execute fast reads and consistent writes, ensuring a seamless user experience
Support multi-region and multi-cloud deployments
After the Starburst team adopted CockroachDB to build out Starburst Galaxy in 2021, they found that CockroachDB was always available even when an entire region failed, meeting their regional survival goals. It was also easy to support customers across different cloud service providers and in different regions. Additionally, even when Stephenson’s team did need to reach out to CockroachDB support or resolve small issues, there was never a major outage, downtime, or data loss.
Today, Starburst operates across 25 cloud regions on AWS, GCP, and Azure. They deploy CockroachDB in six regions and are able to guarantee roundtrip times of 40-80ms for queries coming from their services plane. Matt told the audience that Starburst had been able to spin up additional CockroachDB regions and take regions down without service interruption – key for operational efficiency and resiliency.
See Matt’s star quality shine on stage! Learn about Starburst’s massive 2TB query history table and how they leverage PrivateLink, Flyway and more to ensure the best customer experience in when you check the talk below:
Watch all of the RoachFest talks from our first annual world tour online!
