View all

Until recently, CockroachDB’s SQL was suffering from a serious, long-standing bug – a schrödinbug, in fact – in its handling of table and column references. This blog post outlines how fuzz testing uncovered the error, how we discovered that our way of using Go was partly to blame, and how we addressed the issue using a form of strong typing.

As CockroachDB has grown over the last 30+ months of development, avenues for communicating between users and developers have proliferated. We started with GitHub, then created a Google Group, then an info@ email, then another Google Group, a Gitter room, and on we went. And while getting in touch with our developers is easier than ever (I suppose we don’t have Snapchat yet…), we hadn’t made a concerted effort to centralize our community’s brainpower – but that’s changing! Today we are launching the CockroachDB Community Forum, a place to ask how-tos, query others about best practices, and receive dev notices like our recent upgrade to Go 1.7.

Jessica Edwards

August 10, 2016

CockroachDB’s JOIN: An Early Implementation When our VP of engineering, Peter Mattis, made the decision in 2015 to support SQL, little did he know that the team would get as far as shipping the first implementation of CockroachDB’s JOIN exactly one year after that. A celebration is in order!

When you write data to CockroachDB (for example, if you insert a row into a table through the SQL client), we take care of replication for you. To do this, we use a consensus protocol – an algorithm which makes sure that your data is safely stored on multiple machines, and that those machines agree on the current state even if some of them are temporarily disconnected. In this post, I will give an overview of common implementation concerns and how we address these concerns in CockroachDB. Then I will abandon these earthly constraints and explore how we could improve consensus algorithms. Specifically, what would it take to make them faster?

Note: this blog post was updated on September 21, 2017. Jails are FreeBSD’s native solution to contain and isolate server processes. They are an alternative to (and predate) Linux cgroups, Solaris zones, and other OS-level process isolation technologies (the technologies that underlie Docker, CoreOS and a few others) .

In our most recent beta, we added a new feature: time-travel queries. These are `SELECT` queries where you can specify a timestamp, and the data returned will be the data as it was at that time. This has various uses including backups, undo, and historical reporting. The SQL:2011 standard describes this feature, and a few SQL databases (Oracle, MSSQL) have implemented it, in addition to various non-SQL DBs (Datomic). I’d like to introduce this feature: what it is, why we built it, and details about how it works for those interested in CockroachDB’s lower layers.

Writing good tests is tricky when the system has a lot of moving parts. When using Go’s testing infrastructure, tests that involve multiple modules can cause dependency cycles which are not allowed by the compiler. In this post we will go over a technique we devised to break these dependency cycles.

Adopting a SQL interface for CockroachDB had an unexpected consequence; it forced us to dabble in language design. Most developers working with SQL have heard rumors of a SQL standard, pages upon pages of norms and requirements for all SQL compliant dialects to respect. Based on its existence, it’s natural to draw the conclusion that SQL is fully specified and straightforward to implement. A developer need only carefully follow each step laid out in the standard until they arrive at a working database. It’s a lot like building a couch from IKEA. However, in the months following our decision to create a SQL layer on top of our distributed consistent key-value store, we’ve come to realize that this is far from the truth.

CockroachDB’s support for SQLAlchemy is currently in beta, but we’re actively developing new features to improve the integration. You can find the documentation here. One of the great things about CockroachDB’s support for SQL is the wide variety of frameworks and tools for working with SQL data. Today, we’ll demonstrate this by building a simple application in Python, using SQLAlchemy and Flask.

Ben Darnell

June 1, 2016