View all

If you’ve ever “watched” a busy GitHub repository, your email inbox has discovered what it feels like to step in front of a firehose. If the project in question has active code reviewers, the problem is often worse by an order of magnitude. Every comment yields another email to all watchers. The CockroachDB repository’s weekly average is at 81 pull requests and 440 notification-generating comments. Most of us who once paid close attention to incoming changes have since lost the ability to do so; these days, monitoring the stream requires a superhuman effort. The mere mortals among us can only pay attention to the pull requests we’ve authored or are tasked with reviewing. What’s surprising is that the watching functionality provided by GitHub is so coarse-grained. The dial apparently only has settings for “0” and “11”.

Spencer Kimball

March 23, 2016

As CockroachDB approaches beta, user documentation has become increasingly important, and one of the meatiest requirements is documentation of our SQL implementation. For inspiration, I researched how other databases have documented SQL. The most effective example I found was SQLite’s grammar diagrams.

As we’ve built CockroachDB, correctness has been our primary concern. But as we’ve drawn closer to our beta launch, we’ve had to start paying significantly more attention to performance. The design of CockroachDB always kept performance and scalability in mind, but when you start measuring performance, there are inevitably surprises. This is the story of the detection, investigation, and fix of just one performance bug.

Peter Mattis

March 11, 2016

The goal of CockroachDB is to “make data easy,” and while it seems like a stretch now, we eventually want CockroachDB to be able to act as the entire state layer for web applications. We are currently addressing the SQL layer, and a full-text search like ElasticSearch is somewhere ahead on the product horizon. Since Cockroach Labs offered a flexible policy for work on experimental projects, I decided to use mine to experiment with implementing the Redis protocol on top of CockroachDB, attempting to answer the question: Could CockroachDB ever replace Redis?

Cgo is a pretty important part of Go: It’s your window to calling anything that isn’t Go (or, more precisely, anything that has C bindings). For CockroachDB, cgo lets us delegate a lot of the heavy lifting at the storage layer to RocksDB, for which no suitable replacement within the Go ecosystem exists, at least to the best of our knowledge. After some iterations, we’ve found that the right way to deal with these external libraries – of which we have quite a few – is to outsource them in Go wrapper packages:

The goal of CockroachDB is to make data easy. If you could take all the energy wasted wrestling with database shortcomings, and invest that time, money and engineering into making your company stronger and your product better, everyone would be better off.

Jessica Edwards

October 30, 2015

<!–– Outdated blog post alert! CockroachDB no longer stores each non-primary-key column in a separate value by default. By default, all data in a table is stored in the same column family. Gory details available here. ––!>

Peter Mattis

September 16, 2015

Editor's Note - April 23, 2021: This article was written in 2015 when CockroachDB was pre-beta. The product has evolved significantly since then. We will be updating this post to reflect the current status of CockroachDB. In the meantime, the transaction section of the Architecture Document provides a more current description of CockroachDB's transaction model.

In CockroachDB, we use the Raft consensus algorithm to ensure that your data remains consistent even when machines fail. In most systems that use Raft, such as etcd and Consul, the entire system is one Raft consensus group. In CockroachDB, however, the data is divided into ranges, each with its own consensus group. This means that each node may be participating in hundreds of thousands of consensus groups. This presents some unique challenges, which we have addressed by introducing a layer on top of Raft that we call MultiRaft.

Ben Darnell

June 12, 2015