View all

CockroachDB recently gained support for Unicode collation, a standard for ordering strings in the different ways that our users around the world expect. This post describes the motivation for Unicode collation as well as the implementation challenges in providing collated strings as a first-class type. Collated strings are documented here. Note that CockroachDB doesn’t support every use of collation that PostgreSQL does, due in part to implementation deficiencies that we plan to address and in part because we believe that the bugs and performance problems caused by implicit type conversions outweigh their convenience. We’ve left the door open for full support, however.

Calling all database and infrastructure experts: get ready to nerd out with the best CockroachDB devs in your city! A year after our Beta release and a shortly before CockroachDB 1.0 becomes available for download, we are excited to announce the newly minted CockroachDB User Groups. We are kicking off our user groups April 4th and 5th, with inaugural events in New York City and San Francisco respectively.

With the release of CockroachDB beta-20170223, we’d like to announce a new arbitrary-precision decimal package for Go: apd. This package replaces the underlying implementation of the DECIMAL type in CockroachDB and is available for anyone to fork and use. Here we will describe why we made apd, some of its features, and how it improved CockroachDB.

I recently came across an old piece on The Atlantic on design research. Author and educator Jon Freach wrote, “Design can exist without ‘the research.’ But if we don't study the world, we don't always know how or what to create.” His words resonated with me. Designers are innate problem solvers. Without “the research,” we wouldn’t know what problems to solve and for whom we create solutions. One may argue that people generally don’t know what they want, and it’s up to us creating something new to spark desire. Yet, that creation process isn’t sheer magic being pulled out of thin air. The creation process usually involves painstaking investigation of the world and deep inquiry on how we can make it better. In other words, the shiny new wonderful thing that everyone wants is just a reincarnation of a similar idea but thoroughly interrogated and researched. Smartphones existed before the first iPhone was introduced. Group chat was invented long before Slack was a company. Similarly, databases are not a new topic. When I started several months ago, I wanted to do “the research” to figure out how this new database with a funny name could spark desire. My first quest was to investigate pain points people have with their current database solutions, and how they first hear about, test and adopt alternative options. Having not been properly trained as a researcher, I hacked together a study with the help of GV’s extensive resources and recruited and interviewed a few developers and CTOs. Yet, the data I collected was scattered, with no clear pattern across the demographics. Given how crucial the questions were to understanding of our audience, I decided to reevaluate the research process, and redo the study with some tweaks. Here’s what I’ve learned from doing the same study twice, and why the second study was a lot more effective.

Almost a year ago, we wrote about our use of Jepsen in testing CockroachDB. As we prepare for CockroachDB 1.0, we wanted to get independent verification of our findings, so last fall we hired Kyle Kingsbury, the author of Jepsen, to review our tests and add more of his own. Last week, Kyle published his results. Kyle’s testing found two new bugs: one in CockroachDB’s timestamp cache which could allow inconsistencies whenever two transactions are assigned the same timestamp, and one in which certain transactions could be applied twice due to internal retries. Both of these issues were fixed (the first in September and the second in October), and the expanded Jepsen test suite is now a part of our nightly test runs. Now we can say with more confidence that CockroachDB meets its guarantee of serializability (the highest ANSI SQL isolation level) for all SQL transactions.

Six months ago, we reported our first implementation of SQL joins in CockroachDB. At that point in development, we merely provided a functional proof of concept that was severely limited performance-wise. This is changing, gradually. We are preparing to launch CockroachDB 1.0 later this spring, and SQL joins in CockroachDB 1.0 will be usable, if underpowered.

One would think that designing for creative consumers through a massively popular brand would be a dream come true for any young designer. After all, what more could you want than a great company, compelling product, and an engaged team? A bigger challenge, it turns out.

Building a complex system requires writing and debugging many tests. Developer tests that can be run without any infrastructure or configuration are crucial for allowing fast-paced development while avoiding regressions. Even for a distributed system like CockroachDB, tests that run on a single anode (e.g. as part of go test) can cover many aspects of the system: one technique we use is that of creating a virtual test cluster with multiple CockroachDB nodes, all running within the same process.

CockroachDB is a project I’ve been keeping an eye on for a while. It’s a an open-source, Apache 2 licensed, database system(Github link). At it’s core its a key-value store that scales horizontally. But what makes it really interesting for us though, is that 1) it supports SQL by using the Postgres wire protocol and 2) has full ACID semantics and distributed transactions. If you’re interested in how they achieve this, make sure to read the technical posts at the CockroachLabs blog (I admit, sometimes it’s not for the faint-of-heart ;-)). Do note that it is still a distributed system and thus follows the CAP theorem, more specifically it is a CP system.