Making the right move: Modernizing a monolith with CockroachDB

Cognitive overload = system failure

Prior to using CockroachDB, a majority of Rightmove’s systems were based on Oracle. To achieve high availability and prevent data loss, the Central Platform and Reliability Engineering Team was running an active-active-active datacenter setup and using async replication. 

This design created a lot of operational overhead for their own team, and created complexity for developers when it came to understanding the database architecture and replication (also known as cognitive load). They wanted a single endpoint to be able to write for their applications to reduce this unnecessary complexity. 

At the time, Rightmove was also tied to a two-week release cycle for database changes. This inhibited the velocity and autonomy of their product teams – especially when trying to innovate in new spaces and build greenfield projects. 

The Central Platform and Reliability Engineering Team needed a solution that solved these challenges which meant looking for a highly available, extremely resilient, geographically distributed database that didn’t add complexity for managing replication. They were also interested in application portability and regional resilience for cloud deployments. 

A summary of their requirements included a database that delivered: 

• High availability and high performance 

• Flexibility to evolve with their needs

• Ability to increase innovation, velocity, and autonomy for teams

• Built-in replication to reduce complexity

The Rightmove team ran a number of POCs and in the end found that CockroachDB was best placed to meet their modernization needs and reduce operational overheads for their teams.

Slow and steady migration

As part of their modernization effort over the past few years, Rightmove started to develop microservices and break apart their monolith. They created separate schemas in their Oracle database that have bounded context within the applications. This enables them to migrate their microservices and the associated schema independently (one by one).

When it's time to migrate, they first move the application and all the microservices from Docker Swarm on-premise to Google’s Kubernetes Engine in the cloud which is still pointing to the Oracle endpoint in their data centres. They then stream the data in real-time from Oracle to CockroachDB Cloud which is running on GCP. 

Then they do all the performance testing and all of the quality checks, and then redeploy the application in the cloud with the new CockroachDB endpoint for the pods update. Finally, they start reading and writing from the new database. 

Here’s an overview of what the migration path looks like:

They have around 180 microservices and run the managed version of CockroachDB on Google Cloud and leverage VPC peering. While this setup works well for now, they have plans to run across multiple regions in the future as their business grows. 

Today, CockroachDB serves as the primary relational data store for all of Rightmove’s stateful applications. It supports a number of their data services for specific property industry products, and it’s also the foundation for their brand new commercial real estate proposition.

Developing greenfield apps, faster

Since using CockroachDB, Andrew reports that they’ve been able to “massively reduce complexity and cognitive load for the development teams as they are building new applications.” 

This is due to the fact that they now have a single endpoint to write to, they no longer need to manage replication, and upgrades are automatic and don’t require downtime. They also no longer have to wait 11 days for a database change to go out and they can develop much faster in the greenfield space.  

When it comes to getting started with CockroachDB, Andrew says some things are slightly different depending on what RDBMS you are coming from. When creating an index he says you need to be specific about which columns you might put into that index whereas before you might not have thought about that. 

On the plus side, with CockroachDB they now have the ability to store semi-structured data alongside relational data which Andrew says is a huge advantage that they weren’t originally aware of. He mentioned that this tactic is good for greenfield app development because you can query across the data and iterate really quickly when you're designing a new data model.

What’s next

Rightmove plans to continue porting over their microservices and scaling up the migration process in the near term. They are also working with CockroachDB on a plan to re-architect some of their underlying data structures and other core business entities. 

Rightmove believes that AI presents a wealth of opportunity for the company in both the near term and medium term. They plan to use GenAI to leverage their large volumes of unstructured data which will allow them to build out the platform’s capabilities to deliver more value for customers, enhance the end-user experience, and drive internal efficiency.