When building a large-scale and highly available payments processing application or service, what exactly do you need from your database?
This sounds like a simple question, yet it is far from simple to answer when we’re talking about the data backbone for an enterprise-scale application that may need to handle hundreds thousands of requests per second. One where critical payments functionality needs to work correctly, even if other parts of the system go down, and where every penny of every transaction must always be correct and instantly accounted for. Oh, and the database also needs to be able to scale out smoothly to keep pace with growing workloads.
Not every database can deliver high availability and strong consistency, at scale, with low latency, while also being relatively simple to scale horizontally. In fact, most can’t: traditional SQL databases (MySQL, PostgreSQL, Oracle, Microsoft SQL Server) are reliable, but complex and cumbersome (and also expensive!) to scale.
Many NoSQL databases arose to solve the scaling limitations inherent to traditional RDBMS, but they fail to provide strong consistency and ACID transactions. This is a pretty big “but” for many businesses — and a total deal-breaker for payments processing platforms that simply can’t risk errors, double-spending, or data inaccuracies.
Distributed SQL for powerful payments architectures
Distributed SQL evolved to solve both scalability and consistency in a single database. With relational database architecture that is distributed by design, distributed SQL databases can provide scalability, high availability, geo-distribution, and resilience that traditional SQL databases struggle to match, and the consistency and idempotency that NoSQL databases cannot offer. Distributed SQL is now a mature technology trusted by household-name global enterprise companies for their production workloads, as evidenced by the now-regular appearance of distributed SQL RDBMS solutions in the Gartner Magic Quadrant.
Evaluation criteria for payment processing databases
An enterprise-worthy payments processing database must deliver a never-down system that can deal with payments efficiently and at scale. The following criteria are essential to consider when evaluating different distributed SQL solutions to find the one that best delivers for your particular payments use case.
1. High availability
Payments processing is a mission-critical function that requires continuous operation. Any downtime or disruption in the database can lead to failed transactions, lost revenue, and seriously unhappy customers. A highly available database reduces downtime to the absolute minimum by providing redundancy and failover mechanisms, ensuring that the system remains operational even if one or more components fail.
What to look for: Disasters happen and unplanned downtime is basically inevitable. But a database that allows you to define survival goals for all instances with just a few declarative SQL statements will let your payments processing platform handle availability zone or region loss without impacting performance. Planned downtime, however, is not inevitable. Taking the database offline for database updates and schema changes is still downtime. Seek a database that offers live, online schema changes and rolling upgrades — and the ability to instantly roll back changes, just in case. Your customers will never know you just performed a full version update to your entire database.
2. Strong consistency
In the world of payments, data integrity is paramount. Every transaction must be accurately recorded and stored to maintain financial records and comply with regulatory requirements.
What to look for: A database built so that data is replicated and synchronized across multiple nodes or servers, reducing the risk of data loss or corruption in the event of a failure. Look for a distributed SQL RDBMS that offers strong consistency with serializable isolation, eliminating the risk of errors, double-spending, or data inaccuracies. A database with strong consistency allows you to build an idempotent system that will faithfully maintain the integrity of your payment transactions and account balances.
3. Scalability
As the load on the payments service increases, the database must be able to scale horizontally to handle the increased traffic and maintain consistent performance. Distributed SQL databases are designed to distribute data and workloads across multiple nodes, allowing for seamless scaling and automatic load rebalancing.
What to look for: A database that scales without adding complexity — ie, sharding. You need a database where every node can service both reads and writes, so that you can scale query throughput and database capacity by simply adding more endpoints. Definitely insist on automated load balancing, but you should also look for a database with the built-in ability to detect hotspots and intelligently distribute data to maintain performance.
4. Disaster recovery
You never know when a fire might break out in a data center. Natural disasters, power outages, or other catastrophic events can potentially disrupt the entire system, including the database.
What to look for: A high availability database should include features like geo-replication and automated failover to remote sites, ensuring that data and operations can be recovered quickly in the event of a disaster. It should also have robust backup and restore capabilities that keep RPO and RTO to the absolute minimum: zero.
The right database should also be cloud agnostic and capable of multi-cloud deployments. While it’s possible to deploy almost any database across multiple clouds, the effort and resources required for setting up and operating such a system can be staggering. A truly multi-cloud database can operate single or multiple clusters that span multiple cloud providers while also allowing your application to treat it like a single-instance database operating on a single cloud.
5. Data geo-partitioning
The financial industry is heavily regulated, and payments processing systems must comply with strict requirements around data domiciling. Government regulations like GDPR dictate how the data of a nation’s residents must be collected, cleaned, processed and stored, and set strict rules about transfers of that data to other nations. A payments processing platform needs a database that makes it simple to geo-partition data to stay where it belongs.
What to look for: A database where the ability to geo-locate data is an inherent capability of its architecture. logging, monitoring, and auditing capabilities, helping organizations to meet compliance standards and maintain detailed records of all transactions.
Conclusion
The payment processing industry has become a critical component of the global economy. Just look at the numbers: the global payment processing market, valued at $63.5 billion in 2020, is projected to reach $140.3 billion by 2026 – a CAGR of 13.3% over the forecast period, according to the payment processing industry report by Allied Market Research.
Many of the world’s businesses — and citizens — now depend on fast, accurate, and always-available payment processing services underpinning, well, literally everything we need to do as part of modern life. Deliver the secure and always-available payment experiences that today's customers expect by choosing a highly-available distributed SQL database. And not just any distributed SQL database, but one that delivers guaranteed ACID transactions and the strongest level of isolation while being built on a horizontally scalable, resilient foundation designed to survive any type of failure. A database like no other. A database like CockroachDB.
Learn more about how CockroachDB delivers secure and always-available payment experiences. See how our customer Shipt, a grocery ecommerce company owned by Target, maintains a crucial suite of payment services in our guide, "From Resilience to Scalability: 12 Mission-Critical CockroachDB Use Cases."
Want a webinar? Watch "How to build a scalable payment system" payment expert Andy Kimball (ex-Square), Engineering Operations Lead at Cockroach Labs, joins Cockroach Labs Sales Engineer Jim Hatcher to discuss payment system requirements.
