« | »

CAP Theorem applied to Organizations

It’s well understood today that as you scale out internet platforms you can’t have perfect data consistency and high availability at the same time.

Like system architects that are now forced to adopt alternative eventually-consistent data stores, big organizations are facing a similar dilemma as the demands for information consistency, speed and scale square off. Increasingly, organizations are taking on distributed, high-decision-throughput formations, where control rests within small functionally-integrated teams – often spanning marketing, engineering and operations.

CAP Theorem
The CAP theorem was presented by Eric Brewer back in 2000 based on his findings while working at Inktomi in the late 90′s.  It’s a key tenet of the big platform architectures that emerged in the last decade, including Amazon’s Dynamo and Google’s MapReduce, and the discrediting of traditional hub-and-spoke architectures with a relational database at the center. CAP comes from:

The theorem states that you can only require two of the three properties in a system. For large systems and for large organizations alike, partitioning is inescapable. Big problems have to be broken down into small problems, large organizations into small teams. But once an organization is partitioned, you have the problem of communication between teams. When having the exact same information and shared goals is the absolute priority for an organization, there are generally two solutions:

1. Centralized decision-making (prone to bottlenecks)

2. Committee-based decision-making (prone to delays)

But when the requirement for strong alignment is relaxed, another approach becomes available:

3. Distributed, parallel decision-making with ongoing conflict reconciliation

While it’s true this option introduces the possibility of organizational disharmony, decisions aren’t stuck in committee or caught in hierarchical bottlenecks. This flexibility makes teams more productive and shortens time to market. Many organizations want to expand, go fast, and be consistent, but given the choice of only two, availability and partition-tolerance is the formula for success. Like the distributed systems that power the internet, these organizations expect decision conflicts, and have processes in place to resolve them as they emerge.

Posted by on August 10, 2011.

Categories: Best Practices, Scalability, Technology

« | »

Recent Posts


About R/GA Techblog

More than thirty years ago agency founders Bob and Richard Greenberg experimented with Academy Award-winning optical printing techniques. Today R/GA programmers continue the legacy of using technology to experiment and inspire. Tech Blog is the most recent account of this evolutionary process.  more →