Copy-on-Write for Read-Mostly Data in C++
Table of Contents
The Problem with Locking Reads
When reads dominate and writes are rare, a mutex can become a scalability bottleneck. Every reader still needs to acquire the lock, even when the operation only needs to inspect the data. This becomes particularly problematic when the read operation is expensive. For example, suppose we have a collection of bank accounts and want to generate reports by performing computations over all of them.
A simple implementation might protect the entire container with a mutex:
1struct Account {
2 std::string name;
3 int balance;
4};
5
6class BankAccounts {
7public:
8 int sum() const {
9 auto lock = std::lock_guard{m_mutex};
10
11 return std::accumulate(
12 m_accounts.cbegin(),
13 m_accounts.cend(),
14 0,
15 [](int res, const Account& acc) {
16 return res + acc.balance;
17 });
18 }
19
20 void create_new(std::string name, int balance) {
21 auto lock = std::lock_guard{m_mutex};
22
23 m_accounts.emplace_back(std::move(name), balance);
24 }
25
26private:
27 mutable std::mutex m_mutex;
28 std::vector<Account> m_accounts;
29};The implementation is thread-safe, but there is an important limitation. If several threads call sum() at the same time, they cannot perform the calculation concurrently because each thread must first acquire the same mutex.
The mutex protects the vector, but it also unnecessarily serializes operations that are only reading the data.
Returning a Copy of the Data
One way to avoid holding the mutex while performing expensive calculations is to copy the data while holding the lock, then perform the calculation on the copy.
1class BankAccounts {
2public:
3 std::vector<Account> get() const {
4 auto lock = std::lock_guard{m_mutex};
5 return m_accounts;
6 }
7
8private:
9 mutable std::mutex m_mutex;
10 std::vector<Account> m_accounts;
11};The caller can now release the lock before performing its computation. This means multiple reports can run concurrently. However, we have simply moved the cost somewhere else. Every report now needs to copy the entire vector. If the container is large and writes are rare, repeatedly copying the same data can waste CPU time and memory bandwidth. Ideally, we would like readers to share the same data without copying it, while still allowing writers to update the data safely.
Copy-on-Write with std::atomic and std::shared_ptr
This is where a copy-on-write approach becomes useful. The basic idea is simple: readers never modify the data. Instead, they obtain a shared pointer to an immutable snapshot of the container. When a writer needs to modify the data, it creates a new copy, applies the modification to that copy, and then atomically publishes the new snapshot.
We can represent this using an atomic shared pointer:
1class BankAccounts {
2private:
3 std::atomic<
4 std::shared_ptr<const std::vector<Account>>
5 > m_accounts;
6};There are two important details here.
First, the std::shared_ptr points to a const std::vector<Account>. This prevents readers from modifying the shared snapshot. Second, the std::shared_ptr itself is stored inside an std::atomic. This allows multiple threads to safely publish and acquire snapshots without protecting the pointer with a mutex.
Lock-Free Reads
Once the data is immutable, the read path becomes very simple. A reader only needs to atomically load the current shared pointer.
1class BankAccounts {
2public:
3 int sum() const {
4 auto values =
5 m_accounts.load(std::memory_order_acquire);
6
7 return std::accumulate(
8 values->cbegin(),
9 values->cend(),
10 0,
11 [](int res, const Account& acc) {
12 return res + acc.balance;
13 });
14 }
15
16private:
17 std::atomic<
18 std::shared_ptr<const std::vector<Account>>
19 > m_accounts;
20};Notice that there is no mutex around the actual calculation. Suppose ten threads call sum() simultaneously. Each thread performs an atomic load and receives its own copy of the std::shared_ptr. All ten shared pointers can point to the same underlying vector, so the threads can safely iterate over it concurrently. The important property is that the vector is immutable. Since nobody can modify it while readers are using it, there is no need to synchronize the actual read operation.
Snapshot Lifetime
The use of std::shared_ptr also solves an important lifetime problem. Imagine that a reader loads the current snapshot. Shortly afterwards, a writer publishes a new snapshot. The reader may still be iterating over the old vector. We cannot destroy that vector immediately just because it is no longer the current snapshot. Fortunately, std::shared_ptr handles this naturally. The reader owns its own copy of the shared pointer, so the old vector remains alive until the reader releases it. This gives each reader a consistent snapshot of the data for the lifetime of its shared_ptr.
Handling Writes
Reads are straightforward because they never modify the shared snapshot. Writes are more interesting. A writer cannot simply obtain the current vector and modify it. Doing so would violate the immutability guarantee that makes concurrent reads safe. Instead, every update follows three steps:
First, load the current snapshot. Second, make a deep copy and apply the modification to the copy. Finally, atomically replace the current snapshot with the new one.
1class BankAccounts {
2public:
3 void create_new(std::string name, int balance) {
4 auto old =
5 m_accounts.load(std::memory_order_acquire);
6
7 bool is_updated = false;
8
9 do {
10 // Create a new snapshot.
11 auto new_values =
12 std::make_shared<std::vector<Account>>(*old);
13
14 // Modify the new snapshot.
15 new_values->emplace_back(
16 std::move(name),
17 balance);
18
19 // Publish it only if the snapshot is still current.
20 is_updated =
21 m_accounts.compare_exchange_strong(
22 old,
23 new_values,
24 std::memory_order_release);
25
26 } while (!is_updated);
27 }
28
29private:
30 std::atomic<
31 std::shared_ptr<const std::vector<Account>>
32 > m_accounts;
33};Why compare_exchange_strong Is Necessary
The atomic update is more subtle than simply storing the new pointer. Consider two writers, A and B, trying to create new accounts at the same time.
Both writers might initially load the same snapshot:
Initial snapshot: [Alice, Bob]
Writer A creates: [Alice, Bob, Charlie]
Writer B creates: [Alice, Bob, David]
If both writers blindly store their new snapshots, one update could overwrite the other. The final result might contain Charlie but not David, or vice versa. compare_exchange_strong prevents this. Conceptually, it says: "Replace the current pointer with my new pointer only if the current pointer is still the same pointer I originally loaded."
If another writer has already published a new snapshot, the comparison fails. The old pointer is updated with the latest value, and the loop retries the operation using that newer snapshot. This retry is essential because every successful update must be based on the latest version of the data.
Putting It Together
Combining the read and write paths gives us a container where readers operate on immutable snapshots, while writers publish new snapshots atomically.
1class BankAccounts {
2public:
3 int sum() const {
4 auto values =
5 m_accounts.load(std::memory_order_acquire);
6
7 return std::accumulate(
8 values->cbegin(),
9 values->cend(),
10 0,
11 [](int res, const Account& acc) {
12 return res + acc.balance;
13 });
14 }
15
16 void create_new(std::string name, int balance) {
17 auto old =
18 m_accounts.load(std::memory_order_acquire);
19
20 while (true) {
21 auto new_values =
22 std::make_shared<std::vector<Account>>(*old);
23
24 new_values->emplace_back(
25 std::move(name),
26 balance);
27
28 if (m_accounts.compare_exchange_strong(
29 old,
30 new_values,
31 std::memory_order_release)) {
32 break;
33 }
34 }
35 }
36
37private:
38 std::atomic<
39 std::shared_ptr<const std::vector<Account>>
40 > m_accounts;
41};The resulting design has an important asymmetry: reads are cheap, while writes are expensive. A reader only needs to atomically acquire a shared pointer and can then traverse the immutable snapshot without locking. A writer, however, needs to copy the entire vector before publishing its update. That trade-off is exactly what makes copy-on-write attractive for read-mostly workloads.
When Not to Use Copy-on-Write
Copy-on-write is not a general replacement for mutexes. Its performance characteristics only make sense for certain workloads.
Frequent Writes
Every write requires copying the entire container. If updates are frequent, the cost of repeatedly allocating and copying large containers can easily outweigh the benefits of cheap reads.
Large Containers
The larger the snapshot, the more expensive each update becomes. A single small modification can still require copying the entire data structure.
Expensive-to-Copy Objects
Copy-on-write becomes even less attractive when the objects stored inside the container are expensive to copy. If a write only changes a small part of a large object, copying everything may be wasteful.
The Right Workload
The sweet spot is a workload with many concurrent readers and relatively few writers, where readers benefit significantly from avoiding locks and writers can tolerate the cost of creating a new snapshot.
In other words, copy-on-write deliberately shifts work from the read path to the write path. If your application is read-heavy and write-light, that can be a very useful trade-off.
Summary
The main idea behind copy-on-write is to make shared data immutable for readers. Instead of modifying the current container in place, writers create a new snapshot and atomically publish it.
std::shared_ptr keeps old snapshots alive while readers are still using them, while std::atomic<std::shared_ptr<...>> allows threads to safely acquire and publish snapshots.
The result is a design where concurrent reads can proceed without taking a mutex, at the cost of making writes more expensive.