Engineering Note
Kafka Active-Active Is About Ownership, Not Kafka
This post explains what Kafka can and cannot solve in Active-Active DR architectures, and why ownership, Outbox, Inbox, and failover fencing must be handled at the application level.
When teams discuss an Active-Active DR architecture with Kafka, the conversation often starts with Kafka replication.
Run Kafka in both regions. Replicate topics. Let consumers in each region read events. At first glance, that looks like Active-Active.
But the hard problem is outside Kafka.
Can the same data be modified in both regions at the same time? Which region owns the right to write? How does that ownership move during a failure? How do we stop an old region from writing after it comes back late?
If those questions are not answered, Kafka may deliver events reliably while the data model still becomes inconsistent.
The core idea is simple.
Kafka is the event delivery layer between regions.
Active-Active consistency is decided by the application's ownership model.
The Trap in the Term Active-Active
Active-Active can mean different things.
Sometimes it means both regions serve user traffic. Users connect to a nearby region, and reads are handled by the local database. This is a realistic and useful goal.
Sometimes it means both regions can write the same data at the same time. That is much harder. It requires conflict resolution, versioning, merge rules, compensation, and sometimes user or operator intervention.
So the term needs to be narrowed before designing a Kafka-based Active-Active system.
Both regions are active.
But one piece of data has only one write owner at a given time.
That is the baseline model in this post.
The Base Model: Local Reads, Owned Writes
The most practical baseline is the Home Region model.
Each piece of data, or more precisely each aggregate or tenant, has a write-owning region. Reads can be served from local databases in both regions, but writes for that data go only to its Home Region.
The flow looks like this.
1. Data owned by Region A is written only in Region A.
2. The application commits the business DB change and Outbox record in one transaction.
3. An Outbox relay publishes the event to Kafka.
4. A separate cross-cluster replication layer copies the event to Kafka in the other region.
5. The receiving region commits the Inbox record and business DB change in one transaction.
6. Reads are served from each region's local DB.
Cross-cluster replication is a different layer from a Kafka broker’s replication factor. Apache Kafka can build directional flows such as A -> B and B -> A with MirrorMaker 2. A managed Kafka service’s cluster linking or another replication product can fill the same role.
This does not provide synchronous consistency. The remote database catches up by applying asynchronous events, so stale reads can happen for a short time.
That means “always read locally” should not be treated as an absolute rule. Most reads can use the local DB, but freshness-sensitive operations may need to route to the Home Region or evaluate replication lag and ownership context before responding.
Kafka Is Not the Only Answer
This architecture does not strictly require Kafka. What it needs is an asynchronous messaging layer that carries change events from an Outbox to the other region and lets the receiving side process them at its own pace. A traditional message broker, a managed Pub/Sub service, or another distributed log platform can fill that role depending on the requirements.
Kafka is often called a message queue, but its model is closer to a retained distributed event log than to a traditional queue. A record does not disappear when one consumer reads it. It remains for the configured retention period, and each consumer group manages its own offset. Those properties fit cross-region change-event delivery well.
The following table compares the typical tendencies of the main options. Product capabilities overlap, so this is a design starting point rather than an absolute classification.
| Option | Good fit | Costs and constraints to verify |
|---|---|---|
| Kafka-style distributed log | Events must be retained and replayed by multiple independent consumer groups; key-scoped ordering and a broad CDC and connector ecosystem matter | Cluster and storage operations, partition design, asynchronous replication lag, duplicates, loop prevention, and failover procedures |
| Traditional message broker | Complex routing, work queues, per-message acknowledgement, retry, and dead-letter handling are central | Long retention, large replay, and repeated use of the same event by many independent consumers may require additional design |
| Managed Pub/Sub | The team wants to minimize broker operations and establish cross-region delivery quickly | Retention, ordering and replay limits, regional failure semantics, cost, and cloud-provider coupling |
Kafka’s main advantage here is that multiple consumer groups can read the same change event with independent offsets and replay history after an incident or when rebuilding a projection. Using aggregate_id as the key can preserve order for that aggregate within a partition. Its Outbox, CDC, connector, and stream-processing ecosystem is also practical.
The tradeoff is that Kafka is not a work queue where delivery means the business task is complete. It does not provide global ordering. Cross-region replication is usually asynchronous, and the application and operating model must tolerate duplicate delivery and prevent replication loops. If the main need is complex routing or short-lived command processing, a traditional broker may be simpler. If minimizing operations matters most, a managed Pub/Sub service may be a better choice.
The useful question is therefore not “Does Active-Active require Kafka?” but something closer to this:
How long must events be retained?
Is replay required after a failure?
How many independent consumers need the same change event?
What unit requires ordering?
How much replication lag and duplication can the system tolerate?
Can the team operate the messaging layer itself?
Regardless of the product, write ownership, fencing, idempotency, and failover policy remain application concerns. This post focuses on Kafka not because it is the only answer, but because retained and replayable events make it a strong option for event-driven Active-Active systems.
What Kafka Solves and What It Does Not
Kafka is good at a clear set of jobs.
It stores and delivers events. If a consumer is temporarily slow, it can read again later. Multiple consumer groups can process the same event stream independently. With the right partition key, event order can be preserved within an aggregate.
Kafka’s role is roughly this:
event storage
event delivery
cross-region event replication through a separate replication layer
event reprocessing
loose coupling between services
independent processing by consumer group
But Kafka does not automatically solve these questions:
which region owns write authority
how concurrent writes to the same data are prevented
how ownership moves during failover
how duplicate events become harmless
how much stale read is acceptable
how data is reconciled during failback
Those are application architecture responsibilities, not Kafka configuration settings.
Outbox and Inbox Are Cross-Region Contracts
For cross-region event delivery, Outbox and Inbox are close to a baseline contract.
The producing service should combine the business state change and the intent to publish an event in the same database transaction. That reduces the case where the DB commit succeeds but the event disappears.
The receiving side needs its own protection. Kafka replication, relay retries, and consumer reprocessing can all produce duplicate handling. The receiving region needs an Inbox table or idempotent handler.
The Inbox record and business DB change must also be part of the same local database transaction. If only the Inbox record commits, a failure before the business update removes the basis for retry. If only the business update commits, the same change may run again. A common baseline is a unique constraint on event_id, an atomic commit of the Inbox insert and business state change, and a Kafka offset advance only after that transaction succeeds. Kafka does not automatically extend exactly-once semantics to an external system; the destination system must cooperate in the atomicity model.
An event envelope should carry enough context to operate the system:
{
"event_id": "01JZ7R3W3K3DZ6X7F4Y5Z6A9TQ",
"aggregate_type": "Order",
"aggregate_id": "ORD-1001",
"event_type": "OrderApproved",
"home_region": "region-a",
"source_region": "region-a",
"owner_epoch": 12,
"version": 42,
"occurred_at": "2026-07-11T09:30:00Z"
}
event_id supports deduplication. aggregate_id can be a strong Kafka key candidate. home_region and source_region provide provenance, but the fields alone do not prevent replication loops. MirrorMaker 2 remote-topic naming and cycle detection, or equivalent source aliases and exclusion rules in another replication layer, must enforce that behavior.
owner_epoch and version do not fence a writer by themselves either. They become effective fencing evidence only when every write boundary validates the current owner and epoch against an authoritative ownership store and rejects stale epochs.
Common Write Models
There are several possible write models in an Active-Active design.
The Home Region model assigns one write-owning region to each data item. It is the simplest and easiest to operate. During failover, ownership for that data moves to another region.
The Regional Write model splits write regions by tenant, customer, business domain, or geography.
Tenant A -> Region A
Tenant B -> Region B
Both regions perform writes, but the same data is still modified by only one region. This is realistic when business boundaries are clear and conflicts are naturally rare.
The Event Sourcing model treats the event log as the system of record and databases as projections. It is useful for audit and reprocessing, but retention, compaction, schema evolution, and replay governance must be designed carefully. Kafka can be used as an event log, but “we have Kafka” does not automatically mean “we have Event Sourcing.”
The Multi-Writer model lets both regions directly modify the same data. It looks closest to full Active-Active, but it is the riskiest choice. Conflict policy is mandatory.
version comparison
merge rules
last write wins
compensation
user or operator intervention
For many business systems, Multi-Writer should be the last option, not the default.
The Dangerous Moment Is Failover
During normal operation, the Home Region model can look straightforward. The risk appears during failover.
Assume Region A owns a piece of data and then fails. For Region B to take over writes, changing routing is not enough.
Region A must first be prevented from writing the same data. Otherwise, under network partitions, delayed recovery, or partial failure, both regions may believe they own the same data. That is a split-brain condition.
Safe ownership transfer usually needs these elements:
strongly consistent authoritative ownership store
owner epoch or fencing token
owner and epoch validation at every write boundary
write blocking or lease expiration for the previous owner
replication lag check in the receiving region
Inbox catch-up verification
post-transfer reconciliation
failback procedure
A routing table can send requests to the new region, but it is not an authority that blocks an old writer. If Region A loses access to routing or the ownership store but can still reach its database, it may continue writing. Validation therefore belongs at a boundary the old writer cannot bypass, such as a conditional database write, a common write gateway that checks the latest epoch, or an expiring lease.
The key is not simply “the new region starts writing.” The key is “the old region can no longer write.”
Planned and Unplanned Failover Are Different
During planned maintenance or migration, the system can stop writes and let the pipeline catch up before transferring ownership.
stop new writes in Region A
-> verify that Outbox publication is complete
-> verify cross-cluster replication and Region B Inbox application
-> change owner to B and increment the epoch
-> begin writes in B with the new epoch
An unplanned failure is different. If Region A is completely unreachable, it may be impossible to prove whether its last database transaction and Outbox event reached the other region. The system must either keep writes blocked to preserve consistency or force failover while accepting the possibility of data loss within its defined RPO.
A failover policy therefore needs more than a lag threshold. It must define who may authorize forced failover when lag cannot be measured, what evidence is required, and what RPO is accepted. Kafka availability cannot make that decision for the application.
The Kafka Key Should Follow the Ordering Unit
When Kafka acts as the cross-region event hub, the partition key matters.
Events for the same aggregate should ideally go to the same partition. That makes it easier to preserve event order within that aggregate.
If event order matters per order, order_id may be a good key. If the business truly requires ordering across every change for a tenant, tenant_id may be appropriate. But tenant-level ownership alone is not a reason to put every order for that tenant on one key.
If the key is too coarse, one partition can become hot. If the key is too fine, the ordering needed for business correctness may disappear. Ownership and ordering units must be compatible, but they do not have to be identical. Kafka guarantees order within a partition, not across an entire topic.
Operations Care More About State Transitions Than Topology
An Active-Active DR diagram tends to show Kafka clusters, databases, applications, and replication arrows. In real operations, the more important thing is the state transition.
which region currently owns this data
what the current owner epoch is
how far the opposite region has applied events
whether lag is within an acceptable range
which failover step is currently running
whether an old writer is still alive
Kafka topic configuration alone cannot answer those questions. You need to look at cross-cluster replication lag, Outbox backlog, Inbox backlog, the last event version applied to the receiving database, ownership state, routing state, and application write-rejection logs together.
This operational angle matters for clustered resources such as Kafka. Konduo connects Kafka-like operational targets through plugins so teams can read resource status, metric evidence, diagnostics, and alert response in one operating flow.
Conclusion
Kafka can be a powerful event hub in an Active-Active environment. It stores events, delivers them, and makes them replayable.
But the core challenge in Active-Active is not Kafka. It is ownership.
Before Kafka can do its job, the system must decide which region may write which data, how that authority moves during failure, how stale writers are fenced, and how duplicate events become harmless.
Once those decisions are made, Kafka becomes the layer that reliably carries the resulting change events between regions.
The success of Active-Active is not decided by the Kafka cluster layout.
It is decided by the data write strategy and ownership enforcement model.
Further Reading
- Why Kafka Fits Between Outbox and Inbox Patterns - Explains the application contracts behind dual-write and duplicate handling in async messaging.
- Trace Should Not Break When Kafka Is in the Middle - Covers tracing event flows across service and Kafka boundaries.
- A Kafka Broker Bottleneck Is Not Always Inside Kafka - Shows why Kafka operations often require looking beyond Kafka itself.