Home Region Writes: Keeping Database and Kafka Ownership Aligned

In the previous post, we argued that the hard part of Kafka-based Active-Active is not cluster replication. It is write ownership.

How should that ownership work in an actual service?

Assume that both Seoul and Singapore receive user traffic. The Home Region for tenant-42 is Singapore, but a change request may arrive in Seoul. If the Seoul application first changes its local database and only sends a Kafka event to Singapore afterward, the Kafka path may be connected while service ownership is already broken.

Home Region is not a setting that chooses a Kafka destination.

Which region is currently allowed to change this business entity?

The same answer must govern the path from API ingress through the database transaction, Outbox record, and Kafka event.

This post has three main conclusions.

The region receiving a request is not necessarily the region allowed to commit it.
A cache may guide a request, but it cannot authorize a write.
The previous owner must be fenced before the new owner becomes active.

A handcrafted landscape with four equal regional workstations, where only one glows warmly beside a small caretaker — Every region may be ready, but one entity is changed only in its currently designated Home Region.

Home Region Is a Service Write Policy

A Home Region may be fixed for an entire service, or it may differ by tenant, account, or another business boundary.

tenant-42  -> region-b
tenant-81  -> region-a
account-7  -> region-b

The important concept is not the physical database or Kafka cluster. It is the ownership key. Every write request must identify which ownership boundary it belongs to.

Common keys include:

tenant_id in a SaaS product
account_id in a financial domain
order_id or merchant_id in commerce
workspace_id in a collaboration service

A good ownership key can be extracted reliably from every write, represents a useful transaction boundary, and rarely requires one operation to modify several independent owners.

A key that is too coarse makes failover and load movement unnecessarily large. A key that is too fine increases ownership records and cache churn while making cross-key business transactions harder.

The first design question is therefore not “Which Kafka should receive this event?” It is:

What is the smallest business unit whose write ownership can move independently?

One Ownership Decision Must Govern the Entire Write Path

Users may enter through any region, but the actual change is committed only in the Home Region.

A request arrives in Region A, discovers from the local ownership cache that Region B is the Home Region, is forwarded to Region B, commits the database change and Outbox record together, and then publishes an event to Kafka — Ingress is not ownership. One Home Region decision must continue from routing through the database and Outbox commit to Kafka publication.

A normal write path for a database-centered service looks like this:

1. Extract the ownership key from the request.
2. Read the Home Region and owner epoch from a local cache.
3. Forward the request if the current region is not the owner.
4. Revalidate ownership and epoch at a write boundary the Home Region service cannot bypass.
5. Commit the business change and Outbox record in one transaction.
6. Let an Outbox relay publish the event to Kafka.
7. Let other regions apply the replicated event idempotently through an Inbox.

Kafka carries a change that has already been accepted. When the database is the system of record, its transaction must complete safely before publication.

An event-sourced service reverses part of the order. It may validate the command in the Home Region, append it to an event log such as Kafka, and update databases as projections. The first durable record differs, but the principle does not: one ownership decision governs the entire write path.

Three Ways to Handle a Request in the Wrong Region

A non-owner region must not write locally first and synchronize later. It should choose one of three explicit behaviors.

Approach	Benefit	Operational concern
Server-side forwarding	The client does not need to understand Home Region	Cross-region latency, timeouts, and duplicate retries
HTTP redirect	Avoids an additional server proxy hop	The client must handle redirects and credentials safely
Reject the write	Simple policy and clear ownership violations	Lower availability and a worse user experience

When redirecting POST or PATCH, verify that the client supports 307 Temporary Redirect, which preserves the method and body. Avoid a permanently cached redirect because Home Region can move, and forward credentials only to a trusted Home Region endpoint.

Server-side forwarding is often the easiest behavior for a public API. A timeout, however, is not necessarily a failure. The Home Region may have committed the transaction even if the response did not return.

Remote writes therefore need an idempotency key.

{
  "ownership_key": "tenant-42",
  "idempotency_key": "01K0A7V3Q9T6K8M2J4X5N7P1RC",
  "owner_epoch": 18,
  "command": "ChangeBillingAddress"
}

Whether the client retries or the ingress region forwards again, the Home Region should commit only once for the same idempotency key.

The same request may be retried in a new Home Region immediately after ownership transfer. Deduplication records must outlive the maximum retry window and move or replicate with ownership. Otherwise, each region may process the key only once while the service as a whole commits it twice.

Forwarding loops also need a guard. Region A may send to B using stale cache data while B has stale data pointing back to A. Internal requests should carry the original ingress, forwarding count, and decision epoch, with a hard limit such as one forwarding hop. The receiving Home Region service must make the final ownership check again.

Do Not Put the Control Plane on Every Request’s Hot Path

The simplest design queries the authoritative ownership store synchronously for every write.

request -> global ownership database -> write service -> business database

This design connects every write’s latency and availability to the central store. A slow cross-region network can degrade otherwise healthy local writes.

The data path usually uses a regional ownership cache instead.

OwnershipRecord {
  key: "tenant-42"
  homeRegion: "region-b"
  epoch: 18
  state: "ACTIVE"
  updatedAt: "2026-07-13T02:10:00Z"
}

The Control Plane creates and transfers ownership, then distributes changes to regional caches. A Write Gateway routes most requests using its local copy.

The cache is not an authority, though. Update delay, process pauses, and network partitions can leave it stale. The Home Region and epoch returned by a cache are best treated as a routing hint and input to authorization, not as authorization itself. Final permission belongs at a write boundary that can see current ownership and cannot be bypassed by the previous writer.

An Epoch Becomes a Fencing Token Only When It Is Enforced

Adding an epoch number to an ownership record does not fence an old writer by itself.

A stale Region A cache sends an epoch 17 request, but the write boundary sees that Region B owns epoch 18 and rejects it, while a refreshed epoch 18 request is committed in Region B — A cache can route incorrectly. The write boundary must validate owner and epoch at a point the old writer cannot bypass.

Suppose the current ownership record is:

owner = region-b
epoch = 18

Region A may still have owner=region-a, epoch=17 in its cache. If that request reaches the database unchanged, an epoch field in the cache has achieved nothing.

Every write boundary needs to validate at least:

request.home_region == current.home_region
request.owner_epoch == current.owner_epoch
current.state == ACTIVE

home_region and owner_epoch do not mean trusting arbitrary headers supplied by an external client. They belong in an authenticated internal request or a server-issued write permit between ingress and the Home Region, and the final write boundary must compare them with current authority state.

The enforcement point depends on the storage model.

Shared or Global Database

If ownership and business data can be checked within the same consistency boundary, a conditional update or transaction can validate owner and epoch.

UPDATE account
SET billing_address = :address,
    version = version + 1
WHERE account_id = :account_id
  AND owner_region = :request_region
  AND owner_epoch = :request_epoch;

No affected row means that the request used stale ownership and must not commit.

Independent Database per Region

If the old Region A database still contains epoch=17, a conditional update that reads only that database may accept an epoch 17 request. This is a common trap.

The old region’s write path must be fenced separately before the new region becomes active. Options include:

revoke or rotate the write service’s database credentials
switch the database to read-only
block the old writer through a network policy or firewall
validate an expiring lease in both the service and storage boundary
force all writes through a common Write Gateway that cannot be bypassed and validates current ownership or a valid lease

An epoch acts as a fencing token only when the enforcement point knows current ownership and the previous writer cannot bypass the check.

The Database and Outbox Record the Same Ownership

After a business change succeeds in the Home Region, other regions need to learn about it. Calling the database and Kafka sequentially reintroduces a dual-write problem.

The Home Region transaction should therefore commit the business state and Outbox record together. The Debezium Outbox Event Router documentation describes this pattern as a way to avoid inconsistencies between a service’s persisted state and the events consumed by other services.

{
  "event_id": "01K0A80D4JYV6P2QH9W3B7N5XM",
  "aggregate_type": "Tenant",
  "aggregate_id": "tenant-42",
  "event_type": "BillingAddressChanged",
  "source_region": "region-b",
  "home_region": "region-b",
  "owner_epoch": 18,
  "aggregate_version": 73,
  "occurred_at": "2026-07-13T02:14:31Z"
}

event_id supports Inbox deduplication. owner_epoch identifies the ownership generation that produced the event. aggregate_version helps the receiving region reason about application order for the entity.

An epoch inside a Kafka event cannot undo an invalid database write that already happened. Ownership must be checked at the write boundary before publication.

Kafka also does not automatically make an external database update exactly once. Regardless of Kafka’s delivery semantics, the receiving region should commit its Inbox record and business update in one local transaction and make a repeated event_id harmless. The Outbox and Inbox post covers this contract in more detail.

Home Region Transfer Is a State Machine

Treating a Home Region change as one routing-table update hides the most dangerous part of the operation.

Ownership moves from ACTIVE in Region A at epoch 17 through DRAINING, FENCED, and Region B CATCH-UP before Region B becomes ACTIVE at epoch 18 — A planned transfer drains and fences the previous owner, checks that the target has caught up, and only then activates a new epoch.

A planned transfer can follow this sequence:

1. Put Region A into DRAINING and stop new writes.
2. Wait for in-flight requests to finish.
3. Confirm Outbox publication and cross-region event replication.
4. Move Region A's write path into FENCED state.
5. Confirm that Region B's Inbox and database reached the target version.
6. Change the owner to Region B and increment the epoch to 18.
7. Set Region B to ACTIVE and allow writes.
8. Watch stale-epoch rejections and delayed events closely.

The critical order is FENCED(A) -> ACTIVE(B). Opening the new region before closing the old one creates a period with two writers.

An unplanned failure may make this sequence impossible to prove. If Region A is unreachable, you may not know whether its final database commit and Outbox event were replicated, or whether an old writer has truly stopped.

Technology cannot hide the decision at that point.

Keep writes blocked and prioritize consistency.
Force failover while accepting possible loss within a defined RPO.
Define which operator may approve the transfer and what evidence is required.

Safe automatic failover needs a condition that Region A cannot extend by itself, such as a lease, quorum, or external fencing mechanism. Kubernetes, for example, uses Leases as a coordination mechanism that lets only one instance perform an active role. A lease record alone is still insufficient; an expired holder’s actual writes must be rejected at the enforcement boundary. Without that property, deliberate manual approval may be the more honest and safer design.

Signals That Operations Need to See Together

Home Region writing is both application behavior and operational state. The following signals should be visible together in a dashboard or runbook.

Area	Signals to inspect
Ownership	Current owner, epoch, state, last change time, and operator
Requests	Non-owner ingress rate, forwarding latency and failures, redirects or rejections
Fencing	Stale-epoch rejections, old credential attempts, read-only violations
Database and Outbox	Commit failures, Outbox backlog, age of the oldest unpublished event
Kafka and Inbox	Publish failures, cross-region lag, Inbox backlog, duplicate events
Consistency	Difference between owner version and the last version applied remotely

A stale epoch rejection is not always an error that should be eliminated. Immediately after transfer, it can prove that delayed requests and stale caches are being fenced safely. If it continues, investigate cache propagation, long-running jobs, and retry queues.

Checklist Before Implementation

Which ownership key defines Home Region?
Can one request modify data under several ownership keys?
Will a non-owner region forward, redirect, or reject a write?
Where is the idempotency key created, and how long is it retained?
Can the new Home Region see idempotency records after ownership transfer?
During a Control Plane outage, does the service keep current ownership or stop writes?
Where is a stale cache decision revalidated?
Can the previous region’s direct database write path actually be fenced?
Are the business update and Outbox record committed in one transaction?
Do events contain event_id, owner_epoch, and aggregate_version?
Can operators observe DRAINING, FENCED, and CATCH-UP during planned transfer?
Are the RPO and approval authority for forced failover documented?
Has failback been rehearsed as carefully as failover?

Conclusion

Home Region is broader than a routing rule that sends a request to a particular data center.

It decides which region may change a business entity. The Write Gateway, business service, database transaction, Outbox, Kafka event, and failover procedure must all follow that same decision.

A request may enter through any region.
The change commits only in the Home Region after current ownership is validated.

Kafka is a powerful layer for carrying the result to other regions and services. But Home Region is not enforced if database and Kafka ownership move independently.

The safety of a multi-region service depends less on proving that the new region can write and more on proving that the previous region can no longer write.