Keep Late Events from Overwriting Current State: Fluss Versioned Merge Engine

Suppose an order is already recorded as PAID, then an older CONFIRMED event arrives through a retry or delayed path. If a primary-key table accepts whichever row arrived last, a completed order can appear to move backward.

Apache Fluss’s Versioned Merge Engine changes the rule for choosing current state. Instead of trusting arrival order, it keeps the row with the greatest source version or timestamp for each primary key. A late, older event cannot overwrite a newer source state.

This article makes that behavior observable with a minimal Flink SQL example, then covers how to choose a version column and where the feature stops. The essential distinction is simple: the most recently received value is not always the newest value in the source system.

The Problem Is the Rule for Current State

Events can reach a table in a different order from the source system’s changes. Retries, replay after recovery, and latency across multiple input paths can all deliver an earlier change after a later one.

Imagine that an order service made these changes in this order:

Source-system change order

source_version=19  CONFIRMED
source_version=20  PAID

The PAID row may reach Fluss first, followed by the retried CONFIRMED row. LastRow, the default merge behavior, treats the later table write as current state. That is the right rule only when table-write order is known to match business-change order.

Versioned Merge Engine compares a configured column for each primary key. Once a greater value is stored, a row with a smaller value cannot change the current state. The Versioned Merge Engine documentation describes it as a way to merge out-of-order data where eventual consistency is acceptable, and lists the supported version-column types and limits.

LastRow, FirstRow, and Versioned Answer Different Questions

A Fluss Primary Key Table uses a merge engine to reduce multiple writes for one key into one current row. The important question is not the engine name; it is which ordering should be trusted.

Merge engine	Row kept for one key	Appropriate when
LastRow	The row written to the table last	Arrival order is also the business order
FirstRow	The first row written to the table	One first-seen value should represent duplicate facts
Versioned	The row with the greatest configured version	Retries, replay, or delayed delivery can reorder writes

FirstRow is for retaining a first-observed event during deduplication. Versioned is for selecting the newer source state among several candidates. Both are often described alongside duplicate handling, but they use different evidence: FirstRow trusts first arrival; Versioned trusts a version value. The FirstRow Merge Engine documentation also notes that FirstRow does not support UPDATE, DELETE, or partial updates.

Create a Versioned Table with Flink SQL

Run the following in a Flink SQL session that has already selected a Fluss catalog. order_id is the primary key, and source_version is assumed to be a monotonically increasing version assigned by the order service.

SET 'execution.runtime-mode' = 'batch';
SET 'table.dml-sync' = 'true';

CREATE TABLE order_status (
  order_id STRING NOT NULL,
  status STRING,
  source_version BIGINT NOT NULL,
  source_updated_at TIMESTAMP(3),
  PRIMARY KEY (order_id) NOT ENFORCED
) WITH (
  'bucket.num' = '1',
  'table.merge-engine' = 'versioned',
  'table.merge-engine.versioned.ver-column' = 'source_version'
);

With table.dml-sync set to true, each INSERT finishes before the next command runs. The query that follows therefore cannot overtake a still-running write. source_updated_at is present only to make the source change time visible in results. The merge decision uses source_version. Versioned Merge Engine accepts INT, BIGINT, TIMESTAMP, and TIMESTAMP_LTZ families as version columns. For an order state, though, a source-owned increasing version is often less ambiguous than a timestamp.

Write the newer state first.

INSERT INTO order_status VALUES (
  'o-1042', 'PAID', 20, TIMESTAMP '2026-08-14 09:00:00'
);

Now write the older state from a retry or delayed path. It is later from the table’s perspective, but smaller in the source ordering.

INSERT INTO order_status VALUES (
  'o-1042', 'CONFIRMED', 19, TIMESTAMP '2026-08-14 08:59:00'
);

SELECT order_id, status, source_version
FROM order_status
WHERE order_id = 'o-1042';

The query returns:

+----------+--------+----------------+
| order_id | status | source_version |
+----------+--------+----------------+
| o-1042   | PAID   |             20 |
+----------+--------+----------------+

Fluss does not need to delete the CONFIRMED input or turn it into an error. It simply sees that version 20 is newer for this key, so version 19 cannot replace the current row.

A Version Is Not an Event ID

Treating event_id and source_version as the same field makes this design unclear.

• An event_id answers, “Have I processed this particular event already?” It identifies a repeated copy of the same fact.
• A source_version answers, “Which of these different states is newer?” It orders changes for one entity.

CONFIRMED and PAID can each arrive exactly once and still be different state changes, not duplicate events. An event ID does not decide which state should become current. Conversely, a version does not necessarily reveal whether the exact same event was sent twice. Many systems need both values.

The strongest version column is usually a monotonic value that the source already guarantees: a row revision, change sequence, or database-log position. A source update time can work too, but only after deciding how equal timestamps and clock skew behave. Versioned Merge Engine can replace a row when a later write has the same version, so a timestamp that can tie may not be a complete ordering rule by itself.

What This Feature Does Not Solve

Versioned Merge Engine selects the current row inside a Primary Key Table. It does not solve every concern around that row.

• Keeping every transition: A current-state table that retains PAID does not tell you when CONFIRMED arrived. Keep the source events in an append-only Log Table when audit, replay, or state-transition analysis matters.
• Atomic work with external systems: The merge result does not atomically include an external database write or a payment API call. Those boundaries can still need idempotency, outcome lookup, or an Outbox-style design.
• Partial updates and deletes: Versioned Merge Engine does not support SQL UPDATE, DELETE, or partial updates. Write a complete state row through INSERT instead.

For that reason, a practical design often has two tables:

order_status_events      Log Table
  Keeps every state transition as an append-only fact.

order_status             Versioned Primary Key Table
  Exposes the row with the greatest source version as current state.

Use the Log Table for audit, replay, and transition analysis; use the Versioned Primary Key Table for current-state lookup or joins. Apache Fluss: The Streaming Lakehouse Between Kafka, Flink, and Iceberg explains the distinct roles of Log Tables and Primary Key Tables in more detail.

Questions to Answer Before Adopting It

Versioned Merge Engine is not a generic safety switch for every Primary Key Table. It is valuable when the following questions have clear answers:

• Can the source provide a comparable per-key version or a trustworthy change time?
• Can retries, replay, or multiple paths deliver an old event late?
• Do consumers need current state more often than every intermediate transition?
• If transitions still matter, is there a separate Log Table that keeps them?

In operations, do not assume that late events are automatically harmless. Observe their rate, the number of version inversions, and the share of inputs without a version through application or Flink-job metrics. Those signals reveal input-quality problems that the merge engine may quietly hide from the current-state table. Konduo connects infrastructure status, metric and log evidence, and alert response in one operations platform. A data path using Fluss needs the same perspective: observe both the current state and the inputs that led to it.

Closing Thought

Storing current state in a Primary Key Table does not automatically preserve the newest source state. First decide whether “latest” means the last row to arrive or the greatest source version.

When retry and late delivery are realistic paths, Versioned Merge Engine makes the current-state table much clearer. Do not stretch that guarantee into an event history, external side-effect protocol, or end-to-end processing guarantee. Keep facts in a Log Table, and keep the source-version winner in a Versioned Primary Key Table.

Further Reading

• Apache Fluss: The Streaming Lakehouse Between Kafka, Flink, and Iceberg — Start with the roles of Log Tables and Primary Key Tables.
• Apache Fluss Flink Quickstart: Create and Try Log and Primary Key Tables — A hands-on introduction to creating and reading Fluss tables through Flink SQL.
• What Your Sensor Event Is Missing: Enrich It with fluss-go — See event history and current-state tables used together in a data path.