#architecture
12 posts.
Why Rednote Moved from Kafka to Fluss: Lessons from a Real-Time Indexing Migration
A close look at the read amplification, historical replay bottleneck, Fluss architecture, results, and migration discipline in Rednote's real-time indexing path.
What Is Apache Fluss? A Streaming Lakehouse Between Kafka, Flink, and Iceberg
Understand Apache Fluss by role: how it differs from Kafka, Flink, and Iceberg, and how its Log Tables, Primary Key Tables, and shared metadata connect real-time streams to a lakehouse.
Why Kafka Transactions Cannot Commit with Your Database Yet: KIP-939 and 2PC Participation
Learn what Kafka transactions can commit today, why databases and external systems remain outside that boundary, and how KIP-939 proposes to let Kafka participate in externally coordinated 2PC.
Active-Active DR: Recovery, Reconciliation, and Failback
Learn how to prove a recovery checkpoint, reconcile uncertain data, and execute failback safely after an Active-Active DR failover.
Home Region Writes: Keeping Database and Kafka Ownership Aligned
Learn how to enforce Home Region ownership across request routing, database transactions, Outbox, Kafka, epoch fencing, and ownership transfer in a multi-region service.
Kafka Active-Active Is About Ownership, Not Kafka
This post explains what Kafka can and cannot solve in Active-Active DR architectures, and why ownership, Outbox, Inbox, and failover fencing must be handled at the application level.
Kafka 4.0 Consumer Rebalance: Coordination Moves to the Broker
Apache Kafka 4.0 made the new rebalance protocol generally available. This post explains what changes from the old protocol, why broker-side assignment matters, and what it means operationally.
Kafka Streams and the EIP Aggregator: Finishing Is Harder Than Grouping
When implementing the EIP Aggregator pattern with Kafka Streams, the hard parts are not groupBy and aggregate. They are correlation keys, state stores, completion conditions, and late-event policy.
Why Kafka Fits Between Outbox and Inbox Patterns
This post explains why Outbox and Inbox patterns matter in asynchronous MSA flows, and why Kafka is a strong fit as the message path between them.
The Large Payload Kafka Handles Well, and the One It Should Not Carry
Kafka is strong at high-volume event streams, but large file payloads can create storage amplification across the whole system.
Where Kafka's Disk Is Going: Object Storage and the New Streaming Layer
A practical look at WarpStream, AutoMQ, Apache Fluss, and Kafka Diskless Topics as the Kafka ecosystem moves durable storage away from broker-local disks.
Five Roles Redis Plays in AI Applications
A practical look at how Redis supports AI applications as a cache, session store, retrieval helper, async work layer, and operational control plane.