Still Using Redis Only as a Cache? A Practical Look at Redis Streams

When people think about Redis, they usually picture GET, SET, TTLs, cache-aside patterns, and maybe distributed locks.

That picture is not wrong. Redis is widely used as a fast key-value store, and it is especially strong as a cache. But if you remember Redis only as a cache, you miss one of its more interesting features.

Redis has Streams.

A stream behaves like an ordered event log. Producers append entries, and consumers read them. That may sound like a queue at first. But Redis Streams also provides consumer groups, pending entries, acknowledgements, and message claiming, which are all important pieces of practical message processing.

So the question changes:

Is Redis only a cache?
Or can it also run a small event pipeline?

Illustration of red event pieces moving through a small processing workshop — Redis Streams show Redis as more than a cache for current state: it can also hold an ordered event flow.

This post is a practical answer to the second question.

The short version

Redis Streams are close to an append-only event log inside Redis.
Producers add entries with XADD; consumers read with XREAD or XREADGROUP.
Consumer groups let multiple workers divide work from the same stream.
A message stays pending until it is acknowledged with XACK.
Pending messages can be inspected and recovered with XPENDING, XCLAIM, or XAUTOCLAIM.
Streams can be a good fit for small async jobs, internal event processing, and short event histories.
They are not a universal replacement for Kafka or RabbitMQ.

The key is to stop treating Streams as a small cache side feature.

Redis key-value = fast access to current state
Redis Streams = ordered event flow over time

They live in the same Redis server, but they solve different problems.

What a stream stores

A Redis Stream is a list of entries. Each entry has an ID and a set of field-value pairs.

For example, an order event can look like this:

XADD order-events * orderId 1001 type paid amount 49000

The * tells Redis to generate the stream entry ID. The generated ID is usually time-based.

Consumers can read new entries with XREAD.

XREAD BLOCK 5000 STREAMS order-events $

From this point of view, a stream is closer to an event log than a simple queue. Entries remain in the stream and can be read by range.

XRANGE order-events - +

Redis is still an in-memory system. If a stream keeps growing, memory usage grows too. That means trimming is part of the design.

XTRIM order-events MAXLEN ~ 100000

Once you use streams, “how long do we keep events?” becomes an operating question.

How List, Pub/Sub, and Streams differ

Redis already had features that could be used for messaging before Streams. The common ones are Lists and Pub/Sub.

Lists are easy to use as queues.

LPUSH jobs payload
BRPOP jobs 0

They are still useful for simple job queues. But once you need many workers, processing tracking, failed message recovery, and replay-like behavior, much of the design moves into application code.

Pub/Sub is closer to real-time fan-out.

PUBLISH events payload
SUBSCRIBE events

It is lightweight and fast, but messages published while no subscriber is listening are not available later. It is a “deliver to whoever is listening now” model.

Streams provide a different option.

Feature	List	Pub/Sub	Streams
Message storage	possible	weak	built in
Worker distribution	app-managed	per subscriber	consumer groups
Completion tracking	app-managed	none	`XACK`
Pending inspection	app-managed	none	pending entries
Replay/range reads	limited	none	`XRANGE`, ID-based reads

Streams are much more natural when you want a stored event flow inside Redis.

Consumer groups change the model

Consumer groups are the part that makes Redis Streams especially interesting.

After creating a consumer group, multiple consumers can divide work from the same stream.

XGROUP CREATE order-events workers $

Each worker reads with a consumer name inside the group:

XREADGROUP GROUP workers worker-1 COUNT 10 BLOCK 5000 STREAMS order-events >

The > means “give me new messages that have not yet been delivered to this group.”

After successful processing, the worker acknowledges the entry:

XACK order-events workers 1718610000000-0

The important distinction is that reading and completing are not the same thing.

XREADGROUP delivered the entry
  -> the worker has received it
  -> it has not necessarily processed it successfully

XACK was sent
  -> the entry is considered processed

That gap creates pending entries.

Diagram of Redis Stream entries distributed across consumer group workers with unacknowledged messages tracked as pending — Consumer groups let workers divide stream entries, while unacknowledged messages remain pending until they are acknowledged or claimed.

Why pending entries matter

What happens if a worker reads an entry and crashes before sending XACK?

Redis Streams does not immediately forget that message. It remains in the consumer group’s Pending Entries List, usually called the PEL.

You can inspect pending messages:

XPENDING order-events workers

You can also inspect messages held by a specific consumer:

XPENDING order-events workers - + 10 worker-1

Messages that have been pending for too long can be claimed by another consumer:

XAUTOCLAIM order-events workers worker-2 60000 0-0 COUNT 10

This command finds pending messages that have been idle long enough and transfers ownership to worker-2.

This gives Redis Streams a stronger operating model than “the worker read the message and then it disappeared.”

It does not mean exactly-once processing. Once retries and claiming are involved, the same message may be processed more than once. Handlers should be idempotent.

Think in at-least-once terms.
Tolerate duplicate processing.
Ack only after successful processing.

A small event pipeline example

Imagine an order API that must run several follow-up actions after payment succeeds:

send a notification
award points
update a recommendation model
record an internal audit event

If the API does all of that inline, request latency grows. With Redis Streams, the request path can append an event and return quickly.

Order API
  -> XADD order-events ...
  -> response

Worker group
  -> XREADGROUP
  -> process
  -> XACK

Redis Streams gives you:

event ordering
worker distribution
completion acknowledgement
pending message inspection
recovery after worker failure

For a small internal async pipeline, that can be enough.

When Redis Streams fit well

Redis Streams are attractive when:

You already operate Redis.
Kafka or RabbitMQ would be too much for the current system.
Event retention is short.
Throughput requirements fit within your Redis capacity.
You need consumer-group style worker distribution.
Your application can handle retries and duplicate processing.

Examples include:

webhook delivery queues
notification queues
image or document post-processing
internal domain event fan-out
short audit trails
temporary buffering for near-real-time events

The key question is whether Redis is already part of your trusted operating surface. If it is, Streams can be a fast way to add lightweight event processing.

When another tool is better

It would be too much to say Redis Streams replace Kafka.

Kafka, RabbitMQ, or a managed queue may be a better fit when:

long-term event retention matters
many teams share the event platform
topic partitioning, replay, and schema governance matter
very high throughput and long-term scaling are required
routing, dead-letter exchanges, or complex delivery semantics are central
Redis memory and persistence settings are too risky for the data

Redis Streams are good for small and fast event pipelines. They need a separate review before becoming the backbone of a company-wide event architecture.

What to watch in production

When Redis Streams go into production, operating rules matter more than command syntax.

Paper illustration of hands pruning and organizing a glowing event stream — Operating Streams means watching the event flow, recovering stuck work, and deciding how much history to keep.

First, manage stream length.

XINFO STREAM order-events

Watch XLEN and memory usage. Decide whether XTRIM, MAXLEN, or another retention policy is appropriate.

Second, watch pending messages.

XINFO GROUPS order-events
XPENDING order-events workers

If pending entries keep growing, workers may be slow, repeatedly failing, or forgetting to acknowledge messages.

Third, design for poison messages. A message that always fails can cycle between pending and claim forever. You need retry counts, a dead-letter stream, or explicit failure records.

Fourth, make handlers idempotent. Processing the same entry twice should not corrupt order state or award points twice.

Fifth, understand Redis persistence. RDB, AOF, replication, and failover settings affect loss and recovery behavior. These choices matter much more when Redis stores events than when it only stores disposable cache entries.

That is why Redis Streams need more than simple up/down monitoring in production. Teams need to watch stream length, consumer groups, pending messages, processing delay, and abnormal Redis states together. Pletor’s Konduo includes a Redis plugin for inspecting Redis Streams and consumers, diagnosing abnormal Redis behavior, and supporting operational remediation and failover workflows from the same operating surface.

Redis beyond cache

Redis Streams change the way Redis feels.

Redis is not only a fast key-value cache. At small to medium scope, it can store event flow, distribute work across multiple consumers, track completion, and recover pending messages.

That does not mean Redis Streams should solve every event-processing problem. Some problems belong in Kafka. Some belong in RabbitMQ. Some only need a simple queue.

But if you already operate Redis and need a small async event pipeline, it is worth asking:

Do we really need Kafka here?
Or is Redis Streams enough?

Redis Streams are one of the clearest ways to see Redis beyond cache.