Skip to content

Redis ​

Redis covers all four capabilities: pub/sub for the backplane, Streams for durable feeds and queues, and pub/sub plus a list for RPC. It is the least new infrastructure of any option here — most deployments already have one.

js
const Redis = require('ioredis');
const { createRedisBroker } = require('@alexify/wrpc/broker/redis');

const broker = createRedisBroker({ client: new Redis(process.env.REDIS_URL) });

const server = new Server({ router, backplane: broker.backplane, port: 8000 });

The client is injected, as everywhere in wRPC: ioredis is a devDependency of this repository and never a runtime one. Anything with the same command surface works — Valkey, KeyDB and Dragonfly need no adapter of their own.

OptionDefault
client—Runs every command; never quit by close()
subscriberclient.duplicate()The connection subscriptions use
connectclient.duplicate()Opens the extra connections blocking reads need
prefix'wrpc'Key and channel namespace
blockMs1000How long a blocking read parks before looping
claimIdleMs60000Idle time after which a stopped consumer's messages are claimed
maxLen0XADD MAXLEN ~ n on every log append; 0 never trims
inboxTtl60000TTL of an RPC group's delivery list and presence key

What maps to what ​

CapabilityRedis
backplanePUBLISH / SUBSCRIBE (the same adapter @alexify/wrpc/scaling ships)
logStreams: XADD, XRANGE for catch-up, one blocking XREAD per instance for the tail
queueStreams + consumer groups: XREADGROUP, XACK+XDEL, XAUTOCLAIM for what a stopped consumer held, a sorted set for delayed retries
directPUBLISH for inboxes; a list (RPUSH/BLPOP) for a competing group

Stream ids (<ms>-<seq>) are the feed resume tokens, so a subscriber resumes exactly where it stopped, on any instance.

Connections ​

Redis blocks a whole connection while it waits, so the adapter opens more than the one you inject:

  • one for the shared log tail (all topics multiplexed into a single XREAD BLOCK);
  • one per queue consumer (XREADGROUP BLOCK);
  • one per RPC service group (BLPOP);
  • one for subscriptions, shared by the backplane and by RPC inboxes.

It opens them with client.duplicate() (or your connect) and quits only the ones it opened. Budget accordingly on a managed Redis with a connection cap.

Trimming and retention ​

A log topic is a stream that grows until something trims it. Set maxLen for an approximate cap per append, or run XTRIM on your own schedule — the feed answers 410 (and calls onGap) when a client resumes from an id the trim has passed.

A queue's stream, by contrast, trims itself: an acked, retried or dead-lettered message is XDELed, so a queue that keeps up stays small. A delayed retry rides a sorted set until it is due, which any instance may promote — ZREM then XADD as one EVAL, so an entry either leaves the set and lands on the stream or does neither. A client without eval, or a proxy that refuses scripts, gets the two steps apart, with the entry put back on the set when the stream refuses it.

Keys ​

KeyHolds
<prefix>:log:<topic>a log topic's stream
<prefix>:q:{<queue>}a queue's stream (consumer groups on it)
<prefix>:q:{<queue>}:delayedthe queue's delayed retries (a sorted set, score = due time)
<prefix>:inbox:<address>an RPC inbox's pub/sub channel
<prefix>:inbox:<address>:lista service group's delivery list
<prefix>:inbox:<address>:membersthe group's live members (a sorted set, score = lease expiry)

The braces are a Redis Cluster hash tag: a queue's stream and its delayed set hash to one slot, which is what lets the promotion script touch both. Names are encoded into Redis' alphabet first, at most 200 characters.

Sharp edges ​

  • A consumer group starts at the beginning. The first consumer of a queue creates its group at 0, so work produced before it existed is delivered. The group's position then lives in Redis and survives restarts.
  • One RPC group per address. Competing listeners share one list, so two different groups on one address would compete rather than each receive. The binding uses one group per service, which is exactly the supported shape.
  • A message for a group nobody serves waits on the list until its TTL (inboxTtl, shortened by an RPC request's own timeout) expires. With no listener at all, send refuses with 503 instead — the group's members set is how the sender can tell: each listener holds a lease there (renewed every inboxTtl / 3), and the group is present while any lease is live. Leases are timestamps by the listener's clock, so the instances' clocks must agree within inboxTtl.
  • Pub/sub is at-most-once, which is what the backplane's loss detection exists for. Feeds and queues are Streams and do not share that property.

Running the tests ​

pnpm test runs the Redis suites against an in-repo fake. Against a real server:

bash
pnpm redis:up
REDIS_URL=redis://127.0.0.1:6379 node --test tests/broker/redis.integration.test.js

Released under the MIT License.