Binary streams

Calls carry JSON. For bytes — a file upload, a video download, anything big enough that buffering it would be a mistake — wRPC has a second framing: binary chunks, interleaved on the same connection.

Two objects, one on each end:

• WrpcWritable — the sending side. write(chunk) reports whether the transport accepted it; end() closes the stream.
• WrpcReadable — the receiving side. Async-iterable, or pipe() into a node Writable, or toBlob() in a browser.

The stream id prefixes every chunk, which is what lets several streams interleave over one connection without head-of-line blocking. Streams need a persistent, binary-capable transport: not HTTP, and not SSE.

Uploading

Announce the stream, call the procedure with its id, then push the bytes:

// client
const blob = new Blob([bytes]);
blob.name = 'video.mp4';                  // optional; defaults to 'blob'
const uploader = client.createBlobUploader(blob);

const receiving = client.api.media.receive({ id: uploader.id });
await uploader.upload();
console.log(await receiving);

upload() is paced by the transport — a chunk it did not accept waits for the stream's 'drain' — and rejects with a 503 WrpcError when the connection closes before the last chunk went, instead of resolving with the rest gone nowhere; a source that fails mid-way terminates the stream on the server, which stops waiting for its end.

// server
receive: procedure({
  handler: async (context, { id }) => {
    const stream = context.client.getStream(id);
    for await (const chunk of stream) {
      await sink.write(chunk);
    }
    return { name: stream.name, size: stream.size };
  },
}),

Note the order: the call is issued before the upload finishes. The handler gets the stream as soon as its opening packet has arrived and consumes chunks as they land — the call's promise resolves when the handler returns.

Without a Blob, drive a WrpcWritable yourself:

const drained = (stream) => new Promise((resolve) => stream.once('drain', resolve));

const stream = client.createStream('report.csv', bytes.length);
const receiving = client.api.media.receive({ id: stream.id });
for (const chunk of chunks) {
  if (!stream.write(chunk)) await drained(stream);
}
stream.end();

Downloading

The same machinery, mirrored:

// server
download: procedure({
  handler: async (context, { name }) => {
    const payload = await load(name);
    const stream = context.client.createStream(name, payload.length);
    queueMicrotask(() => {
      stream.write(payload);
      stream.end();
    });
    return { id: stream.id };
  },
}),

// client
const { id } = await client.api.media.download({ name: 'report.csv' });
const readable = client.getStream(id);

const blob = await readable.toBlob('text/csv');   // browser
readable.pipe(fs.createWriteStream('report.csv')); // node
for await (const chunk of readable) { /* ... */ }  // either

createStream() sends the opening packet immediately, so writing from inside the same tick as the return would race the caller's chance to register a consumer — hence the queueMicrotask.

Backpressure

This is the part that makes streams usable for real payloads, and it works end to end:

if (!stream.write(chunk)) {
  await new Promise((resolve) => stream.once('drain', resolve));
}

write() returns the transport's real answer: false means the socket buffer is above its high-water mark. Wait for 'drain' before writing more. A transport with no flow-control reporting (a browser WebSocket, HTTP) always reports accepted.

false can also mean the transport closed, and no 'drain' will ever follow — check stream.closed, or listen for 'close'.

On the receiving side, WrpcReadable applies its own high-water mark (highWaterMark, 32 chunks by default) once a consumer has attached, and the server pauses socket reads while chunks are being consumed. So a slow consumer pushes back through the queue, through the socket, and into TCP — the producer on the other end sees write() return false. Nothing between them buffers without bound.

Every dotted arrow is one hop of the same signal travelling backwards: a slow consumer pauses socket reads, TCP stops moving bytes, the sender's buffer fills, and write() starts answering false. Nothing in between buffers without bound.

Verified, not asserted

tests/perf/stream-memory.perf.js (pnpm test:perf) streams 1 GiB through and fails if RSS grows with it.

Lifecycle

Call	Effect
writable.end()	Normal end. The reader's iteration finishes.
writable.terminate()	Abnormal end — the reader stops where it is.
readable.close()	Waits for the announced size, then closes.
readable.terminate()	Stops now, discarding what is queued.

A disconnect terminates every stream the client held. readable.status is 'active', 'closed' or 'terminated'; bytesRead is what has been consumed so far, against the announced size.

A connection holds at most maxStreams streams at once (256 by default): a stream is announced by the peer, and holds a readable — with what was sent into it — until a handler reads it, so the number is the peer's to choose and the cap the server's. Past it the stream packet answers 429 (stream.capacity in the log, at debug).

Bytes inside a call, not as a stream

A stream is for bytes that do not fit in memory or should not wait for their end. Bytes that do fit — an avatar, a thumbnail, a signature, a serialized model — belong in the call, and they can simply be put there:

await client.api.files.put({ name: 'avatar.png', body: pngBytes }); // a Uint8Array
const { thumbnail } = await client.api.files.get({ id }); // arrives as a Uint8Array
context.client.sendEvent('files/ready', { preview: bytes });

Any typed array, ArrayBuffer or DataView anywhere in a packet's args, result, event data or error details travels as binary attachments: the packet's JSON goes out with null at each byte leaf and an index of where the leaves were, the buffers follow in the same frame, and the other end puts them back — as fresh Uint8Arrays that own their bytes. No base64 (a third more), and none of what JSON.stringify makes of a typed array otherwise: a {"0":137,"1":80,…} object nine times the size that used to arrive silently as a plain object. Every transport carries it — a binary WebSocket frame, a WebTransport or data-channel message, a broker session frame, a worker port, an HTTP body under application/octet-stream — except SSE, which is text-only: it speaks revision 1 to every client, so bytes on it travel as the JSON objects 1.0 made of them.

On by default between two 2.x ends. The frame is revision 2 of the protocol, negotiated per connection, so a 1.0 peer is never sent one: its bytes arrive — and are sent — as the plain objects 1.0 made of them, and client.revision (on the client, and on context.client in a handler) says which kind of connection this is. The cost is a walk of every outbound packet to find the bytes (bench/attachments.js: tens of nanoseconds on a small callback, a few microseconds on a 30 KB one); attachments: false skips it and makes that end speak revision 1 — JSON, to everyone. Under a packet codec it is off by itself — the codec owns the wire. A REST result holding bytes needs codec.rest and answers 501 without it. A room event with bytes crosses the backplane too: its envelope rides as the same frame, base64 under a wrpc-bin: marker (or inside the sealed envelope, under rooms.encryption), and the members on another instance receive bytes.

Two things the walk leaves to JSON. An object with a toJSON() is its projection: what toJSON() answers is what travels, and a typed array kept behind it (a password hash on a user entity, say) is never lifted out and sent — put the bytes in the projection if they should go. And a packet with a cycle in it answers false to the walk and is handed to JSON.stringify, whose "Converting circular structure" TypeError is the one you would have seen anyway; the walk itself stays linear on a graph with back-references (a parent every child points at), where a walk with no memory of what it visited would take 12^16 steps. The frame's layout is on the protocol page.

Chunk framing

Each binary frame is one chunk of one stream:

┌────────┬──────────────────┬──────────────────────────┐
│ 1 byte │  id (idLength)   │        payload           │
│ idLen  │  utf-8 stream id │        bytes             │
└────────┴──────────────────┴──────────────────────────┘

chunkEncode(id, payload) / chunkDecode(chunk) are exported if you need to speak it yourself. The Node build uses Buffer, the browser build TextEncoder/TextDecoder — swapped through the package's browser field, so you never choose. Full details in the wire format reference.