Pipe vs shared memory: what actually happens when you pass a tensor between processes

I use Ray every day at work — submitting tasks, passing tensors between actors, watching the object store fill up. I have a rough mental model of what’s happening, but “rough” has been bothering me. What actually moves when I call ray.get()? Why is it fast? What would happen if I had to build that myself?

So I went back to basics. This post is about one specific question: when you pass a tensor between two Python processes, what actually happens at the OS level? I built a small benchmark — pickle over a pipe versus shared memory — and used it as an excuse to get concrete about virtual memory, kernel buffers, and why the difference matters.

The goal isn’t to reimplement Ray. It’s to understand the OS primitives well enough that Ray’s design stops feeling like magic — and starts feeling like a set of reasonable engineering decisions I can reason about.

The setup

The data is a synthetic sensor frame: a 1080p RGB image (shape 1920×1080×3, ~6MB) and a LiDAR point cloud (65,536 points × 4 floats, ~1MB). Together about 7MB per frame — realistic for an autonomous vehicle simulation tick.

I ran two experiments with the same data:

  • Method A: producer pickles the frame and sends it over multiprocessing.Pipe, consumer unpickles and wraps it in a torch tensor
  • Method B: producer writes into a multiprocessing.shared_memory block, passes only the name and shape metadata over a pipe, consumer attaches to the same block and reads directly

Same data, same two processes, same machine. The only difference is how the bytes travel.

The results (1080p RGB + LiDAR point cloud, ~7.3MB per frame):

MethodThroughputAvg latencyCPU time (10k frames)Effective bandwidth
Pipe + pickle95 fps0.84 ms53.8 s691 MB/s
Python shared memory268 fps0.13 ms1.40 s1,947 MB/s
Python ring buffer263 fps0.12 ms1.30 s1,910 MB/s
C++ lock-free ring buffer261 fps0.11 ms1.23 s1,897 MB/s

The headline number: 2.8× more throughput, 6× lower latency, ~38× less CPU per frame by eliminating pickle and the kernel copies. The C++ ring buffer’s advantage over plain Python shared memory is surprisingly small — because the bottleneck is memory bandwidth (writing 7MB into the shared block), not the coordination mechanism.

(The numbers tell the story — but let’s look at why.)

Method A: Pipe + Pickle

Here’s the producer side:

def producer(conn: Connection, num_frames: int = 300) -> None:
    for _ in range(num_frames):
        rgb, cloud = make_frame()
        payload = pickle.dumps((rgb, cloud), protocol=pickle.HIGHEST_PROTOCOL)
        conn.send_bytes(payload)
    conn.send_bytes(b"__done__")

And the consumer:

def consumer(conn: Connection) -> list[float]:
    while True:
        raw_bytes = conn.recv_bytes()
        if raw_bytes == b"__done__":
            break
        rgb, cloud = pickle.loads(raw_bytes)
        tensor = torch.from_numpy(rgb.copy())

Looks simple. But here’s what the OS is actually doing on every single frame — four distinct steps:

PRODUCER (user space)KERNELCONSUMER (user space)numpy array7.3 MBproducer heaprgb + cloud① pickle.dumps()CPU serializeburns CPU cycles7.3MB bytesserializedproducer heapmalloc’d copy② write()COPY #1syscallcontext switchpipe buffer7.3MB inkernel space2 context switchesper frame③ read()COPY #2syscallcontext switch7.3MB bytesserializedconsumer heapmalloc’d copy④ pickle.loads()CPU deserializeburns CPU cyclesnumpy array7.3 MBconsumer heapready to usePer frame: 2 kernel copies × 7.3MB + 2 CPU serialize/deserialize + 4 context switchesMeasured: 95 fps · 0.84ms avg latency · 691 MB/s · 53.8s CPU per 10,000 frames

There are actually two separate problems bundled together:

  1. Serialization cost — pickle.dumps() and pickle.loads() burn CPU converting Python objects to bytes and back
  2. Two kernel copies — the bytes cross the kernel boundary twice: once on write() into the pipe buffer, once on read() out

Each send_bytes / recv_bytes is a syscall, which forces a context switch from user mode into kernel mode and back. At 60fps that’s 120 context switches per second, before any real work is done.

Full picture — Method A annotated top to bottom:

from multiprocessing import Pipe, Process
import pickle, torch, numpy as np

# ── Setup ─────────────────────────────────────────────────────────────────────
parent_conn, child_conn = Pipe()   # OS creates an anonymous pipe (two file descriptors)

# fork(): child inherits child_conn fd; page table is copied (copy-on-write)
p = Process(target=producer, args=(child_conn, NUM_FRAMES))
p.start()

# ── Producer (child process) ───────────────────────────────────────────────────
def producer(conn, num_frames):
    for _ in range(num_frames):
        rgb, cloud = make_frame()                         # 7.3MB allocated on heap
        payload = pickle.dumps((rgb, cloud), protocol=5) # CPU: walk object graph → malloc new buffer
        conn.send_bytes(payload)                          # syscall write(): 7.3MB → kernel pipe buffer  ← COPY #1
    conn.send_bytes(b"__done__")

# ── Consumer (parent process) ──────────────────────────────────────────────────
def consumer(conn):
    while True:
        raw = conn.recv_bytes()          # syscall read(): blocks until data arrives
                                         # kernel copies 7.3MB → consumer heap              ← COPY #2
        if raw == b"__done__":
            break
        rgb, cloud = pickle.loads(raw)   # CPU: deserialize bytes back to numpy arrays
        tensor = torch.from_numpy(rgb.copy())

# ── Teardown ───────────────────────────────────────────────────────────────────
latencies = consumer(parent_conn)        # blocks here until producer sends __done__
p.join()

The pipe is the only coordination mechanism: data and signal travel together, serialized inside the same bytes.

Method B: Shared Memory

The shared memory version allocates a block once at startup. The producer writes directly into it:

class SharedFrame:
    def write(self, rgb: np.ndarray, cloud: np.ndarray) -> None:
        buf = self._shm.buf
        ptr = np.frombuffer(buf, dtype=np.uint8, count=FRAME_BYTES)
        ptr[:] = rgb.ravel()   # writes directly into shared memory

One subtlety worth noting: ptr[:] = rgb.ravel() is not the same as ptr = rgb.ravel(). The first writes into the shared buffer in-place. The second just rebinds the local variable — the shared buffer is never touched.

The consumer attaches to the same block by name and reads from it:

    def read(self) -> tuple[np.ndarray, np.ndarray]:
        buf = self._shm.buf
        rgb = np.frombuffer(buf, dtype=np.uint8, ...).reshape(FRAME_SHAPE).copy()

What the OS is doing this time:

PRODUCER (user space)PHYSICAL RAMCONSUMER (user space)numpy array7.3MB on heapvaddr: 0x7f1a0000ptr[:] = rgb.ravel()writes into shm① write into shm1 copy · no serializedirect memory writeno syscall neededPhysical RAM7.3MB shared blockphys: 0xABCD0000one allocation · never moves~1,518 × 4KB pages⚡ page fault on first write onlywarm after first frame② mmap view0 copies · 0 serializeOS page table mapssame physical pagesnumpy viewzero-copy readvaddr: 0x7f2b0000np.frombuffer(shm.buf)no new allocationPipe carries only: “zcm_frame_slot” (~20 bytes, not 7.3MB)Per frame: 1 write (heap→shm) · 0 kernel copies · 0 serialize ops · 0 context switchesMeasured: 268 fps · 0.13ms avg latency · 1,947 MB/s · 1.4s CPU per 10,000 frames

The bytes never move. The pipe only carries a short name string — something like "zcm_frame_slot". Both processes map that name to the same physical pages via their own virtual address spaces. Producer’s virtual address A and consumer’s virtual address B are different numbers, but they point to the same physical RAM.

Under the hood, multiprocessing.SharedMemory is wrapping three POSIX calls:

# What Python does for you:
shm_open("zcm_frame_slot", O_CREAT | O_RDWR, 0600)  # create named shm
ftruncate(fd, size)                                   # set size
mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)  # map into VA space

One wrinkle: page faults on first access. The first time the producer writes into the shared block, the OS hasn’t yet allocated physical pages for it — those pages only get faulted in as you touch them. A 1080p RGB frame spans ~1,518 4KB pages, so the first write triggers ~1,518 page faults. Subsequent writes are fast — the OS mapping is already in place. Production pipelines pre-warm the buffer at startup to absorb this cost before real frames arrive.

Full picture — Method B annotated top to bottom:

from multiprocessing import Pipe, Process
from multiprocessing.shared_memory import SharedMemory
import numpy as np, torch

FRAME_BYTES = 1920 * 1080 * 3   # 6.2MB RGB
CLOUD_BYTES = 65536 * 4 * 4     # 1.0MB point cloud
TOTAL_BYTES = FRAME_BYTES + CLOUD_BYTES

# ── Setup ─────────────────────────────────────────────────────────────────────
# shm_open() + ftruncate() + mmap(): OS allocates physical pages, maps into VA space
shm = SharedMemory(name="zcm_frame_slot", create=True, size=TOTAL_BYTES)

parent_conn, child_conn = Pipe()   # pipe carries only signals now, not frame data

p = Process(target=producer_shm, args=(child_conn, NUM_FRAMES))
p.start()

# ── Producer (child process) ───────────────────────────────────────────────────
def producer_shm(conn, num_frames):
    # second mmap to the same backing file → same physical pages, different virtual address
    local_shm = SharedMemory(name="zcm_frame_slot", create=False, size=TOTAL_BYTES)
    buf = np.frombuffer(local_shm.buf, dtype=np.uint8)

    for _ in range(num_frames):
        rgb, cloud = make_frame()
        buf[:FRAME_BYTES] = rgb.ravel()   # user-space write into shared pages — no kernel crossing
        conn.send("ready")                # syscall: ~5 bytes over pipe, NOT the 7.3MB frame  ← signal only

    conn.send(None)
    local_shm.close()

# ── Consumer (parent process) ──────────────────────────────────────────────────
def consumer_shm(shm, conn):
    # same physical pages, consumer's own virtual address
    buf = np.frombuffer(shm.buf, dtype=np.uint8)

    while True:
        msg = conn.recv()              # blocks on the 5-byte signal, not the frame data
        if msg is None:
            break
        # zero-copy read: buf already points at shared pages, no kernel crossing
        rgb = buf[:FRAME_BYTES].reshape(1080, 1920, 3).copy()
        tensor = torch.from_numpy(rgb)

# ── Teardown ───────────────────────────────────────────────────────────────────
latencies = consumer_shm(shm, parent_conn)
p.join()

shm.close()    # munmap: remove this process's virtual mapping
shm.unlink()   # unlink: delete /dev/shm/zcm_frame_slot (backing file)

Pipe carries only the signal ("ready" or None). The 7.3MB frame never touches the kernel.

The signaling problem — and a hidden race condition

Shared memory eliminates copies, but creates two new questions:

  1. How does the consumer know the data is ready?
  2. How does the producer know the consumer has finished reading before it overwrites?

The single-slot code above only solves question 1. It ignores question 2 entirely — and that’s a real bug.

The OS pipe buffers messages in a FIFO queue (~64KB on Linux). The producer never blocks on conn.send("ready") unless that buffer is full. So with a fast producer:

producer: write frame 1 → shm,  send "ready"        # pipe: [ready]
producer: write frame 2 → shm,  send "ready"         # OVERWRITES frame 1  pipe: [ready, ready]
consumer: recv() → "ready", reads shm → gets frame 2  # frame 1 is gone
producer: write frame 3 → shm,  send "ready"         # overwrites again...

One shared slot, multiple buffered signals — every read is racing against the next write.

The naive fix is an explicit ACK — consumer tells producer “I’m done, you can write now”:

# producer
buf[:FRAME_BYTES] = rgb.ravel()
conn.send("ready")
conn.recv()            # ← wait for consumer to finish before overwriting

# consumer
conn.recv()            # wait for signal
rgb = buf[:FRAME_BYTES].reshape(...).copy()
conn.send("ack")       # ← release producer to write next frame

Now producer and consumer strictly take turns. No overwrites, no corrupted reads. But throughput is cut nearly in half — the producer sits idle while the consumer processes, and the consumer sits idle while the producer writes. No pipelining.

Signaling approachRace condition?ThroughputCoordination cost
Single slot + pipe signalYes — producer overwrites before consumer readsFullNone, but data is silently lost
Single slot + explicit ACKNo~Half — producer and consumer take strict turns2 round-trip syscalls per frame
Ring buffer (N slots)NoFull — producer and consumer run concurrentlyLock/atomic on each slot state transition

The ring buffer solves both problems at once. Instead of a single slot, it uses N slots — each with its own state variable embedded in the shared memory itself. Each slot cycles through four states:

EMPTY → WRITING → READY → READING → EMPTY

Here is the complete Python implementation (mutex-based, easier to follow):

EMPTY, WRITING, READY, READING = 0, 1, 2, 3

class PyRingBuffer:
    def __init__(self, name: str, num_slots: int, slot_bytes: int):
        self._num_slots  = num_slots
        self._slot_bytes = slot_bytes
        self._shm        = SharedMemory(name=name, create=True,
                                        size=num_slots * slot_bytes)
        self._states     = Array('i', [EMPTY] * num_slots)  # shared, has built-in lock
        self._write_head = Value('I', 0)
        self._read_head  = Value('I', 0)

    # ── Producer ──────────────────────────────────────────────────

    def acquire_write(self) -> tuple[int, memoryview]:
        while True:
            idx = self._write_head.value % self._num_slots
            with self._states.get_lock():
                if self._states[idx] == EMPTY:
                    self._states[idx] = WRITING
                    self._write_head.value += 1
                    offset = idx * self._slot_bytes
                    return idx, self._shm.buf[offset: offset + self._slot_bytes]
            time.sleep(0)  # yield CPU, wait for consumer to free a slot

    def commit_write(self, slot_idx: int) -> None:
        with self._states.get_lock():
            self._states[slot_idx] = READY

    # ── Consumer ──────────────────────────────────────────────────

    def acquire_read(self) -> tuple[int, memoryview]:
        while True:
            idx = self._read_head.value % self._num_slots
            with self._states.get_lock():
                if self._states[idx] == READY:
                    self._states[idx] = READING
                    self._read_head.value += 1
                    offset = idx * self._slot_bytes
                    return idx, self._shm.buf[offset: offset + self._slot_bytes]
            time.sleep(0)

    def release_read(self, slot_idx: int) -> None:
        with self._states.get_lock():
            self._states[slot_idx] = EMPTY

Here is how the producer and consumer are wired together:

# main process — create ring buffer, spawn producer, run consumer
SLOT_BYTES = FRAME_BYTES + CLOUD_BYTES                        # 7.3MB per slot
rb = PyRingBuffer("zcm_ring", num_slots=8, slot_bytes=SLOT_BYTES)
parent_conn, child_conn = Pipe()

producer_proc = Process(target=producer, args=(rb, child_conn, NUM_FRAMES))
producer_proc.start()
latencies = consumer(rb, parent_conn)
producer_proc.join()
rb.close()
rb.unlink()

# producer process
def producer(rb, conn, num_frames):
    for _ in range(num_frames):
        rgb, cloud = make_frame()
        slot_idx, mv = rb.acquire_write()
        np.copyto(np.frombuffer(mv[:FRAME_BYTES], dtype=np.uint8), rgb.ravel())
        np.copyto(np.frombuffer(mv[FRAME_BYTES:], dtype=np.float32), cloud.ravel())
        rb.commit_write(slot_idx)
        conn.send(slot_idx)         # signal: "slot N is ready"
    conn.send(None)

# consumer process
def consumer(rb, conn):
    while True:
        msg = conn.recv()           # wait for signal
        if msg is None:
            break
        slot_idx, mv = rb.acquire_read()
        rgb = np.frombuffer(mv[:FRAME_BYTES], dtype=np.uint8).reshape(FRAME_SHAPE)
        tensor = torch.from_numpy(rgb.copy())
        rb.release_read(slot_idx)

Notice the pipe is still there — but it now carries a 4-byte slot index, not 7.3MB. The data never touches the kernel. The producer calls acquire_write() to claim a slot, writes directly into the returned memoryview, calls commit_write() to mark it READY, then sends the index as a nudge. The consumer waits for the nudge, calls acquire_read() to get the slot view, reads it zero-copy, and calls release_read() to free the slot.

What Ray actually does

Once you understand Method B, Ray’s Plasma object store is immediately recognisable. The OS mechanism is identical — Ray just productionises the parts I left manual.

Scope: everything below describes workers on the same node as the Plasma store, for large objects (> 100KB). Objects smaller than 100KB are stored directly on the owner worker’s heap (the “in-process store”) and copied by value to any reader — no shared memory involved. For the 7.3MB frames in this post, Plasma always applies. Workers on a different node trigger a network transfer first — the local raylet fetches the object from the remote raylet over gRPC, depositing a copy into the receiving node’s Plasma store. Once that copy exists locally, the same mmap path applies.

My experimentRayname string”zcm_frame_slot”identifierobject ID28-byte unique ID (task-derived, not content hash)shared memory blockraw bytes, 1 block per framestoragePlasma storemanaged pool, Apache Arrow formatnp.frombuffer(shm.buf)zero-copy view into shared pagesreadray.get(object_id)same-node: mmap same pagesmanual close() + unlink()you decide when to freelifetimedistributed reference countingfreed when no ObjectRef in scope

The physical law doesn’t change — for same-node access. A tensor still moves from producer heap into the shared block exactly once. The consumer’s virtual address is still a different number pointing to the same physical pages. Ray wraps all of this in a production API, but the OS primitive underneath is identical to what this experiment does.

Under the hood: how Plasma avoids locks

The key difference from the ring buffer is that Plasma objects are immutable once written. A worker writes into a mutable Plasma buffer; the worker’s own CoreWorker then seals it — a one-way transition that makes it permanently read-only. (The raylet embeds the Plasma store and manages the memory pool, but the seal call is issued by the producing worker’s CoreWorker, not the raylet itself.) After sealing, any number of workers can call ray.get() simultaneously and each gets an mmap view to the same physical pages. Because no writer can ever touch the buffer again, concurrent reads need zero coordination.

One practical consequence of immutability: ray.get() returns numpy arrays as read-only views into the shared buffer. Any attempt to write to them raises an error. That’s why production Ray code nearly always ends with arr = arr.copy() before doing in-place work — the copy buys writability at the cost of one extra allocation, just like the .copy() calls in Method B above.

① Write (single writer)② Seal③ Read (unlimited concurrent)Workertask executorone writer onlyper objectPlasmaMUTABLEwriting…raylet serializescreation requestsseal()one-wayno undoPlasmaIMMUTABLE0xABCD0000no writer cantouch thismmapWorker Avaddr 0x7f1a… → same pagesWorker Bvaddr 0x7f2b… → same pagesWorker Cvaddr 0x7f3c… → same pages0 locks neededSeal makes write and read non-overlapping by construction — no writer exists during reads, so nothing can raceRing buffer: slots reused → needs lock/atomic on every state transitionPlasma: immutable after seal → concurrent reads are trivially safe, no coordination needed

This is the opposite trade-off from the ring buffer:

Ring bufferPlasma (Ray)
Slot lifecycleREADING → EMPTY → WRITING (reused)Write once, sealed → immutable forever
Concurrent readers1 reader per slot at a timeUnlimited — no coordination needed
CoordinationLock or atomic on every state transitionNone after seal
Memory useFixed N-slot pool, boundedObjects live until all refs hit zero across cluster
Best suited forStreaming pipelines with fixed-size framesArbitrary objects shared across many workers

When to free: distributed reference counting

Since objects are never overwritten, Ray needs a distributed protocol to decide when to free the physical pages. Every ObjectRef has an owner — the worker that created it — who tracks five reference count components. The object is freed only when all reach zero across the entire cluster.

Owner WorkerTracks 5 ref count components per ObjectRef:local Python count · submitted task count · borrower set · nested count · lineage countObject freed when ALL = 0 across ALL processes in the clusterpass as task argpass as task argBorrower Amaintains own local ref countnotifies owner when count → 0(async RPC back to owner)Borrower Bmaintains own local ref countnotifies owner when count → 0(can recursively pass to Borrower C)count = 0count = 0Raylet frees /dev/shm pages only after owner + all borrowers (recursively) have reported count = 0

When an ObjectRef is passed to another process as a task argument, that process becomes a borrower. The owner adds it to a borrower set and waits for a “count = 0” report via async RPC. Borrowers chain recursively — if B passes the ref to C, B includes C in its reply to the owner, who then contacts C directly. Only once every count is zero does the raylet free the /dev/shm pages.

Putting it all together

The diagram below maps the five lifecycle steps — allocate, write, signal, read, free — across both approaches side by side. The amber box is where the designs diverge most sharply: a raw pipe signal versus Plasma’s seal primitive.

My Experiment (Method B)shared memory + pipe signalRay (same node)Plasma object store— KEY DIFFERENCE —1AllocateSharedMemory(name=‘zcm’, create=True)shm_open() + ftruncate() + mmap()pages lazy-faulted on first write1AllocatePlasma pool pre-allocated at Ray startupraylet owns shm_open()+mmap() for poolobjects carved from fixed-size pool2Writebuf[:FRAME_BYTES] = rgb.ravel()heap -> shm: 1 user-space copyno kernel crossing2Writedata written to: Plasma buffer (raylet shared memory, /dev/shm)heap -> Plasma: 1 user-space copyno kernel crossing3Signalconn.send(‘ready’) # pipe syscallcarries ~5 bytes, not 7.3 MBno ObjectRef concept — raw signal[!] single slot: race condition risk3Seal + ObjectRefworker seals Plasma buffer: MUTABLE->IMMUTABLEmetadata written to: owner CoreWorker heap ownership table: ID->Plasma addr, rc=1ObjectRef = {28-byte ID + owner addr}multi-reader safe by construction4Readnp.frombuffer(shm.buf).reshape(…)mmap: same physical pages as producerzero-copy, no kernel crossing4Readmetadata read from: owner CoreWorker heap ownership table -> finds Plasma addrdata read from: Plasma buffer via mmap(fd) same physical pages, zero-copy5Freeshm.close() # munmapshm.unlink() # delete /dev/shmmanual: you decide when to free5Freeownership table tracks ref countborrowers report back via async RPCcount=0 across all -> Plasma frees pagesautomatic, distributed GC

Summary

Pipe + pickleShared memory
Serializationpickle dumps/loads (CPU cost)None
Data copies2 (kernel in + kernel out)0
SignalingImplicit — data is the signalExplicit — name string or atomic flag
N producersSafe — pipe handles queuingNeeds MPSC design
BackpressureNone — pipe buffer absorbs burstsExplicit — drop or pause producer
Real-world example—Ray Plasma, PyTorch DataLoader shm mode

The core insight is that “passing a tensor between processes” is not one problem — it’s three: serialization, copying, and signaling. Pipe + pickle bundles all three into one opaque call. Shared memory forces you to handle each one explicitly, which is exactly why it’s worth understanding.