This binding encodes Core Semantics over shared memory. It does NOT redefine the meaning of calls, channels, errors, or flow control — only their representation in this transport.

This binding assumes:

• All processes sharing the segment run on the same architecture (same endianness, same word size, same atomic semantics)
• Cross-process atomics are valid (typically true on modern OSes)
• The shared memory region is cache-coherent

Cross-architecture SHM is not supported.

The hub topology has exactly one host and zero or more guests. The host creates and owns the shared memory segment. Guests attach to communicate with the host.

Guests communicate only with the host, not with each other. Each guest has its own rings and slot pool within the shared segment.

Either the host or a guest can initiate calls. The host can call methods on any guest; a guest can call methods on the host.

A guest's peer_id (u8) is 1 + the index of its entry in the peer table. Peer table entry 0 corresponds to peer_id = 1, entry 1 to peer_id = 2, etc. The host does not have a peer_id (it is not in the peer table).

The maximum number of guests is limited to 255 (peer IDs 1-255). The max_guests field in the segment header MUST be ≤ 255. The peer table has exactly max_guests entries.

SHM encodes request_id as u32. The upper 32 bits of Core's u64 request_id are implicitly zero. Implementations MUST NOT use request IDs ≥ 2^32.

SHM encodes channel_id as u32. The upper 32 bits of Core's u64 channel_id are implicitly zero.

Channel IDs are scoped to the guest-host pair. Two different guests may independently use the same channel_id value without collision because they have separate channel tables.

Within a guest-host pair, channel IDs use odd/even parity to prevent collisions:

• The host allocates even channel IDs (2, 4, 6, ...)
• The guest allocates odd channel IDs (1, 3, 5, ...)

Channel ID 0 is reserved and MUST NOT be used.

Request IDs are scoped to the guest-host pair. Two different guests may use the same request_id value without collision because their rings are separate.

SHM does not use Hello messages. Instead, the segment header fields (max_payload_size, initial_credit, max_channels) serve as the host's unilateral configuration. Guests accept these values by attaching to the segment.

Unlike networked transports, SHM has no negotiation — the host's values are authoritative. A guest that cannot operate within these limits MUST NOT attach.

The segment MUST begin with a header:

The segment header is 128 bytes.

The magic field MUST be exactly RAPAHUB\x01 (8 bytes).

The peer table contains one entry per potential guest:

Peer states:

• Empty (0): Slot available for a new guest
• Attached (1): Guest is active
• Goodbye (2): Guest is shutting down or has crashed
• Reserved (3): Host has allocated slot, guest not yet attached (see r[shm.spawn.reserved-state])

Each guest has two descriptor rings:

• Guest→Host ring: Guest produces, host consumes
• Host→Guest ring: Host produces, guest consumes

At the offset specified by PeerEntry.ring_offset:

1. Guest→Host ring: ring_size * 64 bytes (ring_size descriptors)
2. Host→Guest ring: ring_size * 64 bytes (ring_size descriptors)

Both rings are contiguous. Total size per guest: 2 * ring_size * 64 bytes.

Each ring MUST be aligned to 64 bytes (cache line). Since descriptors are 64 bytes and rings are contiguous, this is naturally satisfied if ring_offset is 64-byte aligned.

On segment creation, all ring memory MUST be zeroed. On guest attach, the guest MUST NOT assume ring contents are valid — it should wait for head != tail before reading.

A ring can hold at most ring_size - 1 descriptors. The ring is full when (head + 1) % ring_size == tail. The ring is empty when head == tail.

If the ring is full, the producer MUST wait before enqueueing. Implementations SHOULD use futex on the tail index to avoid busy-wait. Ring fullness is not a protocol error — it indicates backpressure from a slow consumer.

Each guest has a dedicated pool of slots_per_guest payload slots. Slots are used for payloads that exceed inline capacity.

Slots from a guest's pool are used for messages sent by that guest. After the host processes a message, the slot is returned to the guest's pool.

Each slot pool (host or guest) has the same size: pool_size = slot_pool_header_size + slots_per_guest * slot_size where slot_pool_header_size is the bitmap header rounded up to 64 bytes (see r[shm.slot.pool-header-size]).

The host has its own slot pool for messages it sends to guests. The host slot pool is located at offset slot_region_offset in the segment (position 0), before the per-guest slot pools.

A guest with peer_id = P (where P ≥ 1) has its slot pool at: slot_region_offset + P * pool_size

Message descriptors MUST be exactly 64 bytes (one cache line).

The flags field is reserved for future use and MUST be zero. Receivers SHOULD ignore this field (do not reject non-zero values) to allow forward compatibility.

For inline payloads (payload_slot == 0xFFFFFFFF):

• payload_generation MUST be 0
• payload_offset MUST be 0
• payload_len indicates bytes used in inline_payload

For Request and Response messages, the descriptor's payload contains both metadata and arguments/result, encoded as a single [POSTCARD] value:

struct RequestPayload {
    metadata: Vec<(String, MetadataValue)>,
    arguments: T,  // method arguments tuple
}

struct ResponsePayload {
    metadata: Vec<(String, MetadataValue)>,
    result: Result<T, RoamError<E>>,
}

This differs from other transports where metadata and payload are separate fields in the Message enum.

The limits from r[call.metadata.limits] apply: at most 128 keys, each value at most 16 KB. Violations are connection errors.

The msg_type field identifies the abstract message:

Payloads MUST be [POSTCARD]-encoded.

If payload_len <= 32, the payload MUST be stored inline and payload_slot MUST be 0xFFFFFFFF.

If payload_len > 32, the payload MUST be stored in a slot from the sender's pool.

A slot pool is an array of slots, each slot_size bytes. Before the slots is a slot header:

#[repr(C)]
struct SlotPoolHeader {
    free_bitmap: [AtomicU64; N],  // 1 bit per slot, 1 = free
}

The bitmap size N = ceil(slots_per_guest / 64). Slots are numbered 0 to slots_per_guest - 1. Bit i of word i / 64 represents slot i.

The slot pool header is padded to a multiple of 64 bytes for alignment. Slot 0 begins immediately after the header.

To allocate a slot:

1. Scan the free_bitmap for a set bit (any strategy: linear, random)
2. Atomically clear the bit (CAS from 1 to 0)
3. If CAS fails, retry with another slot
4. Increment the slot's generation counter
5. Write payload to the slot

To free a slot:

1. Set the corresponding bit in free_bitmap (atomic OR)

The receiver frees slots after processing the message. This returns the slot to the sender's pool.

Each slot's first 4 bytes are an AtomicU32 generation counter, incremented on allocation. The usable payload area is slot_size - 4 bytes starting at byte 4 of the slot. The receiver verifies payload_generation matches to detect ABA issues.

The payload_offset field in MsgDesc is relative to the payload area (after the generation counter), not the slot start. A payload_offset of 0 means the payload begins at byte 4 of the slot.

If no free slots are available, the sender MUST wait. Use futex on a bitmap word or poll with backoff. Slot exhaustion is not a protocol error — it indicates backpressure.

When enqueueing a descriptor:

1. Write descriptor and payload with Release ordering
2. Increment ring head with Release ordering

When dequeueing a descriptor:

1. Load head with Acquire ordering
2. If head != tail, load descriptor with Acquire ordering
3. Process message
4. Increment tail with Release ordering

Consumer waiting for messages (ring empty):

• Wait: futex_wait on ring head when head == tail
• Wake: Producer calls futex_wake on head after incrementing it

Producer waiting for space (ring full):

• Wait: futex_wait on ring tail when (head + 1) % ring_size == tail
• Wake: Consumer calls futex_wake on tail after incrementing it

Sender waiting for credit (zero remaining):

• Wait: futex_wait on ChannelEntry.granted_total
• Wake: Receiver calls futex_wake on granted_total after updating

Sender waiting for slots (pool exhausted):

• Wait: futex_wait on a bitmap word (implementation-defined which)
• Wake: Receiver calls futex_wake on that word after freeing a slot

On non-Linux platforms, use polling with exponential backoff or platform-specific primitives (e.g., WaitOnAddress on Windows).

Each guest-host pair has a channel metadata table for tracking active channels. The table is located at a fixed offset within the guest's region:

Each guest's channel table offset is stored in PeerEntry.channel_table_offset. The table size is max_channels * 16 bytes.

The channel_id directly indexes the channel table: channel N uses entry N. This means:

• Channel IDs MUST be < max_channels
• Channel ID 0 is reserved; entry 0 is unused
• Usable channel IDs are 1 to max_channels - 1

When opening a new channel, the allocator MUST initialize the entry:

1. Set granted_total = initial_credit (from segment header)
2. Set state = Active (with Release ordering)

The sender maintains its own sent_total counter locally (not in shared memory).

A channel ID MAY be reused after the channel is closed (Close or Reset received by both peers). To reuse:

1. Sender sends Close or Reset
2. Receiver sets ChannelEntry.state = Free (with Release ordering)
3. Allocator polls for state == Free before reusing

On reuse, the allocator reinitializes per r[shm.flow.channel-activate].

Implementations SHOULD delay reuse to avoid races (e.g., wait for the entry to be Free before reallocating).

Each active channel has a granted_total: AtomicU32 counter in its channel table entry. The receiver publishes; the sender reads.

granted_total is cumulative bytes authorized by the receiver. Monotonically increasing (modulo wrap).

remaining = granted_total - sent_total (wrapping subtraction). Sender MUST NOT send if remaining < payload size.

Interpret granted_total - sent_total as signed i32. Negative or

2^31 indicates corruption.

Update granted_total with Release after consuming data.

Load granted_total with Acquire before deciding to send.

Channels start with granted_total = initial_credit from segment header. Sender's sent_total starts at 0.

Sender waits. Use futex on the counter to avoid busy-wait. Receiver wakes after granting credit.

After Reset, stop accessing the channel's credit counter. Values after Reset are undefined.

To attach, a guest:

1. Opens the shared memory segment
2. Validates magic and version
3. Finds an Empty peer table entry
4. Atomically sets state from Empty to Attached (CAS)
5. Increments epoch
6. Begins processing

If no Empty slots exist, the guest cannot attach (hub is full).

To detach gracefully:

1. Set state to Goodbye
2. Drain remaining messages
3. Complete or cancel in-flight work
4. Unmap segment

The host periodically checks peer states. On observing Goodbye or epoch change (crash), the host cleans up that guest's resources.

A guest signals shutdown by setting its peer state to Goodbye. It MAY send a Goodbye descriptor with reason first.

The host signals shutdown by setting host_goodbye in the header to a non-zero value. Guests MUST poll this field and detach when it becomes non-zero.

A Goodbye descriptor's payload is a [POSTCARD]-encoded String containing the reason. Per r[core.error.goodbye-reason], the reason MUST contain the rule ID that was violated.

The host_goodbye field MUST be accessed atomically (load/store with at least Relaxed ordering). It is written by the host and read by all guests.

The host MUST use an out-of-band mechanism to detect crashed guests. Common approaches:

• Hold a process handle (e.g., pidfd on Linux, process handle on Windows) and detect termination
• Require guests to update a heartbeat field periodically
• Use OS-specific death notifications

If using heartbeats: each PeerEntry contains a last_heartbeat: AtomicU64 field. Guests MUST update this at least every heartbeat_interval nanoseconds (from segment header). The host declares a guest crashed if heartbeat is stale by more than 2 * heartbeat_interval.

Heartbeat values are monotonic clock readings, not wall-clock time. All processes read from the same system monotonic clock, so values are directly comparable without synchronization.

Each process writes its current monotonic clock reading (in nanoseconds) to last_heartbeat. The host compares the guest's value against its own clock reading: if host_now - guest_heartbeat > 2 * heartbeat_interval, the guest is considered crashed.

Platform clock sources:

• Linux: CLOCK_MONOTONIC (via clock_gettime or Instant)
• Windows: QueryPerformanceCounter
• macOS: mach_absolute_time

Guests increment epoch on attach. If epoch changes unexpectedly, the previous instance crashed and was replaced.

On detecting a crashed guest, the host MUST:

1. Set the peer state to Goodbye
2. Treat all in-flight operations as failed
3. Reset rings to empty (head = tail = 0)
4. Return all slots to free
5. Reset channel table entries to Free
6. Set state to Empty (allowing new guest to attach)

For flow control, "bytes" = payload_len of Data descriptors (the [POSTCARD]-encoded element size). Descriptor overhead and slot padding do NOT count.

The host creates the segment as a regular file at a path known to both host and guests. Common locations:

• /dev/shm/<name> (Linux tmpfs, recommended)
• /tmp/<name> (portable but may be disk-backed)
• Application-specific directory

The path MUST be communicated to guests out-of-band (e.g., via command-line argument or environment variable).

To create a segment file:

1. Open or create the file with read/write permissions
2. Truncate to the required total_size
3. Memory-map the entire file with MAP_SHARED
4. Initialize all data structures (header, peer table, rings, slots)
5. Write header magic last (signals segment is ready)

To attach to an existing segment:

1. Open the file read/write
2. Memory-map with MAP_SHARED
3. Validate magic and version
4. Read configuration from header
5. Proceed with guest attachment per r[shm.guest.attach]

The segment file SHOULD have permissions that allow all intended guests to read and write. On POSIX systems, mode 0600 or 0660 is typical, with the host and guests running as the same user or group.

The host SHOULD delete the segment file on graceful shutdown. On crash, stale segment files may remain; implementations SHOULD handle this (e.g., by deleting and recreating on startup).

On POSIX systems, use mmap() with:

• PROT_READ | PROT_WRITE
• MAP_SHARED (required for cross-process visibility)
• File descriptor from open() or shm_open()

On Windows, use:

• CreateFileMapping() to create a file mapping object
• MapViewOfFile() to map it into the process address space
• Named mappings can use Global\<name> for cross-session access

Before spawning a guest, the host:

1. Allocates a peer table entry (finds Empty slot, sets to Reserved)
2. Creates a doorbell pair (see Doorbell section)
3. Prepares a "spawn ticket" containing:
   • hub_path: Path to the segment file
   • peer_id: Assigned peer ID (1-255)
   • doorbell_fd: Guest's end of the doorbell (Unix only)

The peer entry state during spawning:

• Host sets state to Reserved before spawn
• Guest sets state to Attached after successful attach
• If spawn fails, host resets state to Empty

The Reserved state prevents other guests from claiming the slot.

The canonical way to pass spawn ticket information to a guest process is via command-line arguments:

--hub-path=<path>    # Path to segment file
--peer-id=<id>       # Assigned peer ID (1-255)
--doorbell-fd=<fd>   # Doorbell file descriptor (Unix only)

On Unix, the doorbell file descriptor MUST be inheritable by the child process. The host MUST NOT set O_CLOEXEC / FD_CLOEXEC on the guest's doorbell fd before spawning. After spawn, the host closes its copy of the guest's doorbell fd (keeping only its own end).

A spawned guest process:

1. Parses command-line arguments to extract ticket info
2. Opens and maps the segment file
3. Validates segment header
4. Locates its peer entry using peer_id
5. Verifies state is Reserved (set by host)
6. Atomically sets state from Reserved to Attached
7. Initializes doorbell from the inherited fd
8. Begins message processing

A doorbell is a bidirectional notification channel between host and guest that provides:

• Wakeup: Signal the other side to check for work
• Death detection: Detect when the other process terminates

Unlike futex (which requires polling shared memory), a doorbell allows blocking on I/O that unblocks immediately when the peer dies.

On Unix, doorbells are implemented using socketpair():

int fds[2];
socketpair(AF_UNIX, SOCK_STREAM, 0, fds);
// fds[0] = host end, fds[1] = guest end

The host keeps fds[0] and passes fds[1] to the guest via the spawn ticket.

To signal the peer, write a single byte to the socket:

char byte = 1;
write(doorbell_fd, &byte, 1);

The byte value is ignored; only the wakeup matters.

To wait for a signal (with optional timeout):

struct pollfd pfd = { .fd = doorbell_fd, .events = POLLIN };
poll(&pfd, 1, timeout_ms);
if (pfd.revents & POLLIN) {
    // Peer signaled - check for work
    char buf[16];
    read(doorbell_fd, buf, sizeof(buf));  // drain
}
if (pfd.revents & (POLLHUP | POLLERR)) {
    // Peer died
}

When a process terminates, its end of the socketpair is closed by the kernel. The surviving process sees POLLHUP or POLLERR on its end, providing immediate death notification without polling.

Doorbells complement ring-based messaging:

• After enqueueing a descriptor and updating head, signal the doorbell
• The receiver can poll() both the doorbell fd and other I/O
• On doorbell signal, check rings for new messages

This avoids busy-waiting and integrates with async I/O frameworks.

Doorbell support is OPTIONAL. Implementations MAY use only futex-based wakeup (per r[shm.wakeup.*]). Doorbells are recommended when:

• Death detection latency is critical
• Integration with async I/O (epoll/kqueue/IOCP) is desired
• Busy-waiting must be avoided entirely

When adding a peer, the host MAY register a death callback:

type DeathCallback = Arc<dyn Fn(PeerId) + Send + Sync>;

struct AddPeerOptions {
    peer_name: Option<String>,
    on_death: Option<DeathCallback>,
}

The callback is invoked when the guest's doorbell indicates death (POLLHUP/POLLERR) or when heartbeat timeout is exceeded.

The death callback:

• Is called from the host's I/O or monitor thread
• Receives the peer_id of the dead guest
• SHOULD NOT block for long (schedule cleanup asynchronously)
• MAY trigger guest restart logic

Implementations SHOULD use multiple detection methods:

Doorbell provides the best latency on Unix. Process handles (pidfd on Linux, process handle on Windows) provide immediate notification on all platforms.

On Linux 5.3+, use pidfd_open() to get a pollable fd for the child process. On Windows, the process handle from CreateProcess() is waitable. This provides kernel-level death notification without relying on doorbells.

On guest death detection, per r[shm.crash.recovery]:

1. Invoke the death callback (if registered)
2. Set peer state to Goodbye, then Empty
3. Reset rings and free slots
4. Close host's doorbell end
5. Optionally respawn the guest

A variable-size pool consists of multiple size classes, each with its own slot size and count. Example configuration:

The specific configuration is application-dependent.

To allocate a slot for a payload of size N:

1. Find the smallest size class where slot_size >= N
2. Allocate from that class's free list
3. If exhausted, try the next larger class (optional)
4. If all classes exhausted, block or return error

Unlike fixed-size per-guest pools, variable-size pools are typically shared across all guests:

• One pool region for the entire hub
• All guests allocate from the same size classes
• Slot ownership is tracked per-allocation

This allows efficient use of memory when different guests have different payload size distributions.

Each slot tracks its current owner:

struct SlotMeta {
    generation: AtomicU32,  // ABA counter
    state: AtomicU32,       // Free=0, Allocated=1, InFlight=2
    owner_peer: AtomicU32,  // Peer ID that allocated (0 = host)
    next_free: AtomicU32,   // Free list link
}

When a guest crashes, slots with owner_peer == crashed_peer_id are returned to their respective free lists.

Size classes can grow dynamically via extents:

• Each size class starts with one extent of slot_count slots
• When exhausted, additional extents can be allocated
• Extents are appended to the segment file (requires remap)

struct SizeClassHeader {
    slot_size: u32,
    slots_per_extent: u32,
    extent_count: AtomicU32,
    extent_offsets: [AtomicU64; MAX_EXTENTS],
}

Each extent contains:

1. Extent header (class, index, slot count, offsets)
2. Slot metadata array (one SlotMeta per slot)
3. Slot data array (actual payload storage)

┌─────────────────────────────────────────────────┐
│ ExtentHeader (64 bytes)                         │
├─────────────────────────────────────────────────┤
│ SlotMeta[0] │ SlotMeta[1] │ ... │ SlotMeta[N-1] │
├─────────────────────────────────────────────────┤
│ Slot[0] data │ Slot[1] data │ ... │ Slot[N-1]   │
└─────────────────────────────────────────────────┘

Each size class maintains a lock-free free list using a Treiber stack:

struct SizeClassHeader {
    // ...
    free_head: AtomicU64,  // Packed (index, generation)
}

Allocation pops from the head; freeing pushes to the head. The generation counter prevents ABA problems.

To allocate from a size class:

1. Load free_head with Acquire
2. If empty (sentinel value), class is exhausted
3. Load the slot's next_free pointer
4. CAS free_head from current to next
5. On success, increment slot's generation, set state to Allocated
6. On failure, retry from step 1

To free a slot:

1. Verify generation matches (detect double-free)
2. Set slot state to Free
3. Load current free_head
4. Set slot's next_free to current head
5. CAS free_head to point to this slot
6. On failure, retry from step 3