# theprototype.app
> Collaborative peer-to-peer 3D prototyping in the browser. No account needed —
> a session is a direct WebRTC peer mesh (servers only introduce peers and relay
> when a direct link fails), so everything one person does is visible to everyone
> connected. The hosted site adds optional accounts, cloud saves and community pages. Desktop and VR
> (WebXR), a node/flow editor that drives scene behavior, and a module SDK for
> loadable, self-contained play content. Built with SvelteKit (static adapter),
> Svelte 5, Threlte and three.js.
## Modules (write your own)
- [Module SDK guide, inlined](/llms-full.txt): the full guide — the api surface,
the one rule that matters (a module runs on every peer), layout/registration,
and worked examples. Fetch this to write a correct module without guessing the
API.
## Agents: join a session
- An AI agent can join a live session from its link and build alongside people:
`claude mcp add theprototype -- npx -y @theprototype/agent mcp ''`
(any MCP client: `npx -y @theprototype/agent mcp `). The bridge runs the
real app headless and exposes its tools.
- In the page: `window.tpAgent` (only with `&agent` in the hash or
`localStorage.agentApi`) — `tools()` (full JSON schemas), `systemPrompt()`,
`exec(name, args)` (replicated, one undo step), `snapshot()`, `status()`,
`waitForEdit(ms)`, `screenshot({view, uuid, size})`, `pendingOffers()`,
`accept(id)`, `leave()`.
- A plain link waits for the host's Approve; an invite link (`&invite=…`) joins
in one step. A VIEW-ONLY agent's edits are refused — check `status().canEdit`.
- [Agent guide, inlined](/llms-full.txt): the whole contract, the view-only
trap and the replication rules an agent must respect (AGENT-SESSIONS.md).
- [Build with an AI agent](https://docs.theprototype.app/ai/build-with-an-agent/):
the same for people.
## Key concepts
- Everything replicates over the peer mesh: apply the change locally, then
broadcast it; a receiver applies it WITHOUT re-broadcasting.
- Two sync models — deterministic (a pure function of a seed/params plus a
shared synced clock) vs authoritative (one initiator simulates and
broadcasts). Pick one per feature; never mix them.
- A module is a self-contained default export
`{ id, name, version, description, register(api) }` in
`src/modules//module.js`; user modules (zip/URL) must have no imports.
- Script and flow nodes are pure functions of `(object, base, data, time)`
evaluated identically on every peer — never stream their outputs.
- Randomness must be seeded: send the seed, not the result, or peers desync.
## Docs
- [llms-full.txt](/llms-full.txt): this map plus the entire module SDK guide
and the agent guide inline (works offline / without repo access).
- [Source repository](https://github.com/theprototype-app/core): MIT-licensed
code; CLAUDE.md in the repo root documents the architecture, replication
rules and conventions agents should follow when contributing.
- [README](https://github.com/theprototype-app/core#readme): feature overview,
quick start, self-hosting notes.
- [Module SDK guide, raw](https://raw.githubusercontent.com/theprototype-app/core/main/MODULES.md):
the same guide as llms-full.txt, straight from the repo.
- [Releases](https://github.com/theprototype-app/core/releases): changelog per
version.
- [User docs](https://docs.theprototype.app): guides for people (not agents).
---
# Agents: join a session and build
An AI agent (Claude Code, Claude Desktop, or any MCP client) can join a live theprototype.app
session from a link and build in it alongside people. The agent is a peer like everyone else:
everything it does goes through the same replicated, undoable paths a person's edits use.
## The setup: one line
Someone in the session hands you a link, e.g. `https://theprototype.app/#F0FD3`.
```sh
claude mcp add theprototype -- npx -y @theprototype/agent mcp 'https://theprototype.app/#F0FD3'
```
(Claude Desktop: add `npx -y @theprototype/agent mcp ` to `mcpServers`.) Node 20+. The
bridge opens the link in a headless browser (the real app, at the app's own version), joins, and
exposes the app's tools to your MCP client. A bare peer id (`F0FD3`) works too.
Then, from your client: call `status`, call `tools` (or read the MCP tool list), and build with
the tools. Three calls from a link to a first build.
## Getting in: approval, invites, and the view-only trap
- **A plain link** knocks: the host sees an approval card and presses Approve. On cloud builds
(theprototype.app with accounts) Approve may admit you **view-only**.
- **An invite link** (`#F0FD3&agent&invite=…`, from Connect ▸ *Request AI* on the host) joins in
one step with the role the host chose (Edit by default), single-use, short-lived. The token lives
in the URL hash, which browsers never send to servers; it is never printed or logged.
- **View-only means view-only.** A viewer's edits are REFUSED with
`{error: "View-only: …", refused: true, role: "viewer"}` — nothing is created anywhere. Ask the
host to change your role (Users ▸ role) or ask for an invite that grants edit. `status().role`
and `status().canEdit` say where you stand; read tools (`list_scene`) still work.
## The in-page API: `window.tpAgent` (contract 1.x)
The bridge drives this object; any driver that can evaluate JavaScript in the page can too. It
exists ONLY when the URL hash carries `&agent` (`#F0FD3&agent`) or `localStorage.agentApi = '1'`;
a normal session never loads it. Semver: new methods and fields are minor versions.
| member | what it does |
|---|---|
| `version` | the contract version, e.g. `"1.1.0"` |
| `tools()` | the in-app assistant's own tools, OpenAI function shape with FULL JSON schemas, plus the agent-only tools below |
| `systemPrompt()` | the assistant's guidance (coordinates, primitives, flow-node vocabulary, physics rules) — give it to your model |
| `exec(name, args)` | run one tool. Replicated to every peer, ONE undo step per call, calls run one at a time, never throws (`{error}` instead) |
| `snapshot(cap?)` | the scene as the assistant sees it: objects (uuid, name, transform, material, physics, flow graph), scene flow, behaviours, joints |
| `status()` | `{version, appVersion, peerId, host, joined, waiting, role, canEdit, peers, missingTools, missingModules, pendingOffers, inVR, scene, hostScene, withHost, following, loading, objects, behaviourErrors, notices}` — `joined` turns true once the host let you in AND (on a build with roles) its roles reached you, so `role`/`canEdit` are final from then on. `withHost` (1.1) is false when the host went to another scene (a Save as, opening a scene): the agent follows on its own (`following`), and `exec` refuses edits until it is there — they would not reach the host. `loading` (1.2) is true while a scene is still arriving (right after a join, or a follow), with `objects: {arrived, total}` (from the moment you join until the host's snapshot has landed) — a tool called meanwhile waits for it (15 s at most). `hostAway` (1.2): the host LEFT and no person is here — the agent is paused (edits refused, `withHost` false), knocks on the host's door until `hostAway.until` (10 minutes) and never hosts or saves; when a person remains instead it simply continues with them (`host` changes, no new invite). `behaviourErrors` (1.2, only when any) lists what the scene's code behaviours threw (`{name, where, message}`, from whichever peer runs them): fix them with `edit_behaviour`. `create_behaviour`/`edit_behaviour` also return `errors` + a `warning` when the new code threw in its first moments |
| `waitForEdit(ms?)` | resolves `{changed: true, at}` on the next scene change made by someone else, or `{changed: false, timedOut: true}` (default 30 s, max 10 min); your own `exec` never wakes it |
| `screenshot({view, uuid, size, grid})` | a clean PNG data URL (no gizmo, selection, grid or helpers) from a TEMPORARY camera: `view` = `iso` (default) · `top` · `front` · `current` (the page's own camera) · `focus` (frames `uuid`); `size` = longest side, 64–2048 (default 768); `grid: true` draws the editor grid. Never moves anyone's view. `null` in VR |
| `pendingOffers()` / `accept(id, action?)` | questions the app asked with buttons (an admin-role offer, "share or stash"), kept until answered — the toast that showed one may have expired |
| `leave()` | a graceful goodbye: every peer drops you at once, no ghost in Users |
**Agent-only tools** (in `tools()`, run locally, allowed for viewers): `install_modules` (install the
modules other peers run that this page lacks — `status().missingModules`), `pending_offers`,
`accept_offer`.
**The full tool set.** In an agent session the physics tools (`set_physics`, `create_joints`,
`control_simulation` and the physics flow nodes) are always offered. `generate_mesh` is offered
when the HOST has a mesh provider ready (the call runs on the host's provider — you never bring
keys into the page — and the host sees who asked and can cancel). `control_simulation` also runs on
the HOST: an agent page never steps the physics itself (its result carries `onHost: true`).
`status().missingTools` says why any tool is absent. `update_objects` reports a field that does not
apply (a light field on a mesh, a bad material value) in that row's `errors` with `ok: false`.
`create_flow_nodes` refuses a wire whose `toHandle` (or, on a node with several outputs, `fromHandle`) the node
does not have; the error lists the node's real socket names and the call's other wires are still made.
`create_behaviour`'s description carries the behaviour format and a working example; its checker refuses an
unknown `this.` member, an unknown event and a hook-named method nothing calls (there is no per-frame hook).
`list_scene` gives a primitive's shape as `geometry {type, sizes}` (a wall's thickness, a box's depth) and can be
narrowed on a big scene: `brief` (uuids and names only), `name` (a glob such as `"Rock*"`, case-insensitive), `type`,
`near {position, radius}` (world metres); a narrowed read says how many objects `matched`. Positions are LOCAL to the
parent: an object created or moved with `parentUuid` takes its position inside that group, as `list_scene` reports it.
## Rules a builder must respect
- **Use real uuids.** Read `snapshot()` (or `list_scene`) first; edit by uuid.
- **Batch.** Put every object of a request into ONE `create_objects` call — one undo step, one
replication burst. A campfire with six rocks and a flame is one call with seven objects.
- **Y is up, metres, radians.** The ground is y = 0; an object of height h rests at y = h/2.
- **Motion is flow nodes.** A behaviour node (spin, bounce, pathpatrol, blink…) in an OBJECT's
graph with no edges drives that object: one node, zero edges. Logic nodes (latch, delay,
sequence, counter) need wires.
- **Static scenery.** New primitives may spawn dynamic; pin ground, walls and props with
`set_physics` `mode: "static"` before anyone presses Play, or the scene collapses.
- **Never fight a person.** An object locked by another peer is skipped, and the skip names the
object and who holds it (`"Door" () is locked by Ana (peer …) — they are editing it`;
`delete_objects` and `group_objects` list them in `skippedWhy`); do not retry in a loop.
`waitForEdit` is how you notice people's changes.
- **Everything is shared.** Every peer sees each call as it happens and can undo it. There is no
private side channel, by design.
- **Look before you say "done".** `screenshot()` after a build; compare with the request.
## Versions
Two version numbers, and only one of them is about compatibility:
- **`tpAgent.version`** is the CONTRACT (`1.1.0` today). A new method or a new optional field is a
minor version; a removed or renamed method, or a changed shape, is a new MAJOR. Read it first
(`status().version` carries it too) and feature-detect methods rather than comparing minors.
- **`@theprototype/agent`** (the bridge) is versioned with the app release it shipped in, and it
speaks one contract major. Against a page with a different major it still connects and says so
in its log and in the `status` tool (`contractNote`), with the fix: `npx -y @theprototype/agent@latest`.
## The light engine
`npx -y @theprototype/agent mcp --engine=wire ` is a small Node peer without a browser:
objects only (no flow graphs, behaviours, physics or screenshots). Use it where a headless
Chromium cannot run.
---
# Module SDK — full guide (MODULES.md, inlined)
# Writing modules
Modules are self-contained folders under `src/modules//` that plug into the
scene, the flow node editor and the peer mesh. Music instruments, playable game
prototypes, generators, custom controls — anything a user should be able to drop
into a session and every connected peer can see and interact with.
## The one rule that matters
**A module runs on every peer.** There is no game server. Whatever your module does
must end up identical on all clients, one of three ways:
1. **Deterministic from shared inputs** — effects driven by node `data` (already
replicated) and the synced clock. No extra work needed. Prefer this.
2. **Broadcast events** — a discrete thing happened ("button pressed",
"generate with seed 42"). Send a small message with `api.send`, apply the same
change in the sender and in `api.onMessage`. Never re-broadcast from a
receiver.
3. **State sync for late joiners** — someone connects mid-session and needs your
current state (score, generated layout, toggles). Provide
`registerStateSync`; the handshake delivers it automatically.
Randomness must be seeded (send the seed, not the result — see the dungeon
module). `Math.random()` in anything replicated is a desync.
## Layout and registration
```
src/modules/
index.js <- list your module here (all peers need the same list)
mymodule/
module.js <- export default { id, name, version, register(api) }
SomethingNode.svelte (optional custom node UI)
```
```js
// src/modules/mymodule/module.js
export default {
id: 'mymodule', // stable, unique; used to route your messages
name: 'My Module',
version: '1.0.0', // peers toast when versions differ
register(api) {
// wire everything here
}
};
```
Peers exchange `{id, version}` lists when they connect and warn when a module is
missing or a different version on the other side. That's advisory — the session
still works, but replicated behavior of that module may differ.
## The lifecycle: load, unload, reload (34 R6)
A module can be **unloaded and loaded again while the app runs** — switched off in the
Modules manager (core modules too, since 1.20), removed, updated, dev-reloaded from its Dev
URL, or unloaded by a scene switch. `unloadModule(id)` takes down **everything the module
registered**, newest first, and `register(api)` can then run again with fresh code. Nothing
needs a page reload, and nothing is left running for a module that is gone.
**What is undone for you.** Every registration you make through `api` is recorded in the
module's lifecycle registry (one registry, keyed by your module id — `src/lib/sdk/lifecycle.js`)
and undone at unload: node groups, effects, value nodes, node defs you seeded and the user did
not edit, primitives, click/drop handlers, frame tasks, scene-clear hooks, interactive / system /
listed groups (the scene-root group is removed AND its geometries, materials and textures are
freed), the spawn override, message handlers and state sync, menus, toolboxes (closed) and their
shortcuts, VR menu entries, key bindings, input listeners and claims, possess, the follow camera,
VR panels, hit listeners, your music track, game levels / settings rows / help / restart hooks /
the forced menu, LOD handles, quality / flow / game / settings / peer-variable listeners, unwrap /
shader / post backends, post effects, audio device kinds, engine voices (`api.audio.voice`),
transport events (`api.audio.schedule`), your copy of the mic stream, a recording you started,
HUD rows / debug lines / actions / element kinds.
Every `off()` the api hands you still works, and calling it early also drops the registry
entry — subscribe and unsubscribe as often as you like. A call that REPLACES (`api.game.levels`
called on every unlock, `setHelp`, `addSetting` with the same id, a toolbox or VR menu entry
re-registered under the same id) replaces its entry rather than stacking one per call.
**What stays.** What your module created in the SHARED scene (`api.create`, objects in
`objectsGroup`, nodes from `api.flow.addNodes`, audio devices and cables, node data you wrote)
is user content: it stays, replicated, like anything a person made. So do the shared game
variables and your peer-variable row, and what you saved on this device with `api.storage`
(a reinstall keeps progress; `storage.clear()` is your reset).
**What only you know about — four tools:**
```js
// your own teardown: a DOM overlay you appended, a worker, a cache, a raw WebAudio graph
// you built on api.audio.context(). Runs at unload (newest first, so a hook added at the
// end of register() runs while everything before it is still in place). Returns cancel().
api.onUnload(() => overlay.remove());
// timers that die with the module (the window's signatures) + what is pending right now
const h = api.timers.setInterval(refresh, 500);
api.timers.clearInterval(h);
api.timers.setTimeout(fn, ms); api.timers.requestAnimationFrame(loop);
api.timers.pending(); // {timeouts, intervals, frames}
// an event listener removed at unload — window, document, the canvas, anything; returns off()
const off = api.listen(window, 'keydown', onKey, true);
// a scene-root object that is YOURS: removed at unload, its GPU resources freed (shared
// ones kept). Returns the object, so: api.scene().add(api.own(group)). Never for content
// inside objectsGroup (that is shared — it is left alone).
api.own(group);
```
**Installed modules (zip / URL) get timers and window listeners tracked automatically.** The
entry is evaluated with its bare `setTimeout` / `clearTimeout` / `setInterval` /
`clearInterval` / `requestAnimationFrame` / `cancelAnimationFrame` bound to tracked ones (a
one-line prologue on line 1, so line numbers in stack traces are unchanged), and a listener
your code adds to `window` or `document` is recorded too — at unload they are cleared and
removed. Two edges: `window.setTimeout(...)` (or a listener on any other target) is not
tracked — use `api.timers` / `api.listen`; and a module that declares one of those six names
itself at top level is loaded without the prologue (its timers then untracked). A core module
(in `src/modules/`) uses `api.timers` / `api.listen` explicitly.
**Keep state inside `register()`.** The browser keeps every imported module's TOP-LEVEL scope
for the life of the page (an ES module is never unloaded), so module-level variables, caches and
DOM survive an unload — the next load starts from them. State declared inside `register()` is
fresh every time; module-level state you must reset belongs in `api.onUnload`. Each reload
of an installed module also keeps the evaluated source itself, so a large module costs some
heap per reload — fine for a development loop, not something to do every second. (Measured
with `module-lifecycle-repo`, 3 cycles each: 19 of the 20 repo modules reload flat; Waves keeps
one DOM node and ~0.3 MB of heap per reload.)
### Adding an SDK surface (core contributors)
Every member of the api declares what it does to a module's lifecycle, beside its slice
(`src/lib/sdk/.js`):
```js
export function sdkThing(ctx) {
const { moduleId, onDispose, owned } = ctx;
return {
registerThing(spec) {
thingRegistry.set(key, spec);
// the undo, journaled with a KIND (the debug view names it); `{key}` makes a
// re-registration under the same key REPLACE its entry
onDispose(() => thingRegistry.delete(key), 'thing', { key: 'thing:' + key });
},
onThing(fn) {
// a registration that hands back an off(): owned() returns an off that also
// drops the journal entry
return owned('thing.listener', subscribe(fn));
},
thingCount: () => thingRegistry.size
};
}
sdkThing.surface = { registerThing: 'registers', onThing: 'registers', thingCount: 'read' };
```
- Kinds: `registers` (leaves something behind: MUST journal its undo), `action` (does
something now, leaves nothing of the module's), `content` (shared, replicated content that
deliberately survives an unload), `read`, `value` (a non-function).
- A `registers` member needs a fixture in `tests/fixtures/sdkLifecycleFixtures.js`: register
it from a real module, read CORE's registry (never the journal), unload, read it gone. The
vitest `moduleLifecycle` fails for an undeclared member, a stale declaration, a `registers`
member with no fixture, and a fixture whose registration survives `unloadModule`; the
browser suite `module-lifecycle` runs the members node cannot load.
- **Async registrations journal SYNCHRONOUSLY.** A disposer recorded inside an
`import().then` lands in the module's NEXT journal whenever an unload comes first — the
registration then outlives the unload. Record it before the import and let a late
registration undo itself:
`let off = null, gone = false; onDispose(() => { gone = true; off?.(); }, 'kind');
import('../x').then((m) => { off = m.register(...); if (gone) off(); });`
- A core subsystem acting FOR a module outside the api (kit entities, loaded models) records
its undo with `trackModuleResource(moduleId, kind, undo)` from `moduleSDK.js`.
- Debug: `__stores.moduleSDK.registrationsOf(id)` / `allRegistrations()` (live entries by
kind), `moduleScopeDebug()` (an installed module's tracked timers + listeners),
`unloadModule(id)` returns what it disposed, by kind.
- Tests: vitest `moduleLifecycle`; e2e `module-lifecycle` (the browser contract, an installed
probe module, and the leak test: each bundled module unloaded/loaded 3x with handlers,
nodes, audio connections, timers, meshes, materials, GPU memory, DOM, JS listeners and the
heap measured) and `module-lifecycle-repo` (every zip in the modules repo through 3 cycles).
## API reference (v1)
### Flow nodes
```js
api.registerNodeGroup(
{
group: 'Modules', // palette/context-menu group header
items: [
{
type: 'wave', // globally unique node type
label: 'Wave (hello)',
defaults: { amplitude: 0.4 }, // seeds node.data, replicated with the node
params: [ // optional: auto-generated controls
{ key: 'amplitude', kind: 'range', min: 0, max: 1.5, step: 0.05 }
// kind: 'range' (min/max/step), 'select' (options: [...]), 'toggle',
// or 'text' (placeholder?, maxLength?) — a text param writes on COMMIT
// (change/blur), never per keystroke: a node edit replicates the whole node
]
}
]
},
{ wave: MyWaveNode } // optional: custom Svelte components per type
);
```
Items with `params` render with the built-in generic node UI (sliders/selects
that write through `setNodeData`, which replicates). Custom components should
follow the same pattern: render from `data`, write with
`setNodeData(id, {...})` from `$lib/nodesHandler`, never `bind:` to data.
### Per-frame effects (node → Object Selector)
```js
api.registerEffect('wave', (object, base, data, time) => {
// base = the object's logical transform: {pos, rot, scale, visible}
// The runtime restores base before each frame — apply offsets FROM base.
object.rotation.z = base.rot[2] + Math.sin(time * (data.speed ?? 2)) * (data.amplitude ?? 0.4);
});
```
Effects run when an edge connects your node to an Object Selector node. `time`
is wall-clock synced across peers (when "Sync animations" is on), so pure
functions of `(data, time)` stay identical everywhere. Don't accumulate
(`rotation.z += ...` breaks determinism) — always compute from `base` and `time`.
An effect takes an optional **5th argument** `{id, graphId}` — its own node id and
the graph it sits in, so one module can host many instances of the same node type.
It is additive: a four-parameter effect is unchanged.
```js
api.registerEffect(
'wave',
(object, base, data, time, ctx) => {
// ctx.id = this node's id, ctx.graphId = 'scene' or the owning object's uuid
},
{ inputs: { who: 'object' } } // optional: typed named inputs, see below
);
```
### Value nodes: your state into core nodes
A node that **outputs a value**, so module state can drive core nodes — a score into
a HUD Text, a level into Map Range, a flag into a Gate.
```js
api.registerValueNode(
'score',
(data, time, ctx) => Number(data.base ?? 0) * Number(data.mult ?? 1),
{
vtype: 'number', // output socket type; also boolean/vector3/color/object/event
inputs: { mult: 'number' } // typed named inputs, resolved BEFORE fn runs
}
);
```
**The evaluator MUST be a pure function of its arguments** — the same rule Script
nodes follow, and the one way to break a session silently. Values are never sent:
every peer evaluates the node itself from the replicated node data and the shared
clock. Read unreplicated local state here (a mouse position, a plain module `Map`,
`Math.random()`) and every downstream consumer diverges per peer with no error
anywhere. Keep mutable state in a replicated place — `registerStateSync` /
`api.send` — and read *that*, or let the value ride the node's own `data`.
`inputs` matters more than it looks. Without a declaration every handle types as
`number`, which **refuses** an Object Selector wire (`object -> number` is not a
coercion) and renders no target socket on the card at all. `data.` is the
wired value when wired and the node's own param when not. `registerEffect` accepts
the same `{inputs}` option.
### Firing your own events
```js
// pulse every instance of the type...
api.fireNodeTrigger('levelcleared');
// ...or just the ones you mean
api.fireNodeTrigger('goal', (data, id) => data.team === 'blue');
```
Register the node with `{vtype: 'event'}` so it can be wired to a Counter or an
Object Selector. **This REPLICATES**, exactly like `fireObjectClick`: the pulse rides
the existing `nodetrigger` message from one peer's stamp, and every peer then computes
the identical pulse. So call it on the peer where the event happened and **not on all
of them**, or a Counter counts it once per peer. `api.peerIds()` and
`api.physics.isInitiator()` are the usual ways to pick that peer.
R3a: pass `{replicate: false}` as the third argument to keep the pulse in **this
peer's own trigger log** — the per-player mechanism (a per-player pickup hides only
for its collector because the pulse never left their machine). Your effect/value
node then reads its own pulse through `ctx.trigger` (`{stamp, age}` or null), which
core has ALREADY folded through the round rules: stamp a `perRound: true` flag on
your node's data and a round bump (or a return to the menu) retires the read with no
round math of your own. That is the whole latch story — never rebuild it.
### The game shell, per-player rows, and the graph (R3a)
```js
api.game.roundCutoff(); // null (shell unused) | Infinity (menu/over) | startedAt
api.game.roundUnderway(); // true while playing/paused (and when the shell is unused)
api.game.playActive(); // THIS peer plays inside a running round
api.game.getVar('score', 0); // the shared game variable (replicated singleton)
api.game.setVar('score', 7);
api.peerVars.setMine('laps', 3); // MY replicated row — one writer per row, by construction
api.peerVars.mine('laps', 0);
api.peerVars.all('laps'); // [{id, name, value, me, rank}] — the leaderboard shape
api.playerPosition(); // [x, y, z] of the viewer — a touch trigger's read
api.selectObject(uuid); // the manager-row click
api.selectedUuids(); // the selection SET (never the sticky primary)
api.flow.nodes('mytype'); // [{id, type, graphId, data}] across every graph
api.flow.edges(); // [{id, source, target, sourceHandle, targetHandle, graphId}]
api.flow.nodeValue(latchId); // a node's evaluated output (a core Latch's round-aware state)
api.flow.triggerStamp(nodeId); // {stamp, age} | null — a node's own round-aware pulse
api.flow.setNodeData(nodeId, { respawn: 5 }); // replicated MERGE (the editor's nodedata path)
api.flow.addNodes({ // replicated, ONE undo entry for the batch
nodes: [{ type: 'mytype', x: 60, y: 40, data: {} }, { type: 'objectselector', x: 280, y: 40, data: { selected: uuid } }],
edges: [{ from: 0, to: 1 }] // indices into nodes, or existing node id strings
});
api.flow.seedGraph({ // 36 (DEVX #10): the same, ABSENT-ONLY — a wired example
key: 'example', // graph on module load; nothing is added once any node is
nodes: [/* … */], edges: [] // marked with this seed or is one of THIS module's types
});
api.hud.registerDebugLine(() => 'mymod: 3 left'); // a line on the debug HUD pill
api.hud.registerAction({ key: 'showleft', label: 'Show it', group: 'Data',
role: 'drives', node: '', via: { node: 'mytype', data: {}, handle: 'value' } });
```
Graph reads are deterministic because the graph is replicated — treat them exactly
like replicated state (the value-node rule). Two per-node data flags core honours on
YOUR nodes: `perRound` (the trigger reads retire on a round bump) and `whilePlaying`
(an effect node stands down outside play and the restore loop hands its object back —
how a module hides an object without owning the give-back).
### Primitives
```js
api.registerPrimitive(
'Flag', // capitalized: /create Flag 2 1
(w, h) => new THREE.PlaneGeometry(+w || 2, +h || 1),
{ label: 'Flag', command: '/create Flag 2 1' } // spawn button on your manager card
);
```
The builder runs on every peer from the same `/create` command, so it must be
deterministic in its arguments.
### Interaction
```js
api.registerClickHandler((object) => {
if (object.userData.myButton) {
press(object); // apply locally
api.send({ op: 'press', uuid: object.uuid }); // tell peers
return true; // consume the click (no selection)
}
return false;
});
```
Handlers see desktop clicks and VR trigger presses (the exact mesh hit, not the
top-level group). Return `false` to let normal selection continue.
**Modes (1.17).** The editor has two click modes, Edit (every click selects) and
Interact (play-style clicks and drags without starting the game), plus Play. A
handler runs in Interact and Play by default, so an Edit click on your game piece
selects it like any object. Say so when you want something else:
```js
api.registerClickHandler(pressKey, { modes: ['interact', 'play'] }); // the default, spelled out
api.registerClickHandler(pickTool, { modes: ['edit'] }); // an editor TOOL
```
In a headset the same modes apply (1.17, 30b): Play in VR enters Interact, where the
trigger clicks (module handlers, On Click) and selects nothing; in Edit it selects as
before. Handlers also get `ctx = {source, mode}` — see *Game feel* below for the
hold-and-sweep and `{sweep: false}`.
`api.editorMode()` reads the mode on this screen (`'edit'` | `'interact'`) — for a module
whose own pointer listeners run outside click routing (a drag), so it can stand down in Edit.
**Listed in the object list (1.17).** Content you build at the scene root shows in
the object list's *Module content* section — a read-only row that frames it, hides
it locally, or opens your toolbox. Give it a readable name:
```js
api.registerListedGroup('piano-module', { label: 'Piano' });
```
```js
api.registerFrameTask((time) => { /* runs every frame, synced time */ });
// click handlers only see the replicated objects root by default; if your
// module adds its own group at the scene root, register it for clicks:
api.registerInteractiveGroup('pong-module');
// an Explorer AUDIO/TEXT item dropped on one of your meshes (23-C2): take it or not.
// `hit` is the exact mesh under the drop, `item.hash` feeds api.audio.sample(hash).
api.registerDropHandler((hit, item, target) => {
if (item.kind !== 'audio' || !hit.userData.pad) return false;
api.audio.setParams(deviceOf(hit).uuid, { ['pad' + hit.userData.pad]: item.hash });
return true; // consumed
});
```
### Toolboxes: a real UI surface (A5)
```js
const id = api.registerToolbox({
id: 'settings', // namespaced to mod--settings
title: 'Dungeon Kit',
width: 240,
shortcut: 'Ctrl+Shift+D', // optional; also lists in Settings > Shortcuts
playMode: false, // default: hidden in Play mode
sidebar: false, // default true: also a row in the burger menu's Modules
// section. false = viewport menu only (see below)
mount(el) {
const label = document.createElement('div');
label.className = 'tbx-label'; // the shell styles this for you
label.textContent = 'Rooms';
const go = document.createElement('button');
go.className = 'tbx-primary';
go.textContent = 'Generate';
go.onclick = () => regenerate();
el.append(label, go);
return () => { /* your cleanup */ };
}
});
```
Write plain DOM into the node `mount` receives and you inherit the app's own
tool-palette treatment: header drag with position persistence, the width grip, z-band
focus, the bottom SHEET at <=640px, and the whole `.tbx-*` CSS contract (`.tbx-label`,
`.tbx-row`, `.tbx-btn`, `.tbx-seg`, `.tbx-primary`, `.tbx-check`, `.tbx-danger`) with
no CSS of your own. That is the point — before this seam, module controls could only
live behind `registerMenu`, two clicks deep inside the Modules *modal*, which then had
to be closed before the module's own overlay was usable.
The user opens it from the **sidebar's Modules section** and the **viewport menu**
(both from one builder, so they cannot drift), plus your `shortcut` if you name one. It
starts CLOSED: a palette that appears uninvited is the thing `registerMenu` was
avoiding. `onOpen`/`onClose` fire on each transition; the header ✕ closes it.
**Where the way IN belongs is your call.** The burger menu's Modules section is the
app's permanent chrome, so a row there is a standing claim on it — right for a tool a
user reaches for constantly, heavy for a window that belongs to one workflow. Pass
`sidebar: false` to keep the viewport-menu row and drop the permanent one, and open it
from where your module already is:
```js
const id = api.registerToolbox({ id: 'manager', title: 'Collectibles', sidebar: false, mount });
// a button on YOUR card in the Modules manager, beside Update/Remove
api.registerMenu('Open Collectibles', () => api.openToolbox(id));
```
`api.openToolbox(id)` / `closeToolbox(id)` / `toggleToolbox(id)` take the id
`registerToolbox` returned. `openToolbox` also dismisses the Modules manager when it is
open, because the manager is the one piece of chrome that can cover a toolbox — a card
button that opens a window *behind* the dialog it was clicked in is the complaint this
whole seam exists to answer. It is a no-op when the manager is closed.
`mount` returns its cleanup and a re-registration re-runs it, so dev-mode live reload
rebuilds the contents in place. Disabling or removing the module force-closes and
unregisters the toolbox — it never leaves a window behind a dead mount fn.
A toolbox is **LOCAL**: it is this viewer's window, and nothing about it replicates or
is saved with the scene. What it *changes* must still go through the replicated paths
(`api.send`, `api.create`, `api.physics.set`).
### HUD elements: rows, and your own element kind (21-E7)
```js
// fill a HUD List element (kind `list`) by id
api.hud.rows('leaderboard', ['1. Ada 12', '2. Grace 9']);
api.hud.clearRows('leaderboard'); // back to the element's authored rows
// the roster WITH NICKNAMES - what a leaderboard actually needs
api.peerNames(); // [{id, name, label, me}] `label` falls back to 'peer abcd'
```
`api.hud.rows` is one of three doors onto the same store: the element's own **authored**
rows (typed into the properties pane, one per line), a **HUD Rows** flow node
(set / append / clear on a trigger edge), and this. A node or a module always wins over
the authored rows, and clearing puts them back — the authored value is the fallback, not
a second source of truth.
**CALL IT ON EVERY PEER.** Rows are never sent. Like a value node, this writes LOCAL
state that every peer is expected to compute identically, so calling it on one peer
shows the rows to one person. Drive it from your own `registerStateSync` state, or from
something every peer already derives (`api.peerNames()` is exactly that — the roster is
already replicated). Your rows are cleared at teardown, so disabling the module shows
the authored rows again rather than freezing the last thing you pushed.
```js
// YOUR OWN ELEMENT KIND: the registerToolbox contract, one layer in
const kind = api.registerHudElement('gauge', {
label: 'Fuel gauge', // palette + properties-pane heading
icon: 'gauge', // a lucide name
summary: 'A dial that reads a wired number.',
defaultSize: { w: 150, h: 40 },
defaults: { caption: 'Fuel' }, // your params' starting values
fields: [ // the pane renders these; same schema as core kinds
{ key: 'caption', kind: 'text', label: 'caption' },
{ key: 'redline', kind: 'number', label: 'redline', min: 0, max: 100, step: 1 }
],
mount(container, el, runtime) {
const span = document.createElement('span');
span.textContent = el.caption;
container.append(span);
// return a bare cleanup fn, OR this pair to avoid a rebuild on every change:
return {
update(nextEl, nextRuntime) { span.textContent = nextEl.caption; },
destroy() { /* your cleanup */ }
};
}
});
// kind === 'mod--gauge'
```
You write plain DOM and inherit the whole HUD system: the layer's z-tier, the 9-grid
anchoring with pixel offsets, the properties pane, the palette, replication, undo and all
four save paths. `runtime` is what a flow node is currently driving into the element
(`{text, value, min, max, rows, options, pulse}`), so a HUD Text or HUD Bar node pointed
at your element feeds it with no extra wiring.
The kind name is **namespaced** and gets written into a replicated, saved document. A peer
without your module — or the same scene opened after it is gone — reaches an unknown kind,
which the HUD PRESERVES verbatim and skips at render. So nobody's layout is destroyed,
installing the module makes the element appear, and a disable is the same story rather
than a special case. Everything is unregistered and unmounted from the teardown journal.
### Messages and state
```js
api.onMessage((data) => { // {type:'module', moduleId, ...payload}
if (data.op === 'press') press(findByUuid(data.uuid));
});
api.send({ op: 'press', uuid }); // broadcast to all peers
api.registerStateSync({
getState: () => ({ pressed: [...pressedUuids] }), // sent to late joiners
applyState: (state) => state.pressed.forEach(markPressed)
});
```
### Input (K-C)
```js
// declare bindings so they LIST in Settings ▸ Shortcuts (display-only —
// you read the keys yourself via input()/onInput)
api.registerBindings([{ label: 'Drive forward', keys: 'W' }]);
api.registerFrameTask(() => {
const { codes, axes } = api.input(); // codes: Set<'KeyW'...> (event.code),
if (codes.has('KeyW')) drive(1); // axes: {lx,ly,rx,ry} = VR stick axes
});
api.onInput((kind, code) => {}); // 'down'/'up' events; returns unsubscribe
// pause the HOST's use of an input scope while your module drives:
// 'keys' — WASD camera fly + play-mode movement
// 'locomotion' — VR left-stick locomotion
// 'sticks' — (1.19) BOTH VR sticks: left-stick move, right-stick snap/smooth turn + teleport
// (a held thing reeled on Y and scaled on X, the way an Edit grab does)
api.claimInput('keys'); // ALWAYS release when your mode ends
api.releaseInput('keys');
const sticksOk = api.claimInput('sticks') === true; // true = this core knows the scope (older: undefined)
// reading a keydown YOURSELF (a toolbox's own handler)? resolve the key the way the
// editor does — `event.key` when it is an ASCII letter/digit, the physical
// `event.code` position otherwise — so it works on a Cyrillic/Greek/Hebrew layout:
window.addEventListener('keydown', (e) => {
if (api.keyOf(e) === 'G') grab(); // 'G' on QWERTY, AZERTY, Dvorak AND ЙЦУКЕН
if (api.letterOf(e) === 'w') drive(); // lowercase form; named keys ('Escape') unchanged
});
```
### Pointer ray (190)
```js
// where the user is POINTING, as a THREE.Raycaster in WORLD space: the desktop
// mouse over the viewport, or the VR pointer hand's ray. Fresh instance per
// call; null before the first pointer event. The drag recipe: click to pick,
// follow pointerRay() in a frame task, click to drop (works desktop + VR).
api.registerFrameTask(() => {
const ray = api.pointerRay();
if (!ray || !carried) return;
const hit = ray.ray.intersectPlane(dragPlane, tempVec);
if (hit) carried.position.copy(hit);
});
```
### Free-cursor games (1.17)
A board, puzzle or instrument game can play with the real mouse cursor and no
pointer lock: publish `userData.play.cursor = 'free'` on your scene-root group
(or the scene's physics block sets `play.cursor: 'free'`). `api.pointerRay()` is
then the cursor's ray in play; in a locked game it is the CROSSHAIR ray (the view's
centre), never the stale mouse position the lock pinned.
### Storage (1.17)
```js
// JSON on THIS device, namespaced to your module (tp:mod::), 256 KB per
// module; never replicated, never saved into a scene, survives disable/remove.
if (api.storage) {
const progress = api.storage.get('progress', { unlocked: 1 });
api.storage.set('progress', progress); // -> true, or false over the quota
api.storage.keys(); api.storage.bytes(); api.storage.remove('progress');
api.storage.clear(); // your "Reset progress"
}
```
Feature-detect it; on an older app write the SAME key yourself
(`localStorage['tp:mod::'] = JSON.stringify(value)`) so progress carries over.
### Models: `api.loadModel` (1.20, roadmap 34 R7)
Load a `.glb` / `.gltf` through core's OWN loader — the scene's `THREE` (never bundle three's
`GLTFLoader` into a module: it drags in a second three whose classes are not the scene's), with
Draco, Meshopt and KTX2 decoders (the Basis transcoder is fetched only for a file that carries
`KHR_texture_basisu`), parsed ONCE per URL and shared by every module that asks.
```js
// a file your module PACKAGED (list it in manifest.json `files`) — or any URL
const enemy = await api.loadModel('assets/enemy.glb', { castShadow: false });
group.add(enemy.scene); // your own copy (another module's tweaks never reach it)
const next = enemy.instance({ ownMaterials: true }); // another copy: its OWN bones when skinned,
group.add(next); // its own materials (a hit flash paints one)
const mixer = new api.THREE.AnimationMixer(next);
mixer.clipAction(enemy.animations.find((c) => c.name === 'walk')).play();
enemy.info; // {meshes, triangles, materials, textures, skinned}
enemy.release(next); // forget one copy now (out of the scene, LOD dropped, own materials freed)
enemy.dispose(); // forget them all — your module's unload does this for you
```
Options (all optional): `lod` — absent / `'auto'` = automatic levels (31-perf, meshes over 3000
triangles; a URL that is a PACK item whose row lists `lods` uses those files instead), `false` =
none, `{ratios, distances, minTriangles}` = tuned automatic levels, `[{file, ratio}]` = pre-built
level FILES beside the model (contract P1: `'tree.lod1.glb'` next to `'assets/tree.glb'`), drawn
as a LOD group wherever you put the copy. `castShadow` / `receiveShadow` = every mesh (absent =
as the file says, which is off). `collider: 'box'|'sphere'|'capsule'|'cylinder'|'cone'|'hull'` =
the physics shape a copy takes once it is SCENE content with physics (`userData.colliderHint`;
your scene-root visuals have no colliders — publish a `userData.play` raster for walls).
`ownMaterials` = every copy gets its own materials.
Lifetime: everything a module loaded is released when it is disabled or unloaded (a copy you put
in `objectsGroup` stays — it is the scene's). A copy you drop WITHOUT `release()` (its wrapper
removed) is noticed within ~2 s, its LOD entry dropped and only weakly held, so it is collected;
put back, it is picked up again. A load still in flight when the module unloads rejects.
Rejects with the reason when the file is missing or cannot be parsed — keep a fallback look.
Feature-detect it (`typeof api.loadModel === 'function'`): 1.19 and older cores do not have it.
### Performance: quality level, LOD, budgets (1.18, roadmap 31)
```js
// The adaptive quality level on THIS device: 0 = best ... api.quality.max (every step taken).
// A headset session starts at 1 (shadows off) in auto mode; a game's Quality setting can pin it.
if (api.quality) {
const apply = (level) => {
particles.maxCount = level >= 3 ? 40 : 200; // cut YOUR effects, not shared state
torchLights.forEach((l, i) => (l.visible = i < (level >= 1 ? 2 : 6)));
};
apply(api.quality.level);
const off = api.quality.onChange((level, { labels, vr }) => apply(level)); // released on disable
}
// Levels of detail for your own dense geometry (auto LOD already covers dense meshes in the
// shared scene and under the module world root). Built ONCE per asset in a worker
// (meshoptimizer), drawn by distance through a render-time swap: the mesh keeps its full
// geometry for picking, physics and your code. Distances are in the mesh's world RADII.
if (api.lod) {
const lod = api.lod(enemyFigure, { ratios: [0.5, 0.2], distances: [6, 18] });
lod.ready.then((n) => console.log(n, "meshes have levels")); // lod.remove() undoes it
lod.force?.(2); // 1.19: draw level 2 on THIS screen whatever the distance (null = auto)
lod.levels?.(); // 1.19: the level each mesh drew last (0 = full)
}
mesh.userData.lod = false; // keep one mesh out of auto LOD
```
**1.19 — LOD groups on scene objects.** A replicated object may carry a LOD GROUP
(`userData.lod`, edited in its Properties ▸ LOD section, or written by a pack item's `lods`):
`{mode: 'auto'|'forced', forced?, bias?, cull?, levels: [{source: 'self'|'pack'|'generated'|
'explorer'|'object', ref?, ratio?, screenSize, offset?, material?}]}` with thresholds by SCREEN
SIZE (the share of the viewport height the object covers). It is scene data (saved, replicated,
undone) and drawn at render time like `api.lod` — the tree never holds a level. A module that
builds scene objects can write one through the same path the panel uses
(`objectParameters {parameter: 'lod'}` is the wire shape); module-only geometry keeps `api.lod`.
Both are LOCAL (a fact about this machine) — never let them change replicated state, or two
peers on different hardware disagree about the game. Skinned meshes and morph targets get no
levels (the simplifier cannot carry weights): pre-decimate those offline.
**The Quest budget** (roadmap 31, the planner's target for a scene in Interact/Play on a Quest 3):
**<= 150 draw calls, <= 300k triangles visible, <= 2 real-time lights with shadows off, no
per-frame allocations in the hot path.** Measure your game with the probe, the same way every time:
```bash
# dev server running; run it under the exclusive slot (frame times are the point)
APP_URL=https://theprototype.app:5173/ ~/.local/bin/e2e-slot --exclusive -- \
node scripts/perf-games.cjs --label mine --only waves [--vr] [--profile]
```
It prints draw calls + triangles per frame (every render pass summed), geometries, textures and
an MB estimate, lights (+ shadow casters), meshes / instanced / unculled, the LOD and quality
state, and p50/p95/p99 frame ms at CPU throttle x4; `--profile` adds CPU self-time, the nearest
app frame calling the heaviest functions, and allocation per function (garbage included).
The things 1.18's round found, in the order they cost:
- **Never keep a THREE object inside Svelte `$state`** (or pass one through an action parameter
that is a state proxy): Svelte deep-reads a proxy, which walks the whole scene graph through
`parent` — typed arrays included. Use `$state.raw`. This one line was 88% of a game's frame.
- **No whole-scene lookups per frame** — `scene.getObjectByName` is a traversal; find once, keep
the reference, re-find when it leaves the scene.
- **No allocation per frame**: reuse vectors/arrays; a `[a, b].map(...)` in a frame task is
garbage 60-90 times a second, and GC pauses are the stutter you feel in a headset.
- **Lights and shadow casters are draw calls**: each shadow-casting light draws every caster
again; transmission (glass) renders the scene an extra time per camera, per eye.
### Functional pack items: doors, lids, levers — `api.behavior` (1.19, roadmap 33)
A pack item can be FUNCTIONAL: a door that opens on a click, a chest lid, a lever, a fan
(the `behavior` field, see PACKS.md). Core runs them. Nothing plays in Edit; a click, a
knock or walking up triggers them in Interact/Play; the state replicates; a door's
collider follows its leaf. A module can drive them too:
```js
if (api.behavior) {
for (const item of api.behavior.list()) { // [{uuid, type, trigger, open}]
if (item.type === 'door' && !item.open) api.behavior.trigger(item.uuid, true); // open it
}
api.behavior.state(uuid); // {on, at, n} or null (never triggered)
api.behavior.trigger(uuid); // toggle, like a player's click
}
```
`trigger` is REPLICATED (one `behavior` message; every peer poses the door from the same
session-clock stamp), so call it on ONE peer: the authority, or the peer that saw the
cause. It returns false when nothing changed (opening an open door). The state is
runtime: it is never saved, and a peer in Edit shows the rest pose whatever it says. The
item sounds are core game sounds: `door`, `gate`, `slide`, `lever`, `lid` (plus `click`),
which `api.playSound` can play too.
### Game feel: sound, music, haptics, effects, banners (1.17, roadmap 30b)
Everything here is LOCAL to the device it runs on — broadcast your own op
(`api.send`) when every peer should hear, feel or see it — and all of it is
feature-detected, so a module written against it still loads on an older app.
```js
register(api) {
api.registerClickHandler((object, ctx) => {
if (!object.userData.ring) return false;
const at = object.getWorldPosition(new api.THREE.Vector3()).toArray();
api.playSound?.('success', at);
api.effects?.burst(at, { kind: 'sparkle' });
api.announce?.('Ring ' + object.userData.ring + ' reached', { sub: '+50' });
api.hapticPattern?.('success'); // both hands; 'left' | 'right' for one
return true;
});
api.game.onChange?.(() => (api.game.roundUnderway() ? api.music?.play('arcade') : api.music?.stop()));
}
```
- **`api.playSound(name, position?)`** → `true` when a sound started. Twenty
procedural game sounds, no assets: `click` `pop` `whoosh` `success` `fail` `hit`
`kick` `shoot` `laser` `explosion` `coin` `levelup` `goal` `whistle` `cheer`
`step` `ring` `sparkle` `hurt` `portal`, plus the ping chimes `ding` (the default),
`chime`, `pluck`, `bell`. An unknown name is a quiet no-op (it used to play the
ding). `position` spatialises it. Played at the player's "Game sounds" volume.
(1.41) A game-set sound shows a CAPTION to a player who switched Captions on in the game
menu's Accessibility page (`[coin]`; step / click / pop stay quiet); a third argument
`{caption: 'The gate opens'}` says it your way, `{caption: '-'}` says nothing.
- **`api.captions.show(text, {seconds?})`** (1.41) — a caption line for something the player
hears that is not a game-set sound (a voice, your own audio): shown only to players with
Captions on, at their caption size, on a flat screen and in a headset. LOCAL; `seconds`
0.5..15 (default 3). `api.captions.shown()` = are captions on for this player. Input you read
through `api.onInput` / `api.input` already follows the player's remapped keys and buttons.
- **`api.music.play(preset, {volume?})` / `api.music.stop()`** — a looping
procedural track under the effects, at the player's "Music" volume: `arcade`
`ambient` `dungeon` `stadium` `space` `puzzle` `studio`. It is TEMPO-SYNCED to the
session clock (two peers on one preset hear the same bar), plays only in Interact
or Play (`false` from the editor) and stops by itself when the player goes back to
editing. `api.music.current()` names what is playing; `api.music.presets()` lists them.
- **`api.hapticPattern(name, hand?)`** — `tap` `bump` `hit` `success` `fail`
`rumble` `heartbeat` on the VR controllers. `api.haptic(intensity, ms, hand?)`
still exists. BOTH are silent in Edit mode (vibration is for playing), and core
already plays the defaults for you in Interact/Play: a `tap` when the laser enters
something clickable, a `bump` when a press lands, a `hit` on a grab, and a knock
sized by how hard the hand hit.
- **`api.effects.burst([x, y, z], {kind?, color?, count?})`** — a short, pooled
particle burst at the scene root: `sparkle` (default), `confetti`, `smoke`,
`sparks`; `count` 1..96. Nothing is saved or replicated.
- **`api.announce(text, {sub?, ms?, color?})`** — a big centred banner ("GOAL!",
"Level 3"): the desktop HUD, and in a headset a banner fixed in front of the
player. A new one replaces the one showing; `ms` 300..15000 (default 1800).
- **`api.setSpawn([x, y, z], yaw, {teleport?})`** — where the player starts: `y` is the
FEET, `yaw` is three's `rotation.y` (0 faces −Z). Entering Interact or Play puts the player
there (a headset's rig lands its feet on it, a desktop its camera); without `teleport` it is a
checkpoint and nobody moves until then. A module's spawn overrides the scene's
`play.spawn`. `api.respawnPlayer()` sends the player back to it now.
### Moving around in a game: locomotion, bounds, VR panels (1.18, roadmap 31)
In Interact and Play a VR player WALKS (collisions, gravity, a 0.3 m step, snap turn)
and nothing else unless the scene's play block — or your module's `userData.play` on
its scene-root group, field by field — allows more:
```js
// feature-detect: an older core has no api.locomotion and would ignore the bounds
if (api.locomotion?.boundedTeleport) {
group.userData.play = {
locomotion: { teleport: true, worldGrab: false, fly: false },
bounds: { min: [-12, -0.5, -12], max: [12, 0.4, 12] }, // in THIS group's local frame
colliders: [{ min: [2, 0, -6], max: [2.3, 3, 6] }] // optional: solid boxes, same frame
};
}
```
- **`locomotion.teleport: true`** — the right stick's arc teleport, BOUNDED: the arc stops
at the first surface it meets; the landing must face up (normal y ≥ 0.7) or be the floor
plane, must lie inside `bounds`, and the straight line 1.1 m above your feet and the
landing must not cross a collider box, a mesh, or (with a dungeon raster published) a wall
cell. A refused landing draws the arc RED and releasing the stick does nothing; a valid one
is green and puts your FEET on it. Without `bounds` the play area is the scene's content
box pulled in 0.3 m. Edit mode's teleport is unchanged (it lands anywhere).
- **`locomotion.worldGrab: true`** — the grips move, rotate and SCALE the world the way they
do in Edit (two grips scale/rotate, the right grip alone pans) whenever a grip does NOT
start on something the player may hold (a grip on a dynamic body under
`interaction: 'grab'` still takes the body). For board games and instruments: Untangle,
the Jam Room.
- **`locomotion.fly: true`** — the left stick flies along the controller's aim, no gravity.
- **`bounds: {min: [x, y, z], max: [x, y, z]}`** — a sibling of `locomotion`. The scene's
play block gives it in scene coordinates; a module's `userData.play.bounds` is read in
that group's local frame. Keep `max.y` just above the floor if box tops and ledges must
not be landing spots.
- `vrControls.teleportVerdict(from, to, normalY?)` (on the debug hook
`window.__stores.vrControls`) answers `{ok, reason}` — `ok` `steep` `outside` `off-floor`
`wall-cell` `blocked` — for your e2e; `from`/`to` are world FEET points `[x, y, z]`.
`vrControls.teleportPreview()` reads the live arc (`{engaged, bounded, valid, reason, target}`).
**Your own VR menu is a panel.** A board, a level bar or buttons you draw in THREE for the
headset should be registered with **`api.vrPanel?.(group)`** (returns the undo, also run
when the module is disabled): it is then drawn OVER the scene — a floor, a base or a wall
between the player and it can never hide its buttons — and the controller laser ends on it
with its dot, even through whatever stands in front. Hit testing is unchanged. Core's own
panels (the radial menu, the game board, the wrist card) already work this way, and the
laser ends on your `registerInteractiveGroup` content too.
**In VR, your game's HUD is in the player's hands.** A screen with `input: 'menu'`,
a control on it, or bound to the `menu` / `paused` / `over` game state is drawn on
a board ~1.2 m in front of the player that follows their head lazily; its buttons
work with the laser and trigger and with a poke. The other screens (score, timer,
level) read as short lines on a card on the left wrist and a strip across the top
of the view. The board and the wrist card always carry an **Edit mode** button.
Nothing to do on your side — author the HUD once and it works on both.
**Hold the trigger and sweep (VR, Interact/Play).** While the trigger is held, the
controller TIP clicks each clickable it passes over — the one it is NEAREST to, so a
glide along a keyboard plays one key at a time — once, and again after it leaves and
comes back: piano keys, pads, toggles, HUD buttons; the laser does the same for things
out of reach. "Clickable" (also what earns the hover `tap`) means: under a group you
passed to `registerInteractiveGroup`, a part of an audio device (`userData.device` on
an ancestor — each key/pad is its own control), an object an On Click node targets, or
an object you mark `userData.clickable = true`. Your handler gets a second argument
`ctx = {source, mode}`: `source` is `'click'` (desktop), `'trigger'` (the VR press
itself) or `'sweep'` (a later entry while held). A control that must not be swept
(a knob you drag, a dot you carry) opts out:
```js
api.registerClickHandler(pickDot, { sweep: false }); // presses still reach it; sweeps do not
```
### The game shell: pause menu, levels, per-game settings (1.18, roadmap 31 K3)
Every game gets ONE pause menu from core — **Resume · Restart · Levels · Settings · How to
play · Main menu** — on the desktop (Escape in Play, or the corner Menu button) and in VR
(the left controller's **X**, or Menu on the game board / wrist card). You do not draw it;
you feed it. A scene counts as a game when it has a state-bound HUD screen, a spawn, or a
module publishing `userData.play` — or when your module registers levels. All of it is
feature-detected (`api.game.levels?.(…)`), LOCAL, and torn down with your module.
**Levels** — the picker on the desktop and on the VR board. Re-call it whenever a level
unlocks or earns stars; `onPick` never hears a locked level.
```js
const refresh = () =>
api.game.levels?.({
list: LEVELS.map((l, i) => ({ id: l.id, label: 'Level ' + (i + 1), locked: i > unlocked, stars: best[l.id] ?? 0 })),
current: currentLevel.id,
onPick: (id) => loadLevel(id)
});
refresh();
onLevelWon(() => { unlocked++; refresh(); });
```
**Your own settings rows** — shown under the core rows, persisted per game on this device
(`tp:game::`). `type` is `'toggle'`, `'choice'` (`options`, optional `optionLabels`)
or `'range'` (`min`, `max`, `step`). A core id is refused.
```js
api.game.addSetting?.({
id: 'board', label: 'Board', type: 'choice',
options: ['globe', '2d'], optionLabels: ['Globe', '2D board'], default: 'globe',
onChange: (v) => setBoard(v)
});
setBoard(api.game.setting?.('board') ?? 'globe'); // the stored choice at load
// (1.19) a CHOICE that decides which levels you see: `onLevels: true` also draws it as tabs
// above the Levels page's grid (desktop + headset) — Untangle's Board [Globe] [2D board]
api.game.setSetting?.('board', '2d'); // your in-game button writes the same row
api.game.onSettingsChange?.((values) => console.log(values.board, values.sfx));
```
**The core rows every game has** — `music`, `musicVolume` (0..100), `sfx`, `sfxVolume`,
`haptics`, `showFps`, `turning` (`default` | `snap` | `smooth` | `off`), `turnAngle`
(`default` | `15` | `30` | `45` | `90`), `vignette`, `quality` (`auto` | `low` | `medium` |
`high`). Core obeys them itself: the per-game volumes land on the audio BUSES (so
`api.playSound`, `api.music`, flow sound nodes and the scene's track all follow), haptics,
VR turning, the comfort vignette and the quality governor read them live. A module that
plays audio through its OWN WebAudio graph should read them:
```js
const gain = ctx.createGain();
const apply = () => (gain.gain.value = api.game.setting?.('sfx') === false ? 0 : (api.game.setting?.('sfxVolume') ?? 100) / 100);
apply();
api.game.onSettingsChange?.(apply);
```
**How to play** — your words first, then the device's controls (keyboard or controllers).
Without it the menu shows the game's Games-tab description.
```js
api.game.setHelp?.([
'Drag the dots until no two lines cross.',
'Finish a level to unlock the next — stars for fewer moves.'
]);
```
**Restart** — the menu resets the game shell (host / alone; others carry on as it is),
respawns the player and runs your hook:
```js
api.game.onRestart?.(() => { resetBoard(); spawnWave(1); });
```
**Your own Menu button** (a module HUD, a VR board of yours):
```js
api.game.openMenu?.(); // only while playing a game; false otherwise
api.game.closeMenu?.();
if (api.game.menuOpen?.()) pauseMyTimers();
```
### The game kit: rules, round, levels, score, pickups — `api.kit` (1.20, roadmap 34)
The logic every game used to re-build, done ONCE in core. Each piece is `api.kit.` AND a
**Kit: ** node group, both generated from one spec (`src/lib/kit/.spec.js`), so a
graph author and a code author get the same behaviour and the same words.
**The model — one writer.** The kit's shared state is ONE document that only the **authority**
peer changes (the physics initiator, else the lowest peer id — the peer Towers and Football
already pick). Call an action on ANY peer: elsewhere it is a request the authority applies,
exactly once. A kit **node** on a replicated pulse is asked by every peer with the same id, so
one press is one change (the `setvariable add` double count cannot happen). Reads are
replicated values; `on(fn)` fires on EVERY peer (play your sound there) and returns
`off`; registrations (`levels.define`, `pickups.register`, `rules.onGrabRequest`) are torn down
with your module. Late joiners get the document in the handshake and hear no history. A scene
clear resets the kit.
```js
const { round, levels, score, pickups, rules } = api.kit;
rules.set({ reach: 1.3, jump: 1.0 }); // every grab path + the VR jump obey
rules.onGrabRequest((req) => { if (req.name === 'Star' && !starFree) req.refuse('Build to the ring first'); });
levels.define({ id: 'mygame', list: [{ id: '1', label: 'Easy', par: { time: 60 } }, { id: '2', label: 'Hard' }] });
round.configure(3, 120, 'lose', 2); // 3 s intro, 2 min limit, lose on time, 2 s outro
round.onGo(() => api.announce('Go!'));
pickups.register({ id: gem.uuid, score: 10, respawn: 8, grants: { time: 5 } });
pickups.onCollected(({ by }) => api.playSound('coin'));
round.onWon(() => levels.complete(true, score.total())); // stars, unlocks, saved per device
levels.select('1'); round.start();
```
| piece | actions | reads | events |
|---|---|---|---|
| `rules` | `setReach(m)` `setJump(m)` `setBounds(min, max)` `clearRules()` · `set({reach, jump, bounds})` | `reach()` `jump()` `current()` `inside(p)` `clamp(p)` `checkGrab(req)` | `refused` (local) · `onGrabRequest(fn)` veto |
| `round` | `configure(intro, limit, 'lose'\|'win', outro)` `start()` `restart()` `pause()` `resume()` `win(reason)` `lose(reason)` `extend(s)` `toMenu()` | `phase()` `playing()` `elapsed()` `remaining()` `countdown()` `number()` `outcome()` `state()` `running()` | `started` `go` `paused` `resumed` `won` `lost` `results` `menu` |
| `levels` | `select(id)` `next()` `complete(won, score, time, level?, detail?)` `setMode(m)` · `define({id, list, unlock?, stars?, store?, merge?})` | `current()` `currentLabel()` `index()` `starsOf(id)` `unlocked(id)` `totalStars()` `mode()` `table()` `progress()` `resumeLevel()` | `selected` `completed` `unlockedNext` |
| `score` | `add(n, player?)` `set(n, player?)` `reset()` · `configure({autoReset})` `useGame(id)` | `total()` `mine()` `best()` `leader()` `of(id)` `leaderboard(n)` `results()` | `scored` `newBest` (local) |
| `pickups` | `collect(id, score?, respawn?)` `resetPickups()` · `register({id, score, respawn, radius, grants})` | `available(id)` `taken()` `left()` `takenBy(id)` | `collected` `respawned` `allCollected` |
- **`round` drives core's game singleton** (`intro`/`playing` → playing with a fresh core round,
so perRound content resets; `paused`; `won`/`lost` → over with the reason as outcome; `menu`),
so HUD `showWhile` screens and the pause menu keep working, and it ADOPTS a change it did not
make (a Set Game State node pausing). `restart()` works while playing; the pause menu's Restart
restarts a kit round by itself.
- **`levels`**: progress is per DEVICE (every peer saves what it saw earned); unlocks read it
merged with this session's results; `setMode` keeps the current level (progress belongs to the
level, not the mode). The table feeds the pause menu's level picker — do not also call
`api.game.levels`. `store: {get, set}` keeps your own save key (Towers keeps
`tp:mod:towers:progress`).
- **`score`** credits the asking peer unless a player is named (a shared pulse is asked by every
peer — name the player, or use a per-player trigger, when it matters who). A new kit round
zeroes it; the device best is checked when a round ends.
- **`pickups`**: TOUCH is built in — a player walking into a registered pickup takes it (their
own body, ~10x a second, in Interact/Play). A `Kit: Pickups ▸ Collect pickup` node with no
object wired takes its own graph's object.
- Prove your rules on the **headless logic sim** (`tests/unit/sim/logicSim.js`: N fake peers, a
fake clock, the real wire validator) — `createSim({peers: ['a', 'b']})`, then
`sim.peer('a').kit.impls.round.start(); sim.advance(3000)` — milliseconds, not e2e minutes.
### Behaviours: game logic as a small file, with a live node view (1.20, roadmap 34 R3)
A **behaviour** is game logic written as ONE self-contained JavaScript file — the code half of
"nodes vs code" (proposal §4): plain JS for what is a loop or a rule, a derived node view for
seeing it run. Add a **Behaviour** node (palette: Logic) to the scene graph (or an object's
graph — then `this.object` is that object) and write the file into it; **Open view** shows it.
```js
export default behaviour({
name: 'Waves spawner',
params: { interval: { value: 3, min: 0, max: 30, step: 0.5, unit: 's' }, waves: 5 },
state: { wave: 0, alive: 0 }, // replicated: every peer reads it
on: {
go() { this.startWave(1); }, // kit.round's "go"
died({ entity }) { // kit.health's "died", typed payload
if (!entity.is('robot')) return;
if (--this.state.alive === 0) this.after(this.params.interval, 'startWave', this.state.wave + 1);
}
},
startWave(n) {
if (n > this.params.waves) return kit.round.win('All waves cleared');
this.state.wave = n;
this.state.alive = n + 1;
kit.spawner.spawn({ kind: 'robot', template: this.find('Robot')?.uuid, count: n + 1, mover: { speed: 1.3 } });
}
});
```
- **params** — a literal (`waves: 5`) or `{value, min, max, step, unit}`; read as `this.params.x`.
In the node view each is a KNOB: dragging previews on your device, releasing rewrites the
literal in the file (one edit, one undo entry) and every peer reloads with it.
- **state** — plain JSON. Handlers run on ONE peer, the kit's **authority**; after each one the
state is sent to everybody (`bhv`, latest-wins). A late joiner gets it in the handshake; when
the authority leaves, the next one carries on from it. A source edit keeps it.
- **on** — event handlers: `start` (once per session), `grabRequest({piece, distance, refuse})`
(the kit.rules veto, asked on the GRABBING peer — read-only there), and every kit event as
`'piece.event'`, `pieceEvent` or a short alias (`roundStart`, `go`, `won`, `lost`, `died`,
`damaged`, `spawned`, `emptied`, `scored`, `collected`, `stuck`, `grabRefused`). An entity in a
payload has `is(glob)` / `hasTag(tag)`.
- **methods** — any other function: `this.name(...)`.
- **this** — `params`, `state`, `kit` (also bare `kit`), `after(s, 'method', ...args)` (session
clock; a method NAME survives the authority leaving, a closure does not), `cancel(key)`,
`rand()` / `randInt(a, b)` / `pick(list)` (seeded per event — never `Math.random`), `now()`,
`find(glob)` / `findAll(glob)` (scene objects by name or tag: `{uuid, name, pos, tags}`),
`object`, `isAuthority()`, `me()`, `log(...)`. Bare helpers: `dist(a, b)`, `clamp`, `lerp`.
- **The lint** stops a file from loading (on every peer, with the line) when it uses what would
make peers disagree or reach outside the game: `Math.random`, `Date`/`Date.now`/
`performance.now`, storage, the DOM/`window`/`globalThis`, network, `eval`/`Function`, bare
timers, `import`. Loops run under the loop guard (a runaway handler throws, the game goes on).
- **Lifecycle (T2)** — each behaviour is a module of its own (`behaviour:`): its kit
listeners and the entities it spawned are tracked; deleting or stopping the node takes them
away. Editing the source does NOT: the definition is swapped, the game keeps running.
- **The node view** (derived, read-only): ⚡ events → ƒ handlers/methods → ▣ state, ⊕ kit calls
(named exactly as the generated **Kit:** nodes), ⏱ timers, ✋ payload actions; ◆ params with
knobs. Live values 10x a second, a glow on what just fired (on every peer), timer countdowns,
errors with their line. Logic edits happen in the code (the view's **Code** panel).
- Prove a behaviour on the **logic sim**: `tests/unit/sim/behaviourSim.js` —
`const sim = createBehaviourSim({peers: ['a', 'b']}); await sim.load('w', source);` then drive
the kit and assert `sim.states('w')`. Examples: `static/behaviours/waves-spawner.js`,
`static/behaviours/towers-reach.js`.
### Water: `api.water` (1.22, roadmap 36)
```js
if (api.water) {
// a tank in the shared scene (replicated like Create -> Water; kinds tank | pool | ocean | cylinder)
const tank = await api.water.create({ kind: 'tank', preset: 'aquarium', size: [3, 1.5, 1.2], at: [0, 0.75, 0] });
api.water.configure(tank, { look: { deepColor: '#0b3d4f' }, bubbles: { enabled: true, rate: 12 } });
api.water.preset(tank, 'toxic'); // lava, swamp, toxic, ice, ocean, lake, river, pool, aquarium
const hit = api.water.query([0, 0.5, 0]); // {uuid, depth, surfaceY, flow} | null (waves included)
api.water.disturb(tank, [0.2, 1.5, 0], 0.4, 0.8); // a LOCAL ripple — call it on every peer
const off = api.water.onChange((volumes) => {}); // appear / edit / move / remove; off() or module unload
}
```
An object is water when `userData.water` holds a water blob (contract W1, `src/lib/water/volumes.js`): `{version: 1,
shape: 'box'|'plane'|'cylinder', level (local Y; null = top), density, linearDrag, angularDrag, flow: [x,y,z] (local
m/s), preset, look, waves, bubbles}`. `configure` merges a patch (look/waves/bubbles one level deep) and is replicated
and undoable like an Inspector edit; on a dry object it takes a whole blob. `query` is deterministic (the waves run
on the shared clock), so every peer's buoyancy agrees. `disturb` is visual only and NOT sent — drive it from shared
physics on every peer. `burst(uuid)` fires the bubble burst for everyone (the moment rides the blob). Feature-detect
it (`api.water`): 1.21 and older cores do not have it.
### Physics (P-A)
All mutations are INITIATOR-ONLY — the peer that started the simulation steps
the world (golden rule: authoritative, never mixed with deterministic). The
blessed recipe for driven physics (pong's paddle pattern): every peer forwards
its INPUT via `api.send({op:'drive', ...})` at ~20Hz, and only the peer where
`api.physics.isInitiator()` is true applies it.
```js
api.physics.isInitiator(); // true while THIS peer runs the sim
api.physics.applyImpulse(uuid, [0, 5, 0]); // push a dynamic body (initiator-only)
api.physics.applyTorqueImpulse(uuid, [0, 2, 0]); // spin a dynamic body (world axes)
api.physics.setJointMotor(jointId, vel, maxForce); // drive a revolute joint
api.physics.joints(); // Promise
```
The **car module** (now in the [modules repo](https://github.com/theprototype-app/modules), `modules/car/`) is the worked example: replicated
primitives + motorized revolute joints, click-to-claim (pong's paddle
pattern), driver forwards `{op:'drive', throttle, steer}` at ~20Hz and only
the initiator applies wheel motors. Driving + the chase camera engage only
in Play mode with a running simulation (the module claims `'keys'` and uses
possess's `startFollowCam` while engaged; the claim itself works anytime).
### Building in the shared scene (17-A)
```js
// the replicated /create, handing back what appeared
const [uuid] = await api.create('/create Box 1 1 1');
await api.create('/create Box 0.6 0.6 0.6', { at: [x, y, z] });
api.moveObject(uuid, { pos, rot, scale }); // the editor's replicated move
api.physics.set(uuid, { mode: 'dynamic', mass: 30 });
api.physics.createJoint('revolute', a, b, 'x', { vel: 0, maxForce: 120 });
api.physics.running(); // a sim runs somewhere in the session
api.isPlaying(); // Play mode active
api.peerIds(); // roster - free a departed peer's state
// LOCAL only (a peer's module must never yank your viewpoint):
api.flyTo([x, y, z], [lx, ly, lz]);
api.playSound('pluck', [x, y, z]);
api.followCam(uuid); api.stopFollowCam();
```
### Possess (K-D)
```js
// drive any object with WASD/arrows or the VR left stick (tank controls),
// chase camera by default; Esc releases. Possessing SELECTS the object
// (selection = lock), suspends its flow effects, and records ONE undo entry
// on release. Movement replicates as plain throttled moves.
api.possess(api.selectedUuid(), { camera: 'chase' }); // 'chase'|'orbit'|'none'|'first'
api.releasePossess();
api.possessModes; // this build's camera modes — feature-detect 'first' here
// (an unknown camera value degrades silently on old builds)
// 17-A1 first person: eye at the object + eyeHeight; mouseLook = pointer lock
// (X turns the OBJECT so movement follows the look, Y pitches the camera;
// leaving the lock — Esc — releases the possession)
api.possess(uuid, { camera: 'first', eyeHeight: 1.7, mouseLook: true });
```
### Audio devices (23-A5)
An instrument, an effect or a speaker is a DEVICE: an object carrying
`userData.device = {kind, params}`, with a WebAudio subgraph the engine builds for it.
Your module supplies the kind; core supplies the object, the replication, undo, saving,
the cables and the clock.
```js
const kind = await api.registerAudioDevice({
kind: 'piano', // namespaced to mod--piano
label: 'Piano', icon: '🎹', group: 'Keys',
ports: { in: [], out: [{ id: 'out', kind: 'audio' }] }, // 'audio' | 'cv' | 'midi'
params: [{ key: 'level', kind: 'range', min: 0, max: 1, step: 0.01, default: 0.8 }],
build(ctx, node, params) { // ctx = the SHARED AudioContext; runs per object
const out = ctx.createGain();
out.gain.value = params.level;
return { output: out, out, dispose() { out.disconnect(); } };
},
onParam(h, key, value) { if (key === 'level') h.out.gain.value = value; },
onNote(h, { note, velocity, at }) { // at = a WALL-CLOCK stamp
const v = api.audio.voice({ freq: 440 * 2 ** ((note - 69) / 12), gain: velocity * 0.5, destination: h.out });
const t = api.audio.timeFor(at); // never Date.now() maths of your own
v.start(t); v.stop(t + 0.5);
},
mesh: (THREE) => new THREE.Mesh(new THREE.BoxGeometry(1, 0.2, 0.4), new THREE.MeshStandardMaterial({ color: '#222' }))
});
api.audio.addDevice('piano', { position: [0, 1, 0] }); // also in the viewport Add menu > Devices
```
- `params` render in the Inspector and the toolbox; every write replicates and undoes.
Keep `onParam` cheap and `build` pure: a peer WITHOUT your module holds the same
object as an inert placeholder with its document intact, and rebuilds it the moment
your module registers.
- Children of your mesh named `port:` are where cables attach.
- Sound goes NOWHERE until cabled: a speaker device connects its `input` to
`api.audio.bus('instruments')`; `api.audio.cable({from: {uuid, port}, to: {uuid, port}})`
plugs one in.
```js
api.audio.voice({ freq: 220, type: 'sawtooth', filter: { freq: 900 } }); // {output, start(t), stop(t), dispose()}
api.audio.voice({ buffer, loop: true, destination: 'instruments' }); // a sample voice
await api.audio.sample(hash); // decoded AudioBuffer for an Explorer content hash,
// pulled from a peer if missing (null after 30 s)
api.audio.bus('sfx'); // a bus to connect to
api.audio.context(); // the shared AudioContext — never make your own
api.audio.transport(); // {bpm, beat, bar, step, phase, playing, loopBeats, swing}
api.audio.play(); api.audio.play(false); api.audio.setBpm(100); // the SHARED transport
const cancel = api.audio.schedule(0, ({ beat, at }) => hit(at), { every: 1 }); // a metronome:
// called ~100 ms EARLY with the exact audio time;
// start voices at `at`; cancelled at teardown
api.audio.note(uuid, { note: 64, velocity: 0.8 }); // replicated; every peer synthesizes it
api.audio.setParams(uuid, { level: 0.5 }); // one undo step, replicated
const before = api.audio.device(uuid); // capture the document when a gesture STARTS
api.audio.previewParams(uuid, { level: 0.5 }); // a live gesture: replicated, NO undo entry -
// throttle it while scrubbing, then commit once:
api.audio.setParams(uuid, { level: 0.5 }, { before }); // one undo entry that restores `before`
api.audio.captureMic(); // Promise: the RAW mic, a separate capture
// from voice chat (no AEC/NS/AGC, never gated by PTT)
const item = await api.audio.record({ maxSeconds: 8 }); // a take -> Explorer item (content-hashed,
// shared to peers); null when refused BEFORE it starts
api.audio.stopRecording(); // end the take early
const dest = api.audio.context().createMediaStreamDestination(); // bounce a NODE instead:
myInputGain.connect(dest); // a looper records what is patched into it
await api.audio.record({ maxSeconds: 8, stream: dest.stream, name: 'loop' });
// declare what a kind references by hash so the Scene manifest (and a .tpscene export)
// carries the bytes: `assets(params) -> [{hash, name}]` on the registerAudioDevice spec
api.audio.device(uuid); // {kind, params} or null
```
The rule behind `schedule` and `onNote`: **anything scheduled on the transport must be a
pure function of its arguments.** Every peer runs the same pattern from the same
`startedAt`; an impure callback desyncs silently, per peer, with no error anywhere.
### Misc
```js
api.registerMenu('Generate dungeon', () => generate()); // sidebar "Modules" section
api.registerVRMenuEntry({ // sector in the VR radial menu
id: 'spawn', group: 'root', label: 'Dungeon', // group 'root' = base ring;
action: () => generate(), closes: true // other names make a sub-ring
}); // (id gets prefixed moduleId:)
api.sceneAssets(); // Promise<[{group, name, kind, hash}]> — what the shared
// scene uses right now (audio/config/textures manifest)
api.scene(); // THREE.Scene
api.objectsGroup(); // the replicated objects root (add scene content here)
api.peerId(); // our peer id (undefined before the mesh is up)
api.toast('hi'); // toast in the corner
api.haptic(0.6, 60); // buzz the VR controllers (no-op on desktop and in Edit);
api.haptic(0.6, 60, 'right'); // optional hand targets one controller (17-A1)
api.hapticPattern('success'); // a named pattern (1.17, see Game feel)
api.isVR(); // true inside a VR session
api.vrHand('left'); // one hand's WORLD pose + buttons, or null:
// {position, quaternion, trigger, gripped, connected}
api.fireObjectClick(uuid); // pulse On Click flow nodes targeting the object
// (replicated) — user graphs react to module events
api.now(); // the runtime clock in seconds — stamp replicated
// timestamps with this, never Date.now() directly
```
Objects your module adds to `objectsGroup()` are part of the shared scene:
they get GLTF-synced to late joiners, appear in the object list, and can be
moved/deleted by anyone. Objects that are *derived* state (e.g. a generated
dungeon) are better added outside `objectsGroup()` and rebuilt from your
module state — then they can't drift.
## Walkthrough: the hello module
`src/modules/hello/module.js` is the smallest complete module: one node group
+ one effect, zero messages (fully deterministic). Open the flow editor, drag
in **Wave (hello)** and an **Object Selector**, pick an object, connect them —
the object rocks on every peer.
## Walkthrough: build a door (button module)
The button module (`src/modules/button/`) shows the interactive pattern:
1. Add a **Button** from the sidebar Modules group (or use any object).
2. Add a **Wall/Cube** where the door should be.
3. Open the flow editor: drag in **Button trigger** and an **Object Selector**.
4. In the Button trigger node pick the button object; in the Object Selector
pick the door object; connect trigger → selector.
5. Click the button in the viewport (or pull the VR trigger on it): the door
slides up by `height` on every peer. Toggle mode keeps it open, push mode
springs back after 1.5 s.
How it replicates: the click writes `{pressed, at: api.now()}` into the node's
data (replicated like any node edit); the slide is a pure function of
`(data, time)` running on each peer — no motion messages at all. The
`registerClickHandler` consumes the click so pressing doesn't select the button.
## Walkthrough: the towers module (a game's rules as a core module)
`src/modules/towers/` runs the **Towers** template's twelve levels. It shows how a game with
real rules splits: the scene (the template) is the arena, the piece templates, the HUD and a
small graph; the module is the rules. Worth copying:
- **The rules as a pure leaf** (`levels.js`) — the level table, the star rule, the unlocks,
the verdict stepper — tested with vitest, no browser.
- **One authority** decides anything shared (the physics initiator, else the lowest peer id):
it deals the pieces, judges the round and writes the result into game variables
(`api.game.setVar`); every peer derives its HUD words from those through a value node wired
into HUD Text's format (`api.registerValueNode`).
- **HUD buttons without a presser**: a press is a replicated stamp on a `hudbutton` node, so
every peer watches `api.flow.triggerStamp` and only the authority acts (first sight = history).
- **Progress on this device** with `api.storage` (stars, unlocks — never replicated).
- **The shell's level picker, help and restart** (`api.game.levels`, `setHelp`, `onRestart`),
feature-detected so the module also runs on an app without them.
- **Since 1.20 it runs on the game kit** (`api.kit`): its levels, unlock chain, stars and saved
progress are `kit.levels` (with Towers' own store and star rule), its round is `kit.round`
(Restart while playing), its tallest tower is the round's `kit.score`, and its reach + jump
are `kit.rules` — what stays in the module is Towers itself.
## Manager, dev mode & gallery (17-A2/A3)
Modules install, update, disable and remove **live** — `unloadModule` (33's
`deactivateModule`) runs the module's lifecycle registry (see **The lifecycle**
above), so nothing needs a page reload. Since 1.20 that includes CORE modules: their
hooks outside the api (vrsleeve's VR hook registries) undo through `api.onUnload`.
Every user-module card carries a **Dev URL** row: **Reload** fetches fresh code
(cache-busted), evaluates it FIRST, then tears down + re-registers — a broken
body keeps the old instance running. **Auto** polls (~2s) and reloads on
change. Peers toast the `{id,version}` mismatch while you iterate (correct —
the dev peer differs). The **Browse** tab lists the community repo
(`theprototype-app/modules`, `index.json` via jsDelivr — `moduleGallery.js`);
installs go through `installUrl` with the entry's source folder, so Update and
the dev reload keep working on gallery installs.
## Checklist before you ship one
- [ ] Everything a user can do through your module looks the same on a second
connected browser (test with two windows).
- [ ] A peer who connects *after* your module did something catches up
(state sync or derivable-from-scene).
- [ ] No `Math.random()` without a broadcast seed; no accumulation in effects.
- [ ] Receiving a message never re-broadcasts it.
- [ ] `id` unique, node `type`s unique, version bumped on behavior changes.
- [ ] Switch it off and on again in the Modules manager (or dev-reload it): no timer,
listener, sound, overlay or scene object of the old instance is left, and the new one
works from scratch (`__stores.moduleSDK.registrationsOf(id)` is `{}` while it is off).