# theprototype.app > Collaborative peer-to-peer 3D prototyping in the browser. No accounts and no > server beyond PeerJS signaling — a session is a direct WebRTC peer mesh, so > everything one person does is visible to everyone connected. Desktop and VR > (WebXR), a node/flow editor that drives scene behavior, and a module SDK for > loadable, self-contained play content. Built with SvelteKit (static adapter), > Svelte 5, Threlte and three.js. ## Modules (write your own) - [Module SDK guide, inlined](/llms-full.txt): the full guide — the api surface, the one rule that matters (a module runs on every peer), layout/registration, and worked examples. Fetch this to write a correct module without guessing the API. ## Key concepts - Everything replicates over the peer mesh: apply the change locally, then broadcast it; a receiver applies it WITHOUT re-broadcasting. - Two sync models — deterministic (a pure function of a seed/params plus a shared synced clock) vs authoritative (one initiator simulates and broadcasts). Pick one per feature; never mix them. - A module is a self-contained default export `{ id, name, version, description, register(api) }` in `src/modules//module.js`; user modules (zip/URL) must have no imports. - Script and flow nodes are pure functions of `(object, base, data, time)` evaluated identically on every peer — never stream their outputs. - Randomness must be seeded: send the seed, not the result, or peers desync. ## Docs - [llms-full.txt](/llms-full.txt): this map plus the entire module SDK guide inline (works offline / without repo access). - [Source repository](https://github.com/theprototype-app/core): MIT-licensed code; CLAUDE.md in the repo root documents the architecture, replication rules and conventions agents should follow when contributing. - [README](https://github.com/theprototype-app/core#readme): feature overview, quick start, self-hosting notes. - [Module SDK guide, raw](https://raw.githubusercontent.com/theprototype-app/core/main/MODULES.md): the same guide as llms-full.txt, straight from the repo. - [Releases](https://github.com/theprototype-app/core/releases): changelog per version. - [User docs](https://docs.theprototype.app): guides for people (not agents). --- # Module SDK — full guide (MODULES.md, inlined) # Writing modules Modules are self-contained folders under `src/modules//` that plug into the scene, the flow node editor and the peer mesh. Music instruments, playable game prototypes, generators, custom controls — anything a user should be able to drop into a session and every connected peer can see and interact with. ## The one rule that matters **A module runs on every peer.** There is no server. Whatever your module does must end up identical on all clients, one of three ways: 1. **Deterministic from shared inputs** — effects driven by node `data` (already replicated) and the synced clock. No extra work needed. Prefer this. 2. **Broadcast events** — a discrete thing happened ("button pressed", "generate with seed 42"). Send a small message with `api.send`, apply the same change in the sender and in `api.onMessage`. Never re-broadcast from a receiver. 3. **State sync for late joiners** — someone connects mid-session and needs your current state (score, generated layout, toggles). Provide `registerStateSync`; the handshake delivers it automatically. Randomness must be seeded (send the seed, not the result — see the dungeon module). `Math.random()` in anything replicated is a desync. ## Layout and registration ``` src/modules/ index.js <- list your module here (all peers need the same list) mymodule/ module.js <- export default { id, name, version, register(api) } SomethingNode.svelte (optional custom node UI) ``` ```js // src/modules/mymodule/module.js export default { id: 'mymodule', // stable, unique; used to route your messages name: 'My Module', version: '1.0.0', // peers toast when versions differ register(api) { // wire everything here } }; ``` Peers exchange `{id, version}` lists when they connect and warn when a module is missing or a different version on the other side. That's advisory — the session still works, but replicated behavior of that module may differ. ## The lifecycle: load, unload, reload (34 R6) A module can be **unloaded and loaded again while the app runs** — switched off in the Modules manager (core modules too, since 1.20), removed, updated, dev-reloaded from its Dev URL, or unloaded by a scene switch. `unloadModule(id)` takes down **everything the module registered**, newest first, and `register(api)` can then run again with fresh code. Nothing needs a page reload, and nothing is left running for a module that is gone. **What is undone for you.** Every registration you make through `api` is recorded in the module's lifecycle registry (one registry, keyed by your module id — `src/lib/sdk/lifecycle.js`) and undone at unload: node groups, effects, value nodes, node defs you seeded and the user did not edit, primitives, click/drop handlers, frame tasks, scene-clear hooks, interactive / system / listed groups (the scene-root group is removed AND its geometries, materials and textures are freed), the spawn override, message handlers and state sync, menus, toolboxes (closed) and their shortcuts, VR menu entries, key bindings, input listeners and claims, possess, the follow camera, VR panels, hit listeners, your music track, game levels / settings rows / help / restart hooks / the forced menu, LOD handles, quality / flow / game / settings / peer-variable listeners, unwrap / shader / post backends, post effects, audio device kinds, engine voices (`api.audio.voice`), transport events (`api.audio.schedule`), your copy of the mic stream, a recording you started, HUD rows / debug lines / actions / element kinds. Every `off()` the api hands you still works, and calling it early also drops the registry entry — subscribe and unsubscribe as often as you like. A call that REPLACES (`api.game.levels` called on every unlock, `setHelp`, `addSetting` with the same id, a toolbox or VR menu entry re-registered under the same id) replaces its entry rather than stacking one per call. **What stays.** What your module created in the SHARED scene (`api.create`, objects in `objectsGroup`, nodes from `api.flow.addNodes`, audio devices and cables, node data you wrote) is user content: it stays, replicated, like anything a person made. So do the shared game variables and your peer-variable row, and what you saved on this device with `api.storage` (a reinstall keeps progress; `storage.clear()` is your reset). **What only you know about — four tools:** ```js // your own teardown: a DOM overlay you appended, a worker, a cache, a raw WebAudio graph // you built on api.audio.context(). Runs at unload (newest first, so a hook added at the // end of register() runs while everything before it is still in place). Returns cancel(). api.onUnload(() => overlay.remove()); // timers that die with the module (the window's signatures) + what is pending right now const h = api.timers.setInterval(refresh, 500); api.timers.clearInterval(h); api.timers.setTimeout(fn, ms); api.timers.requestAnimationFrame(loop); api.timers.pending(); // {timeouts, intervals, frames} // an event listener removed at unload — window, document, the canvas, anything; returns off() const off = api.listen(window, 'keydown', onKey, true); // a scene-root object that is YOURS: removed at unload, its GPU resources freed (shared // ones kept). Returns the object, so: api.scene().add(api.own(group)). Never for content // inside objectsGroup (that is shared — it is left alone). api.own(group); ``` **Installed modules (zip / URL) get timers and window listeners tracked automatically.** The entry is evaluated with its bare `setTimeout` / `clearTimeout` / `setInterval` / `clearInterval` / `requestAnimationFrame` / `cancelAnimationFrame` bound to tracked ones (a one-line prologue on line 1, so line numbers in stack traces are unchanged), and a listener your code adds to `window` or `document` is recorded too — at unload they are cleared and removed. Two edges: `window.setTimeout(...)` (or a listener on any other target) is not tracked — use `api.timers` / `api.listen`; and a module that declares one of those six names itself at top level is loaded without the prologue (its timers then untracked). A core module (in `src/modules/`) uses `api.timers` / `api.listen` explicitly. **Keep state inside `register()`.** The browser keeps every imported module's TOP-LEVEL scope for the life of the page (an ES module is never unloaded), so module-level variables, caches and DOM survive an unload — the next load starts from them. State declared inside `register()` is fresh every time; module-level state you must reset belongs in `api.onUnload`. Each reload of an installed module also keeps the evaluated source itself, so a large module costs some heap per reload — fine for a development loop, not something to do every second. (Measured with `module-lifecycle-repo`, 3 cycles each: 19 of the 20 repo modules reload flat; Waves keeps one DOM node and ~0.3 MB of heap per reload.) ### Adding an SDK surface (core contributors) Every member of the api declares what it does to a module's lifecycle, beside its slice (`src/lib/sdk/.js`): ```js export function sdkThing(ctx) { const { moduleId, onDispose, owned } = ctx; return { registerThing(spec) { thingRegistry.set(key, spec); // the undo, journaled with a KIND (the debug view names it); `{key}` makes a // re-registration under the same key REPLACE its entry onDispose(() => thingRegistry.delete(key), 'thing', { key: 'thing:' + key }); }, onThing(fn) { // a registration that hands back an off(): owned() returns an off that also // drops the journal entry return owned('thing.listener', subscribe(fn)); }, thingCount: () => thingRegistry.size }; } sdkThing.surface = { registerThing: 'registers', onThing: 'registers', thingCount: 'read' }; ``` - Kinds: `registers` (leaves something behind: MUST journal its undo), `action` (does something now, leaves nothing of the module's), `content` (shared, replicated content that deliberately survives an unload), `read`, `value` (a non-function). - A `registers` member needs a fixture in `tests/fixtures/sdkLifecycleFixtures.js`: register it from a real module, read CORE's registry (never the journal), unload, read it gone. The vitest `moduleLifecycle` fails for an undeclared member, a stale declaration, a `registers` member with no fixture, and a fixture whose registration survives `unloadModule`; the browser suite `module-lifecycle` runs the members node cannot load. - **Async registrations journal SYNCHRONOUSLY.** A disposer recorded inside an `import().then` lands in the module's NEXT journal whenever an unload comes first — the registration then outlives the unload. Record it before the import and let a late registration undo itself: `let off = null, gone = false; onDispose(() => { gone = true; off?.(); }, 'kind'); import('../x').then((m) => { off = m.register(...); if (gone) off(); });` - A core subsystem acting FOR a module outside the api (kit entities, loaded models) records its undo with `trackModuleResource(moduleId, kind, undo)` from `moduleSDK.js`. - Debug: `__stores.moduleSDK.registrationsOf(id)` / `allRegistrations()` (live entries by kind), `moduleScopeDebug()` (an installed module's tracked timers + listeners), `unloadModule(id)` returns what it disposed, by kind. - Tests: vitest `moduleLifecycle`; e2e `module-lifecycle` (the browser contract, an installed probe module, and the leak test: each bundled module unloaded/loaded 3x with handlers, nodes, audio connections, timers, meshes, materials, GPU memory, DOM, JS listeners and the heap measured) and `module-lifecycle-repo` (every zip in the modules repo through 3 cycles). ## API reference (v1) ### Flow nodes ```js api.registerNodeGroup( { group: 'Modules', // palette/context-menu group header items: [ { type: 'wave', // globally unique node type label: 'Wave (hello)', defaults: { amplitude: 0.4 }, // seeds node.data, replicated with the node params: [ // optional: auto-generated controls { key: 'amplitude', kind: 'range', min: 0, max: 1.5, step: 0.05 } // kind: 'range' (min/max/step), 'select' (options: [...]), 'toggle', // or 'text' (placeholder?, maxLength?) — a text param writes on COMMIT // (change/blur), never per keystroke: a node edit replicates the whole node ] } ] }, { wave: MyWaveNode } // optional: custom Svelte components per type ); ``` Items with `params` render with the built-in generic node UI (sliders/selects that write through `setNodeData`, which replicates). Custom components should follow the same pattern: render from `data`, write with `setNodeData(id, {...})` from `$lib/nodesHandler`, never `bind:` to data. ### Per-frame effects (node → Object Selector) ```js api.registerEffect('wave', (object, base, data, time) => { // base = the object's logical transform: {pos, rot, scale, visible} // The runtime restores base before each frame — apply offsets FROM base. object.rotation.z = base.rot[2] + Math.sin(time * (data.speed ?? 2)) * (data.amplitude ?? 0.4); }); ``` Effects run when an edge connects your node to an Object Selector node. `time` is wall-clock synced across peers (when "Sync animations" is on), so pure functions of `(data, time)` stay identical everywhere. Don't accumulate (`rotation.z += ...` breaks determinism) — always compute from `base` and `time`. An effect takes an optional **5th argument** `{id, graphId}` — its own node id and the graph it sits in, so one module can host many instances of the same node type. It is additive: a four-parameter effect is unchanged. ```js api.registerEffect( 'wave', (object, base, data, time, ctx) => { // ctx.id = this node's id, ctx.graphId = 'scene' or the owning object's uuid }, { inputs: { who: 'object' } } // optional: typed named inputs, see below ); ``` ### Value nodes: your state into core nodes A node that **outputs a value**, so module state can drive core nodes — a score into a HUD Text, a level into Map Range, a flag into a Gate. ```js api.registerValueNode( 'score', (data, time, ctx) => Number(data.base ?? 0) * Number(data.mult ?? 1), { vtype: 'number', // output socket type; also boolean/vector3/color/object/event inputs: { mult: 'number' } // typed named inputs, resolved BEFORE fn runs } ); ``` **The evaluator MUST be a pure function of its arguments** — the same rule Script nodes follow, and the one way to break a session silently. Values are never sent: every peer evaluates the node itself from the replicated node data and the shared clock. Read unreplicated local state here (a mouse position, a plain module `Map`, `Math.random()`) and every downstream consumer diverges per peer with no error anywhere. Keep mutable state in a replicated place — `registerStateSync` / `api.send` — and read *that*, or let the value ride the node's own `data`. `inputs` matters more than it looks. Without a declaration every handle types as `number`, which **refuses** an Object Selector wire (`object -> number` is not a coercion) and renders no target socket on the card at all. `data.` is the wired value when wired and the node's own param when not. `registerEffect` accepts the same `{inputs}` option. ### Firing your own events ```js // pulse every instance of the type... api.fireNodeTrigger('levelcleared'); // ...or just the ones you mean api.fireNodeTrigger('goal', (data, id) => data.team === 'blue'); ``` Register the node with `{vtype: 'event'}` so it can be wired to a Counter or an Object Selector. **This REPLICATES**, exactly like `fireObjectClick`: the pulse rides the existing `nodetrigger` message from one peer's stamp, and every peer then computes the identical pulse. So call it on the peer where the event happened and **not on all of them**, or a Counter counts it once per peer. `api.peerIds()` and `api.physics.isInitiator()` are the usual ways to pick that peer. R3a: pass `{replicate: false}` as the third argument to keep the pulse in **this peer's own trigger log** — the per-player mechanism (a per-player pickup hides only for its collector because the pulse never left their machine). Your effect/value node then reads its own pulse through `ctx.trigger` (`{stamp, age}` or null), which core has ALREADY folded through the round rules: stamp a `perRound: true` flag on your node's data and a round bump (or a return to the menu) retires the read with no round math of your own. That is the whole latch story — never rebuild it. ### The game shell, per-player rows, and the graph (R3a) ```js api.game.roundCutoff(); // null (shell unused) | Infinity (menu/over) | startedAt api.game.roundUnderway(); // true while playing/paused (and when the shell is unused) api.game.playActive(); // THIS peer plays inside a running round api.game.getVar('score', 0); // the shared game variable (replicated singleton) api.game.setVar('score', 7); api.peerVars.setMine('laps', 3); // MY replicated row — one writer per row, by construction api.peerVars.mine('laps', 0); api.peerVars.all('laps'); // [{id, name, value, me, rank}] — the leaderboard shape api.playerPosition(); // [x, y, z] of the viewer — a touch trigger's read api.selectObject(uuid); // the manager-row click api.selectedUuids(); // the selection SET (never the sticky primary) api.flow.nodes('mytype'); // [{id, type, graphId, data}] across every graph api.flow.edges(); // [{id, source, target, sourceHandle, targetHandle, graphId}] api.flow.nodeValue(latchId); // a node's evaluated output (a core Latch's round-aware state) api.flow.triggerStamp(nodeId); // {stamp, age} | null — a node's own round-aware pulse api.flow.setNodeData(nodeId, { respawn: 5 }); // replicated MERGE (the editor's nodedata path) api.flow.addNodes({ // replicated, ONE undo entry for the batch nodes: [{ type: 'mytype', x: 60, y: 40, data: {} }, { type: 'objectselector', x: 280, y: 40, data: { selected: uuid } }], edges: [{ from: 0, to: 1 }] // indices into nodes, or existing node id strings }); api.flow.seedGraph({ // 36 (DEVX #10): the same, ABSENT-ONLY — a wired example key: 'example', // graph on module load; nothing is added once any node is nodes: [/* … */], edges: [] // marked with this seed or is one of THIS module's types }); api.hud.registerDebugLine(() => 'mymod: 3 left'); // a line on the debug HUD pill api.hud.registerAction({ key: 'showleft', label: 'Show it', group: 'Data', role: 'drives', node: '', via: { node: 'mytype', data: {}, handle: 'value' } }); ``` Graph reads are deterministic because the graph is replicated — treat them exactly like replicated state (the value-node rule). Two per-node data flags core honours on YOUR nodes: `perRound` (the trigger reads retire on a round bump) and `whilePlaying` (an effect node stands down outside play and the restore loop hands its object back — how a module hides an object without owning the give-back). ### Primitives ```js api.registerPrimitive( 'Flag', // capitalized: /create Flag 2 1 (w, h) => new THREE.PlaneGeometry(+w || 2, +h || 1), { label: 'Flag', command: '/create Flag 2 1' } // spawn button on your manager card ); ``` The builder runs on every peer from the same `/create` command, so it must be deterministic in its arguments. ### Interaction ```js api.registerClickHandler((object) => { if (object.userData.myButton) { press(object); // apply locally api.send({ op: 'press', uuid: object.uuid }); // tell peers return true; // consume the click (no selection) } return false; }); ``` Handlers see desktop clicks and VR trigger presses (the exact mesh hit, not the top-level group). Return `false` to let normal selection continue. **Modes (1.17).** The editor has two click modes, Edit (every click selects) and Interact (play-style clicks and drags without starting the game), plus Play. A handler runs in Interact and Play by default, so an Edit click on your game piece selects it like any object. Say so when you want something else: ```js api.registerClickHandler(pressKey, { modes: ['interact', 'play'] }); // the default, spelled out api.registerClickHandler(pickTool, { modes: ['edit'] }); // an editor TOOL ``` In a headset the same modes apply (1.17, 30b): Play in VR enters Interact, where the trigger clicks (module handlers, On Click) and selects nothing; in Edit it selects as before. Handlers also get `ctx = {source, mode}` — see *Game feel* below for the hold-and-sweep and `{sweep: false}`. `api.editorMode()` reads the mode on this screen (`'edit'` | `'interact'`) — for a module whose own pointer listeners run outside click routing (a drag), so it can stand down in Edit. **Listed in the object list (1.17).** Content you build at the scene root shows in the object list's *Module content* section — a read-only row that frames it, hides it locally, or opens your toolbox. Give it a readable name: ```js api.registerListedGroup('piano-module', { label: 'Piano' }); ``` ```js api.registerFrameTask((time) => { /* runs every frame, synced time */ }); // click handlers only see the replicated objects root by default; if your // module adds its own group at the scene root, register it for clicks: api.registerInteractiveGroup('pong-module'); // an Explorer AUDIO/TEXT item dropped on one of your meshes (23-C2): take it or not. // `hit` is the exact mesh under the drop, `item.hash` feeds api.audio.sample(hash). api.registerDropHandler((hit, item, target) => { if (item.kind !== 'audio' || !hit.userData.pad) return false; api.audio.setParams(deviceOf(hit).uuid, { ['pad' + hit.userData.pad]: item.hash }); return true; // consumed }); ``` ### Toolboxes: a real UI surface (A5) ```js const id = api.registerToolbox({ id: 'settings', // namespaced to mod--settings title: 'Dungeon Kit', width: 240, shortcut: 'Ctrl+Shift+D', // optional; also lists in Settings > Shortcuts playMode: false, // default: hidden in Play mode sidebar: false, // default true: also a row in the burger menu's Modules // section. false = viewport menu only (see below) mount(el) { const label = document.createElement('div'); label.className = 'tbx-label'; // the shell styles this for you label.textContent = 'Rooms'; const go = document.createElement('button'); go.className = 'tbx-primary'; go.textContent = 'Generate'; go.onclick = () => regenerate(); el.append(label, go); return () => { /* your cleanup */ }; } }); ``` Write plain DOM into the node `mount` receives and you inherit the app's own tool-palette treatment: header drag with position persistence, the width grip, z-band focus, the bottom SHEET at <=640px, and the whole `.tbx-*` CSS contract (`.tbx-label`, `.tbx-row`, `.tbx-btn`, `.tbx-seg`, `.tbx-primary`, `.tbx-check`, `.tbx-danger`) with no CSS of your own. That is the point — before this seam, module controls could only live behind `registerMenu`, two clicks deep inside the Modules *modal*, which then had to be closed before the module's own overlay was usable. The user opens it from the **sidebar's Modules section** and the **viewport menu** (both from one builder, so they cannot drift), plus your `shortcut` if you name one. It starts CLOSED: a palette that appears uninvited is the thing `registerMenu` was avoiding. `onOpen`/`onClose` fire on each transition; the header ✕ closes it. **Where the way IN belongs is your call.** The burger menu's Modules section is the app's permanent chrome, so a row there is a standing claim on it — right for a tool a user reaches for constantly, heavy for a window that belongs to one workflow. Pass `sidebar: false` to keep the viewport-menu row and drop the permanent one, and open it from where your module already is: ```js const id = api.registerToolbox({ id: 'manager', title: 'Collectibles', sidebar: false, mount }); // a button on YOUR card in the Modules manager, beside Update/Remove api.registerMenu('Open Collectibles', () => api.openToolbox(id)); ``` `api.openToolbox(id)` / `closeToolbox(id)` / `toggleToolbox(id)` take the id `registerToolbox` returned. `openToolbox` also dismisses the Modules manager when it is open, because the manager is the one piece of chrome that can cover a toolbox — a card button that opens a window *behind* the dialog it was clicked in is the complaint this whole seam exists to answer. It is a no-op when the manager is closed. `mount` returns its cleanup and a re-registration re-runs it, so dev-mode live reload rebuilds the contents in place. Disabling or removing the module force-closes and unregisters the toolbox — it never leaves a window behind a dead mount fn. A toolbox is **LOCAL**: it is this viewer's window, and nothing about it replicates or is saved with the scene. What it *changes* must still go through the replicated paths (`api.send`, `api.create`, `api.physics.set`). ### HUD elements: rows, and your own element kind (21-E7) ```js // fill a HUD List element (kind `list`) by id api.hud.rows('leaderboard', ['1. Ada 12', '2. Grace 9']); api.hud.clearRows('leaderboard'); // back to the element's authored rows // the roster WITH NICKNAMES - what a leaderboard actually needs api.peerNames(); // [{id, name, label, me}] `label` falls back to 'peer abcd' ``` `api.hud.rows` is one of three doors onto the same store: the element's own **authored** rows (typed into the properties pane, one per line), a **HUD Rows** flow node (set / append / clear on a trigger edge), and this. A node or a module always wins over the authored rows, and clearing puts them back — the authored value is the fallback, not a second source of truth. **CALL IT ON EVERY PEER.** Rows are never sent. Like a value node, this writes LOCAL state that every peer is expected to compute identically, so calling it on one peer shows the rows to one person. Drive it from your own `registerStateSync` state, or from something every peer already derives (`api.peerNames()` is exactly that — the roster is already replicated). Your rows are cleared at teardown, so disabling the module shows the authored rows again rather than freezing the last thing you pushed. ```js // YOUR OWN ELEMENT KIND: the registerToolbox contract, one layer in const kind = api.registerHudElement('gauge', { label: 'Fuel gauge', // palette + properties-pane heading icon: 'gauge', // a lucide name summary: 'A dial that reads a wired number.', defaultSize: { w: 150, h: 40 }, defaults: { caption: 'Fuel' }, // your params' starting values fields: [ // the pane renders these; same schema as core kinds { key: 'caption', kind: 'text', label: 'caption' }, { key: 'redline', kind: 'number', label: 'redline', min: 0, max: 100, step: 1 } ], mount(container, el, runtime) { const span = document.createElement('span'); span.textContent = el.caption; container.append(span); // return a bare cleanup fn, OR this pair to avoid a rebuild on every change: return { update(nextEl, nextRuntime) { span.textContent = nextEl.caption; }, destroy() { /* your cleanup */ } }; } }); // kind === 'mod--gauge' ``` You write plain DOM and inherit the whole HUD system: the layer's z-tier, the 9-grid anchoring with pixel offsets, the properties pane, the palette, replication, undo and all four save paths. `runtime` is what a flow node is currently driving into the element (`{text, value, min, max, rows, options, pulse}`), so a HUD Text or HUD Bar node pointed at your element feeds it with no extra wiring. The kind name is **namespaced** and gets written into a replicated, saved document. A peer without your module — or the same scene opened after it is gone — reaches an unknown kind, which the HUD PRESERVES verbatim and skips at render. So nobody's layout is destroyed, installing the module makes the element appear, and a disable is the same story rather than a special case. Everything is unregistered and unmounted from the teardown journal. ### Messages and state ```js api.onMessage((data) => { // {type:'module', moduleId, ...payload} if (data.op === 'press') press(findByUuid(data.uuid)); }); api.send({ op: 'press', uuid }); // broadcast to all peers api.registerStateSync({ getState: () => ({ pressed: [...pressedUuids] }), // sent to late joiners applyState: (state) => state.pressed.forEach(markPressed) }); ``` ### Input (K-C) ```js // declare bindings so they LIST in Settings ▸ Shortcuts (display-only — // you read the keys yourself via input()/onInput) api.registerBindings([{ label: 'Drive forward', keys: 'W' }]); api.registerFrameTask(() => { const { codes, axes } = api.input(); // codes: Set<'KeyW'...> (event.code), if (codes.has('KeyW')) drive(1); // axes: {lx,ly,rx,ry} = VR stick axes }); api.onInput((kind, code) => {}); // 'down'/'up' events; returns unsubscribe // pause the HOST's use of an input scope while your module drives: // 'keys' — WASD camera fly + play-mode movement // 'locomotion' — VR left-stick locomotion // 'sticks' — (1.19) BOTH VR sticks: left-stick move, right-stick snap/smooth turn + teleport // (a held thing reeled on Y and scaled on X, the way an Edit grab does) api.claimInput('keys'); // ALWAYS release when your mode ends api.releaseInput('keys'); const sticksOk = api.claimInput('sticks') === true; // true = this core knows the scope (older: undefined) // reading a keydown YOURSELF (a toolbox's own handler)? resolve the key the way the // editor does — `event.key` when it is an ASCII letter/digit, the physical // `event.code` position otherwise — so it works on a Cyrillic/Greek/Hebrew layout: window.addEventListener('keydown', (e) => { if (api.keyOf(e) === 'G') grab(); // 'G' on QWERTY, AZERTY, Dvorak AND ЙЦУКЕН if (api.letterOf(e) === 'w') drive(); // lowercase form; named keys ('Escape') unchanged }); ``` ### Pointer ray (190) ```js // where the user is POINTING, as a THREE.Raycaster in WORLD space: the desktop // mouse over the viewport, or the VR pointer hand's ray. Fresh instance per // call; null before the first pointer event. The drag recipe: click to pick, // follow pointerRay() in a frame task, click to drop (works desktop + VR). api.registerFrameTask(() => { const ray = api.pointerRay(); if (!ray || !carried) return; const hit = ray.ray.intersectPlane(dragPlane, tempVec); if (hit) carried.position.copy(hit); }); ``` ### Free-cursor games (1.17) A board, puzzle or instrument game can play with the real mouse cursor and no pointer lock: publish `userData.play.cursor = 'free'` on your scene-root group (or the scene's physics block sets `play.cursor: 'free'`). `api.pointerRay()` is then the cursor's ray in play; in a locked game it is the CROSSHAIR ray (the view's centre), never the stale mouse position the lock pinned. ### Storage (1.17) ```js // JSON on THIS device, namespaced to your module (tp:mod::), 256 KB per // module; never replicated, never saved into a scene, survives disable/remove. if (api.storage) { const progress = api.storage.get('progress', { unlocked: 1 }); api.storage.set('progress', progress); // -> true, or false over the quota api.storage.keys(); api.storage.bytes(); api.storage.remove('progress'); api.storage.clear(); // your "Reset progress" } ``` Feature-detect it; on an older app write the SAME key yourself (`localStorage['tp:mod::'] = JSON.stringify(value)`) so progress carries over. ### Models: `api.loadModel` (1.20, roadmap 34 R7) Load a `.glb` / `.gltf` through core's OWN loader — the scene's `THREE` (never bundle three's `GLTFLoader` into a module: it drags in a second three whose classes are not the scene's), with Draco, Meshopt and KTX2 decoders (the Basis transcoder is fetched only for a file that carries `KHR_texture_basisu`), parsed ONCE per URL and shared by every module that asks. ```js // a file your module PACKAGED (list it in manifest.json `files`) — or any URL const enemy = await api.loadModel('assets/enemy.glb', { castShadow: false }); group.add(enemy.scene); // your own copy (another module's tweaks never reach it) const next = enemy.instance({ ownMaterials: true }); // another copy: its OWN bones when skinned, group.add(next); // its own materials (a hit flash paints one) const mixer = new api.THREE.AnimationMixer(next); mixer.clipAction(enemy.animations.find((c) => c.name === 'walk')).play(); enemy.info; // {meshes, triangles, materials, textures, skinned} enemy.release(next); // forget one copy now (out of the scene, LOD dropped, own materials freed) enemy.dispose(); // forget them all — your module's unload does this for you ``` Options (all optional): `lod` — absent / `'auto'` = automatic levels (31-perf, meshes over 3000 triangles; a URL that is a PACK item whose row lists `lods` uses those files instead), `false` = none, `{ratios, distances, minTriangles}` = tuned automatic levels, `[{file, ratio}]` = pre-built level FILES beside the model (contract P1: `'tree.lod1.glb'` next to `'assets/tree.glb'`), drawn as a LOD group wherever you put the copy. `castShadow` / `receiveShadow` = every mesh (absent = as the file says, which is off). `collider: 'box'|'sphere'|'capsule'|'cylinder'|'cone'|'hull'` = the physics shape a copy takes once it is SCENE content with physics (`userData.colliderHint`; your scene-root visuals have no colliders — publish a `userData.play` raster for walls). `ownMaterials` = every copy gets its own materials. Lifetime: everything a module loaded is released when it is disabled or unloaded (a copy you put in `objectsGroup` stays — it is the scene's). A copy you drop WITHOUT `release()` (its wrapper removed) is noticed within ~2 s, its LOD entry dropped and only weakly held, so it is collected; put back, it is picked up again. A load still in flight when the module unloads rejects. Rejects with the reason when the file is missing or cannot be parsed — keep a fallback look. Feature-detect it (`typeof api.loadModel === 'function'`): 1.19 and older cores do not have it. ### Performance: quality level, LOD, budgets (1.18, roadmap 31) ```js // The adaptive quality level on THIS device: 0 = best ... api.quality.max (every step taken). // A headset session starts at 1 (shadows off) in auto mode; a game's Quality setting can pin it. if (api.quality) { const apply = (level) => { particles.maxCount = level >= 3 ? 40 : 200; // cut YOUR effects, not shared state torchLights.forEach((l, i) => (l.visible = i < (level >= 1 ? 2 : 6))); }; apply(api.quality.level); const off = api.quality.onChange((level, { labels, vr }) => apply(level)); // released on disable } // Levels of detail for your own dense geometry (auto LOD already covers dense meshes in the // shared scene and under the module world root). Built ONCE per asset in a worker // (meshoptimizer), drawn by distance through a render-time swap: the mesh keeps its full // geometry for picking, physics and your code. Distances are in the mesh's world RADII. if (api.lod) { const lod = api.lod(enemyFigure, { ratios: [0.5, 0.2], distances: [6, 18] }); lod.ready.then((n) => console.log(n, "meshes have levels")); // lod.remove() undoes it lod.force?.(2); // 1.19: draw level 2 on THIS screen whatever the distance (null = auto) lod.levels?.(); // 1.19: the level each mesh drew last (0 = full) } mesh.userData.lod = false; // keep one mesh out of auto LOD ``` **1.19 — LOD groups on scene objects.** A replicated object may carry a LOD GROUP (`userData.lod`, edited in its Properties ▸ LOD section, or written by a pack item's `lods`): `{mode: 'auto'|'forced', forced?, bias?, cull?, levels: [{source: 'self'|'pack'|'generated'| 'explorer'|'object', ref?, ratio?, screenSize, offset?, material?}]}` with thresholds by SCREEN SIZE (the share of the viewport height the object covers). It is scene data (saved, replicated, undone) and drawn at render time like `api.lod` — the tree never holds a level. A module that builds scene objects can write one through the same path the panel uses (`objectParameters {parameter: 'lod'}` is the wire shape); module-only geometry keeps `api.lod`. Both are LOCAL (a fact about this machine) — never let them change replicated state, or two peers on different hardware disagree about the game. Skinned meshes and morph targets get no levels (the simplifier cannot carry weights): pre-decimate those offline. **The Quest budget** (roadmap 31, the planner's target for a scene in Interact/Play on a Quest 3): **<= 150 draw calls, <= 300k triangles visible, <= 2 real-time lights with shadows off, no per-frame allocations in the hot path.** Measure your game with the probe, the same way every time: ```bash # dev server running; run it under the exclusive slot (frame times are the point) APP_URL=https://theprototype.app:5173/ ~/.local/bin/e2e-slot --exclusive -- \ node scripts/perf-games.cjs --label mine --only waves [--vr] [--profile] ``` It prints draw calls + triangles per frame (every render pass summed), geometries, textures and an MB estimate, lights (+ shadow casters), meshes / instanced / unculled, the LOD and quality state, and p50/p95/p99 frame ms at CPU throttle x4; `--profile` adds CPU self-time, the nearest app frame calling the heaviest functions, and allocation per function (garbage included). The things 1.18's round found, in the order they cost: - **Never keep a THREE object inside Svelte `$state`** (or pass one through an action parameter that is a state proxy): Svelte deep-reads a proxy, which walks the whole scene graph through `parent` — typed arrays included. Use `$state.raw`. This one line was 88% of a game's frame. - **No whole-scene lookups per frame** — `scene.getObjectByName` is a traversal; find once, keep the reference, re-find when it leaves the scene. - **No allocation per frame**: reuse vectors/arrays; a `[a, b].map(...)` in a frame task is garbage 60-90 times a second, and GC pauses are the stutter you feel in a headset. - **Lights and shadow casters are draw calls**: each shadow-casting light draws every caster again; transmission (glass) renders the scene an extra time per camera, per eye. ### Functional pack items: doors, lids, levers — `api.behavior` (1.19, roadmap 33) A pack item can be FUNCTIONAL: a door that opens on a click, a chest lid, a lever, a fan (the `behavior` field, see PACKS.md). Core runs them. Nothing plays in Edit; a click, a knock or walking up triggers them in Interact/Play; the state replicates; a door's collider follows its leaf. A module can drive them too: ```js if (api.behavior) { for (const item of api.behavior.list()) { // [{uuid, type, trigger, open}] if (item.type === 'door' && !item.open) api.behavior.trigger(item.uuid, true); // open it } api.behavior.state(uuid); // {on, at, n} or null (never triggered) api.behavior.trigger(uuid); // toggle, like a player's click } ``` `trigger` is REPLICATED (one `behavior` message; every peer poses the door from the same session-clock stamp), so call it on ONE peer: the authority, or the peer that saw the cause. It returns false when nothing changed (opening an open door). The state is runtime: it is never saved, and a peer in Edit shows the rest pose whatever it says. The item sounds are core game sounds: `door`, `gate`, `slide`, `lever`, `lid` (plus `click`), which `api.playSound` can play too. ### Game feel: sound, music, haptics, effects, banners (1.17, roadmap 30b) Everything here is LOCAL to the device it runs on — broadcast your own op (`api.send`) when every peer should hear, feel or see it — and all of it is feature-detected, so a module written against it still loads on an older app. ```js register(api) { api.registerClickHandler((object, ctx) => { if (!object.userData.ring) return false; const at = object.getWorldPosition(new api.THREE.Vector3()).toArray(); api.playSound?.('success', at); api.effects?.burst(at, { kind: 'sparkle' }); api.announce?.('Ring ' + object.userData.ring + ' reached', { sub: '+50' }); api.hapticPattern?.('success'); // both hands; 'left' | 'right' for one return true; }); api.game.onChange?.(() => (api.game.roundUnderway() ? api.music?.play('arcade') : api.music?.stop())); } ``` - **`api.playSound(name, position?)`** → `true` when a sound started. Twenty procedural game sounds, no assets: `click` `pop` `whoosh` `success` `fail` `hit` `kick` `shoot` `laser` `explosion` `coin` `levelup` `goal` `whistle` `cheer` `step` `ring` `sparkle` `hurt` `portal`, plus the ping chimes `ding` (the default), `chime`, `pluck`, `bell`. An unknown name is a quiet no-op (it used to play the ding). `position` spatialises it. Played at the player's "Game sounds" volume. - **`api.music.play(preset, {volume?})` / `api.music.stop()`** — a looping procedural track under the effects, at the player's "Music" volume: `arcade` `ambient` `dungeon` `stadium` `space` `puzzle` `studio`. It is TEMPO-SYNCED to the session clock (two peers on one preset hear the same bar), plays only in Interact or Play (`false` from the editor) and stops by itself when the player goes back to editing. `api.music.current()` names what is playing; `api.music.presets()` lists them. - **`api.hapticPattern(name, hand?)`** — `tap` `bump` `hit` `success` `fail` `rumble` `heartbeat` on the VR controllers. `api.haptic(intensity, ms, hand?)` still exists. BOTH are silent in Edit mode (vibration is for playing), and core already plays the defaults for you in Interact/Play: a `tap` when the laser enters something clickable, a `bump` when a press lands, a `hit` on a grab, and a knock sized by how hard the hand hit. - **`api.effects.burst([x, y, z], {kind?, color?, count?})`** — a short, pooled particle burst at the scene root: `sparkle` (default), `confetti`, `smoke`, `sparks`; `count` 1..96. Nothing is saved or replicated. - **`api.announce(text, {sub?, ms?, color?})`** — a big centred banner ("GOAL!", "Level 3"): the desktop HUD, and in a headset a banner fixed in front of the player. A new one replaces the one showing; `ms` 300..15000 (default 1800). - **`api.setSpawn([x, y, z], yaw, {teleport?})`** — where the player starts: `y` is the FEET, `yaw` is three's `rotation.y` (0 faces −Z). Entering Interact or Play puts the player there (a headset's rig lands its feet on it, a desktop its camera); without `teleport` it is a checkpoint and nobody moves until then. A module's spawn overrides the scene's `play.spawn`. `api.respawnPlayer()` sends the player back to it now. ### Moving around in a game: locomotion, bounds, VR panels (1.18, roadmap 31) In Interact and Play a VR player WALKS (collisions, gravity, a 0.3 m step, snap turn) and nothing else unless the scene's play block — or your module's `userData.play` on its scene-root group, field by field — allows more: ```js // feature-detect: an older core has no api.locomotion and would ignore the bounds if (api.locomotion?.boundedTeleport) { group.userData.play = { locomotion: { teleport: true, worldGrab: false, fly: false }, bounds: { min: [-12, -0.5, -12], max: [12, 0.4, 12] }, // in THIS group's local frame colliders: [{ min: [2, 0, -6], max: [2.3, 3, 6] }] // optional: solid boxes, same frame }; } ``` - **`locomotion.teleport: true`** — the right stick's arc teleport, BOUNDED: the arc stops at the first surface it meets; the landing must face up (normal y ≥ 0.7) or be the floor plane, must lie inside `bounds`, and the straight line 1.1 m above your feet and the landing must not cross a collider box, a mesh, or (with a dungeon raster published) a wall cell. A refused landing draws the arc RED and releasing the stick does nothing; a valid one is green and puts your FEET on it. Without `bounds` the play area is the scene's content box pulled in 0.3 m. Edit mode's teleport is unchanged (it lands anywhere). - **`locomotion.worldGrab: true`** — the grips move, rotate and SCALE the world the way they do in Edit (two grips scale/rotate, the right grip alone pans) whenever a grip does NOT start on something the player may hold (a grip on a dynamic body under `interaction: 'grab'` still takes the body). For board games and instruments: Untangle, the Jam Room. - **`locomotion.fly: true`** — the left stick flies along the controller's aim, no gravity. - **`bounds: {min: [x, y, z], max: [x, y, z]}`** — a sibling of `locomotion`. The scene's play block gives it in scene coordinates; a module's `userData.play.bounds` is read in that group's local frame. Keep `max.y` just above the floor if box tops and ledges must not be landing spots. - `vrControls.teleportVerdict(from, to, normalY?)` (on the debug hook `window.__stores.vrControls`) answers `{ok, reason}` — `ok` `steep` `outside` `off-floor` `wall-cell` `blocked` — for your e2e; `from`/`to` are world FEET points `[x, y, z]`. `vrControls.teleportPreview()` reads the live arc (`{engaged, bounded, valid, reason, target}`). **Your own VR menu is a panel.** A board, a level bar or buttons you draw in THREE for the headset should be registered with **`api.vrPanel?.(group)`** (returns the undo, also run when the module is disabled): it is then drawn OVER the scene — a floor, a base or a wall between the player and it can never hide its buttons — and the controller laser ends on it with its dot, even through whatever stands in front. Hit testing is unchanged. Core's own panels (the radial menu, the game board, the wrist card) already work this way, and the laser ends on your `registerInteractiveGroup` content too. **In VR, your game's HUD is in the player's hands.** A screen with `input: 'menu'`, a control on it, or bound to the `menu` / `paused` / `over` game state is drawn on a board ~1.2 m in front of the player that follows their head lazily; its buttons work with the laser and trigger and with a poke. The other screens (score, timer, level) read as short lines on a card on the left wrist and a strip across the top of the view. The board and the wrist card always carry an **Edit mode** button. Nothing to do on your side — author the HUD once and it works on both. **Hold the trigger and sweep (VR, Interact/Play).** While the trigger is held, the controller TIP clicks each clickable it passes over — the one it is NEAREST to, so a glide along a keyboard plays one key at a time — once, and again after it leaves and comes back: piano keys, pads, toggles, HUD buttons; the laser does the same for things out of reach. "Clickable" (also what earns the hover `tap`) means: under a group you passed to `registerInteractiveGroup`, a part of an audio device (`userData.device` on an ancestor — each key/pad is its own control), an object an On Click node targets, or an object you mark `userData.clickable = true`. Your handler gets a second argument `ctx = {source, mode}`: `source` is `'click'` (desktop), `'trigger'` (the VR press itself) or `'sweep'` (a later entry while held). A control that must not be swept (a knob you drag, a dot you carry) opts out: ```js api.registerClickHandler(pickDot, { sweep: false }); // presses still reach it; sweeps do not ``` ### The game shell: pause menu, levels, per-game settings (1.18, roadmap 31 K3) Every game gets ONE pause menu from core — **Resume · Restart · Levels · Settings · How to play · Main menu** — on the desktop (Escape in Play, or the corner Menu button) and in VR (the left controller's **X**, or Menu on the game board / wrist card). You do not draw it; you feed it. A scene counts as a game when it has a state-bound HUD screen, a spawn, or a module publishing `userData.play` — or when your module registers levels. All of it is feature-detected (`api.game.levels?.(…)`), LOCAL, and torn down with your module. **Levels** — the picker on the desktop and on the VR board. Re-call it whenever a level unlocks or earns stars; `onPick` never hears a locked level. ```js const refresh = () => api.game.levels?.({ list: LEVELS.map((l, i) => ({ id: l.id, label: 'Level ' + (i + 1), locked: i > unlocked, stars: best[l.id] ?? 0 })), current: currentLevel.id, onPick: (id) => loadLevel(id) }); refresh(); onLevelWon(() => { unlocked++; refresh(); }); ``` **Your own settings rows** — shown under the core rows, persisted per game on this device (`tp:game::`). `type` is `'toggle'`, `'choice'` (`options`, optional `optionLabels`) or `'range'` (`min`, `max`, `step`). A core id is refused. ```js api.game.addSetting?.({ id: 'board', label: 'Board', type: 'choice', options: ['globe', '2d'], optionLabels: ['Globe', '2D board'], default: 'globe', onChange: (v) => setBoard(v) }); setBoard(api.game.setting?.('board') ?? 'globe'); // the stored choice at load // (1.19) a CHOICE that decides which levels you see: `onLevels: true` also draws it as tabs // above the Levels page's grid (desktop + headset) — Untangle's Board [Globe] [2D board] api.game.setSetting?.('board', '2d'); // your in-game button writes the same row api.game.onSettingsChange?.((values) => console.log(values.board, values.sfx)); ``` **The core rows every game has** — `music`, `musicVolume` (0..100), `sfx`, `sfxVolume`, `haptics`, `showFps`, `turning` (`default` | `snap` | `smooth` | `off`), `turnAngle` (`default` | `15` | `30` | `45` | `90`), `vignette`, `quality` (`auto` | `low` | `medium` | `high`). Core obeys them itself: the per-game volumes land on the audio BUSES (so `api.playSound`, `api.music`, flow sound nodes and the scene's track all follow), haptics, VR turning, the comfort vignette and the quality governor read them live. A module that plays audio through its OWN WebAudio graph should read them: ```js const gain = ctx.createGain(); const apply = () => (gain.gain.value = api.game.setting?.('sfx') === false ? 0 : (api.game.setting?.('sfxVolume') ?? 100) / 100); apply(); api.game.onSettingsChange?.(apply); ``` **How to play** — your words first, then the device's controls (keyboard or controllers). Without it the menu shows the game's Games-tab description. ```js api.game.setHelp?.([ 'Drag the dots until no two lines cross.', 'Finish a level to unlock the next — stars for fewer moves.' ]); ``` **Restart** — the menu resets the game shell (host / alone; others carry on as it is), respawns the player and runs your hook: ```js api.game.onRestart?.(() => { resetBoard(); spawnWave(1); }); ``` **Your own Menu button** (a module HUD, a VR board of yours): ```js api.game.openMenu?.(); // only while playing a game; false otherwise api.game.closeMenu?.(); if (api.game.menuOpen?.()) pauseMyTimers(); ``` ### The game kit: rules, round, levels, score, pickups — `api.kit` (1.20, roadmap 34) The logic every game used to re-build, done ONCE in core. Each piece is `api.kit.` AND a **Kit: ** node group, both generated from one spec (`src/lib/kit/.spec.js`), so a graph author and a code author get the same behaviour and the same words. **The model — one writer.** The kit's shared state is ONE document that only the **authority** peer changes (the physics initiator, else the lowest peer id — the peer Towers and Football already pick). Call an action on ANY peer: elsewhere it is a request the authority applies, exactly once. A kit **node** on a replicated pulse is asked by every peer with the same id, so one press is one change (the `setvariable add` double count cannot happen). Reads are replicated values; `on(fn)` fires on EVERY peer (play your sound there) and returns `off`; registrations (`levels.define`, `pickups.register`, `rules.onGrabRequest`) are torn down with your module. Late joiners get the document in the handshake and hear no history. A scene clear resets the kit. ```js const { round, levels, score, pickups, rules } = api.kit; rules.set({ reach: 1.3, jump: 1.0 }); // every grab path + the VR jump obey rules.onGrabRequest((req) => { if (req.name === 'Star' && !starFree) req.refuse('Build to the ring first'); }); levels.define({ id: 'mygame', list: [{ id: '1', label: 'Easy', par: { time: 60 } }, { id: '2', label: 'Hard' }] }); round.configure(3, 120, 'lose', 2); // 3 s intro, 2 min limit, lose on time, 2 s outro round.onGo(() => api.announce('Go!')); pickups.register({ id: gem.uuid, score: 10, respawn: 8, grants: { time: 5 } }); pickups.onCollected(({ by }) => api.playSound('coin')); round.onWon(() => levels.complete(true, score.total())); // stars, unlocks, saved per device levels.select('1'); round.start(); ``` | piece | actions | reads | events | |---|---|---|---| | `rules` | `setReach(m)` `setJump(m)` `setBounds(min, max)` `clearRules()` · `set({reach, jump, bounds})` | `reach()` `jump()` `current()` `inside(p)` `clamp(p)` `checkGrab(req)` | `refused` (local) · `onGrabRequest(fn)` veto | | `round` | `configure(intro, limit, 'lose'\|'win', outro)` `start()` `restart()` `pause()` `resume()` `win(reason)` `lose(reason)` `extend(s)` `toMenu()` | `phase()` `playing()` `elapsed()` `remaining()` `countdown()` `number()` `outcome()` `state()` `running()` | `started` `go` `paused` `resumed` `won` `lost` `results` `menu` | | `levels` | `select(id)` `next()` `complete(won, score, time, level?, detail?)` `setMode(m)` · `define({id, list, unlock?, stars?, store?, merge?})` | `current()` `currentLabel()` `index()` `starsOf(id)` `unlocked(id)` `totalStars()` `mode()` `table()` `progress()` `resumeLevel()` | `selected` `completed` `unlockedNext` | | `score` | `add(n, player?)` `set(n, player?)` `reset()` · `configure({autoReset})` `useGame(id)` | `total()` `mine()` `best()` `leader()` `of(id)` `leaderboard(n)` `results()` | `scored` `newBest` (local) | | `pickups` | `collect(id, score?, respawn?)` `resetPickups()` · `register({id, score, respawn, radius, grants})` | `available(id)` `taken()` `left()` `takenBy(id)` | `collected` `respawned` `allCollected` | - **`round` drives core's game singleton** (`intro`/`playing` → playing with a fresh core round, so perRound content resets; `paused`; `won`/`lost` → over with the reason as outcome; `menu`), so HUD `showWhile` screens and the pause menu keep working, and it ADOPTS a change it did not make (a Set Game State node pausing). `restart()` works while playing; the pause menu's Restart restarts a kit round by itself. - **`levels`**: progress is per DEVICE (every peer saves what it saw earned); unlocks read it merged with this session's results; `setMode` keeps the current level (progress belongs to the level, not the mode). The table feeds the pause menu's level picker — do not also call `api.game.levels`. `store: {get, set}` keeps your own save key (Towers keeps `tp:mod:towers:progress`). - **`score`** credits the asking peer unless a player is named (a shared pulse is asked by every peer — name the player, or use a per-player trigger, when it matters who). A new kit round zeroes it; the device best is checked when a round ends. - **`pickups`**: TOUCH is built in — a player walking into a registered pickup takes it (their own body, ~10x a second, in Interact/Play). A `Kit: Pickups ▸ Collect pickup` node with no object wired takes its own graph's object. - Prove your rules on the **headless logic sim** (`tests/unit/sim/logicSim.js`: N fake peers, a fake clock, the real wire validator) — `createSim({peers: ['a', 'b']})`, then `sim.peer('a').kit.impls.round.start(); sim.advance(3000)` — milliseconds, not e2e minutes. ### Behaviours: game logic as a small file, with a live node view (1.20, roadmap 34 R3) A **behaviour** is game logic written as ONE self-contained JavaScript file — the code half of "nodes vs code" (proposal §4): plain JS for what is a loop or a rule, a derived node view for seeing it run. Add a **Behaviour** node (palette: Logic) to the scene graph (or an object's graph — then `this.object` is that object) and write the file into it; **Open view** shows it. ```js export default behaviour({ name: 'Waves spawner', params: { interval: { value: 3, min: 0, max: 30, step: 0.5, unit: 's' }, waves: 5 }, state: { wave: 0, alive: 0 }, // replicated: every peer reads it on: { go() { this.startWave(1); }, // kit.round's "go" died({ entity }) { // kit.health's "died", typed payload if (!entity.is('robot')) return; if (--this.state.alive === 0) this.after(this.params.interval, 'startWave', this.state.wave + 1); } }, startWave(n) { if (n > this.params.waves) return kit.round.win('All waves cleared'); this.state.wave = n; this.state.alive = n + 1; kit.spawner.spawn({ kind: 'robot', template: this.find('Robot')?.uuid, count: n + 1, mover: { speed: 1.3 } }); } }); ``` - **params** — a literal (`waves: 5`) or `{value, min, max, step, unit}`; read as `this.params.x`. In the node view each is a KNOB: dragging previews on your device, releasing rewrites the literal in the file (one edit, one undo entry) and every peer reloads with it. - **state** — plain JSON. Handlers run on ONE peer, the kit's **authority**; after each one the state is sent to everybody (`bhv`, latest-wins). A late joiner gets it in the handshake; when the authority leaves, the next one carries on from it. A source edit keeps it. - **on** — event handlers: `start` (once per session), `grabRequest({piece, distance, refuse})` (the kit.rules veto, asked on the GRABBING peer — read-only there), and every kit event as `'piece.event'`, `pieceEvent` or a short alias (`roundStart`, `go`, `won`, `lost`, `died`, `damaged`, `spawned`, `emptied`, `scored`, `collected`, `stuck`, `grabRefused`). An entity in a payload has `is(glob)` / `hasTag(tag)`. - **methods** — any other function: `this.name(...)`. - **this** — `params`, `state`, `kit` (also bare `kit`), `after(s, 'method', ...args)` (session clock; a method NAME survives the authority leaving, a closure does not), `cancel(key)`, `rand()` / `randInt(a, b)` / `pick(list)` (seeded per event — never `Math.random`), `now()`, `find(glob)` / `findAll(glob)` (scene objects by name or tag: `{uuid, name, pos, tags}`), `object`, `isAuthority()`, `me()`, `log(...)`. Bare helpers: `dist(a, b)`, `clamp`, `lerp`. - **The lint** stops a file from loading (on every peer, with the line) when it uses what would make peers disagree or reach outside the game: `Math.random`, `Date`/`Date.now`/ `performance.now`, storage, the DOM/`window`/`globalThis`, network, `eval`/`Function`, bare timers, `import`. Loops run under the loop guard (a runaway handler throws, the game goes on). - **Lifecycle (T2)** — each behaviour is a module of its own (`behaviour:`): its kit listeners and the entities it spawned are tracked; deleting or stopping the node takes them away. Editing the source does NOT: the definition is swapped, the game keeps running. - **The node view** (derived, read-only): ⚡ events → ƒ handlers/methods → ▣ state, ⊕ kit calls (named exactly as the generated **Kit:** nodes), ⏱ timers, ✋ payload actions; ◆ params with knobs. Live values 10x a second, a glow on what just fired (on every peer), timer countdowns, errors with their line. Logic edits happen in the code (the view's **Code** panel). - Prove a behaviour on the **logic sim**: `tests/unit/sim/behaviourSim.js` — `const sim = createBehaviourSim({peers: ['a', 'b']}); await sim.load('w', source);` then drive the kit and assert `sim.states('w')`. Examples: `static/behaviours/waves-spawner.js`, `static/behaviours/towers-reach.js`. ### Physics (P-A) All mutations are INITIATOR-ONLY — the peer that started the simulation steps the world (golden rule: authoritative, never mixed with deterministic). The blessed recipe for driven physics (pong's paddle pattern): every peer forwards its INPUT via `api.send({op:'drive', ...})` at ~20Hz, and only the peer where `api.physics.isInitiator()` is true applies it. ```js api.physics.isInitiator(); // true while THIS peer runs the sim api.physics.applyImpulse(uuid, [0, 5, 0]); // push a dynamic body (initiator-only) api.physics.applyTorqueImpulse(uuid, [0, 2, 0]); // spin a dynamic body (world axes) api.physics.setJointMotor(jointId, vel, maxForce); // drive a revolute joint api.physics.joints(); // Promise ``` The **car module** (now in the [modules repo](https://github.com/theprototype-app/modules), `modules/car/`) is the worked example: replicated primitives + motorized revolute joints, click-to-claim (pong's paddle pattern), driver forwards `{op:'drive', throttle, steer}` at ~20Hz and only the initiator applies wheel motors. Driving + the chase camera engage only in Play mode with a running simulation (the module claims `'keys'` and uses possess's `startFollowCam` while engaged; the claim itself works anytime). ### Building in the shared scene (17-A) ```js // the replicated /create, handing back what appeared const [uuid] = await api.create('/create Box 1 1 1'); await api.create('/create Box 0.6 0.6 0.6', { at: [x, y, z] }); api.moveObject(uuid, { pos, rot, scale }); // the editor's replicated move api.physics.set(uuid, { mode: 'dynamic', mass: 30 }); api.physics.createJoint('revolute', a, b, 'x', { vel: 0, maxForce: 120 }); api.physics.running(); // a sim runs somewhere in the session api.isPlaying(); // Play mode active api.peerIds(); // roster - free a departed peer's state // LOCAL only (a peer's module must never yank your viewpoint): api.flyTo([x, y, z], [lx, ly, lz]); api.playSound('pluck', [x, y, z]); api.followCam(uuid); api.stopFollowCam(); ``` ### Possess (K-D) ```js // drive any object with WASD/arrows or the VR left stick (tank controls), // chase camera by default; Esc releases. Possessing SELECTS the object // (selection = lock), suspends its flow effects, and records ONE undo entry // on release. Movement replicates as plain throttled moves. api.possess(api.selectedUuid(), { camera: 'chase' }); // 'chase'|'orbit'|'none'|'first' api.releasePossess(); api.possessModes; // this build's camera modes — feature-detect 'first' here // (an unknown camera value degrades silently on old builds) // 17-A1 first person: eye at the object + eyeHeight; mouseLook = pointer lock // (X turns the OBJECT so movement follows the look, Y pitches the camera; // leaving the lock — Esc — releases the possession) api.possess(uuid, { camera: 'first', eyeHeight: 1.7, mouseLook: true }); ``` ### Audio devices (23-A5) An instrument, an effect or a speaker is a DEVICE: an object carrying `userData.device = {kind, params}`, with a WebAudio subgraph the engine builds for it. Your module supplies the kind; core supplies the object, the replication, undo, saving, the cables and the clock. ```js const kind = await api.registerAudioDevice({ kind: 'piano', // namespaced to mod--piano label: 'Piano', icon: '🎹', group: 'Keys', ports: { in: [], out: [{ id: 'out', kind: 'audio' }] }, // 'audio' | 'cv' | 'midi' params: [{ key: 'level', kind: 'range', min: 0, max: 1, step: 0.01, default: 0.8 }], build(ctx, node, params) { // ctx = the SHARED AudioContext; runs per object const out = ctx.createGain(); out.gain.value = params.level; return { output: out, out, dispose() { out.disconnect(); } }; }, onParam(h, key, value) { if (key === 'level') h.out.gain.value = value; }, onNote(h, { note, velocity, at }) { // at = a WALL-CLOCK stamp const v = api.audio.voice({ freq: 440 * 2 ** ((note - 69) / 12), gain: velocity * 0.5, destination: h.out }); const t = api.audio.timeFor(at); // never Date.now() maths of your own v.start(t); v.stop(t + 0.5); }, mesh: (THREE) => new THREE.Mesh(new THREE.BoxGeometry(1, 0.2, 0.4), new THREE.MeshStandardMaterial({ color: '#222' })) }); api.audio.addDevice('piano', { position: [0, 1, 0] }); // also in the viewport Add menu > Devices ``` - `params` render in the Inspector and the toolbox; every write replicates and undoes. Keep `onParam` cheap and `build` pure: a peer WITHOUT your module holds the same object as an inert placeholder with its document intact, and rebuilds it the moment your module registers. - Children of your mesh named `port:` are where cables attach. - Sound goes NOWHERE until cabled: a speaker device connects its `input` to `api.audio.bus('instruments')`; `api.audio.cable({from: {uuid, port}, to: {uuid, port}})` plugs one in. ```js api.audio.voice({ freq: 220, type: 'sawtooth', filter: { freq: 900 } }); // {output, start(t), stop(t), dispose()} api.audio.voice({ buffer, loop: true, destination: 'instruments' }); // a sample voice await api.audio.sample(hash); // decoded AudioBuffer for an Explorer content hash, // pulled from a peer if missing (null after 30 s) api.audio.bus('sfx'); // a bus to connect to api.audio.context(); // the shared AudioContext — never make your own api.audio.transport(); // {bpm, beat, bar, step, phase, playing, loopBeats, swing} api.audio.play(); api.audio.play(false); api.audio.setBpm(100); // the SHARED transport const cancel = api.audio.schedule(0, ({ beat, at }) => hit(at), { every: 1 }); // a metronome: // called ~100 ms EARLY with the exact audio time; // start voices at `at`; cancelled at teardown api.audio.note(uuid, { note: 64, velocity: 0.8 }); // replicated; every peer synthesizes it api.audio.setParams(uuid, { level: 0.5 }); // one undo step, replicated const before = api.audio.device(uuid); // capture the document when a gesture STARTS api.audio.previewParams(uuid, { level: 0.5 }); // a live gesture: replicated, NO undo entry - // throttle it while scrubbing, then commit once: api.audio.setParams(uuid, { level: 0.5 }, { before }); // one undo entry that restores `before` api.audio.captureMic(); // Promise: the RAW mic, a separate capture // from voice chat (no AEC/NS/AGC, never gated by PTT) const item = await api.audio.record({ maxSeconds: 8 }); // a take -> Explorer item (content-hashed, // shared to peers); null when refused BEFORE it starts api.audio.stopRecording(); // end the take early const dest = api.audio.context().createMediaStreamDestination(); // bounce a NODE instead: myInputGain.connect(dest); // a looper records what is patched into it await api.audio.record({ maxSeconds: 8, stream: dest.stream, name: 'loop' }); // declare what a kind references by hash so the Scene manifest (and a .tpscene export) // carries the bytes: `assets(params) -> [{hash, name}]` on the registerAudioDevice spec api.audio.device(uuid); // {kind, params} or null ``` The rule behind `schedule` and `onNote`: **anything scheduled on the transport must be a pure function of its arguments.** Every peer runs the same pattern from the same `startedAt`; an impure callback desyncs silently, per peer, with no error anywhere. ### Misc ```js api.registerMenu('Generate dungeon', () => generate()); // sidebar "Modules" section api.registerVRMenuEntry({ // sector in the VR radial menu id: 'spawn', group: 'root', label: 'Dungeon', // group 'root' = base ring; action: () => generate(), closes: true // other names make a sub-ring }); // (id gets prefixed moduleId:) api.sceneAssets(); // Promise<[{group, name, kind, hash}]> — what the shared // scene uses right now (audio/config/textures manifest) api.scene(); // THREE.Scene api.objectsGroup(); // the replicated objects root (add scene content here) api.peerId(); // our peer id (undefined before the mesh is up) api.toast('hi'); // toast in the corner api.haptic(0.6, 60); // buzz the VR controllers (no-op on desktop and in Edit); api.haptic(0.6, 60, 'right'); // optional hand targets one controller (17-A1) api.hapticPattern('success'); // a named pattern (1.17, see Game feel) api.isVR(); // true inside a VR session api.vrHand('left'); // one hand's WORLD pose + buttons, or null: // {position, quaternion, trigger, gripped, connected} api.fireObjectClick(uuid); // pulse On Click flow nodes targeting the object // (replicated) — user graphs react to module events api.now(); // the runtime clock in seconds — stamp replicated // timestamps with this, never Date.now() directly ``` Objects your module adds to `objectsGroup()` are part of the shared scene: they get GLTF-synced to late joiners, appear in the object list, and can be moved/deleted by anyone. Objects that are *derived* state (e.g. a generated dungeon) are better added outside `objectsGroup()` and rebuilt from your module state — then they can't drift. ## Walkthrough: the hello module `src/modules/hello/module.js` is the smallest complete module: one node group + one effect, zero messages (fully deterministic). Open the flow editor, drag in **Wave (hello)** and an **Object Selector**, pick an object, connect them — the object rocks on every peer. ## Walkthrough: build a door (button module) The button module (`src/modules/button/`) shows the interactive pattern: 1. Add a **Button** from the sidebar Modules group (or use any object). 2. Add a **Wall/Cube** where the door should be. 3. Open the flow editor: drag in **Button trigger** and an **Object Selector**. 4. In the Button trigger node pick the button object; in the Object Selector pick the door object; connect trigger → selector. 5. Click the button in the viewport (or pull the VR trigger on it): the door slides up by `height` on every peer. Toggle mode keeps it open, push mode springs back after 1.5 s. How it replicates: the click writes `{pressed, at: api.now()}` into the node's data (replicated like any node edit); the slide is a pure function of `(data, time)` running on each peer — no motion messages at all. The `registerClickHandler` consumes the click so pressing doesn't select the button. ## Walkthrough: the towers module (a game's rules as a core module) `src/modules/towers/` runs the **Towers** template's twelve levels. It shows how a game with real rules splits: the scene (the template) is the arena, the piece templates, the HUD and a small graph; the module is the rules. Worth copying: - **The rules as a pure leaf** (`levels.js`) — the level table, the star rule, the unlocks, the verdict stepper — tested with vitest, no browser. - **One authority** decides anything shared (the physics initiator, else the lowest peer id): it deals the pieces, judges the round and writes the result into game variables (`api.game.setVar`); every peer derives its HUD words from those through a value node wired into HUD Text's format (`api.registerValueNode`). - **HUD buttons without a presser**: a press is a replicated stamp on a `hudbutton` node, so every peer watches `api.flow.triggerStamp` and only the authority acts (first sight = history). - **Progress on this device** with `api.storage` (stars, unlocks — never replicated). - **The shell's level picker, help and restart** (`api.game.levels`, `setHelp`, `onRestart`), feature-detected so the module also runs on an app without them. - **Since 1.20 it runs on the game kit** (`api.kit`): its levels, unlock chain, stars and saved progress are `kit.levels` (with Towers' own store and star rule), its round is `kit.round` (Restart while playing), its tallest tower is the round's `kit.score`, and its reach + jump are `kit.rules` — what stays in the module is Towers itself. ## Manager, dev mode & gallery (17-A2/A3) Modules install, update, disable and remove **live** — `unloadModule` (33's `deactivateModule`) runs the module's lifecycle registry (see **The lifecycle** above), so nothing needs a page reload. Since 1.20 that includes CORE modules: their hooks outside the api (vrsleeve's VR hook registries) undo through `api.onUnload`. Every user-module card carries a **Dev URL** row: **Reload** fetches fresh code (cache-busted), evaluates it FIRST, then tears down + re-registers — a broken body keeps the old instance running. **Auto** polls (~2s) and reloads on change. Peers toast the `{id,version}` mismatch while you iterate (correct — the dev peer differs). The **Browse** tab lists the community repo (`theprototype-app/modules`, `index.json` via jsDelivr — `moduleGallery.js`); installs go through `installUrl` with the entry's source folder, so Update and the dev reload keep working on gallery installs. ## Checklist before you ship one - [ ] Everything a user can do through your module looks the same on a second connected browser (test with two windows). - [ ] A peer who connects *after* your module did something catches up (state sync or derivable-from-scene). - [ ] No `Math.random()` without a broadcast seed; no accumulation in effects. - [ ] Receiving a message never re-broadcasts it. - [ ] `id` unique, node `type`s unique, version bumped on behavior changes. - [ ] Switch it off and on again in the Modules manager (or dev-reload it): no timer, listener, sound, overlay or scene object of the old instance is left, and the new one works from scratch (`__stores.moduleSDK.registrationsOf(id)` is `{}` while it is off).