Reactor Runtime turns an inference pipeline into a real-time, interactive media and data stream. You write load() and generate(); the runtime drives them one step at a time and handles the session lifecycle, the media transport (WebRTC, or a LiveKit room), and the wire protocol that connects clients to your model. Viewers watch frames as they are generated and change what the model is doing mid-stream, with no restart and no re-queue.
- 📡 Real-time streaming. Frames reach clients over WebRTC as your model produces them, not after a whole video is done. The runtime paces playout from the time each step took, so a model needs no rate limiter of its own.
- 🎮 Live interaction. Clients send commands mid-generation: change a prompt, move a camera, adjust a parameter. The next frame reflects it.
- 🔌 No transport code. You never import a WebRTC library, manage a WebSocket, or encode video. The runtime ships its own media engine as a wheel, so a plain Python container is all a model needs.
- 🏠 LiveKit rooms. A session can join a LiveKit room: the model's tracks are published there, and every other participant in the room is a client.
- ✅ Typed, validated commands. Declare the commands your model accepts with standard Python types and constraints. The runtime validates every payload before your handler runs and compiles the surface into an OpenAPI schema that drives typed client SDKs.
- 🗂️ Step results. Start a session with its input and a number of steps, and keep each step's output as a folder of files.
- 🔎 Traceable logs.
get_logger()writes structured records, and every record a session writes carries its id, so one filter recovers a whole run. - 📦 One container, anywhere. The
reactorCLI scaffolds a workspace, builds a small image, and runs it locally. The same image deploys to Reactor's GPU cloud unchanged.
You ship one ReactorApp subclass: the application the runtime drives and the client talks to. Declare the media it sends and the state a client can set, load your weights once, and write what one step of generation does:
from pathlib import Path
from reactor_runtime import InputField, InputState, Output, ReactorApp, Video
class MyState(InputState):
prompt: str = InputField(default="a sunny meadow", moderate=True, description="Scene to render.")
paused: bool = InputField(default=False, description="Hold generation on the last frame.")
class MyOutput(Output):
main_video: Video
class MyModel(ReactorApp):
state: MyState
def load(self, config_path: Path | None) -> None:
self.pipe = load_my_pipeline()
def generate(self, input: MyState) -> MyOutput:
return MyOutput(main_video=self.pipe.forward(prompt=input.prompt))That is a complete application. The runtime calls generate() in a loop for as long as someone is watching and emits what it returns. Every public field on MyState is a command the client can send: here set_prompt and set_paused, validated from the fields, and the next step reads the new values.
A step is three calls, and generate() is the one you must write. process_input() runs before it and decides whether a step can happen: return the input generate() gets, or raise ApplicationError("reason") to skip the step. process_output(outcome) runs after it, with the result or the error, and returns the media to emit; a message sent there with await self.send() reaches the client before the step's frames. Both have defaults, so the model above writes neither.
from reactor_runtime import ApplicationError, MessageField, ModelMessage, StepOutcome
class FrameReady(ModelMessage):
prompt: str = MessageField(description="The prompt this frame was rendered from.")
class MyModel(ReactorApp):
state: MyState
async def process_input(self) -> MyState:
if self.state.paused:
raise ApplicationError("paused")
return self.state
def generate(self, input: MyState) -> MyOutput: ...
async def process_output(self, outcome: StepOutcome) -> Output | None:
if outcome.error is not None:
raise outcome.error
await self.send(FrameReady(prompt=self.state.prompt))
return outcome.to_output()generate() fails by raising, and the exception reaches process_output() as outcome.error. Recover an error you expect there and return None to go on to the next step. Re-raise anything else to end the session with an error, as the default process_output() does.
Commands and lifecycle hooks such as @session_started and @connected run between steps, never during one. A command handler refuses by raising CommandError(code, message), which fails the caller's own request. For a loop that is not one step per emit, override run() and write it against emit(), send(), and complete_step().
A session can also start with its setup in hand and run a set number of steps: POST /start_session takes a starting_input of state values and commands, and steps, and the session ends once the model has reported that many. Each step can be kept as a folder of files, turned on in reactor.yaml:
runtime:
step_results:
enabled: true
video: {codec: h264, preset: veryfast, crf: 23} # or codec: h265
audio: {codec: aac, bitrate_kbps: 128}
queue: 8A step's tracks are saved as one output.mp4, with a result.json that lists the step's files, and GET /sessions/{id}/steps serves the folders.
A session can join a LiveKit room. The room connection publishes the model's output tracks, and every other participant in the room is a client of the session. The model's reactor.yaml says which RPC methods, data topics, and byte streams reach which of its commands, messages, and input tracks:
runtime:
livekit:
rpc:
- method: lk.clear_buffer # a call to this method runs clear_buffer
command: clear_buffer
- method: lk.playback_finished # sending playback_finished calls this method
message: playback_finished
topics: [avatar] # a data packet on avatar.<command> runs <command>
media_inputs:
lk.audio_stream: agent_audio # byte streams on this topic feed agent_audioThe LiveKit SDK is an optional part of the runtime, so a model that joins rooms lists reactor-runtime[livekit] in its requirements.txt. A running session joins a room through a connection of its own:
POST /sessions/{session_id}/transport/livekit/connections
{"url": "wss://example.livekit.cloud", "token": "eyJhbGciOi..."}
Scaffold, build, and run it with the CLI:
reactor init my-model
cd my-model
reactor runreactor run builds a container with the runtime inside and serves WebRTC signaling on port 8080. Point a browser at it with the JS SDK, or connect from the Reactor Sandbox and watch frames stream immediately.
Log from the same import, passing context as keyword arguments:
from reactor_runtime import get_logger
logger = get_logger(__name__)
logger.info("scene changed", prompt=self.prompt)Records render as key=value text, or as one JSON object per line under REACTOR_LOG_FORMAT=json. Every record written during a session carries its id and the runtime's lifecycle state, including the records of logging.getLogger() and of the libraries your model imports.
Everything runs through the reactor CLI. There is nothing to install on your host but the CLI and Docker; the runtime ships inside the image the CLI builds for your workspace.
brew install reactor-team/tools/reactor-cliNot on macOS, or pinning a release in CI? See Install the CLI.
- Quickstart: from zero to a model deployed on Reactor's GPUs
- Runtime overview: what the runtime handles, and the outline of a model
- Model anatomy: every member of a
ReactorApp, line by line - The step loop:
process_input(),generate(),process_output(), and the frame rate - Events and messages: commands, refusals with
CommandError, and messages to clients - Video and audio tracks: output and input tracks, and how input is buffered
- Starter example: the model
reactor initscaffolds: one class,generate()alone, the smallest completeReactorApp - Echo example: the client's webcam and microphone in, an effect applied, both sent back, in batches
- Waypoint example: a world model on a GPU, seeded from an upload and steered live
This repository holds the runtime package itself: the authoring interface, the session runner, the media transport, and the wire protocol. To work on it, use mise, which pins the toolchain and forwards every task through a thin make shim:
mise run install # install deps, generate wire bindings, and git hooks
mise run lint # ruff check, ruff format --check, and mise.lock drift
mise run format # apply ruff formatting
mise run typecheck # ty (strict)
mise run test # unit tests on the floor Python
mise run test-matrix # unit tests on every supported PythonLicensed under the Apache License, Version 2.0.