Robot command and event protocol

Connect and recover

Open a WebSocket to the Robot WebUI origin's /ws. HTTPS deployments require wss://. The server sends system.hello with protocolVersion: 1, then initial performance.runtime and runtime.settings snapshots, followed by retained events of other types. Retention is latest-per-type, not an event history. Events marked retain: false are transient and are not replayed on reconnect.

const base = new URL('https://ROBOT_WEBUI_HOST/');
base.protocol = base.protocol === 'https:' ? 'wss:' : 'ws:';
base.pathname = '/ws';
const socket = new WebSocket(base);
socket.onopen = () => socket.send(JSON.stringify({
  type: 'client.ping', requestId: 'connect-check-1'
}));
socket.onmessage = event => console.log(JSON.parse(event.data));

A server.pong preserves requestId and includes server time. Reconnecting clients should reacquire snapshots and restore subscriptions. Do not automatically replay motion, speech or performance commands: acknowledgements can be lost after execution. The server does not provide a general exactly-once/idempotency guarantee.

Command envelopes

Most commands use type and optional client-generated requestId. Results use command.result, requestId, ok, and optional error/operation-specific fields. POST /api/teleops/command accepts the same object and generates a request ID if missing. Results can indicate failure even when HTTP returns 200.

Command	Fields	Effect
`behavior.set_state`	`state` required	Publish behavior state switch
`behavior.set_situation`	`situation` required	Publish situation and state-switch commands
`teleops.speech.say`	`text`, `language`, `voice`	Request speech through ROS TTS
`teleops.chat.send`	`text`, `language`, `source`	Submit user text; routes directly to ElevenLabs when that agent is enabled
`teleops.elevenlabs.userMessage`	`text`	Send text to current ElevenLabs session
`teleops.elevenlabs.contextualUpdate`	`text`	Update ElevenLabs context
`teleops.elevenlabs.restart`	`reason`, default web	Restart ElevenLabs session
`teleops.asr.force_send`	None	Finalize/send current ASR input
`teleops.tts.control`	`action`: `shutup` or `clear`	Interrupt/clear speech according to implementation
`teleops.arf`	None	Trigger interaction ARF
`teleops.head_tilt`	`angle`	Publish head tilt
`teleops.face_target`, `teleops.gaze_target`	`x`, `y`, `z`, `speed`	Publish face/eye target
`teleops.expression.set`	`name`, `magnitude`, `duration`	Publish expression
`teleops.animation.play`, `teleops.arm_animation.play`	`name`, `repeat`, `speed`, `magnitude`	Publish animation
`teleops.scene.set`	`name`	Select scene/ARF
`teleops.arm_mode.set`, `teleops.arm_pose.set`	`mode` or `pose`	Update arm configuration
`teleops.arm_playback.set`	`action`: play/pause/stop/fast_forward	Change arm player state
`teleops.language.set`	`language`, default `en-US`	Update recognizer alternative language
`teleops.voice.set`	`language`, `voice`	Update language-specific voice selection
`teleops.setting.set`	`key`, `value`, optional `agentId`	Update supported teleoperation setting
`teleops.animation_mode.set`	`mode`	Publish animation mode
`teleops.suggestion.accept`	`responseId`, `source`	Accept a chatbot suggestion
`teleops.hybrid_mouse.set`	`enabled`	Change hybrid mouse state
`runtime.setting.set`	`key`, `value`, optional `commandId` (falls back to `requestId`)	Update current runtime setting and emit result/state

Use /api/teleops/catalog and /api/teleops/settings to discover valid names/options. Model names and coordinates depend on current configuration; a schema-free JSON handler is not a promise that arbitrary values are meaningful.

command.stop is unsupported in the current ROS 2 adapter. It returns ok: false and an explicit error. It must not be used as an emergency-stop mechanism.

Canonical command names above are preferred. The adapter normalizes speech/tts, chat, gesture, expression/emotion/pose, scene, target and suggestion-accept aliases. However, /ws first filters raw command prefixes. Unprefixed animation.play, expression.set, scene.set and similar aliases can reach the adapter through the REST wrapper but are rejected by the socket filter. Use canonical teleops.* names on both transports.

Suggestion function keys are resolved by the current UI to teleops.suggestion.accept. The ROS2 adapter does not implement teleops.suggestion.shortcut; sending that string returns an unsupported-command result.

Defaults that affect physical behavior

The direct expression/animation handlers use value or 1 for magnitude, duration, speed and repeat as applicable. An explicit zero is therefore replaced by one on those paths. Zero magnitude is not a stop or suppression command. Face/gaze targets default to x=1, y=0, z=0, with speed zero; head tilt defaults to zero. These handlers convert values to ROS fields but do not implement a universal calibrated physical limit check.

Speech voice selection accepts voiceOverride, voice_override, then voice in that order; language accepts language, then lang, then en-US. Chat routes to the active ElevenLabs session when enabled, including session-reset handling, rather than always publishing the generic ASR/chat topic.

Performance state machine

Use performance.command with commandId and action. Supported actions include load, loadInline, loadAndRun, run, pause, resume, stop, unload, setAutopause and previewNode. Operation fields include performanceId, performance, source, startTime and options; include only fields appropriate to the selected action.

Results use performance.commandResult with commandId, ok, optional error and an authoritative runtime snapshot. loadAndRun sends the full draft once. Running progress arrives through performance.tick, which does not carry the full timeline.

performance.runtime: authoritative revision, contentRevision, state, metadata-only performance, playback, autopause, last command ID, error and server time.
performance.tick: transient runtimeRevision, sequence, current time, active timeline ID/index and server time.
Discard ticks for a different runtime revision.
Fetch GET /api/performances/current when content revision changes to obtain the full active payload, including unsaved preview data.
setAutopause updates performance.autopause through the runtime settings store in the current ROS adapter.

The current ROS2 performance handler has no seek action. Choose startTime when running or loading-and-running; moving the UI indicator alone selects a position and does not invoke a seek service. Unsupported actions return a failed performance command result and error state.

Stopping a performance is scoped to that runtime. It is different from the unsupported global robot stop.

Events for clients

Family	Useful events and handling
Connection	`robot.connection`; distinguish WebSocket connected from backend connected
Behavior/monitoring	`behavior.state`, `system.heartbeat`, `monitor.status`; heartbeat can be transient
Speech	`tts.utterance`, `tts.audio`, `tts.control`; stop local playback when control requests it
Chat	`chat.message`, `chat.message.metadata`, suggestion events; correlate metadata patches by `messageId` without submitting a second chat message
Settings	`runtime.settings`; update controls from authoritative values
Faces	Live candidate/recognition data via associated REST and adapter state

Do not assume every event listed in historical protocol notes is emitted by this ROS adapter: some notes describe simulated behavior. Inspect the active adapter's publications when implementing a consumer.

Evidence: webui/api/webui_api/main.py, event_hub.py, adapters/base.py, adapters/ros2.py, and webui/COMMUNICATION_PROTOCOL.MD in head-ros-nodes/ros.