Skip to content

TeleOps operator workflow

Before issuing commands

Open Robot WebUI at #/teleops. Check API connectivity, ROS adapter status and the intended robot identity. The page combines head/behavior controls, chat and exact speech, compact performance playback, poses, animations and arms. Desktop columns can be hidden with the side toggles; compact layouts rearrange these panels rather than creating a separate robot state.

The UI can show optimistic values while commands are pending. Wait for command/setting results and authoritative events before assuming a change succeeded. A disconnected socket can leave previously displayed state visible.

Chat versus exact speech

  • Chat input submits a user message to the conversational pipeline. It can produce suggestions or an automatic response depending on hybrid/provider settings.
  • Speech input asks the robot to say the supplied text using selected language/voice.
  • Repeat user message submits that input again; it is a new conversation action.
  • Repeat speech requests the displayed robot utterance again.
  • Force send finalizes the current recognizer message where supported.
  • Shutup/Clear send TTS control; they also affect browser playback through control events.
  • Reset conversation submits the :reset command through the chat path. It is not a full platform reset.

Select the intended language and voice before speaking. Choices come from the live catalog, not a fixed list in this handbook. If exact speech works but chat does not, inspect chatbot/provider configuration rather than changing audio output.

Suggestions and hybrid operation

Hybrid mode controls whether operator selection is involved. Suggestions carry response IDs and sources; accept the desired suggestion or use its assigned shortcut. Acceptance and final spoken output are separate events. A stale suggestion from a prior interaction should not be treated as a current response.

Hybrid mouse mode changes the operator interaction mode. Review the displayed mode before using target controls. Interim recognition can be manually repeated/sent as final; do this deliberately to avoid submitting the same utterance twice.

Quick Settings and keyboard speech selection

Open the wand button to show Quick Settings. Response, Length and Scene choices come from the current settings/catalog. Selecting an option sends a runtime setting immediately. Agent buttons toggle agent.enabled for the displayed agent ID; an empty agent list means the robot has not supplied available agents.

Capability switches control Vision (visualProcessing), RAG, DB Keeper and Metaprompting. These change runtime configuration; successful acknowledgement does not prove a provider is configured or healthy. Inspect the configuration service for persistence and underlying parameter paths.

The Instant prompt text is a draft while typing. Leaving the field or closing the popup commits a changed draft. Check the command result before assuming the new prompt has taken effect.

In the speech input, Up/Down cycles available TTS suggestions, Escape dismisses the list, and Enter speaks the selected suggestion or the typed text when none is selected. Choosing a suggestion speaks it immediately. Conversation response shortcuts resolve to suggestion acceptance; they are separate from speech-input completion. Chat history preserves a scrolled-back reading position rather than forcing every new message into view.

Speech recognition and browser audio

Use the ASR enable switch, provider selector and language selector to configure recognition through runtime settings. Auto language selection differs from selecting one supported language. Soniox diarization and Oruk delayed metadata can appear alongside transcripts; metadata updates do not create new user turns.

The header microphone and speaker switches are session-local browser controls. Speaker monitoring starts off. Enabling it subscribes this tab to robot PCM; it does not reroute the physical robot's speaker. The microphone captures PCM through the audio bridge and requires browser permission and input ownership.

If another tab owns input, the UI can request an explicit takeover. The old owner loses its lease. Closing/disconnecting releases ownership; a missing heartbeat also expires it. Check audio protocol and diagnosis if microphone permission is granted but no input reaches the recognizer.

ElevenLabs status is provider/session state, not just socket connectivity. The Hangup control disables that agent through runtime settings. A stale status display requires checking fresh session events before retrying.

Head, behavior and field of view

The Head panel can be collapsed. It exposes the current behavior state/situation, target controls and head tilt. On the states backend, state buttons issue behavior.set_state; the manager backend uses behavior.set_situation.

Field-of-view imagery comes from the perception feed. Pointer targeting sends face-target coordinates through the adapter; it can move the head. Verify that the image/target calibration is appropriate for the current robot. A missing/stale image should be diagnosed before treating the pointer as an accurate spatial target.

The UI does not provide a supported whole-robot emergency stop through command.stop. Use the robot's actual physical safety procedure.

Poses and animations

Poses select an expression plus duration and magnitude. The visible pose controls allow duration 0–8 seconds and magnitude 0–1. Animations expose speed and magnitude controls; TeleOps speed spans 0.2–2 and magnitude 0–1 in the inspected source. Names/grouping/favorites come from the catalog and helper classification.

Selecting an item sends a runtime action immediately. It does not save a performance or change the animation library. Use reduced magnitude and a prepared robot when checking a newly configured action. Do not assume expression names from a different robot map correctly.

Arms

Arm controls include pose/mode selection, torque off, manual, playback play/pause, fast forward and stop. Pose selection first changes runtime arm mode to animations, then selects pose. These are state-changing commands and can affect support/torque behavior.

Arm animations use the same current speed/magnitude controls and are grouped by catalog/favorites and sitting/standing/left/right/both naming where available. Stop applies to the arm player; it is not documented as stopping every robot process.

Compact performance panel

The TeleOps performance column is a read-only compact editor/player. Browse/select content, use playback controls and watch authoritative runtime/ticks. For editing timelines/nodes open the full Performances page. Hidden columns do not stop a running performance.

Recovery checklist

If a control does nothing, inspect its pending/error result, connection state, selected catalog name, runtime owner and backend capability. If a change reverts, compare live runtime setting acknowledgement with saved configuration. If speech duplicates, distinguish repeated input, suggestion acceptance, speech replay and separate browser/physical audio outputs.

Evidence: head-ros-nodes/ros/webui/frontend/src/{App.jsx,pages/TeleOpsPage.jsx,pages/actionCatalog.js}, operator chat components and runtime API clients. Wire protocol.