Browser audio bridge

Role and transport

The bridge transports browser microphone PCM into ROS and robot output PCM back to browser clients. It is implemented in ros_chatbot/scripts/browser_audio_bridge.py, not the separate audio-stream source directory. The main ROS 2 stack enables it through enable_browser_audio_bridge.

Listener	Default	Configuration
Audio WebSocket	`0.0.0.0:3033`	`--ws-host`, `--ws-port`, `BROWSER_AUDIO_WS_HOST`, `BROWSER_AUDIO_WS_PORT`
Helper browser page	`0.0.0.0:3034`	`--http-host`, `--http-port`, matching environment variables
WebUI audio proxy	`/ws/audio`	`WEBUI_AUDIO_UPSTREAM_WS`, default `ws://127.0.0.1:3033`

--no-http disables the helper page. --static-dir/BROWSER_AUDIO_STATIC_DIR selects its assets. Prefer the same-origin WebUI socket through HTTPS for browser microphone use. Direct bridge ports need their own network/access policy.

Protocol version 2 handshake

Send a JSON text frame first:

{"type":"audio.hello","version":2,"clientId":"operator-tab-1","outputChannel":"monitor"}

outputChannel selects monitor or internal; invalid/absent values fall back to internal. The ready message advertises:

{
  "type":"audio.ready",
  "version":2,
  "input":{"sampleRate":16000,"channels":1,"encoding":"pcm_s16le","chunkMs":20},
  "leaseSeconds":6.0,
  "heartbeatSeconds":2.0,
  "playbackBufferMs":{"min":100,"max":1000}
}

The server also sends input.owner. An unsupported version returns audio.error with code: unsupported-version and supported versions. Other v2 controls before hello return hello-required.

Microphone ownership

Send {"type":"input.acquire"} and wait for input.owner with granted: true. Only one connection owns input. Another connection receives occupied unless it explicitly requests takeover: true. Takeover notifies the former owner and updates the ROS input override state.

Owner state includes granted, occupied, ownerClientId, reason and leaseSeconds. Reasons include acquired, occupied, taken-over, released, disconnect and lease-expired.

The six-second lease is renewed by input frames or owner pings. Send ping around the advertised two-second interval, including id and clientSentAt; pong echoes these and reports server time in milliseconds. Release explicitly with input.release when the microphone stops. A one-second monitor loop releases expired ownership; disconnect also releases it.

Binary input format

Once ownership is granted, send signed 16-bit little-endian PCM, 16,000 samples/second, mono. A 20 ms chunk contains 320 samples or 640 bytes. The bridge rejects empty or odd-byte frames with invalid-pcm-frame; it rejects nonowner frames with input-not-owned. Before hello, binary frames are ignored with a log warning.

The server checks frame byte alignment, not whether the PCM actually has the advertised sampling characteristics. Browser clients must resample and encode correctly. Raw microphone capture rates such as 48 kHz cannot be sent unchanged.

Output subscriptions and stream lifecycle

Send {"type":"output.subscribe","enabled":true}. Wait for output.start before interpreting binary audio. It carries streamId, sampleRate, channels and encoding: pcm_s16le. Use that announced format; output need not match the input sample rate.

output.end identifies the stream and reason. output.clear means queued playback should be discarded. Disabling subscription triggers a clear with reason unsubscribed. New stream IDs or formats can replace an active stream. Client jitter buffering should account for RTT/arrival jitter within the advertised 100–1000 ms range.

Client telemetry consists of playback.started, playback.stats and playback.ended. The first two are logged; internal playback completion can release speech gating. A monitor client's completion deliberately does not finish the robot's playback gate.

ROS integration

Purpose	Topic
Browser microphone	`/hr/control/browser_audio_in`
Input override	`/hr/control/browser_audio_override`
Internal playback status	`/hr/control/browser_audio_playback`
Internal audio / control	`/hr/control/browser_audio_out`, `/hr/control/browser_audio_out_control`
Operator monitor audio / control	`/hr/control/webui_audio_out`, `/hr/control/webui_audio_out_control`

Microphone publication uses AudioStream and stream QoS. Monitor and internal channels share transport and ownership but maintain separate stream/completion state. Monitoring alone does not enable acoustic echo cancellation; the internal browser needs both capture and speaker playback for AEC.

UI behavior and troubleshooting

The WebUI speaker switch is tab-local and off by default. It subscribes that tab to monitor output. It does not change the physical robot's speaker choice. The microphone switch captures through an AudioWorklet and sends PCM; it does not use browser Web Speech Recognition.

For a silent monitor, check socket connection, output subscription, output.start, browser autoplay restrictions, and the correct output channel. For no transcripts, verify ownership, actual PCM rate, ROS input topic and the recognizer's browser input selection. If input stops after a short idle interval, inspect ping/lease renewal. After reconnection perform hello, subscription and ownership negotiation again; do not assume state survived.

Evidence: head-ros-nodes/ros/ros_chatbot/scripts/browser_audio_bridge.py, browser helper assets, WebUI main.py and speech frontend modules. No microphone capture or robot playback was executed for this guide.

Helper HTTP status

The optional HTTP listener also serves GET /status as JSON. Its status payload reports client/input/output state and audio activity timing. It is useful for distinguishing a live bridge from an active microphone stream. Static helper assets use Cache-Control: no-store across release changes. This listener is separate from the WebUI /api/health endpoint.