Speech recognition

Pipeline

The current online recognizer in asr/scripts/recognizer_node.py exposes a provider-neutral ROS 2 interface. Audio can come from a local PyAudio device or the browser audio bridge; provider-final transcripts, interim transcripts, speech activity and delayed metadata are separate outputs. The recognizer also coordinates robot-speech suppression and input-ownership overrides.

Configuration namespace: /hr/perception/speech_recognizer. Providers are google (default), soniox, and oruk. Changing provider is applied live and does not silently fall back to another provider.

Provider and language settings

Setting	Behavior
provider	Select Google, Soniox or Oruk
enable	Enable recognition lifecycle
audio_input	Local pyaudio or browser PCM path
language, alt_language, recent_languages	Locale streams/hints
selected_language	Empty for automatic selection; explicit locale constrains supported provider behavior
phrases / context service	Adaptation phrases or provider context terms
continuous, activity/inactivity settings	Recognition session lifecycle
Robot speech suppression / skip_first_audio	Audio gating and startup handling
enable_speaker_diarization	Soniox labels; off by default, ignored by Google
final_result_timeout	Google fallback finalization or Soniox finalize behavior

Google uses GOOGLE_APPLICATION_CREDENTIALS; Soniox uses SONIOX_API_KEY; Oruk uses ORUK_API_KEY. Configure credentials on the runtime host/container, not in a browser request. Oruk performs automatic language recognition and can publish phrase-emotion metadata after the transcript.

Google can use separate locale streams with result ranking. Soniox uses one multilingual stream with ordered language hints. A selected supported Soniox locale becomes a strict hint; unsupported choices clear back to Auto. Primary language-code normalization includes Hebrew and Chinese aliases. Ranking/bias settings retained for Google can be saved no-ops for Soniox; changing them need not change Soniox results.

Remote interfaces

Interface	Role
/hr/perception/hear/sentence	Final ChatMessage
/hr/perception/hear/interim_speech	Interim ChatMessage
/hr/perception/hear/asr_metadata	Delayed operator metadata correlated to a result
/hr/perception/hear/event	Recognition/activity event
/hr/control/browser_audio_in	Browser PCM input
/hr/control/browser_audio_override	Input ownership override
/hr/control/webui/force_send_asr	Force-send request
/hr/control/speech/event	Robot speech gating input
/hr/control/speech/robot_said	Language/speech history input

Active HR profile settings can override topic names. Use the ASR declaration reference and running graph for exact types and context service name.

ros2 topic info -v /hr/perception/hear/sentence
ros2 interface show hr_msgs/msg/ChatMessage
ros2 topic echo /hr/perception/hear/interim_speech --qos-reliability best_effort

Final/interim distinction matters: do not submit every interim as a new user turn. result_id/message correlation lets delayed metadata update the existing transcript. WebUI metadata patches must not create or resend input.

Browser microphone workflow

Open the WebUI through a browser context that permits microphone capture, connect /ws/audio, negotiate v2, acquire input, and send 16 kHz mono PCM16. The lease must stay alive. The bridge publishes input override so recognition/provider input selection can follow the owner. Merely enabling the browser's microphone permission does not prove bridge ownership or provider connectivity.

In TeleOps use the ASR provider and language controls, confirm accepted runtime settings, then speak and observe interim/final results. If recognition is disabled or speech is suppressed, inspect current settings and robot-speech state before reconnecting audio. Audio protocol details.

Recordings

Provider-finalized transcripts save mono 16 kHz signed-16-bit WAV under ASR_TMP_OUTPUT_DIR or $HR_DIR/asr/tmp. Saved audio matches provider input and is cropped to usable word/token timestamps. Startup moves previous root recordings into dated archive folders, retained for 14 days. Interim-only/discarded/shutdown sessions are not saved; recording errors are logged without blocking transcript publication.

ChatMessage.audio_path identifies saved audio. The ASR Files UI lists current and archived recordings. These files may contain private conversations; they belong to robot storage and backups, not the public documentation artifact.

Failure isolation

Check capture/device or browser ownership, PCM format, ROS input activity, enable/gating settings, provider credentials/network, provider response and final publication in order. A speech activity indicator is not proof of a final result. If the correct transcript appears twice, inspect client handling of interim/final/metadata and input routing before blaming the provider.

Evidence: head-ros-nodes/ros/asr/README.md, scripts/recognizer_node.py, ASR source providers/configuration, ros_chatbot/scripts/browser_audio_bridge.py, and WebUI speech modules. No live transcription was performed for this guide.