Skip to content

Behavior, attention and event processing

Backends and selection

The main ROS 2 launch selects behavior through behavior_backend, defaulting to states. Other current source paths include the behavior manager and rules-engine integration. Select the backend deliberately; launching two owners of the same behavior state or animation outputs can create conflicting commands.

The states backend publishes /hr/behavior/current_state, writes the matching shared parameter and listens for state switches, operator commands, speech, people/objects and performance events. Main state families include idle, interacting, presenting, photography and gallery. Interacting and presenting have substates for interested/listening/speaking/thinking/scripted/posing/selfie/gallery and waiting/performing.

A state name accepted by the browser must still be supported by the selected backend. TeleOps chooses situation commands for the manager backend and state commands for the states backend.

Remote commands and observability

Interface Role
/hr/behavior/state_switch State selection
/hr/behavior/operator_command Operator behavior command
/hr/behavior/current_state Retained state output
/commands/set_situation Situation selection used by manager path
/hr/animation/set_face_target Head/face attention target
/hr/animation/set_gaze_target Eye attention target
/events/main Structured JSON domain events

Use WebUI command envelopes for HTTP/WebSocket clients. Direct ROS clients should discover current types and QoS; state uses the platform's best-effort transient-local profile. A reliable-only subscriber can be incompatible with a best-effort publisher.

Attention and animation support

Attention consumes tracked perception and speech activity to select head/eye targets, with runtime configuration controlling tracking/idle behavior. Its output is motion, not a read-only visualization. Animation support publishes expression/gesture/arm-animation requests using the configured libraries; favorite-animation and speech-motion support can also generate motion.

If a manual pose is being overridden, inspect active behavior/attention/animation publishers and actuator control mode. Sending a direct command without changing the relevant owner does not ensure exclusive control.

Presence and object reducers

people_presence_node turns tracked people into stable domain events. Constructor defaults require two frames in a four-frame window and two seconds of absence. It emits person_entered_scene, person_left_scene, and recognized_person_detected; recognized events are correlated by tracker and face ID.

objects_reducer_node similarly aggregates object observations with confidence/frame/absence gates and periodic context. Its constructor defaults include confidence 0.6, three present frames in a five-frame window, absence 1.5 seconds and context cadence 2 Hz. Check launch overrides before treating these as deployed values.

Domain event envelopes contain eventType, eventSource, timestamp, and event_payload. Presence messages are semantic summaries, not one event for each camera frame. Downstream rules should tolerate repeated context and disappearing/reused tracker IDs.

Scene and rules integration

scene_description_node subscribes to visual-processing prompt configuration and emits scene_description events for extracted text, suppressing consecutive duplicate text. It is not itself a camera/VLM inference service.

Rules-engine adapter, Redis bridge, NLP-event, supervisor and manager source paths connect domain events to decisions and operator situations. Their topics, parameters and launch registration are enumerated in the behavior source reference. The selected backend determines which paths are active; source presence does not mean every node runs by default.

Operating and troubleshooting

  1. Read the effective backend, current state and selected situation.
  2. Confirm incoming perception or speech events separately from state output.
  3. Inspect reducer thresholds and staleness before changing behavior rules.
  4. Compare active attention/animation settings with manual control expectations.
  5. Review performance events when presentation state appears stuck.
  6. Persist intended configuration only after confirming live behavior.

A WebUI state-command acknowledgement means publication/acceptance at that layer; observe the resulting authoritative state. The unsupported WebUI global stop is not a behavior safety mechanism.

Evidence: head-ros-nodes/ros/r2_behavior/scripts/{states.py,attention.py,animation.py,behavior_manager_node.py,people_presence_node.py,objects_reducer_node.py,scene_description_node.py}, config nodes and hr_ros2 launch files.