Skip to content

Faces, image enrollment and recognition

Data flow

Robot WebUI stores face records, uploaded images and embeddings under FACES_DATA_DIR (defaulting beneath the configured storage root). Jetson perception computes quality-gated embeddings; the WebUI matches identities and publishes feedback. The Faces UI displays stored people separately from live tracker candidates.

REST operations

Operation Input / result
GET /api/faces Returns faces array
POST /api/faces id, display_name or name, info, enabled; returns saved face
DELETE /api/faces/{face_id} Deletes stored face; returns ok and ID
GET /api/faces/images/{image_path} Image bytes; invalid/absent paths return 404
POST /api/faces/{face_id}/images Multipart file; saves image and attempts embedding
POST /api/faces/{face_id}/regenerate Rebuild embeddings for existing person; unknown ID returns 404
POST /api/faces/regenerate Rebuild all stored people
POST /api/faces/search Nonempty embedding, optional threshold and model
GET /api/faces/live-candidates Current adapter live tracker candidates
GET/POST /api/faces/recognition-settings Read/update recognition threshold

Threshold must parse to a finite number between 0 and 1. Missing threshold on settings POST returns HTTP 400 with missing_threshold; invalid values return invalid_threshold. Search without an embedding returns 400. Search defaults to the current recognition threshold, or 0.45 if absent.

Enroll from a remote client

First create/update the person record, then upload an image:

curl --fail-with-body "$ROBOT_UI_URL/api/faces" \
  -H 'Content-Type: application/json' \
  --data '{"id":"example-person","display_name":"Example Person","enabled":true}'
curl --fail-with-body "$ROBOT_UI_URL/api/faces/example-person/images" \
  -F 'file=@example-person.jpg'

Use authorized enrollment images. Do not upload them to the public documentation site. The image upload persists the image before extraction; it can return ok: true, embedded: false, and embedding_error. That means image storage succeeded but recognition enrollment did not. Inspect the result rather than treating the upload status as a successful embedding.

Successful extraction returns the updated face, image metadata and embedding metadata; the API omits the full vector from that response's embedding object. The extraction adapter waits for the ROS service and returns its result/error.

Regeneration and recovery

Regeneration clears existing target embeddings first, then reads each stored image and attempts extraction. The result includes regenerated, failures, and current faces. It can return ok: true with partial or total failures. Do not run regeneration casually when the extraction service is unavailable: old embeddings have already been cleared.

The UI toast distinguishes successful regeneration, partial success and total failure. Live candidate rows distinguish acceptance/rejection reasons, latest recognition, best recognition and call count. A tracker being visible does not mean its crop passed quality gates or matched an enrolled identity.

UI workflow

  1. Open Faces, select an existing record or start a new one, and save identity/display information.
  2. Upload suitable images and inspect embedding status individually.
  3. Confirm the embedding service is available before regenerating stored images.
  4. Review live candidate reasons and latest/best matches while testing recognition.
  5. Adjust threshold only after comparing intended matches and false matches; save and verify the current setting.
  6. Back up the face store before bulk regeneration or deletion. These operations change persistent data.

Evidence: webui/api/webui_api/api_routes.py, face_store.py, adapters/ros2.py, and frontend/src/pages/FacesPage.jsx under head-ros-nodes/ros, plus perception embedding source.

Matching details and result interpretation

Stored/query embeddings are normalized to float vectors before matching. Zero-norm vectors raise an error rather than becoming a valid face. Search skips disabled people, empty/dimension-mismatched vectors and embeddings with a conflicting nonempty model when a model filter was supplied. A stored embedding without model metadata is not excluded solely by that filter.

The implementation selects the greatest dot-product score, beginning from zero, and marks matched when a face was selected and score meets the threshold. Results include matched, face_id, display_name, score, threshold and model. A best candidate can be present while matched is false; consumers must inspect that boolean rather than interpreting any nonempty name as an accepted identity. Do not mix embeddings from unrelated models merely because their dimensions match.