engagement_recognition Documentation

engagement_recognition: Recognizing Engagement Behaviors in Human-Robot Interaction

Human-robot engagement package developed at Worcester Polytechnic Institute by Prof. Charles Rich et al.
Currently contains the recognition node which is responsible for recognizing directed gaze, mutual facial gaze, adjacency pairs, and backchannels during an interaction.

Author(s): Brett Ponsler (bponsler (at) wpi (dot) edu)
Review Status: unreviewed
Review Notes:
Homepage: http://sourceforge.net/projects/ros-engagement/
Downloadable files

engagement is "the process by which two (or more) participants establish, maintain and end their perceived connection during interactions they jointly undertake" (Sidner, C.; Lee, C.; Kidd, C.; Lesh, N.; and Rich, C. 2005. Explorations in Engagement for Humans and Robots. Artificial Intelligence 166(1-2): 140-164).

ROS API

List of nodes:

engagement_recognition/recognition

recognition

The recognition node receives incoming messages pertaining to actions performed by actors and provides services which allow a particular connection event to be recognized (adjacency pair, directed gaze, and mutual facial gaze). Keeps track of the engagement statistics during an interaction and publishes messages in the event that human actors initiate connection events.

Usage

$ recognition [standard ROS args]

ROS topics

Subscribes to:

Topics pertaining to actors performing actions

"performing/look": [engagement_msgs::Performance] An actor has started or stopped looking at a set of objects.
"performing/point": [engagement_msgs::Performance] An actor has started or stopped pointed at a set of objects.
"performing/shake": [engagement_msgs::Performance] An actor shook his head as an acknowledgment behavior.
"performing/nod": [engagement_msgs::Performance] An actor nodded his head as an acknowledgment behavior.
"performing/utterance": [engagement_msgs::Speech] The start or end of speech for an actor was detected.

Topics pertaining to vision information

"vision/face": [engagement_msgs::ActorIDStatus] A human face was detected.

Publishes to:

"recognition/human/status": [engagement_msgs::ActorIDStatus] The current engagement status of an actor.
"recognition/human/adjacency_pair": [engagement_msgs::HumanAdjacencyPair] An actor initiated an adjacency pair.
"recognition/human/backchannel": [engagement_msgs::HumanBackchannel] An actor performed a backchannel.
"recognition/human/directed_gaze": [engagement_msgs::HumanDirectedGaze] An actor initiated a directed gaze.
"recognition/human/mutual_facial_gaze": [engagement_msgs::HumanMutualFacialGaze] An actor initiated a mutual facial gaze.

ROS services

"recognition/statistics": [engagement_srvs::ActorStatistics] The goal to return the current engagement statistics.
"recognition/robot/adjacency_pair": [engagement_srvs::RobotAdjacencyPair] The goal to establish adjacency pair, with the success being returned.
"recognition/robot/directed_gaze": [engagement_srvs::RobotDirectedGaze] The goal to establish directed gaze with a set of objects, with the success being returned.
"recognition/robot/mutual_facial_gaze": [engagement_srvs::RobotMutualFacialGaze] The goal to establish mutual facial gaze, with the success being returned.

ROS parameters

Reads the following parameters from the parameter server

"recognition/conf/degraded": [boolean] True to run recognition in degraded mode. Defaults to false.

"recognition/conf/verbosity": [int] The verbosity used for debug print statements (anything with a log level <= verbosity will be printed when in Debug mode).
"recognition/conf/window_size": [double] The window size for computing engagement statistics.
"recognition/conf/log_stats": [string] The file in which to save recognition statistics.
"recognition/conf/log_stats_directory": [string] The directory in which to save the statistics file.

"recognition/conf/recog_fg_color": [int] The foreground color used for printing statements pertaining to the recognition node.
"recognition/conf/recog_bg_color": [int] The background color used for printing statements pertaining to the recognition node.

"recognition/conf/actor_fg_color": [int] The foreground color used for printing statements pertaining to actors.
"recognition/conf/actor_bg_color": [int] The background color used for printing statements pertaining to actors.

"recognition/conf/state_fg_color": [int] The foreground color used for printing statements pertaining to states.
"recognition/conf/state_bg_color": [int] The background color used for printing statements pertaining to states.

"recognition/conf/sema_fg_color": [int] The foreground color used for printing statements pertaining to semaphores.
"recognition/conf/sema_bg_color": [int] The background color used for printing statements pertaining to semaphores.

"recognition/conf/adjacency_pair/max_delay": [double] The maximum delay for the adjacency pair recognizer.
"recognition/conf/adjacency_pair/fg_color": [int] The foreground color used for printing statements pertaining to the adjacency pair recognizer.
"recognition/conf/adjacency_pair/bg_color": [int] The background color used for printing statements pertaining to the adjacency pair recognizer.

"recognition/conf/backchannel/fg_color": [int]The foreground color used for printing statements pertaining to the backchannel recognizer.n
"recognition/conf/backchannel/bg_color": [int] The background color used for printing statements pertaining to the backchannel recognizer.

"recognition/conf/directed_gaze/max_delay": [double] The maximum delay for the directed gaze recognizer.
"recognition/conf/directed_gaze/fg_color": [int] The foreground color used for printing statements pertaining to the directed gaze recognizer.
"recognition/conf/directed_gaze/bg_color": [int] The background color used for printing statements pertaining to the directed gaze recognizer.

"recognition/conf/mutual_facial_gaze/max_delay": [double] The maximum delay for the mutual facial gaze recognizer.
"recognition/conf/mutual_facial_gaze/fg_color": [int] The foreground color used for printing statements pertaining to the mutual facial gaze recognizer.
"recognition/conf/mutual_facial_gaze/bg_color": [int] The background color used for printing statements pertaining to the mutual facial gaze recognizer.

.launch files

This section describes the launch file used for launching this node:

"recognition.launch": Used to launch the recognition node. Contains all the parameters used for the recognition node.
"degraded-recognition.launch": Used to launch the recognition node in degraded mode. Contains all the parameters used for the recognition node.
"test_recognition.launch": Used to launch the recognition tests. Contains all parameters used for testing the recognition node.

Usage

$ roslaunch recognition.launch

Usage

$ roslaunch degraded-recognition.launch

Usage

$ roslaunch test_recognition.launch