The module managing all the components of the IOL application.

Description

This module is responsible for coordinating all the components that form the overall State Machine Handler application. To this end, it receives input from the human operator and then forwards proper requests to the classifier, the blob detector, the motion caputer, the motor layer to let the robot achieve the goal.

The commands sent as bottles to the module port /<modName>/human:rpc are the following:

(notation: [.] identifies a vocab, <.> specifies a number, "." specifies a string)

HOME
format: [home]
action: brings the robot back to its resting state.

CALIB_TABLE
format: [cata]
action: let the robot discover the table height.

CALIB_KINEMATICS
This command is splitted in two consecutive sub-commands:

format subcmd1: [caki] [start] [left]/[right] "object"
action: the robot reaches the object "object" with the specified hand and waits for the interaction with human based on force control in order to store the kinematic offsets corresponding to the given object.

format subcmd2: [caki] [stop]
action: terminate the calibration phase.

TRACK
format: [track] [start]/[stop]
action: let the robot track any moving object.

NAME
format: [name] "object"
action: let the robot learn a name to be associated with the object that was tracked right before.

FORGET
format: [forget] "object"
action: remove the object from the internal memory.
The special key "all" is used to purge the whole memory.

WHERE
format: [where] "object"
action: ask the robot to point at the given object.
If no_object/wrong is recognized then the robot enters the learning phase, where further commands are envisaged: i.e. [ack], [nack], [skip], ...

WHAT
format: [what]
action: ask the robot to say the name of the pointed object.
In case of a mistake the robot enters the learning phase where further commands are envisaged: i.e. [ack], [nack], [skip], [name], ...

THIS
format: [this] "object" ["click"]
action: tell the robot the name of the pointed object.
In case the option "click" is specified, then the location clicked in the clickable viewer will be used rather than the one resulting from the pointing action.

EXPLORE
format: [explore] "object"
action: let the robot explore the object from many different view points in order to improve its knowledge.

REINFORCE
format: [reinforce] "object" (<x> <y> <z>)
action: let the robot improve the recognition rate of the specified object whose 3d coordinates are provided.

MOTOR_COMMANDS
format: [take]/[grasp]/[push]/[touch]/[hold]/[drop] "object"
action: ask the robot to perform some motor commands on the given object.

ATTENTION
format: [attention] [start]/[stop]
action: switch on/off the attention system.

SAY
format: [say] "sentence"
action: let the robot utter the specified sentence.

The commands sent as bottles to the module port /<modName>/rpc are the following:

(notation: [.] identifies a vocab, <.> specifies a double, "." specifies a string)

STATUS
format: [status]
reply: "ack" "idle" | "busy" whether an action is currently being processed or not.

Libraries

YARP libraries.

Ports Created

/ <modName>/img:i receives the image acquired from the camera previously specified through the command-line parameters.
/ <modName>/img:o streams out the image containing recognized object. The image is updated whenever an action is required to be executed.
/ <modName>/imgLoc:o streams out the images for real-time objects localization.
/ <modName>/imgHistogram:o streams out the histogram of classification scores.
/ <modName>/histObjLocation:i receives the pixel in the image from which retrieve a cartesian position for displaying the classification histograms of the closest blob.
/ <modName>/recog:o streams out information about the object just recognized; the format is: ("label" "object-name") ("position_3d" (<x> <y> <z>)) ("type" "recognition"|"creation").
/ <modName>/imgClassifier:o used to pass images to the classifier.
/ <modName>/human:rpc receives requests for actions execution.
/ <modName>/rpc receives check requests.
/ <modName>/blobs:rpc used to forward requests to the blob detector for image segmentation.
/ <modName>/classify:rpc sends out requests for object classification.
/ <modName>/motor:rpc sends out motor commands.
/ <modName>/motor_stop:rpc used to interrupt/restore motor capabilities.
/ <modName>/motor_stop:i receives the [icub-stop] trigger from the verbal commands interpreter.
/ <modName>/point:i receives the latest pointed location within the image from the motion capture module.
/ <modName>/speak:o streams out the robot's sentences that need to be spoken.
/ <modName>/memory:rpc used to communicate with the objects properties collector.

Parameters

–name name

specify the module stem-name, which is iolStateMachineHandler by default. The stem-name is used as prefix for all open ports.

–rt_localization_period period

specify the period (given in [ms]) of the thread devoted to real-time objects localization. The default value is 30 ms.

–exploration_period period

specify the period (given in [ms]) of the thread devoted to images acquisition during objects exploration phase. The default value is 30 ms.

–memory_update_period period

specify the period (given in [ms]) of the thread devoted to updating the objects properties database. The default value is 60 ms.

–blobs_detection_timeout tmo

specify the timeout (given in [s]) for disabling blob detection. The default valus is 0.2 s.

–camera [left|right]

specify the camera used to localized object. The default camera is "left".

–skim_blobs_x_bounds (min max)

to reduce blob detection within the given x bounds on the table.

–skim_blobs_y_bounds (min max)

to reduce blob detection within the given y bounds on the table.

–classification_threshold t

to establish the default threshold t used by the classification algorithm.

–improve_train_period T

to extend the training instance of T seconds within which collect further relevant images.

–train_flipped_images [on|off]

allow training over flipped images to improve accuracy; default is "off".

–train_burst_images [on|off]

allow acquiring a burst of images over which carry out training later on; default is "off".

–skip_learning_upon_success [on|off]

avoid refining the objects knowledge when the recognition is successful; default is "off".

–hist_filter_length len

allow selecting the length of the moving average filter used to smooth out the quickly varying scores.

–block_eyes ver

for ver>0 specify to block eyes vergence at ver angle before gazing at the object to power-grasp.

–drop_position (x y z)

if specified, force dropping the object at a given location.

–tracker_type type

specify the opencv tracker type. Default value is BOOSTING.

–tracker_timeout tmo

specify the timeout (given in [s]) for stopping tracking. Default value is 5.0 s.

–tracker_min_blob_size (min max)

specify the minimum size of the blobs in terms of (width height) that can be tracked.

–attention [on|off]

enable/disable attention system at startup (default is on).

Tested OS

Windows, Linux

Author: Ugo Pattacini