tool-affordances

Repo for Affordance project related stuff


Project maintained by robotology Hosted on GitHub Pages — Theme by mattgraham

Overview

This repository contains modules concerning the exploration and learning of tool affordances. The general learning schema consists of 3 phases:

  1. Exploration: After being provided a tool, the iCub extracts a descriptor vector to represent it, and executes a series of actions, recording the achieved effects.
  2. Learning: After enough tool affordance data (that is, the effects achieved by each action with each tool) has been recorded, it is sent to an affordance learning model, which learns the relationship between the tool, the action, and the achieved effects.
  3. Prediction: Given any tool similar to the ones used diring training, the iCub extracts its descriptive features and feeds them to the trained affordance model, which in turn is able to predict the expected effect for any given action. This predictions can be used to select the best action for any task acheived through the action repertoire, or even to select another tool.

A diagram of the described learning schema can be observed below: learnSchema

This general schema was used in several studies on tool affordances with iCub, listed below.

Moreover, it provides the motor and coordination modules to run the tool use demo (click on image to see video on Youtube): tool use demo video

Modules

The modules are located under the modules directory. Given the number of different behaviors achieved by the modules in this repository, they can be divided according to their function:

Perception:

Motor:

Memory

Coordination

Managers: (Relay and organize commands, but do not execute any behavior)

State Machines: (Upon execution generate a given behavior based on inputs and states)

Data analysis and learning

The MATLAB folder contains all the functions for data analysis and visualization applied in the studies listed below. These functions are conveniently organized in the following folders:

Use

The modules in this repository are intended for two main uses:

How to compile

In Linux systems code can be compiled as follows:

git clone https://github.com/robotology/tool-affordances.git
cd tool-affordances
mkdir build; cd build
ccmake ..
make install

Dependencies

All modules require

The experiments using 2D features require:

3D tool representation as well as pose and tooltip estimation, used in the demo and in all experiments using 3D information, are provided by the modules on:

The methods executed by the modules in this repository have been described in detail in the following papers:

As well as in Tanis Mar’s PhD Thesis:

“Affordances for and from Manipulation: A Developmental Approach to Tool Use Learning on iCub.”

License

Material included here is Copyright of iCub Facility - Istituto Italiano di Tecnologia and is released under the terms of the GPL v2.0 or later. See the file LICENSE for details.