visual-tracking-control

⚙️ visual-tracking-control

The visual-tracking-control project is a suite of cross-platform applications for visual tracking and visual servoing for the humanoid robot platform iCub.

The suite includes:

• hand-tracking: a visual end-effector tracker using a 3D model-aided particle filter [1];
• visual-servoing: a visual servoing YARP plugin to control the pose (position and orientation) of the end-effector using image feedback [2].
  • The visual-servoing consist of two plugins, a client and server version. On one hand, the server must be always running via yarpdev and is responsible to command the robot. On the other hand, the client is dynamically loaded by any application using visual servoing interfaces and has the sole purpose of communicating commands to the server. This architecture enable distributed computation over different machine and decouples client and server implementations.

The output of the hand-tracking application can be visualized by means of the iCubProprioception module. iCubProprioception provides an augmented-reality application which superimposes the 3D model of the end-effector onto the camera images, using the estimated pose provided by hand-tracking.

Overview

• 🎛 Dependencies
• 🔨 Build the suite
• 📝 API documentation and example code
• 📑 Reference

🎛 Dependencies

visual-tracking-control suite depends on

• BayesFilters - version >= 0.6
• iCub
• iCubContrib
• OpenCV - version >= 3.3, built with CUDA >= 8.0
• SuperimposeMesh - version >= 0.9
• YARP

🔨 Build the suite

Use the following commands to build, install and link the library.

Build

With make facilities:

$ git clone https://github.com/robotology/visual-tracking-control
$ cd visual-tracking-control
$ mkdir build && cd build
$ cmake -DBUILD_HAND_TRACKING=ON -DBUILD_VISUAL_SERVOING_CLIENT=ON -DBUILD_VISUAL_SERVOING_SERVER=ON ..
$ make
$ [sudo] make install

With IDE build tool facilities:

$ git clone https://github.com/robotology/visual-tracking-control
$ cd visual-tracking-control
$ mkdir build && cd build
$ cmake -DBUILD_HAND_TRACKING=ON -DBUILD_VISUAL_SERVOING_CLIENT=ON -DBUILD_VISUAL_SERVOING_SERVER=ON ..
$ cmake --build . --target ALL_BUILD --config Release
$ cmake --build . --target INSTALL --config Release

📝 API documentaion and example code

Doxygen-generated documentation is available here.

📑 References

[1] C. Fantacci, U. Pattacini, V. Tikhanoff and L. Natale, “Visual end-effector tracking using a 3D model-aided particle filter for humanoid robot platforms”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 2017. arXiv preprint arXiv:1703.04771.
[2] C. Fantacci, G. Vezzani, U. Pattacini, V. Tikhanoff and L. Natale, “Markerless visual servoing on unknown objects for humanoid robot platforms”, IEEE International Conference on Robotics and Automation (ICRA), Brisbane, AU, 2018. arXiv preprint arXiv:1710.04465.