The module performs a sequence of coding and pooling operator and retrieves a hierarchical image representation.

The module performs a sequence of coding and pooling operator and retrieves a hierarchical image representation.

Description

This module is responsible for coding images into feature vectors. To this end, we extract local appearance descriptors (sparse or dense). These descriptors are coded using a learned dictionary and the pooled together via a spatial pyramid representation.

The commands sent as bottles to the module port /sparseCoder/rpc are the following:

(notation: [.] identifies a vocab, <.> specifies a double, "." specifies a string)

HELP format: [help] action: it prints the available commands.

DUMP_SIFT format: [dump] "a" action: starts to dump local descriptors that can be used for dictionary learning. "a" is an optional parameters that stands for "append" in case you want to keep old descriptors previously dumped.

DICTIONARY_LEARNING format: [learn] <dictionarySize> "usePCA" <dimPCA> action: starts to learn a dictionary from previously dumped local descriptors. <dictionarySize> is the number of atoms of the dictionary, "usePCA" performs standard PCA analysis on low-level descriptors before the encoding and <dimPCA> indicates the number of relevant eigenvectors that you want use for the projection.

Libraries

• YARP libraries.
• OpenCV 2.2 libraries.

Ports Created

• /sparseCoder/img:i receives the image acquired from the camera.
• /sparseCoder/img:o streams out the image containing the extracted local descriptors.
• /sparseCoder/code:o streams out the vector containing the hierarchical image representation.
• /sparseCoder/rpc receives requests for SIFT dump or dictionary learning.

Parameters

–dictionary_file file

• specify the dictionary file.ini where atoms are stored.

–coding_mode coding

• specify the coding method used. Possible values are "BOW" for Bag of Words, "SC" for Sparse Coding, "BCE" for Best Codes Entries.

–nearest_neighbor knn

• specify the number of nearest neighbors used for the encoding with BCE.

–dense_descriptors [1|0]

• specify is a regular grid of dense descriptors must be used (1) or sparse keypoints must be detected (0).

–grid_scale scale

• specify the scale of local descriptors extracted from a dense grid.

–grid_step step

• specify the sampling step among descriptors in a dense grid.

Tested OS

Linux

Author

Sean Ryan Fanello, Carlo Ciliberto