Writing a new test
Create a folder with the name of your test case in the icub-tests/src/ folder to keep your test codes:
$ mkdir icub-tests/src/example-test
Create a child test class inherited from the YarpTestCase:
#ifndef _EXAMPLE_TEST_H_
#define _EXAMPLE_TEST_H_
#include <yarp/robottestingframework/TestCase.h>
class ExampleTest :
public yarp::robottestingframework::TestCase {
public:
virtual bool setup(yarp::os::Property& property);
virtual void tearDown();
virtual void run();
};
#endif
This is just an example test, use it as a reference to implement new tests.
Implement the test case:
#include "ExampleTest.h"
#include <robottestingframework/dll/Plugin.h>
#include <robottestingframework/TestAssert.h>
using namespace std;
using namespace robottestingframework;
using namespace yarp::os;
ExampleTest::ExampleTest() : yarp::robottestingframework::TestCase("ExampleTest") {
}
ExampleTest::~ExampleTest() { }
bool ExampleTest::setup(yarp::os::Property &property) {
if(property.check("name"))
setName(property.find("name").asString());
string example = property.check("example", Value("default value")).asString();
ROBOTTESTINGFRAMEWORK_TEST_REPORT(Asserter::format("Use '%s' for the example param!",
example.c_str()));
return true;
}
void ExampleTest::tearDown() {
}
void ExampleTest::run() {
int a = 5; int b = 3;
ROBOTTESTINGFRAMEWORK_TEST_CHECK(a<b, "a smaller then b");
ROBOTTESTINGFRAMEWORK_TEST_CHECK(a>b, "a bigger then b");
ROBOTTESTINGFRAMEWORK_TEST_CHECK(a==b, "a equal to b");
}
Notice: The ROBOTTESTINGFRAMEWORK_TEST_CHECK, ROBOTTESTINGFRAMEWORK_TEST_REPORT do NOT threw any exception and are used to add failure or report messages to the result collector. Instead, all the macros which include _ASSERT_ within their names (e.g., ROBOTTESTINGFRAMEWORK_ASSERT_FAIL) throw exceptions which prevent only the current test case (Not the whole test suite) of being proceed. The error/failure messages thrown by the exceptions are caught. (See Basic Assertion macros).
The report/assertion macros store the source line number where the check/report or assertion happen. To see them, you can run the test case or suite with --detail parameter using the robottestingframework-testrunner (See Running test case plug-ins using robottestingframework-testrunner).
Create a cmake file to build the plug-in:
# iCub Robot Unit Tests (Robot Testing Framework)
#
# Copyright (C) 2015-2019 Istituto Italiano di Tecnologia (IIT)
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 2.1 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
if(NOT DEFINED CMAKE_MINIMUM_REQUIRED_VERSION)
cmake_minimum_required(VERSION 3.5)
endif()
# add the required cmake packages
find_package(RobotTestingFramework 2 COMPONENTS DLL)
find_package(YARP 3.5.1 COMPONENTS os robottestingframework)
# add the source codes to build the plugin library
# add required libraries
target_link_libraries(${PROJECT_NAME} RobotTestingFramework::RTF
RobotTestingFramework::RTF_dll
YARP::YARP_os
YARP::YARP_init
YARP::YARP_robottestingframework)
# set the installation options
install(TARGETS ${PROJECT_NAME}
EXPORT ${PROJECT_NAME}
COMPONENT runtime
LIBRARY DESTINATION lib)
Call your cmake file from the icub-test/CMakeLists.txt to build it along with the other other test plugins. To do that, adds the following line to the icub-test/CMakeLists.txt
# Build example test
add_subdirectory(src/example-test)
Please check the icub-tests/example folder for a template for developing tests for the iCub.
Running a single test case
As it is documented here (Running test case plug-ins using robottestingframework-testrunner) you can run a single test case or run it with the other tests using a test suite. For example, to run a single test case:
robottestingframework-testrunner --verbose --test plugins/
ExampleTest.so --param
"--name MyExampleTest"Notice that this test require the yarpserver to be running and it contains tests that are programmed to succeed and some that are programmed to fail.
or to run the iCubSim camera test whith the test configuration file:
robottestingframework-testrunner --verbose --test plugins/
CameraTest.so --param
"--from camera_right.ini" --environment
"--robotname icubSim"Check if a camera is publishing images at desired framerate.
This runs the icubSim right-camera test with the parameters specified in the right_camera.ini which can be found in icub-tests/suites/contexts/icubSim folder. This test assumes you are running yarpserver and the iCub simulator (i.e. iCub_SIM).
Notice that the environment parameter --robotname icubSim is used to locate the correct context (for this examples is icubSim) and also to update the variables loaded from the right_camera.ini file.
Running multiple tests using a test suite
You can update one of the existing suite XML files to add your test case plug-in and its parameters or create a new test suite which keeps all the relevant test cases. For example the basic-icubSim.xml test suite keeps the basic tests for cameras and motors:
<?xml version="1.0" encoding="UTF-8"?>
<suite name="Basic Tests Suite">
<description>Testing robot's basic features</description>
<environment>--robotname icubSim</environment>
<fixture param="--fixture icubsim-fixture.xml"> yarpmanager </fixture>
<!-- Camera -->
<test type="dll" param="--from right_camera.ini"> CameraTest </test>
<test type="dll" param="--from left_camera.ini"> CameraTest </test>
<!-- Motors -->
<test type="dll" param="--from test_right_arm.ini"> MotorTest </test>
<test type="dll" param="--from test_left_arm.ini"> MotorTest </test>
</suite>
Then you can run all the test cases from the test suite:
robottestingframework-testrunner --verbose --suite icub-tests/suites/basics-icubSim.xml
The robottestingframework-testrunner, first, launches the iCub simulator and then runs all the tests one after each other. After running all the test cases, the tesrunner stop the simulator. If the iCub simulator crashes during the test run, the robottestingframework-testrunner re-launchs it and continues running the remaining tests.
How robottestingframework-testrunner knows that it should launch the iCub simulator before running the tests? Well, this is indicated by <fixture param="--fixture icubsim-fixture.xml"> yarpmanager </fixture>. The robottestingframework-testrunner uses the yarpmanager fixture plug-in to launch the modules which are listed in the icubsim-fixture.xml. Notice that all the fixture files should be located in the icub-tests/suites/fixtures folder.
