@#$%%^&*~IAO

Robot Testing Framework

What is it?

 

Robot Testing Framework is a generic and multi-platform unit testing framework for the test driven development (TDD) which has been initially designed for the robotic systems. Robot Testing Framework can deal with different levels of unit testing in the real or simulated environment. The robot testing framework also provides facilities for developing and running middleware and language independent tests; thus can be used for any TDD system.

Philosophy

 

Robot Testing Framework implementaion follows xUnit design pattern. It provides functionalities for developing and running unit tests in a language and middleware independent manner. The test cases can be developed as independent plug-ins (i.e., using scripts such as Lua, Python, Ruby or built as DLLs using C++, ADA) to be loaded and executed by an automated test runner. A fixture manager also prepares the setup and actively monitors that all the requirements for running the tests are satisfied during the execution of the tests.

Installation

 

Robot Testing Framework library does not depend on any external library. The Robot Testing Framework framework has a test runner utility to easily run the test cases which are built as plug-ins. Test cases can be organized in test suites using simple xml files to be executed by the test runner. The build system checks for the installation of TinyXml and, in case, it cannot find any installed version of that, it uses the internal version which is delivered with the Robot Testing Framework.

Licensing

 

Robot Testing Framework is an open-source software package which has been developed under GNU Lesser General Public License (LGPL).

Simple Examples

#include <stdio.h>
#include <TestCase.h>
#include <TestResult.h>
#include <ConsoleListener.h>
#include <TestAssert.h>

using namespace robottestingframework;

class MyTest : public TestCase {
public:
    MyTest() : TestCase("MyTest") {

    }

    virtual ~MyTest() { }

    virtual bool setup(int argc, char **argv) {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("running MyTest::setup...");
        return true;
    }

    virtual void tearDown() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("running MyTest::teardown...");
        ROBOTTESTINGFRAMEWORK_ASSERT_ERROR("this is just for example!");
    }

    virtual void run() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing integers");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(2<3, "is not smaller");
        int a = 5;
        int b = 3;
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(a<b, Asserter::format("%d is not smaller than %d.", a, b));
    }

};

int main(int argc, char** argv)
{
    // create a test listener to collect the result
    // and enbale the verbose mode
    ConsoleListener listener(true);

    // create a test result and add the listeners
    TestResult result;
    result.addListener(&listener);

    // calling a test case
    MyTest atest;
    atest.TestCase::run(result);

    return 0;
}

A simple test case 

#include <stdio.h>
#include <TestCase.h>
#include <TestResult.h>
#include <TestRunner.h>
#include <ConsoleListener.h>
#include <TestAssert.h>

using namespace robottestingframework;

class MyTest : public TestCase {
public:
    MyTest() : TestCase("MyTest") {

    }

    virtual ~MyTest() { }

    virtual bool setup(int argc, char** argv) {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("running MyTest::setup...");
        return true;
    }

    virtual void tearDown() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("running MyTest::teardown...");
        ROBOTTESTINGFRAMEWORK_ASSERT_ERROR("this is just for example!");
    }

    virtual void run() {

        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing integers");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(2<3, "is not smaller");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(5<3, "is not smaller");
    }

};

int main(int argc, char** argv)
{
    // create a test listener to collect the result
    ConsoleListener listener(false);

    // create a test result and add the listeners
    TestResult result;
    result.addListener(&listener);

    // create a test runner
    TestRunner runner;
    MyTest atest;
    runner.addTest(&atest);
    runner.run(result);

    return 0;
}

Using test runner 

#include <stdio.h>
#include <TestCase.h>
#include <TestResult.h>
#include <TestRunner.h>
#include <TestSuite.h>
#include <ConsoleListener.h>
#include <TestAssert.h>
#include <TestResultCollector.h>
#include <TextOutputter.h>

using namespace robottestingframework;

class MyTest1 : public TestCase {
public:
    MyTest1() : TestCase("MyTest1") { }

    virtual void run() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing smaller");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(3<5, "is not smaller");
    }
};

class MyTest2 : public TestCase {
public:
    MyTest2() : TestCase("MyTest2") { }

    virtual void run() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing equality");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(5==3, "are not equal");
    }
};


int main(int argc, char** argv)
{
    // create a test listener to print out the result
    ConsoleListener listener(false);

    // create a test result collector to collect the result
    TestResultCollector collector;

    // create a test result and add the listeners
    TestResult result;
    result.addListener(&listener);
    result.addListener(&collector);


    // create a test suite and the test cases
    TestSuite suite("MyTestSuite");
    MyTest1 test1;
    MyTest2 test2;
    suite.addTest(&test1);
    suite.addTest(&test2);

    // create a test runner
    TestRunner runner;    
    runner.addTest(&suite);
    runner.run(result);

    // print out some simple statistics
    printf("\n-------- results ---------\n");
    printf("Total number of tests : %d\n", collector.testCount());
    printf("Number of passed tests: %d\n", collector.passedCount());
    printf("Number of failed tests: %d\n", collector.failedCount());

    // store the results in a text file
    TextOutputter outputter(collector);
    outputter.write("./result.txt");
    return 0;
}

Using result collector 

#include <stdio.h>
#include <TestCase.h>
#include <TestResult.h>
#include <TestRunner.h>
#include <TestSuite.h>
#include <ConsoleListener.h>
#include <TestAssert.h>

using namespace robottestingframework;

class MyTest1 : public TestCase {
public:
    MyTest1() : TestCase("MyTest1") { }

    virtual void run() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing smaller");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(3<5, "is not smaller");
    }
};

class MyTest2 : public TestCase {
public:
    MyTest2() : TestCase("MyTest2") { }

    virtual void run() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing equality");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(5==3, "are not equal");
    }
};


int main(int argc, char** argv)
{
    // create a test listener to collect the result
    ConsoleListener listener(false);

    // create a test result and add the listeners
    TestResult result;
    result.addListener(&listener);

    // create a test suite and the test cases
    TestSuite suite("MyTestSuite");
    MyTest1 test1;
    MyTest2 test2;
    suite.addTest(&test1);
    suite.addTest(&test2);

    // create a test runner
    TestRunner runner;    
    runner.addTest(&suite);
    runner.run(result);

    return 0;
}

Using test suite 

#include <stdio.h>
#include <TestCase.h>
#include <TestResult.h>
#include <TestRunner.h>
#include <TestSuite.h>
#include <FixtureManager.h>
#include <ConsoleListener.h>
#include <TestAssert.h>

using namespace robottestingframework;

class MyTest1 : public TestCase {
public:
    MyTest1() : TestCase("MyTest1") { }

    virtual void run() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing smaller");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(3<5, "is not smaller");
    }
};

class MyTest2 : public TestCase {
public:
    MyTest2() : TestCase("MyTest2") { }

    virtual void run() {
        ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing equality");
        ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(3==3, "are not equal");
    }
};

class MyFixture : public FixtureManager {
public:
    MyFixture(FixtureEvents* dispatcher)
        : FixtureManager(dispatcher) { }

    bool setup(int argc, char** argv) {
        // setup and initialize the fixture        
        // ...
        printf("Myfixture setup!\n");        
        // return true if everything is fine.
        return true;
    }


    void tearDown() {
        // uninitialize the fixture
        // ...
        printf("Myfixture tear down!\n");
        // for an example if there is any error during tear down,
        // throw an exception.
        throw FixtureException(TestMessage("MyFixture cannot tear down!"));
    }

};

int main(int argc, char** argv)
{
    // create a test listener to collect the result
    ConsoleListener listener(false);

    // create a test result and add the listeners
    TestResult result;
    result.addListener(&listener);

    // create a test suite
    TestSuite suite("MyTestSuite");

    // create a fixture manager for the test suite
    MyFixture fixture(&suite);
    suite.setFixtureManager(&fixture);

    // creates test cases and add them to the suite
    MyTest1 test1;
    MyTest2 test2;
    suite.addTest(&test1);
    suite.addTest(&test2);

    // create a test runner and run the tests
    TestRunner runner;    
    runner.addTest(&suite);
    runner.run(result);

    return 0;
}

Using fixture manager

Developing test plug-in in C++

#ifndef _MYTEST_H_
#define _MYTEST_H_
#include <TestCase.h>

class MyTest : public TestCase {
public:

    MyTest();

    virtual ~MyTest();

    virtual bool setup(int argc, char** argv);

    virtual void tearDown();

    virtual void run();
};
#endif //_MYTEST_H_

MyTest.h 

#include <TestAssert.h>
#include <Plugin.h>
#include "MyTest.h"

using namespace robottestingframework;
ROBOTTESTINGFRAMEWORK_PREPARE_PLUGIN(MyTest)

MyTest::MyTest() : TestCase("MyTest") { }

MyTest::~MyTest() { }

bool MyTest::setup(int argc, char** argv) {
    ROBOTTESTINGFRAMEWORK_TEST_REPORT("running MyTest::setup...");
    return true;
}

void MyTest::tearDown() {
    ROBOTTESTINGFRAMEWORK_TEST_REPORT("running MyTest::teardown...");
    // assert an arbitray error for example.
    ROBOTTESTINGFRAMEWORK_ASSERT_ERROR("this is just for example!");
}

void MyTest::run() {
    ROBOTTESTINGFRAMEWORK_TEST_REPORT("testing integers");
    ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(2<3, "is not smaller");
    int a = 5;
    int b = 3;
    ROBOTTESTINGFRAMEWORK_TEST_FAIL_IF_FALSE(a<b, Asserter::format("%d is not smaller than %d.", a, b));
}

MyTest.cpp 

cmake_minimum_required(VERSION 2.8.9)

find_package(RobotTestingFramework)
find_package(RobotTestingFramework COMPONENTS DLL)

include_directories(${CMAKE_SOURCE_DIR}
                    ${RobotTestingFramework_INCLUDE_DIRS})

add_library(mytest MODULE MyTest.cpp MyTest.h)

CMakeList.txt 

$ testrunner --verbose --test libmytest.so 
Staring test runner.
Test case MyTest started...
[INFO]  (MyTest) reports: running MyTest::setup...
[INFO]  (MyTest) reports: testing integers
[FAIL]  (MyTest) checking (a<b): 5 is not smaller than 3.
[INFO]  (MyTest) reports: running MyTest::teardown...
[ERROR] (MyTest) asserts error with exception: this is just for example!
Test case MyTest failed!
Ending test runner.

-------- results ---------
Total number of tests : 1
Number of passed tests: 0
Number of failed tests: 1

Running the test

Developing test plug-ins in Lua, Python, Ruby, ...

--
-- TestCase table is used by the lua plugin loader
-- to invoke the corresponding methods:
--
-- TestCase.setup = function(options) ... return true end
-- TestCase.run = function() ... end 
-- TestCase.tearDown = function() ... end 
--
-- The following methods are for reporting, failures or assertions: 
--
-- robottestingframework.setName(name)             : sets the test name
-- robottestingframework.testReport(msg)           : reports a informative message
-- robottestingframework.testCheck(condition, msg) : reports a failure message
-- robottestingframework.assertError(msg)          : throws an error exception with message
-- robottestingframework.asserFail(msg)            : throws a failure exception with message
--

--
-- setup is called before the test run to setup 
-- user defined fixture
-- @return Boolean (true/false uppon success or failure)
--
TestCase.setup = function(parameter)
    robottestingframework.setName("MyTest")
    robottestingframework.testReport("Preparing setup...")
    return true;
end

--
-- The implementation of the test goes here
-- @return Boolean
--
TestCase.run = function()
    robottestingframework.testReport("Checking bigger...")
    robottestingframework.testCheck(5>3, "5 is not bigger than 3.")
    robottestingframework.testReport("Checking smaller...")
    robottestingframework.testCheck(5<3, "5 is not smaller than 3.")
end


--
-- tearDown is called after the test run to tear down
-- user defined fixture
--
TestCase.tearDown = function()
    robottestingframework.testReport("Tearing down...")
end

mytest.lua 

'''
 robottestingframework module is automatically imported by the python plugin loader
 to invoke the corresponding test case methods. To develop a new 
 test case simply implement the following class; (setup and tearDown 
 methods are optional) :

 class TestCase:
     def setup(self, param):
         return True

     def run(self):

     def tearDown(self):


 The following methods are for reporting, failure or assertions: 

 robottestingframework.setName(name)             : sets the test name (defualt is the test filename)
 robottestingframework.testReport(msg)           : reports a informative message
 robottestingframework.testCheck(condition, msg) : reports a failure message
 robottestingframework.assertError(msg)          : throws an error exception with message
 robottestingframework.asserFail(msg)            : throws a failure exception with message
'''

class TestCase:
    # setup is called before the test's run to setup 
    # the user defined fixture
    # @return Boolean (True/False uppon success or failure)
    def setup(self, param):
        robottestingframework.testReport("Preparing setup...")
        return True
   
    # The implementation of the test goes here
    def run(self):
        robottestingframework.testReport("Checking bigger...")
        robottestingframework.testCheck(5>3, "5 is not bigger than 3.")
        robottestingframework.testReport("Checking smaller...")
        robottestingframework.testCheck(5<3, "5 is not smaller than 3.")

    # tearDown is called after the test's run to tear down
    # the user defined fixture
    def tearDown(self):
        robottestingframework.testReport("Tearing down...")

mytest.py 

# RobotTestingFramework module is automatically imported by the ruby plugin loader
# to invoke the corresponding test case methods. To develop a new 
# test case simply implement the following class; (setup and tearDown 
# methods are optional) :
#
# class TestCase
#     def setup(param)
#         ...
#         return true
#     end
#
#     def run ... end
#
#     def tearDown ... end
# end
#
# The following methods are for reporting, failure or assertions: 
#
# RobotTestingFramework::setName(name)             : sets the test name (defualt is the test filename)
# RobotTestingFramework::testReport(msg)           : reports a informative message
# RobotTestingFramework::testCheck(condition, msg) : reports a failure message
# RobotTestingFramework::assertError(msg)          : throws an error exception with message
# RobotTestingFramework::asserFail(msg)            : throws a failure exception with message
#

class TestCase
    # setup is called before the test's run to setup 
    # the user defined fixture
    # @return Boolean (True/False uppon success or failure)
    def setup(param)
        RobotTestingFramework::testReport("Preparing setup...")
        return true
    end

    # The implementation of the test goes here
    def run
        RobotTestingFramework::testReport("Checking bigger...")
        RobotTestingFramework::testCheck(5>3, "5 is not bigger than 3.")
        RobotTestingFramework::testReport("Checking smaller...")
        RobotTestingFramework::testCheck(5<3, "5 is not smaller than 3.")
    end

    # tearDown is called after the test's run to tear down
    # the user defined fixture
    def tearDown
        RobotTestingFramework::testReport("Tearing down...")
    end
end

mytest.rb 

$ testrunner --verbose --test mytest.lua
Staring test runner.
Test case  started...
[INFO]  (MyTest) reports: Preparing setup...
[INFO]  (MyTest) reports: Checking bigger...
[INFO]  (MyTest) reports: Checking smaller...
[FAIL]  (MyTest) checking (false): 5 is not smaller than 3.
[INFO]  (MyTest) reports: Tearing down...
Test case MyTest failed!
Ending test runner.

-------- results ---------
Total number of tests : 1
Number of passed tests: 0
Number of failed tests: 1

Running the test

Developing test plug-ins in ADA

With robottestingframework.TestCase;   use robottestingframework.TestCase;

package MyTest is

    type MyTest is new TestCase with null record;

    overriding function Setup (Self : in out MyTest;
                               Parameters : String) return Boolean;
    overriding procedure TearDown (Self : in out MyTest);
    overriding procedure Run(Self : in out MyTest);

    procedure Create;
    pragma Export (C, Create, robottestingframework.Test_Create_Symbol);

end MyTest;

MyTest.ads 

with robottestingframework; use robottestingframework;
With robottestingframework.Asserter;
With MyTest; use MyTest;

package body MyTest is
    
    procedure Create is 
    begin
        SetTest(new MyTest, "MyTest");
    end;

    function Setup (Self : in out MyTest;
                    Parameters : String) return Boolean is
        pragma Unreferenced (Self);
        pragma Unreferenced (Parameters);
    begin
        Asserter.TestReport("Preparing setup...");
        return True;
    end Setup;

    procedure TearDown (Self : in out MyTest) is
         pragma Unreferenced (Self);  
    begin
        Asserter.TestReport("Tearing down...");
    end;

    procedure Run (Self : in out MyTest) is
         pragma Unreferenced (Self);  
    begin
        Asserter.TestReport("Checking bigger...");
        Asserter.TestCheck(5>3, "5 is not bigger than 3.");
        Asserter.TestReport("Checking smaller...");
        Asserter.TestCheck(5<3, "5 is not smaller than 3.");
    end;

end MyTest;

MyTest.adb