![]() Boost Test Library: Unit Test Framework The acceptance test makes the customer satisfied that the XP maxim Home The Test Case The Framework integration Introduction The Boost Test Library's Unit Test Framework provides
facilities to simplify writing test cases using Test Tools
and organizing them into test suites hierarchy. The framework relieves users from
messy a error detection, reporting duties and parameter processing. It provided
main() function that initialize the framework, setups parameters based on command
line arguments and/or environment variables, calls the user-supplied function init_unit_test_suite(argc,
argv) and than runs the provided test suite. The framework keeps track of all passed/failed
Test Tools assertions, provides an ability to check
the testing progress based on the amount of test cases run as part of the total
amount of test cases and generates the result report in several different formats.
The Unit Test Framework intended to be used both for a simple testing needs and
a complex non trivial testing. It is not intended to be used with the production
code, where the Program Execution Monitor could
be used. This was one of the design rationale to make the library implemented offline
vs. an inline implementation to allows us to speed up a compilation at an expense
of a runtime efficiency. The Unit Test Framework should be preferred over Test
Execution Monitor while working on creating new test programs. Getting StartedSee Getting Started to use Unit Test Framework The Framework componentsThe Unit Test Framework consists of several cooperating components. All components are located in the namespace boost::unit_test_framework. The Test Case component encapsulate the notion of a simple testing unit. The Test Suite component allows to unite several related testing units into one bundle that could be considered an a compound testing unit. To manage output generated during testing one could use the Test Log component. The Test Result component is responsible for testing results presentation. The Test CaseThe Unit Test Framework provides an ability for the user to create the Test Case based on a free function or a user's class method. There are four types of test cases: function_test_case, class_test_case, parametrized_function_test_case, parametrized_class_test_case. All of them implements test case interface defined in the abstract base class test_case. Test case interfaceDefinitiondefined in unit_test_suite.hpp Synopsisclass test_case { public: void set_timeout( int timeout ); void set_expected_failures( unit_test_counter exp_fail ); void run(); }; DescriptionAbstract class test_case define the test case interface. Use method test_case:set_timeout(...) to set the timeout value for the test case. See method execution_monitor::execute(...) for more details about the timeout value. Use method test_case::set_expected_failures(...) to set the expected amount of Test Tools failures in the test case. In most cases it's more convenient to set these parameters while adding this test_case to a test_suite. See the method test_suite::add(...) for more details. Use method test_case::run() to start the test case processing. ConstructionYou will never need to create instances of the class test_case. ![]() Free function based test casesDefinitiondefined in unit_test_suite.hpp Synopsisclass function_test_case : public test_case { public: function_test_case( void (*function_type)(), char const* name ); ... // Implementation }; DescriptionSimplest and most widely used form of test cases. Instances of the class function_test_case are created by the framework for supplied by the user pointer to free function with the following specification: void (*fct)(). Note void return type. To report a testing results one should use the Test Tools instead of returing result code. ConstructionTo create a test case based on your test function use the following macro: BOOST_TEST_CASE( &free_function ). BOOST_TEST_CASE creates a new instance of the class function_test_case and returns a pointer to the base class test_case. In most cases you will use it as an argument to the method test_suite::add(...). Examplesvoid test_feature1()
{
...
}
...
ts->add( BOOST_TEST_CASE( &test_feature1 ) );
_____________________
#include <boost/test/unit_test.hpp> using boost::unit_test_framework::test_suite; // most frequently you implement test cases as a free functions void free_test_function() { BOOST_CHECK( 2 == 1 ); } test_suite* init_unit_test_suite( int argc, char* argv[] ) { test_suite* test= BOOST_TEST_SUITE( "Example" ); test->add( BOOST_TEST_CASE( &free_test_function ) return test; } ![]() Class member function based test casesDefinitiondefined in unit_test_suite.hpp Synopsistemplate<class UserTestClass> class class_test_case : public test_case { public: class_test_case( void (UserTestCase::*function_type)(), char const* name, boost::shared_ptr<UserTestCase>& user_test_case ); ... // Implementation }; DescriptionInstances of the class class_test_case are created by the framework for the supplied member function and instance of the user's test class with the following specification: void (UserTestClass::*fct)(). To allow sharing on the same instance, so that one test case could use results of another, class_test_case is constructed based on boost::shared_ptr to the user test case class instance. Note, though, that we could achive similar effect with use of free function like this: void compount_test() {
UserTestCase instance;
instance.test1();
instance.test2();
...
};
The only drawback of this version that you could not separate results of one test
case from another. Another reason to use class member function based tast cases is
to test non default constructible user test case. In other words, if iser test case
need command line arguments or some other parameters to be constructed. As a general
rule it is recommended to use the class_test_case only if you can't implement a test
logic in a free function. Due to usage of templates in an implementation of the class_test_case,
a compilation can be longer would you use the function_test_case.
ConstructionTo create an instance of the class class_test_case use the following macro: BOOST_CLASS_TEST_CASE( function, shared_test_case_instance ). BOOST_CLASS_TEST_CASE creates a new instance of the class class_test_case and returns a pointer to the base class test_case. In most cases you will use it as an argument to the method test_suite::add(...). Examplesclass my_complex_test { public: void test_feature1() { ... } }; ... ts->add( BOOST_TEST_CASE( &my_complex_test::test_feature1 ) );_______________________________ class class_under_test { public: // i should be positive; throw an exception otherwise explicit class_under_test( int i ); ... // access methods int get_value() const; }; class compound_test { public: void test_construction() { BOOST_CHECK_THROW( new class_under_test( -1 ) ); v = new class_under_test( 1 ); BOOST_CHECK( v is valid ); ... } void test_access_methods() { BOOST_CHECK_EQUAL( v->get_value(), 1 ); ... } private: class_under_test* v; }; ... boost::shared_ptr<compound_test> instance( new compound_test ); ts->add( BOOST_TEST_CASE( &compound_test::constructor, instance ) ); ts->add( BOOST_TEST_CASE( &compound_test::test_access_methods, instance ) ); ![]() Parameterized free function test casesDefinitiondefined in unit_test_suite.hpp Synopsistemplate <typename ParamIterator, typename ParameterType> class parametrized_function_test_case : public test_case { ... // Implementation }; DescriptionInstances of the class parametrized_function_test_case are created by the framework for the supplied user's free function with the following specification: void (*fct)( ParameterType ). ConstructionTo create an instance of the class parametrized_function_test_case use the following macro: BOOST_PARAM_TEST_CASE( free_function, first_parameter, last_parameter ). Here first_parameter and last_parameter are begin and end iterators for the list of parameters accordingly. BOOST_PARAM_TEST_CASE creates a new instance of the class parametrized_function_test_case and returns a pointer to the abstract class test_case. In most cases you will use it as an argument to the method test_suite::add(...). Be aware that the parametrized_function_test_case does not store list of parameters internally. The user should make sure that parameters list will not be destroyed until the test case is run. That's why it not recommended to create a parameters list as local variable in init_unit_test_suite. A simple way to handle a parameters list lifetime is to place it into a user defined test suite class. Examplevoid test_file_reader( std::string const& file_name ) { ... } struct reader_testing : public boost::unit_test_framework::test_suite { void reader_testing() { files_to_test.push_back( "file 1" ); ... files_to_test.push_back( "file N" ); add( BOOST_TEST_CASE( &test_file_reader, files_to_test.begin(), files_to_test.end() ); } std::list<std::string> files_to_test; }; ... ts->add( new reader_testing ); ![]() Parameterized class member function based test casesDefinitiondefined in unit_test_suite.hpp Synopsistemplate<class UserTestClass, typename ParamIterator, typename ParameterType> class parametrized_class_test_case : public test_case { ... // Implementation }; DescriptionInstances of the class parametrized_class_test_case are created by the framework for the supplied method of the user's test class with the following specification: void (UserTestClass::*fct)( ParameterType ). The parametrized_class_test_case is responsible for the creation and the destroying of the user's test class instance. ConstructionTo create an instance of the class parameterized_class_test_case use the following macro: BOOST_PARAM_TEST_CASE( test_class_method, first_parameter, last_parameter ). Here first_parameter and last_parameter are begin and end iterators for the list of parameters accordingly. BOOST_TEST_CASE creates a new instance of the class parametrized_class_test_case and returns a pointer to the abstract class test_case. In most cases you will use it as an argument to the method test_suite::add(...). Be aware that the parametrized_class_test_case does not store list of parameters internally. The user should make sure that parameters list will not be destroyed until the test case is run. That's why it not recommended to create a parameters list as local variable in init_unit_test_suite. For example, you can place parameters list in file scope. Example:class my_complex_test { void test_assignment( double tolerance ) { ... } }; ... std::list<double> possible_tolerances; ts->add( BOOST_TEST_CASE( &my_complex_test::test_assignment, possible_tolerances.begin(), possible_tolerances.end() ) ); ![]() The Test SuiteDefinitiondefined in unit_test_suite.hpp Synopsisclass test_suite : public test_case { public: void add( test_case* tc, unit_test_counter expected_failures = 0, int timeout = 0 ); ... // Implementation }; DescriptionThe Unit Test Framework provides an ability for the user
to combine created test cases into a test suite and create a hierarchy of test
suites of any depth. To add a test_case to the test_suite use the method test_suite::add(...).
The first argument is a pointer to a new test_case, second - expected_failured
defines the amount of Test Tools assertions that
are expected to fail in the test case, and third - timeout defines the timeout
value for the test case. Note that the expected number of failures is calculated
automatically for the test suites, so you do not need to specify them while adding
test suites to a hierarchy. See the method execution_monitor::execute(...)
for more details about the timeout value. Last two arguments are optional and will
not be set if absent. In this case values defined in the test_case are used. ConstructionTo create instances of the class test_suite you can use the following macro: BOOST_TEST_SUITE( test_suite_name ). BOOST_TEST_SUITE creates an instance of the class test_suite and returns a pointer to it. test_suite is a test_case, that allows to generate a multilevel hierarchy. The Test resultDefinitiondefined in unit_test_result.hpp Synopsisclass unit_test_result { static unit_test_result& instance(); void confirmation_report( std::ostream& where_to ); void short_report( std::ostream& where_to ); void detailed_report( std::ostream& where_to ); int result_code(); }; DescriptionThe Unit Test Framework keeps testing results as the instances of the class unit_test_result. Class unit_test_result provides three methods for the result reporting. Use the method unit_test_result::confirmation_report(...) to report the success/failure confirmation massage only. Use the method unit_test_result::short_report(...) to report testing results for the current test_case. The report includes an amount of passed and failed tests and an amount of passed and failed Test Tools assertions. Use the method unit_test_result::detailed_report(...) to report testing results for the current test_case and all test cases it consist of. In most cases you will not need to use this interface directly. The framework provides a report for overall testing results automatically. ConstructionTo access instance of class unit_test_result holding results
for the current test case use static method unit_test_result::instance().
The Test logDefinitiondefined in unit_test_log.hpp Synopsisclass unit_test_log { static unit_test_log& instance(); void set_log_stream( std::ostream& str ); void set_log_threshold_level_by_name( char const* lev ); }; DescriptionTo manage the testing output stream the Unit Test Framework uses a singleton class unit_test_log. To change the test output stream use method unit_test_log::set_log_stream(...). The default stream is std::cout. To set a log level use the method unit_test_log::set_log_level_by_name(...). In most cases you will not need to use this interface directly. The framework provides a simple way to set the log level from out of the test program. For more details and for the list of acceptable values see Framework parameters. ConstructionTo access the only instance of the class unit_test_log
use method unit_test_log::instance(). The Test monitorTo monitor a test_case run the Unit Test Framework uses a class unit_test_monitor. The unit_test_monitor allows run test_case methods under control of a execution_monitor and extends the Execution Monitor functionality to translate execution_monitor exceptions into designated error codes. For more details about the class unit_test_monitor see the Boost Test Library Design. In most cases you will not need to utilize unit_test_monitor because the framework already monitor a test case run. The Framework integrationDescriptionAs mentioned above Unit Test Framework is responsible for supplying main function that initializing testing environment and taking care about results reporting. The main function also includes a hook for the function that should be supplied by the user. So, to integrate test program with the framework you should provide this function with the following specification: boost::unit_test_framework::test_suite* init_unit_test_suite ( int argc, char* argv[] ) This function should create and initialize top level instance of the class test_suite. The NULL pointer returned by the function is treated as a non-initialized test_suite. In other case the framework runs provided instance of the class test_suite and then destroy it the end of program, so it should be allocated dynamically. Command line arguments passed to the function guarantied not to have any framework-specific command line arguments. After test cases run the framework reports the results and returns the result code. Here the list of values returned by the testing programs integrated with the unit test framework:
Exampletest_suite* init_unit_test_suite( int argc, char* argv[] ) { test_suite* test= BOOST_TEST_SUITE( "Master test suite" ); test->add( BOOST_TEST_CASE( &my_test_function ) ); return test; } ![]() The Framework parametersThe Unit Test Framework provides two ways to configure test program run parameters externally: using command line arguments and setting environment variables. The following tables provides a compiled information about the framework parameters. Note that default parameter's value (i.e. value of the parameter if it is not defined either in the environment or among command-line arguments) for each parameter is bolded.
The Framework compilationThe Unit Test Framework is supplied as an offline library and should be compiled and linked with a test program. Following files, that are located in the Boost Test Library src directory, compose the framework: execution_monitor.cpp You also have a choice to include all files constituting the framework directly into your test module. Use <boost/test/included/unit_test_framework.hpp> for this porpose. Example and Test Programs
unit_test_example1 RationaleWhat is the first thing to do when you start working on new library/class/program? Exactly - we need the unit test module. Writing of the unit test module should be simple and obvious. On the other hand the framework should allow us to do a lot of non-trivial things. We want to be able to have a lot of small test cases. We want to be able to group them in test suites. At the beginning of the development we want to see as much descriptive error message as possible, while during regression testing we just want to know is there any errors at all. For small test programs a run time should prevail over a compilation time - who want to wait a 1 min to start the test that run a 1 sec?. For a long and complex test we want to be able to see the test progress. Design![]() The Boost Test Library Design document describes the relationship between Boost Test Library components. |