let’s dev GmbH & Co. KG - The brand for groundbreaking custom software

Blog

Top!

let’s dev | Scroll to top
let’s dev | Scroll to next content item

Hello

let’s dev | Scroll to previous content item
let’s dev Blog | Automated testing of C++ code with frameworks - part 1
by Arne
20. Februar 2019

Automated testing of C++ code with frameworks - part 1

Testing applications is also an important part of the development process in the mobile environment. For libraries written in C++, this is made possible by gtest and gmock. We present the use of these with simple examples.

Development tests

Automated tests have become an indispensable part of modern development processes. They not only help to ensure the quality of the software, but also enable the developer to detect and eliminate errors and problems at an early stage.

Different types of tests are to be distinguished here. As examples unit tests for testing a component and integration tests, which examine the interaction of several components, are mentioned. For most programming languages there are in addition frameworks, which facilitate the production and execution significantly, so also for C++.

Frameworks gtest and gmock

The Google frameworks gtest and gmock were initially developed in separate projects, but have since been merged. gtest is a testing framework for C++, which is intended to facilitate the testing and assessing of applications. Included functionalities are e.g. the organization of tests in test cases and the creation of fixtures for reuse. With gmock, on the other hand, mock objects can be created, e.g. to test whether a function can be called with the expected parameters. In the first part of this article, we will first take a closer look at the gtest framework.

gtest and gmock compared with other C++ unit testing frameworks

The large number of C++ unit testing frameworks on the market makes the right choice considerably more difficult. It therefore makes sense to compare the frameworks with each other in advance from various points of view. Already in 2004 some of these frameworks like CppUnit or Boost.Test were compared. Even though gtest did not exist at that time, it is still interesting to take a look at the metrics used:

These questions will now be answered in the course of this article. It should be said in advance that gtest fulfills all the requirements mentioned. Nevertheless, the choice of framework always depends on the project in which it is to be used and must be determined individually.

gtest

This post is about gtest, but all the general topics are valid for gmock as well, since the functionality has been merged into one library.

A unique nomenclature must be defined for the description. gtest uses the designation TEST() or Test for individual tests, the grouping of tests for a component is called Test Case. A term that will appear more frequently in the further course is that of the Test Fixture. This is a class that contains functions and data that are reused in multiple tests.

Setup

The framework works on a variety of platforms as well as with different compilers. Supported systems include Windows, Mac OS X, and Linux, among others. It is possible to build and link gtest as a static and dynamic library. The use of cmake simplifies integration. Thus, gtest can be built as a stand-alone or integrated into other projects.

add_subdirectory (${ CMAKE_CURRENT_BINARY_DIR }/ googletest -src ${
CMAKE_CURRENT_BINARY_DIR }/ googletest - build EXCLUDE_FROM_ALL )

Before the actual implementation, as shown above, the download from GitHub must take place. Here you have the option of integrating the code directly or always loading it automatically as a dependency.

General structure

In principle, it makes sense to set up tests similar to the project structure. This makes it easier for other developers to find tests for a particular class. It is also quicker to find out whether tests exist for a class. Tests that require the same resources should be written in one file. This makes it very easy to use the fixtures and prepare the resources only in one place.

Test examples

A simple test is written in a macro:

TEST ( TestCaseName , TestName ) {
                ... test body ...
}

The TestCaseName as well as the TestName are freely selectable. The names of the tests may also be repeated in different TestCases, only within a TestCase the TestName must be unique.

Fixture tests are used to access the same resources in different test cases. This can be a dummy object, which is filled with test data, but it can also be a complex initialization. To be able to use this functionality, a class must be created beforehand, which then serves as a fixture.

(1) Test Fixture MyFixture.h
# include <gtest / gtest .h>
class MyFixture : public testing :: Test {
protected :
void SetUp () override ;
bool myFunction ();
};
(2) Test Fixture MyFixture.cpp
void BaseTest :: SetUp () {
Test :: SetUp ();
... my additional setup ...
}
TEST_F ( MyFixture , TestName ) {
bool myBool = myFunction ();
... my aditional testing ...
}

In the header the function "myFunction" has the visibility "protected". All functions and properties called in the test macro must be visible "protected" or "public". The SetUp function can be used to perform complex initializations and is executed before each test. Analogously, a "TearDown" can also be implemented. The macro in (2) has changed slightly compared to (1). TEST has become TEST_F. The F stands for Fixture. The name of the fixture can be freely chosen, it only has to be identical in the fixture class and in the macro.

Reviews (Assertions)

Assertions are one of the most important functions of any test framework. gtest provides a number of different checks that can be used to compare values or objects.

Assertions are available as fatal or non-fatal checks. A fatal check terminates the currently running test if the comparison is judged to be false. A non-fatal check allows the test to finish, but subsequently marks it as failed.

Fatal comparisons start with "ASSERT_", non-fatal ones with "EXPECT_". If possible, non-fatal checks should always be made.

The following examples show the use of some checks. However, it is only a small section, of the function provided by Google.

(1) Use of non-fatal checks
EXPECT_TRUE ( MyClass :: myBoolFunc ());
std :: string expectedResult (" this result ");
EXPECT_STREQ ( expectedResult . c_str () , MyClass :: myStringFunc (). c_str ());
EXPECT_EQ (42 , MyClass :: myIntFunc ());
EXPECT_GT (0, MyClass :: myLongFunc ());

The various macros in (1) test multiple conditions. The macro with _STREQ checks two strings for equality, via _EQ the equality of numbers is matched and _GT checks whether the first number is greater than the second.

(2) Use of fatal and non-fatal checks
User * user = MyClass :: getMyObject ();
ASSERT_NE ( nullptr , user );
EXPECT_EQ (25 , user ->age);
EXPECT_EQ (female , user -> gender );

In (2) the existence of the object is checked with ASSERT_NE. Of course, it does not make sense to let the test continue if the object does not exist. The check of the age, on the other hand, is done via EXPECT_EQ, since other properties can also be checked here, even if the age is not correct.

In the following, both variants will not always be discussed. Whenever Assert or Expect is mentioned, both variants are meant, if not explicitly mentioned.

It is possible to give an individual output to a check. This is also output in the log in case of a failure and makes it easier to identify the cause. For this purpose, the message is appended to the assertion using the stream operator «.

TEST_F ( MyTestFixture , checkitemCount ) {
int itemCount = _sut -> getItemCount ();
EXPECT_EQ (0, itemCount ) << " Item count was not the expected size .";
}

Death Tests

Death tests can be used to check whether the program crashes on a particular execution. But why would one want to test this? In C++ it is common to secure preconditions by checks with abort(). Especially within the st1 or other libraries this kind of backup is used.

To use Death Tests, the death or crash of the program must be expected within the test:

EXPECT_DEATH (_sut -> myDeadlyFunction () , ".* this did happen .*")

When using the macro, two parameters are used. The first parameter must define the call that will cause the program to crash. The second argument is a regex that describes the expected error message. Thus the test fails under two conditions: First, if the program does not crash when myDeadlyFunction is called, and second, if the error message does not satisfy the passed regex.

Why is the error message crucial? If the software did not terminate with the expected message, the logic that was supposed to be tested was not addressed. A condition was therefore not fulfilled or a system error occurred before or after. Thus it is unclear whether the condition to be tested is fulfilled. The regex makes it possible to terminate the program exactly at the expected point.

Exception Tests

Basically, this is a group of asserts and not a test type of its own. However, since exceptions are also becoming more and more important for C++, it should be noted that a check with gtest is also possible for these. The assertions also exist as fatal and non-fatal variants.

(1) Function with Exception
EXPECT_THROW (_sut -> myThrowingFunc () , std :: exception );
ASSERT_THROW (_sut -> myThrowingFunc () , std :: exception );
(2) Function without Exception
EXPECT_NO_THROW (_sut -> myNotThrowingFunc ());
ASSERT_NO_THROW (_sut -> myNotThrowingFunc ());

Floating point comparisons

Floating Point Due to internal representations and rounding problems with floating point numbers, a comparison of these is usually difficult, because two floating point numbers will almost never have the exact same value. Therefore the comparison with ASSERT_EQ will fail. To solve this problem there is a specific macro for comparing floating point numbers. These match on four units in the last place (ULP). There are specific versions for float and double available:

ASSERT_FLOAT_EQ (0.0f, _sut -> myFloat ());
ASSERT_DOUBLE_EQ (0.0 , _sut -> myDouble ());
EXPECT_FLOAT_EQ (0.0f, _sut -> myFloat ());
EXPECT_DOUBLE_EQ (0.0 , _sut -> myDouble ());

Assertion with Matchers

gmock brings own matchers to gtest. Since it is possible to write your own matchers, individual comparisons can also be created. So complex comparisons can be outsourced to matchers and provided with proper error messages. Multiple use of matchers is also conceivable.

For example, if you want to compare the content of a vector of pointers, there is no standard comparison for this. So the user has to iterate the vector manually and compare the content. If one has several tests that compare these vectors, a lot of duplicate code quickly results. If the code is outsourced to functions, you get cluttered tests. Matchers can be used to create a comparison operation for the vectors that works with gtest. A simple example for this looks as follows:

MATCHER_P ( vectorContentEq , expectedVector , " Expected vector content does
not match .") {
bool result = (arg. size () == expectedVector . size ());
for ( int i = 0; i < std :: min(arg . size () , expectedVector . size ()); ++i) {
auto firstItem = arg.at(i);
auto secondItem = expectedVector .at(i);
if( firstItem != secondItem ){
result = false ;
break ;
}
}
return result ;
}

MATCHER_P defines a new matcher named vectorContentEq. The P stands for a matcher that expects a paramater. In this case the parameter is expectedVector. The length of the vectors and the content of the elements are compared.

There are also more matchers than assertion macros. So there are situations where testing with the matchers provided by Google makes more sense than with macros. For example, some matchers can compare parts of strings:

EXPECT_THAT ( myString , StartsWith ("My string Starts with "));

To use matchers, the EXPECT_THAT macro is used and the matcher is passed in the second argument. However, matchers are not a unique implementation of gmock. A similar concept can be found in newer jUnit versions. There, too, the expression matcher is used.

Parameterized tests

Testing functions with input parameters is one of the everyday tasks of a developer. If the function should behave the same with different parameters, the same test must be written several times. With gtest you pass parameters to a test and then specify with which values in the parameter the tests are to be executed.

Definition of a parameterized TestCase
TEST_P ( MyParamTest , TestName ) {
std :: string myParam = GetParam ();
... normal testing with usage of myParam ...
}

There are some small differences to be seen. The macro ends on P for parameterised test. In the implementation the call GetParam() was used to access the parameter. For readability reasons the parameter is assigned to a local variable. However, this procedure is not mandatory necessary. If the test class is now executed, no result is to be expected. The reason for this is that it must first be defined with which values the tests are to be executed. Several tests can be executed with the same parameters without any problems.

Definition of the parameterized values
INSTANTIATE_TEST_CASE_P ( myParamTestInstance , MyParamTest , :: testing ::
Values (" value1 ", " value2 ", " value3 "));

Now all tests in the test case "MyParamTest" are executed three times, once with each parameter.

Assertion placement

In contrast to other frameworks, assertions in gtest can be written in almost any function. Only two conditions must be met. First, the include command of gtest must be included, and second, fatal assertions must be handled separately. This applies to all assertions beginning with "ASSERT_" as well as the direct macro "Fail", which can only be used in functions with the return type "void".

Assertions in functions

It is good practice to outsource repetitive blocks of assertions to auxiliary classes. An example of this is a user object with multiple properties. If you want to check this, a whole series of assertions are available. It is obvious to move this block to an auxiliary class or auxiliary function. However the problem arises that the possible output of a failed assertion is no longer very meaningful. Because it is no longer at first sight from which test the assertion failed.

gtest has introduced the macro "SCOPED_TRACE" for this case, which appends a message with file and line number to the log in the current context. If the current scope is left, the message is also removed. For the example from just now this looks as follows:

TEST_F ( MyTestFixture , testUserWithSomething ) {
if( something ()){
SCOPED_TRACE (" Something ");
User * user = sut -> getUser ();
testUser ( user );
}
User * user = sut -> getUser ();
testUser ( user );
}

Without the "SCOPED_TRACE" macros it would be very difficult to find out in case of assertion errors in the "testUser" function which of the two cases has led to the failure. With the macro, however, a log of this type would be displayed in case of a failure in the "if":

path / MyTestFixture .cpp :198: Something

However, if the test fails only on the second call, the message is not present in the log.

Change test output

The standard output of gtest looks like this:

... a lot mor tests and logs ...
[ OK ] MyTestCase . myTest (4 ms)
[----------] 4 tests from DeviceOnboardingManagerTest (18 ms total )
[----------] Global test environment tear - down
[==========] 452 tests from 40 test cases ran. (16823 ms total )
[ PASSED ] 452 tests .

However, the fact that the gtest outputs can be mixed with the actual outputs on "std::count" is problematic here. However, it is possible to modify these outputs. The following adjustments can be made:

It should also be mentioned here that gtest is supported by integrated development environments. For example, it is possible to execute only individual tests or test cases. In the CLion development environment, this looks like this:

Figure 1: gtest in CLion

Figure 1: gtest in CLion

Restrictions

gtest is considered a threadsafe on systems that provide "pthreads". The attribute provides information about whether it is safe to call program code from different threads. However, on other systems, such as Windows, it is not currently safe to execute assertions from two threads. However, this is not important for most applications.


In the second part of the blog series, we introduce the mock framework gmock and give our take on automated testing with the two featured frameworks. Stay tuned!

More articles from our blog

let’s dev Blog |

Technical

by Julian

2021-10-14

Read more
let’s dev Blog | Apple Developer Program: What is it used for and what content does it offer me as a member?

Corporate

Apple Developer Program: What is it used for and what content does it offer me as a member?

by Julian

2021-09-30

Read more
let’s dev Blog | Sketch, Figma & Co. - We take a look at the most popular UI and Prototyping Tools in 2021

Corporate

Sketch, Figma & Co. - We take a look at the most popular UI and Prototyping Tools in 2021

by Ellen

2021-07-15

Read more
let’s dev Blog | Tailwind: An innovative project for the future use of old wind turbines

Corporate

Tailwind: An innovative project for the future use of old wind turbines

by Karl

2021-06-24

Read more
let’s dev Blog |

Corporate

by Julian

2021-06-10

Read more
let’s dev Blog | Smart Prognosis of Energy with Allocation of Resources

Corporate

Smart Prognosis of Energy with Allocation of Resources

by Karl

2021-02-18

Read more
let’s dev Blog | Dasoman - Data-Sovereignty-Manager

Corporate

Dasoman - Data-Sovereignty-Manager

by Karl

2021-01-11

Read more
let’s dev Blog | We look back on the past months - And wish all the best for the coming year 2021!

Corporate

We look back on the past months - And wish all the best for the coming year 2021!

by Julian

2020-12-17

Read more
let’s dev Blog | iOS User Interface Tests

Technical

iOS User Interface Tests

by Nicolas

2020-11-12

Read more
let’s dev Blog | Adobe Max - Online for the first time

Corporate

Adobe Max - Online for the first time

by Julia

2020-10-29

Read more
let’s dev Blog | CAN2BLE

Technical

CAN2BLE

by Raphael

2020-09-24

Read more
let’s dev Blog | Mensch und Computer 2020 - Digital Change in the Flow of Time

Corporate

Mensch und Computer 2020 - Digital Change in the Flow of Time

by UX Team

2020-09-18

Read more
let’s dev Blog | Neumorphism – A new era of user interface design?

Technical

Neumorphism – A new era of user interface design?

by Julian

2020-08-13

Read more
let’s dev Blog | UX Research Part 3 - UX Methods

Technical

UX Research Part 3 - UX Methods

by Elena

2020-05-28

Read more
let’s dev Blog | UX Research Part 2 - What is UCD and what does User Research have to do with it?

Technical

UX Research Part 2 - What is UCD and what does User Research have to do with it?

by Elena

2020-04-23

Read more
let’s dev Blog | go-digital promotes establishment of home office workstations

Corporate

go-digital promotes establishment of home office workstations

by Karl

2020-03-19

Read more
let’s dev Blog | Google Passes - Card Management on Android Devices

Technical

Google Passes - Card Management on Android Devices

by Michelle

2020-03-12

Read more
let’s dev Blog | 100% code coverage in software testing - a reasonable goal?

Technical

100% code coverage in software testing - a reasonable goal?

by Raphael

2020-03-06

Read more
let’s dev Blog | Swift UI - Simple and fast implementation of user interfaces

Technical

Swift UI - Simple and fast implementation of user interfaces

by Tobias

2020-03-02

Read more
let’s dev Blog | In dialog with the business juniors - Exciting insights into business start-ups and digital transformation

Corporate

In dialog with the business juniors - Exciting insights into business start-ups and digital transformation

by Julian

2020-02-27

Read more
let’s dev Blog | Simplified testing of iOS push notifications in the simulator with Xcode 11.4

Technical

Simplified testing of iOS push notifications in the simulator with Xcode 11.4

by Manuel

2020-02-26

Read more
let’s dev Blog | National meeting of the consortium of the SPEAR research project at let's dev in Karlsruhe

Corporate

National meeting of the consortium of the SPEAR research project at let's dev in Karlsruhe

by Karl

2020-01-27

Read more
let’s dev Blog | UX Research Part 1 - Why User Research is so important

Technical

UX Research Part 1 - Why User Research is so important

by Elena

2020-01-23

Read more
let’s dev Blog | Dark Mode

Technical

Dark Mode

by Elisa

2020-01-09

Read more
let’s dev Blog | We wish you a Merry Christmas - And a Happy New Year!

Corporate

We wish you a Merry Christmas - And a Happy New Year!

by Julian

2019-12-20

Read more
let’s dev Blog | Exchange on the topic of digitization with the Business Club Luxembourg at the Embassy of Luxembourg in Berlin

Corporate

Exchange on the topic of digitization with the Business Club Luxembourg at the Embassy of Luxembourg in Berlin

by Karl

2019-12-17

Read more
let’s dev Blog | DaSoMan at the Internet+ Expo in Foshan (China)

Corporate

DaSoMan at the Internet+ Expo in Foshan (China)

by Karl

2019-12-13

Read more
let’s dev Blog | Google Play Console: Pre-Launch Reports

Technical

Google Play Console: Pre-Launch Reports

by Fabian

2019-12-11

Read more
let’s dev Blog | DevFest 2019 in Hamburg

Technical

DevFest 2019 in Hamburg

by Julian

2019-12-05

Read more
let’s dev Blog | Vernissage digital art in the media theater of the Humboldt University Berlin

Corporate

Vernissage digital art in the media theater of the Humboldt University Berlin

by Karl

2019-11-21

Read more
let’s dev Blog | World Usability Day 2019 in Karlsruhe - let's dev supports as main sponsor

Corporate

World Usability Day 2019 in Karlsruhe - let's dev supports as main sponsor

by Aileen

2019-11-11

Read more
let’s dev Blog | Gutted - Open Day at the Alter Schlachthof Karlsruhe 2019

Corporate

Gutted - Open Day at the Alter Schlachthof Karlsruhe 2019

by Julian

2019-09-26

Read more
let’s dev Blog | Mensch und Computer 2019 - Conference on User Experience and Usability in Hamburg

Corporate

Mensch und Computer 2019 - Conference on User Experience and Usability in Hamburg

by Elena

2019-09-17

Read more
let’s dev Blog | Business and Enterprise App Distribution on iOS

Technical

Business and Enterprise App Distribution on iOS

by Aileen

2019-08-05

Read more
let’s dev Blog | Digital Transformation - Chances and Challenges in the Automotive Industry, Agriculture and New Technologies

Corporate

Digital Transformation - Chances and Challenges in the Automotive Industry, Agriculture and New Technologies

by Karl

2019-07-17

Read more
let’s dev Blog | let's dev supports runners at the 7th KIT Championship

Corporate

let's dev supports runners at the 7th KIT Championship

by Karl

2019-07-05

Read more
let’s dev Blog | Automated testing of C++ code with Google Test and Google Mock - Part 2

Technical

Automated testing of C++ code with Google Test and Google Mock - Part 2

by Arne

2019-06-13

Read more
let’s dev Blog | Apple WWDC 2019: These are the highlights of the keynote

Technical

Apple WWDC 2019: These are the highlights of the keynote

by Nicolas

2019-06-05

Read more
let’s dev Blog | App Builders 2019

Technical

App Builders 2019

by Nicolas

2019-05-23

Read more
let’s dev Blog | Official opening of the Consolidation and Expansion Center (FUX)

Corporate

Official opening of the Consolidation and Expansion Center (FUX)

by Helena

2019-04-15

Read more
let’s dev Blog | Delegation from Nottingham to visit the Alter Schlachthof in Karlsruhe

Corporate

Delegation from Nottingham to visit the Alter Schlachthof in Karlsruhe

by Helena

2019-04-14

Read more
let’s dev Blog | The time has come: We are moving!

Corporate

The time has come: We are moving!

by Helena

2019-03-26

Read more
let’s dev Blog | The app in the Google Play Store

Technical

The app in the Google Play Store

by Elisa

2019-01-24

Read more
let’s dev Blog | „UX Day“ 2018

Corporate

„UX Day“ 2018

by Aileen

2018-12-17

Read more
let’s dev Blog | let's dev supports SG Siemens volleyball players from Karlsruhe

Corporate

let's dev supports SG Siemens volleyball players from Karlsruhe

by Helena

2018-12-04

Read more
let’s dev Blog | SMEs shape digitalization - SME Conference 2018

Corporate

SMEs shape digitalization - SME Conference 2018

by Helena

2018-11-12

Read more
let’s dev Blog | Apple Wallet

Technical

Apple Wallet

by Maik

2018-10-26

Read more
let’s dev Blog | „Mensch und Computer“ 2018

Corporate

„Mensch und Computer“ 2018

by Judith

2018-09-24

Read more
let’s dev Blog | State Design Pattern in Android

Technical

State Design Pattern in Android

by Thomas

2018-09-17

Read more
let’s dev Blog | let's dev is an authorized consulting company in the „go-digital“ funding program

Corporate

let's dev is an authorized consulting company in the „go-digital“ funding program

by Helena

2018-09-01

Read more
let’s dev Blog | App Design & Development Conference 2018

Corporate

App Design & Development Conference 2018

by Helena

2018-08-14

Read more
let’s dev Blog | iOS 12: The top new features at a glance

Technical

iOS 12: The top new features at a glance

by Nicolas

2018-07-17

Read more
let’s dev Blog | let's dev at CEBIT

Corporate

let's dev at CEBIT

by Karl

2018-06-11

Read more
let’s dev Blog | Introduction to User Interface (UI) Testing with Espresso

Technical

Introduction to User Interface (UI) Testing with Espresso

by Raphael

2018-06-07

Read more
let’s dev Blog | The app in the Apple App Store: what information is needed?

Technical

The app in the Apple App Store: what information is needed?

by Aileen

2018-04-27

Read more
let’s dev Blog | Smart Pointer in C++

Technical

Smart Pointer in C++

by Matthias

2018-04-01

Read more
let’s dev Blog | User interface design for iPhone X: all innovations at a glance

Technical

User interface design for iPhone X: all innovations at a glance

by Helena

2018-02-07

Read more
let’s dev Blog | WebVR - Virtual Reality Experience in the Browser with the A-Frame Framework

Technical

WebVR - Virtual Reality Experience in the Browser with the A-Frame Framework

by Judith

2018-01-10

Read more
let’s dev Blog | Deutsche Bahn Open Data Hackathon

Corporate

Deutsche Bahn Open Data Hackathon

by Karl

2015-03-31

Read more
let’s dev Blog | Blur effects under iOS 7

Technical

Blur effects under iOS 7

by Katja

2014-04-24

Read more
let’s dev Blog | Beyond App Store - iOS application distribution

Technical

Beyond App Store - iOS application distribution

by Karl

2012-08-27

Read more
let’s dev Blog | Front-end architecture - Model View Presenter and Message Bus

Technical

Front-end architecture - Model View Presenter and Message Bus

by Karl

2011-03-08

Read more