2

I've read

C++ unit testing framework

which is closed as too broad; and answer-wise, while it lists several testing frameworks it doesn't really compare them or discuss their features.

Now, I'm interested in adding a framework for unit tests for an existing project of mine. It actually already includes a binary which performs a certain kind of unit tests, or perhaps I should something between unit and subsystem tests - but its' not a unit testing framework in itself and not intended to test all of the code. Which is why I want a proper framework. It may (or may not) also allow me to drop some of the custom code I have right now.

A significant requirement that I have is that the framework be Modern-C++-oriented. That is, that the syntax for using it will be C++11'ish (could be C++14 or C++17, but the latter option might give me some trouble due to CUDA compatibility issues), rather than just supporting the testing of C++11 code as an afterthought. So when I read, for example, that Google Test only needs a C++98 compiler, I get worried.

Other key features (partly adapted from this article):

  • Minimal amount of work needed to add new tests.
  • Easy to modify and port - but not for the reasons mentioned here! That is, it can and should depend on advanced C++ language and standard library features, as long as they're standard; and I don't mind dependencies on something like, say, CMake, or some scripting language etc, as long as it's tolerant to the difference between OSes and OS distributions.
  • Supports setup/teardown steps (fixtures).
  • Flexible and robust w.r.t. exceptions, crashes and asserts.
  • Nice pretty-printed/ASCII-graphics-enhanced console output. I don't care for any specific such features, e.g. colors vs monochrome or whether or not progress and succeses/failures are animated, but it should be pleasingly readable.
  • Integration with IDEs is a big plus

and of course:

  • Free and Open Source license
  • Gratis
  • Support for different output formats, for use by various post-mortem tools

I would really like a comparison of strengths and weaknesses w.r.t. the features I listed as well as a comparison on a single framework.

1

If you have Boost as a dependency, then Boost.Test is a no-brainer choice. Consistent, feature-complete, as minimalistic or as rich as you need it to be.

  • Can you explain how it is oriented towards modern C++? I was under the impression it's a bit old. – einpoklum - reinstate Monica Jan 24 '18 at 22:31
  • @einpoklum - correct word is "mature", not "old". Why don't you point which aspect it is outdated in and we discuss? – bobah Jan 26 '18 at 12:21
  • I meant "old" as in "implemented in pre-2011 C++". – einpoklum - reinstate Monica Jan 26 '18 at 12:23
  • been testing C++1xyz code since time it was called C++0x (late 2010), never felt Boost.Test blocking me, you do want to build some project-specific test rigs library around it but that's about it - it does not force you to degrade the syntax ( or use boost::mpl and boost::fusion unnecessarily :-) ) – bobah Jan 26 '18 at 12:28
0

Mettle

A C++14-and-up framework!

Example:

suite<> basic("a basic suite", [](auto &_) {
  _.test("a test", []() {
    expect(true, equal_to(true));
  });

  for(int i = 0; i < 4; i++) {
    _.test("test number " + std::to_string(i), [i]() {
      expect(i % 2, less(2));
    });
  }

  subsuite<>(_, "a subsuite", [](auto &_) {
    _.test("a sub-test", []() {
      expect(true, equal_to(true));
    });
  });
});

Note: I've not tried it myself.

0

lest

  • Stated design goals: Modern and C++11-native, single-file, header-only, small size
  • Framework code: https://github.com/martinmoene/lest
  • Missing: Suites of tests, parameterised tests, templated tests, test data generators, built-in hamcrest matchers.

Example:

const lest::test specification[] =
{
    CASE( "Empty string has length zero (succeed)" )
    {
        EXPECT( 0 == string(  ).length() );
        EXPECT( 0 == string("").length() );    
   },
}

Note: I haven't tried it myself.

0

Bandit

"A human-friendly unit testing for C++11"

Apparently, this framework "bends over backwards" to make your test's source code read like a natural-language description of a test. For example, here is a test of a guitar, which makes sure it sounds distorted in distortion mode:

describe("in distorted mode", [&]() {
    before_each([&]() { fuzzbox->flip(); });

    it("sounds distorted", [&]() {
        AssertThat(guitar->sound(), Equals(sounds::distorted));
    });
});

Note: I haven't tried this one myself.

0

DocTest

Presented in CppCon 2017 by Victor Kirilov

This actively-developed framework is not actually modern-C++-targeted, but it's aiming to be with its next major release (2.0; it's at 1.2 right now).

  • Motto: "The fastest feature-rich C++98/C++11 single-header testing framework for unit tests and TDD"
  • Framework code: https://github.com/onqtam/doctest
  • Ambitious roadmap, including IDE integrations
  • Purportedly very fast - with [benchmarks] against the Catch framework (but I have no idea if Catch is faster than Gtest, CppUnit etc.)

Note: I haven't tried this myself.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.