Contributing PowerDNS tests: unit tests

Each and every PowerDNS change is tested by our Jenkins continuous integration setup, as hosted on https://autotest.powerdns.com. The more tests we have, the more valuable this setup becomes, and in this post you can find easy to follow instructions on how to contribute unit tests.

Our automated testing consists of two parts: regression-tests and unit tests. The regression tests stress full operational scenarios, with actually running PowerDNS binaries being queried for records, doing AXFRs, signing records, the works. We exercise hundreds of scenarios this way.

In addition, we run ‘unit tests’ which stress out particular pieces of code, so that we know that they do what we think they do. Both of these testing strategies (‘whole binary’, ‘component’) have their merits. This post is about adding unit tests.

Forking the repository

Step one is to fork PowerDNS via our GitHub repository. This requires a GitHub account. Forking is explained here: https://help.github.com/articles/fork-a-repo. After you’ve forked us, clone your fork & create a branch for your work:

$ git clone https://github.com/YourUsername/pdns
$ cd pdns
$ git checkout -b Base64UnitTest

Next, make sure that you meet the basic dependencies for building PowerDNS. On Debian 7, we have verified that “apt-get install autoconf automake bison flex g++ libboost-all-dev libtool make pkg-config ragel" does the job.

Then run:

$ cd pdns
$ ./bootstrap
$ ./configure --with-modules="" --enable-unit-tests --without-lua
$ make

This will generate PowerDNS binaries and libraries, allowing us to build and run the ‘testrunner’:

$ cd pdns
$ make testrunner
$ ./testrunner
Running 9 test cases...
*** No errors detected

Nine tests is not enough by far, so let’s go about adding a new one. PowerDNS uses the Boost Unit Test framework, and by convention all test files are called ‘test-filename_extension.cc’. So, to add a test for base64.cc, we create ‘test-base64_cc.cc’:

#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_NO_MAIN
#include "base64.hh"
#include <boost/test/unit_test.hpp>
BOOST_AUTO_TEST_SUITE(base64_cc)
// tests go here
BOOST_AUTO_TEST_SUITE_END()

Next, add ‘test-base64_cc.cc’ to the testrunner_SOURCES variable in Makefile.am, re-run bootstrap and configure, and run ‘make testrunner’.  If we now actually execute our new testrunner, it still says ‘Running 9 testcases’, and to fix this, we’ll actually need to hook up a testcase (where we said // tests go here):

BOOST_AUTO_TEST_CASE(test_Base64_Roundtrip) {
    std::string before("Some Random String"), after;
    std::string encoded = Base64Encode(before);
    B64Decode(encoded, after);
    BOOST_CHECK_EQUAL(before, after);
}

And if we now rebuild testrunner & run it, it does indeed report 10 tests that passed! Within the PowerDNS Jenkins framework, testrunner is called with parameters that make it emit the status of each test as XML, which is then displayed visually on https://autotest.powerdns.com/.

Now that we’ve built a working test & hooked it up, we need to submit it to GitHub so that we over at PowerDNS can integrate your work at the touch of a button:

$ git add test-base64_cc.cc
$ git commit -a -m "create and hook up base64 tests"
$ git push origin Base64UnitTest

Your  code is at GitHub now, so head over there and go to your fork (in our example, this would be https://github.com/YourUsername/pdns).

There you will find a panel that lists “Your recently pushed branches”, including Base64UnitTest. You can press ‘compare’ to see that it indeed matches what you did, and then press the “Pull Request” button. This allows you to describe your work, and it pays to spend some time on this as it makes our decision to merge your fork easier. Once the description & title are ok, press submit.

Quite soon you should be getting feedback that your test is now part of each and every PowerDNS build!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s