testr - Manual Usage

Capturing arbitrary code

While the basic usage vignette deals with the introduction of testR capabilies, explanation of its limitations and a walkthrough over the two most common usecases, testR also exports the functions required to capture and generate regressions from any code. These functions will be discussed in this document. To illustrate the low level API for genthat we will need a function, we can use the isPrime() function from testgenthat package:

isPrime <- function (x) {
    for (i in 2:(x - 1)) if (x%%i >= 1) 
        return(F)
    return(T)
}

Capturing Functions

First we must determine which functions we want to capture. Enabling capture for functions is done using the start_capture function that takes the functions to be captured as its arguments. You can use either package qualified function names, such as base::any, or you specify the same in a character vector ("base::any"):

start_capture(isPrime, verbose = testr_options("verbose"))

As with all other functions, the verbose flag is used to add more output. If there are no namespace clashes for given functions, you can omit the package name from the function definition, i.e. we could have written just start_capture(isPrime) to the same effect.

Once the functions are captured, we must make sure to execute the code that actuall calls them to generate the call snapshots. The simplest way to do this is call them manually, but many other ways are possible:

isPrime(2)
isPrime(3)
isPrime(10)

When the code has exected, we should stop capturing the functions. Specific function can be stopped using the stop_capture function, or all currently captured functions can be stopped:

stop_capture_all()

Generating tests

After the data has been captured, it is now time to generate the tests:

generate("isPrime_tests")

The generate function analyzes the captured information and generates tests out of it. In its simplest invocation only the output directory to which the tests should be generated is needed, as in the example above. Other options include specifying where to look for the captured data (root), whether a timestamp should be added to the tests directory (timed) or if the capture data should be retained so that tests from it can be generated again (clear_capture).

The generator creates a file called bad_arguments in which it puts all invocations that cannot be reproduced (because of external dependencies, environment, user input, etc.) and then for each captured function a directory, in which tests are stored, one test per file. Our example has generated three tests in the testthat format:

library(testthat)

context("testgenthat:::isPrime")

test_that("0", {
    expected <- FALSE
    expect_equal({
        testgenthat:::isPrime(x = 1)
    }, expected)
})

Shorthands

The following steps could be accomplished by calling a shorthand function:

gen_from_code({ isPrime(1); isPrime(2); isPrime(10) }, "isPrime_tests", isPrime)

will have the same effect. The function takes first the code to be executed, then the directory where to put the tests and finally list of functions to be captured. Similar function called gen_from_source has the same purpose, but instead of code to run expects a source file to execute.

Pruning the tests

Usually, genthat generates way more tests than are necessary as it record each invocation of captured functions. To prune the generated tests based on code coverage, we may use the prune function:

prune("isPrime_tests", "pruned", isPrime)

This tells genthat to take all tests in isPrime_tests directory and prune them, looking into code coverage for function isPrime. The tests that increase the coverage will be copied into the pruned directory. Additional arguments may specify whether the remove the unpruned tests (remove_tests), whether to compact the pruned tests into a file per function (compact) or, instead of specifying the functions to be checked against code coverage, it may specify an entire package to be checked (package_path).

After the pruning, in our tiny example, we will have only two tests out of the original three (one that covers prime numbers and one that covers the others).