Verify output

This is a regression test that records interwoven code and output into a file, in a similar way to kniting an .Rmd (but see caveats below).

verify_output() designed particularly for testing print methods and error messages, where the primary goal is to ensure that the output is helpful to a human. Obviously, you can't test that with code, so the best you can do is make the results explicit by saving them to text file. This makes the output easy to see in code reviews, and ensures that you don't change the output accidentally.

verify_output() is designed to be used with git: to see what has changed from the previous run, you'll need to use git diff or similar.

verify_output(
  path,
  code,
  width = 80,
  crayon = FALSE,
  unicode = FALSE,
  env = caller_env()
)

Arguments

path	Path to record results. This should usually be a call to test_path() to ensures that same path is used when run interactively (when the working directory is typically the project root), and when run as an autmated test (when the working directory will be tests/testthat).
code	Code to execute. This will usually be a multiline expression contained within {} (similarly to test_that() calls).
width	Width of console output
crayon	Enable crayon package colouring?
unicode	Enable cli package UTF-8 symbols? If you set this to TRUE, call skip_if(!cli::is_utf8_output()) to disable the test on your CI platforms that don't support UTF-8 (e.g. Windows).
env	The environment to evaluate code in.

Syntax

verify_output() can only capture the abstract syntax tree, losing all whitespace and comments. To mildy offset this limitation:

• Strings are converted to R comments in the output.

• Strings starting with # are converted to headers in the output.

CRAN

On CRAN, verify_output() will never fail, even if the output changes. This avoids false positives because tests of print methods and error messages are often fragile due to implicit dependencies on other packages, and failure does not imply incorrect computation, just a change in presentation.

Examples

# The first argument would usually be `test_path("informative-name.txt"`)
# but that is not permitted in examples
path <- tempfile()
verify_output(path, {
   head(mtcars)
   log(-10)
   "a" * 3
})
#> Warning: Creating reference output
writeLines(readLines(path))
#> > head(mtcars)
#>                    mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
#> Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
#> Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
#> Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
#> Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2
#> Valiant           18.1   6  225 105 2.76 3.460 20.22  1  0    3    1
#> 
#> > log(-10)
#> Warning in log(-10): NaNs produced
#> 
#> [1] NaN
#> 
#> > "a" * 3
#> Error in "a" * 3: non-numeric argument to binary operator
#> 

# Use strings to create comments in the output
verify_output(tempfile(), {
   "Print method"
   head(mtcars)

   "Warning"
   log(-10)

   "Error"
   "a" * 3
})
#> Warning: Creating reference output

# Use strings starting with # to create headings
verify_output(tempfile(), {
   "# Base functions"
   head(mtcars)
   log(-10)
   "a" * 3
})
#> Warning: Creating reference output