Whole file snapshot testing is designed for testing objects that don't have
a convenient textual representation, with initial support for images
(.png
, .jpg
, .svg
), data frames (.csv
), and text files
(.R
, .txt
, .json
, ...).
The first time expect_snapshot_file()
is run, it will create
_snaps/{test}/{name}.{ext}
containing reference output. Future runs will
be compared to this reference: if different, the test will fail and the new
results will be saved in _snaps/{test}/{name}.new.{ext}
. To review
failures, call snapshot_review()
.
We generally expect this function to be used via a wrapper that takes care of ensuring that output is as reproducible as possible, e.g. automatically skipping tests where it's known that images can't be reproduced exactly.
expect_snapshot_file(
path,
name = basename(path),
binary = lifecycle::deprecated(),
cran = FALSE,
compare = NULL,
transform = NULL,
variant = NULL
)
announce_snapshot_file(path, name = basename(path))
compare_file_binary(old, new)
compare_file_text(old, new)
Path to file to snapshot. Optional for
announce_snapshot_file()
if name
is supplied.
Snapshot name, taken from path
by default.
Should these expectations be verified on CRAN? By default, they are not, because snapshot tests tend to be fragile because they often rely on minor details of dependencies.
A function used to compare the snapshot files. It should take
two inputs, the paths to the old
and new
snapshot, and return either
TRUE
or FALSE
. This defaults to compare_file_text
if name
has
extension .r
, .R
, .Rmd
, .md
, or .txt
, and otherwise uses
compare_file_binary
.
compare_file_binary()
compares byte-by-byte and
compare_file_text()
compares lines-by-line, ignoring
the difference between Windows and Mac/Linux line endings.
Optionally, a function to scrub sensitive or stochastic text from the output. Should take a character vector of lines as input and return a modified character vector as output.
If not-NULL
, results will be saved in
_snaps/{variant}/{test}/{name}.{ext}
. This allows you to create
different snapshots for different scenarios, like different operating
systems or different R versions.
Paths to old and new snapshot files.
testthat automatically detects dangling snapshots that have been
written to the _snaps
directory but which no longer have
corresponding R code to generate them. These dangling files are
automatically deleted so they don't clutter the snapshot
directory. However we want to preserve snapshot files when the R
code wasn't executed because of an unexpected error or because of a
skip()
. Let testthat know about these files by calling
announce_snapshot_file()
before expect_snapshot_file()
.
# To use expect_snapshot_file() you'll typically need to start by writing
# a helper function that creates a file from your code, returning a path
save_png <- function(code, width = 400, height = 400) {
path <- tempfile(fileext = ".png")
png(path, width = width, height = height)
on.exit(dev.off())
code
path
}
path <- save_png(plot(1:5))
path
#> [1] "/tmp/RtmpiNHhw1/fileebd7fb1844a.png"
if (FALSE) {
expect_snapshot_file(save_png(hist(mtcars$mpg)), "plot.png")
}
# You'd then also provide a helper that skips tests where you can't
# be sure of producing exactly the same output
expect_snapshot_plot <- function(name, code) {
# Other packages might affect results
skip_if_not_installed("ggplot2", "2.0.0")
# Or maybe the output is different on some operation systems
skip_on_os("windows")
# You'll need to carefully think about and experiment with these skips
name <- paste0(name, ".png")
# Announce the file before touching `code`. This way, if `code`
# unexpectedly fails or skips, testthat will not auto-delete the
# corresponding snapshot file.
announce_snapshot_file(name = name)
path <- save_png(code)
expect_snapshot_file(path, name)
}