covr thinks I'm CRAN!

testthat is a great tool to test that R package code actually does what it is supposed to do. And keep doing the right thing after multiple package updates, edits and ‘fixes’.

covr helps show what part of the codebase is actually tested with testthat.

It is possible to use testthat with shiny.

One annoying problem I had is that devtools::test() might work :

> devtools::test()
Loading DTedit
Testing DTedit
✓ |  OK F W S | Context
✓ |   1       | callbacks_actionButtons [12.5 s]
✓ |   1       | dtedit_demo [10.0 s]
✓ |   1       | dtedit_demo_modular [5.2 s]
✓ |   5       | error_test [6.5 s]
✓ |   1       | fileInput_modular [12.2 s]
✓ |   1       | password [6.9 s]
✓ |   1       | reactive [2.7 s]
✓ |   1       | reactive_demo [2.7 s]
✓ |   1       | selectInputReactive [8.7 s]
✓ |   1       | selectInputReactive_demo [13.6 s]
✓ |   1       | simple [13.6 s]
✓ |   1       | simple_modular [13.7 s]

══ Results ═════════════════════════════════════════════════════════════════════
Duration: 113.0 s

OK:       16
Failed:   0
Warnings: 0
Skipped:  0

but testthat::test_package() skips all the tests :

> testthat::test_package("DTedit", quiet = FALSE)
══ testthat results  ═══════════════════════════════════════════════════════════
[ OK: 0 | SKIPPED: 12 | WARNINGS: 0 | FAILED: 0 ]

with similar results from covr::codecov() :

> covr::codecov(quiet = FALSE)
* installing *source* package ‘DTedit’ ...
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package ‘DTedit’
    dtedit                                  example
    dteditmodUI                             example
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (DTedit)
Running specific tests for package ‘DTedit’
  Running ‘testthat.R’
DTedit Coverage: 0.00%
R/dtedit_demo.R: 0.00%
R/dtedit_test.R: 0.00%
R/dtedit.R: 0.00%

ouch! no coverage!

The problem is that, strangely, the default setting of various R packages is to think that testing is occurring on a CRAN computer!

So tests will be skipped if testthat::skip_on_cran() is used in the tests.

There a few possible solutions, but one solution described by Randy Lai is to use withr::with_envvar :

> withr::with_envvar(c("NOT_CRAN" = "true"), covr::codecov(quiet = FALSE))
* installing *source* package ‘DTedit’ ...
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package ‘DTedit’
    dtedit                                  example
    dteditmodUI                             example
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (DTedit)
Running specific tests for package ‘DTedit’
  Running ‘testthat.R’
DTedit Coverage: 95.07%
R/dtedit.R: 94.99%
R/dtedit_test.R: 95.04%
R/dtedit_demo.R: 100.00%
$uploaded
[1] TRUE