On this page:
13.1 Test Configuration by Submodule
13.2 Test Configuration by "info.rkt"
13.3 Responsible-Party and Varying-Output Logging

13 raco test: Run tests

The raco test command requires and runs the (by default) test submodule associated with each path given on the command line. Command-line flags can control which submodule is run, whether to run the main module if no submodule is found, and whether to run tests directly, in separate processes (the default), or in separate places. The current directory is set to a test file’s directory before running the file.

When an argument path refers to a directory, raco test recursively discovers and runs all files within the directory that end in a module suffix (see get-module-suffixes, but the suffixes always include ".rkt", ".scrbl", ".ss", and ".scm") or have a (possibly empty) list of command-line arguments provided by test-command-line-arguments in an "info.rkt" file, or as directed by test-include-paths in an "info.rkt" file. At the same time, raco test omits files and directories within a directory as directed by test-omit-paths in an "info.rkt" file.

A test is counted as failing if it logs a failing test code via test-log!, causes Racket to exit with a non-zero exit code, or (when -e or --check-stderr is specified) if it produces output on the error port.

The raco test command accepts several flags:

Changed in version 1.1 of package compiler-lib: Added --heartbeat.
Changed in version 1.4: Changed recognition of module suffixes to use get-module-suffixes, which implies recognizing ".ss" and ".rkt".
Changed in version 1.5: Added ++ignore-stderr.
Changed in version 1.6: Added ++arg and ++args.
Changed in version 1.8: Added --output and -o.

13.1 Test Configuration by Submodule

When raco test runs a test in a submodule, a config sub-submodule can provide additional configuration for running the test. The config sub-submodule should use the info module language to define the following identifiers:

In order to prevent evaluation of a file for testing purposes, it suffices to create a submodule that does not perform any tests and does not trigger the evaluation of the enclosing module. So, for instance, a file might look like this:

#lang racket
 
(/ 1 0)
 
; don't run this file for testing:
(module test racket/base)

Changed in version 1.5 of package compiler-lib: Added ignore-stderr support.

13.2 Test Configuration by "info.rkt"

Submodule-based test configuration is preferred (see Test Configuration by Submodule). In particular, to prevent raco test from running a particular file, normally the file should contain a submodule that takes no action.

In some cases, however, adding a submodule is inconvenient or impossible (e.g., because the file will not always compile). Thus, raco test also consults any "info.rkt" file in the candidate test file’s directory. In the case of a file within a collection, "info.rkt" files from any enclosing collection directories are also consulted for test-omit-paths and test-include-paths. Finally, for a file within a package, the package’s "info.rkt" is consulted for pkg-authors to set the default responsible parties (see Responsible-Party and Varying-Output Logging) for all files in the package.

The following "info.rkt" fields are recognized:

Changed in version 1.5 of package compiler-lib: Added test-ignore-stderrs support.

13.3 Responsible-Party and Varying-Output Logging

When a test has a declared responsible party, then the test’s output is prefixed with a

  raco test:which @(test-responsible 'responsible)

line, where which is a space followed by an exact non-negative number indicating a parallel task when parallelism is enabled (or empty otherwise), and responsible is a string, symbol, or list datum.

When a test’s output (as written to stdout) is expected to vary across runs—aside from varying output that has the same form as produced by timethen it should be declared as varying. In that case, the test’s output is prefixed with a

  raco test:which @(test-random #t)

line.