When ctest repeats tests with its --repeat option, it repeats each test
on its own. A fixture's setup and cleanup tests therefore run all of
their repetitions back to back, and the tests they bracket repeat inside
a single setup/cleanup pair:
setup -> setup -> test -> test -> cleanup -> cleanup
Add a FIXTURE_REPEAT_MODE test property to select how a fixture behaves
when its tests are repeated:
* AROUND_ALL_REPEATS: the fixture runs once, around all repetitions of
the tests requiring it.
* AROUND_EACH_REPEAT: the fixture and the tests requiring it repeat
together, so every repetition gets a fresh setup and its own cleanup.
* EACH_TEST_SEPARATELY: every test repeats on its own, as before.
The property describes the fixture rather than the test carrying it, so
setting it on any one of a fixture's setup or cleanup tests is enough.
In AROUND_EACH_REPEAT mode the tests of a fixture form a repeat group that
ctest re-queues as a whole once every test in it has finished. The --repeat
condition then applies to the group the way it applies to an individual
test: until-fail repeats while the whole group passes, until-pass repeats
while any of it does not, and after-timeout repeats while any of it times
out. Fixtures that share a test repeat together, so a test requiring two of
them still runs once per repetition.
A group is recorded the way a repeating test is: only once it stops
repeating, and with the results of its last repetition. A group that
until-pass makes pass therefore reports a pass rather than the failure that
made it repeat, a test that DEPENDS on one of the group's tests waits for
the last repetition rather than the first, and `ctest -F` resumes an
interrupted group by running it again from the beginning.
Fixtures that repeat together have to agree on the mode: a test cannot
repeat with one fixture but not with another it takes part in, and a fixture
whose setup and cleanup tests disagree has no coherent behavior. Report an
error and run nothing in those cases rather than pick an order in which a
test repeats after a fixture it requires has been cleaned up.
Add policy CMP0224 to select AROUND_EACH_REPEAT as the default for fixtures
whose setup and cleanup tests choose no mode themselves. Record the mode
the policy chose in the generated test file under its own
_CMAKE_DEFAULT_FIXTURE_REPEAT_MODE keyword, so that ctest reads a mode
rather than the policy settings behind it, and so that a mode requested on
one of a fixture's tests wins over the default recorded for its siblings.
Only NEW needs recording: with nothing recorded, ctest already uses the
behavior of CMake 4.4 and below. Fixtures are common, and the choice of
mode matters only to those who run ctest --repeat, so warn about the unset
policy only when the CMAKE_POLICY_WARNING_CMP0224 variable asks for it.
discover_tests() and gtest_discover_tests() create their tests while ctest
runs or at build time, too late for the policy to reach them, so carry the
setting in effect at their call sites through to the tests they create.
Report the repetition a grouped test belongs to in the "(run N/M)" suffix
of its "Start" line, as ctest already does for a test repeating on its own.
Co-authored-by: Tyler Yankee <tyler.yankee@kitware.com>
Fixes: #21438
Revert commit a324c2bb58 (GoogleTest: Base on builtin command
discover_tests, 2026-03-01, v4.4.0-rc1~30^2). The `discover_tests`
implementation does not persist already-discovered tests across ctest
runs, making it much slower on projects whose test binaries take a while
just to start up and list tests. Revert to the prior implementation
pending caching support in `discover_tests`.
Fixes: #28053
Issue: #28060
Commit c3c95f3010 (GoogleTest: Avoid generation error on duplicate
target test discovery, 2026-07-14, v4.4.1~24^2) has added a level of
indentation which made rebasing other changes harder than necessary.
Avoiding an else branch after a return command allows keeping the
original indentation.
Prior to porting `gtest_discover_tests` to `discover_tests` it silently
tolerated an unpaired property name. `discover_tests` does not support
that, so remove the unpaired argument. Also add a warning to help
projects avoid such usage.
Fixes: #28004
The discovery script created by commit 3748ca9f (GoogleTest: generate
single discovery script, 2026-04-08, v4.4.0-rc1~302^2~1) creates
generate-time collisions if gtest_discover_tests() is called on the
same target in the same directory with the same arguments multiple
times. Handle the case with a warning to preserve compatibility with
existing projects.
Fixes: #27940
Restore the logic from commit 6680df042e (GoogleTest: Avoid POST_BUILD
race condition for gtest_discover_tests(), 2025-10-26, v4.2.0-rc2~15^2)
that was lost in the refactoring in commit 3748ca9fc7 (GoogleTest:
generate single discovery script, 2026-04-08, v4.4.0-rc1~302^2~1). Add
a regression test.
Fixes: #27939, #27319, #27972
Since commit 1cdceae8e3 (GoogleTest: Parse discovered test list from
JSON output if supported, 2025-05-02, v4.2.0-rc1~533^2~2) we've placed
the discovery JSON files in the working directory provided by the user.
Place them in the current build tree folder instead.
Fixes: #27840
Introduce a mechanism (via the private and undocumented `ISSUE_WARNING`
option to the `cmake_policy` command) to allow built-in modules to issue
policy warnings. Update modules to use this.
This allows modules to issue policy warnings with the `CMD_POLICY`
diagnostic type, which is consistent with internally issued diagnostics.
This also tends to simplify the logic for issuing such warnings in much
the way the helpers introduced in the previous commit did for the C++
code.
Generate a script that contains a `gtest_discover_tests_impl` call
and use that script for both discovery modes: Either execute it via
`-P` for the POST_BUILD mode, or include it in the test script for
the PRE_TEST mode.
In fc7aa3cd69 (tests: Preserve empty arguments in test command lines,
2024-09-30), the CMP0174 policy setting had to be explicitly handled in
GoogleTestAddTests.cmake because it was part of the same release.
Now that we're past that release, we can remove that and update the
policies based on the CMake version instead. This removes the TODO
added in that original commit.
We also update the policy settings in GoogleTest.cmake so that we are
using consistent policy settings across the GoogleTest implementation.
For single config generators, don't write a file at configure time
as it will be rewritten at generation time.
For multi config generators, use `$<CONFIG>` in the output instead
of looping over `CMAKE_CONFIGURATION_TYPES`.
Covers almost all blocks containing actual code, except:
* Parsed-literal blocks can't be highlighted, including many command
summaries and substitution-heavy docs like find_... commands.
This is a Sphinx limitation.
* Code with errors, like CMP0049, DEPLOYMENT_ADDITIONAL_FILES,
DEPLOYMENT_REMOTE_DIRECTORY, @PACKAGE_INIT@ substitution in the
tutorial, bracket arguments/comments in cmake-language.7 and
cmake-developer.7.
* FindQt4 module, which needs reformatting.
Revise commit fc7aa3cd69 (tests: Preserve empty arguments in test
command lines, 2024-09-30, v3.31.0-rc1~33^2) to delay variable expansion
until code evaluation. Otherwise we may parse the working directory
path as CMake language code.
Test names with square brackets can't be safely parsed in the
list returned through TEST_LIST, so they are omitted from the
list. This was implemented in 8b85b10fb3 (GoogleTest: Add
handling for square brackets in test names, 2022-01-05), but the
documentation wasn't updated as part of that change.
This will now preserve empty values in the TEST_LAUNCHER and
CROSSCOMPILING_EMULATOR target properties for tests added by:
- The add_test() command.
- The ExternalData_Add_Test() command from the ExternalData module.
- The gtest_add_tests() or gtest_discover_tests() commands from the
GoogleTest module.
For the gtest_add_tests() and gtest_discover_tests() commands,
empty elements in the values passed after the EXTRA_ARGS keyword
are also now preserved.
Policy CMP0178 is added to provide backward compatibility with the
old behavior where empty values were silently discarded from the
above cases.
Fixes: #26337
The prose previously gave the impression that discovery was always done
as a post-build operation, but later in the documentation the
DISCOVERY_MODE option is explained where pre-test can also be specified.
Update the earlier prose to mention both methods equally.
- Use an "arg" prefix for cmake_parse_arguments() rather than
an empty string or uppercase ARGS so that usages of parsed
values don't start with a bare underscore or look fully
uppercase. This makes it harder to miss them or mistake
them for keywords.
- Use lowercase for names of positional function arguments
so they don't get mistaken for keywords.
We run test executables to discover tests. Use the same launchers that
are used to run the tests. We already handle `CROSSCOMPILING_EMULATOR`.
Update the logic to account for the `TEST_LAUNCHER` property added by
commit 1ec0372ed4 (add_test: Optionally use a launcher for tests running
in-project targets, 2023-11-11), and for the `CROSSCOMPILING_EMULATOR`
behavior change in commit ca5a300d7f (add_test: Honor
CROSSCOMPILING_EMULATOR only when cross-compiling, 2023-11-02).
Fixes: #25603
Co-authored-by: Brad King <brad.king@kitware.com>
Signed-off-by: Ralf Habacker <ralf.habacker@freenet.de>
The internal helper variable '_GOOGLETEST_DISCOVER_TESTS_SCRIPT' can have gone
out-of-scope when 'gtest_discover_tests()' is called, depending on where the
GoogleTest module is actually included. This leads to a silent failure of
dynamic test discovery, since the custom post-build commands actually does
nothing (it basically invokes 'cmake -P ""'). Ctest will then fail to run the
tests, considering them to be 'not built'.
Fix this by determining the path to the GoogleTest module based on
'${CMAKE_ROOT}' instead, which is always available.
A new test case was added to test suite 'RunCMake/GoogleTest' to ensure that
'gtest_discover_tests()' works correctly when invoked in a different variable
scope.
Fixes: #25477
d4d1f85297 Help: Move a note about arguments and genexes into include file
968224484f Help: Drop unused `PROPERTY_GENEX` replace from `prop_tgt/INTERFACE_*` files
65457e4fae Help: Add _See Also_ sections to related CMake commands
c86ec79d07 Help: Use direct refs to genexes instead of ``$<GENEX-NAME>``
8c6274326d Help: Use CMake highlighting for command synopsis
c356a94512 Help: Use monospaced font for a genex name in a section title
f98c8707f4 Help: Use `code-block:: cmake` instead of `::` for sub-commands descriptions
2ba351ca4f Help: ``CMake`` → CMake
...
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !7873
The `TEST_FILTER` argument can be used to filter tests during the
discovery phase. It combines `--gtest_filter=<expr>` with the
`--gtest_list_tests` argument when invoking the test excutable for
listing defined tests.
Fixes: #17493
When using a file system which only has second resolution timestamps,
there is a reasonably high likelihood of timestamps being the same.
The IS_NEWER_THAN test returns true when timestamps are the same,
so don't redo test discovery when they match exactly.
The generated ..._include.cmake file contained an inclusion of
GoogleTestAddTests without any path. In general, this is a good
approach since it enables to correctly catch possibly
user-customized files.
However, in this case, it didn’t work this way since the
..._include.cmake file is evaluated by a separate CMake call under
a custom command. Because of this, the CMAKE_MODULE_PATH is not set
as expected by the user and the said inclusion catches CMake own
version of the module rather than the possibly user-customized one.
This change fixes this by making the inclusion with an absolute path
determined upon the ..._include.cmake file generation.
Taylor Braun-JonesandTyler Yankee
Daniel Pfeifer
Ottmar Zittlau
Matthew Woehlke
Brad King
Máté Ferenc Nagy-Egri
Craig Scott
Peter Kokot
Kitware Robot
Nikita Nemkin
Ihor Sofiichenko
Alex Turbov
Cedric MartineauandCraig Scott
Daniel Sim
Ralf HabackerandBrad King
Moritz Haase
DavidPerretSN
Oleksii Odynochenko
Ashish Sadanandan
Adam Badura