mirror of
https://gitlab.kitware.com/cmake/cmake.git
synced 2026-09-25 04:09:36 +03:00
Taylor Braun-JonesandTyler Yankee
5cced02b9d
When ctest repeats tests with its --repeat option, it repeats each test on its own. A fixture's setup and cleanup tests therefore run all of their repetitions back to back, and the tests they bracket repeat inside a single setup/cleanup pair: setup -> setup -> test -> test -> cleanup -> cleanup Add a FIXTURE_REPEAT_MODE test property to select how a fixture behaves when its tests are repeated: * AROUND_ALL_REPEATS: the fixture runs once, around all repetitions of the tests requiring it. * AROUND_EACH_REPEAT: the fixture and the tests requiring it repeat together, so every repetition gets a fresh setup and its own cleanup. * EACH_TEST_SEPARATELY: every test repeats on its own, as before. The property describes the fixture rather than the test carrying it, so setting it on any one of a fixture's setup or cleanup tests is enough. In AROUND_EACH_REPEAT mode the tests of a fixture form a repeat group that ctest re-queues as a whole once every test in it has finished. The --repeat condition then applies to the group the way it applies to an individual test: until-fail repeats while the whole group passes, until-pass repeats while any of it does not, and after-timeout repeats while any of it times out. Fixtures that share a test repeat together, so a test requiring two of them still runs once per repetition. A group is recorded the way a repeating test is: only once it stops repeating, and with the results of its last repetition. A group that until-pass makes pass therefore reports a pass rather than the failure that made it repeat, a test that DEPENDS on one of the group's tests waits for the last repetition rather than the first, and `ctest -F` resumes an interrupted group by running it again from the beginning. Fixtures that repeat together have to agree on the mode: a test cannot repeat with one fixture but not with another it takes part in, and a fixture whose setup and cleanup tests disagree has no coherent behavior. Report an error and run nothing in those cases rather than pick an order in which a test repeats after a fixture it requires has been cleaned up. Add policy CMP0224 to select AROUND_EACH_REPEAT as the default for fixtures whose setup and cleanup tests choose no mode themselves. Record the mode the policy chose in the generated test file under its own _CMAKE_DEFAULT_FIXTURE_REPEAT_MODE keyword, so that ctest reads a mode rather than the policy settings behind it, and so that a mode requested on one of a fixture's tests wins over the default recorded for its siblings. Only NEW needs recording: with nothing recorded, ctest already uses the behavior of CMake 4.4 and below. Fixtures are common, and the choice of mode matters only to those who run ctest --repeat, so warn about the unset policy only when the CMAKE_POLICY_WARNING_CMP0224 variable asks for it. discover_tests() and gtest_discover_tests() create their tests while ctest runs or at build time, too late for the policy to reach them, so carry the setting in effect at their call sites through to the tests they create. Report the repetition a grouped test belongs to in the "(run N/M)" suffix of its "Start" line, as ctest already does for a test repeating on its own. Co-authored-by: Tyler Yankee <tyler.yankee@kitware.com> Fixes: #21438
2 lines
87 B
Plaintext
2 lines
87 B
Plaintext
^Test project .*/Tests/RunCMake/CTestCommandLine/repeat-fixture-conflict-shared-build$
|