mirror of
https://gitlab.kitware.com/cmake/cmake.git
synced 2026-09-25 04:09:36 +03:00
Taylor Braun-JonesandTyler Yankee
5cced02b9d
When ctest repeats tests with its --repeat option, it repeats each test on its own. A fixture's setup and cleanup tests therefore run all of their repetitions back to back, and the tests they bracket repeat inside a single setup/cleanup pair: setup -> setup -> test -> test -> cleanup -> cleanup Add a FIXTURE_REPEAT_MODE test property to select how a fixture behaves when its tests are repeated: * AROUND_ALL_REPEATS: the fixture runs once, around all repetitions of the tests requiring it. * AROUND_EACH_REPEAT: the fixture and the tests requiring it repeat together, so every repetition gets a fresh setup and its own cleanup. * EACH_TEST_SEPARATELY: every test repeats on its own, as before. The property describes the fixture rather than the test carrying it, so setting it on any one of a fixture's setup or cleanup tests is enough. In AROUND_EACH_REPEAT mode the tests of a fixture form a repeat group that ctest re-queues as a whole once every test in it has finished. The --repeat condition then applies to the group the way it applies to an individual test: until-fail repeats while the whole group passes, until-pass repeats while any of it does not, and after-timeout repeats while any of it times out. Fixtures that share a test repeat together, so a test requiring two of them still runs once per repetition. A group is recorded the way a repeating test is: only once it stops repeating, and with the results of its last repetition. A group that until-pass makes pass therefore reports a pass rather than the failure that made it repeat, a test that DEPENDS on one of the group's tests waits for the last repetition rather than the first, and `ctest -F` resumes an interrupted group by running it again from the beginning. Fixtures that repeat together have to agree on the mode: a test cannot repeat with one fixture but not with another it takes part in, and a fixture whose setup and cleanup tests disagree has no coherent behavior. Report an error and run nothing in those cases rather than pick an order in which a test repeats after a fixture it requires has been cleaned up. Add policy CMP0224 to select AROUND_EACH_REPEAT as the default for fixtures whose setup and cleanup tests choose no mode themselves. Record the mode the policy chose in the generated test file under its own _CMAKE_DEFAULT_FIXTURE_REPEAT_MODE keyword, so that ctest reads a mode rather than the policy settings behind it, and so that a mode requested on one of a fixture's tests wins over the default recorded for its siblings. Only NEW needs recording: with nothing recorded, ctest already uses the behavior of CMake 4.4 and below. Fixtures are common, and the choice of mode matters only to those who run ctest --repeat, so warn about the unset policy only when the CMAKE_POLICY_WARNING_CMP0224 variable asks for it. discover_tests() and gtest_discover_tests() create their tests while ctest runs or at build time, too late for the policy to reach them, so carry the setting in effect at their call sites through to the tests they create. Report the repetition a grouped test belongs to in the "(run N/M)" suffix of its "Start" line, as ctest already does for a test repeating on its own. Co-authored-by: Tyler Yankee <tyler.yankee@kitware.com> Fixes: #21438
111 lines
3.0 KiB
C++
111 lines
3.0 KiB
C++
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
|
|
file LICENSE.rst or https://cmake.org/licensing for details. */
|
|
#pragma once
|
|
|
|
#include "cmConfigure.h" // IWYU pragma: keep
|
|
|
|
#include <cstddef>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "cmListFileCache.h"
|
|
#include "cmPolicies.h"
|
|
#include "cmPropertyMap.h"
|
|
#include "cmValue.h"
|
|
|
|
class cmMakefile;
|
|
|
|
/** \class cmTest
|
|
* \brief Represent a test
|
|
*
|
|
* cmTest is representation of a test.
|
|
*/
|
|
class cmTest
|
|
{
|
|
public:
|
|
/**
|
|
*/
|
|
cmTest(cmMakefile* mf);
|
|
~cmTest();
|
|
|
|
//! Set the test name
|
|
void SetName(std::string const& name);
|
|
std::string GetName() const { return this->Name; }
|
|
|
|
void SetCommand(std::vector<std::string> const& command);
|
|
std::vector<std::string> const& GetCommand() const { return this->Command; }
|
|
|
|
void SetBuildDependencies(std::vector<std::string> deps);
|
|
std::vector<std::string> const& GetDependencies() const
|
|
{
|
|
return this->BuildDependencies;
|
|
}
|
|
|
|
//! Set/Get a property of this source file
|
|
void SetProperty(std::string const& prop, cmValue value);
|
|
void SetProperty(std::string const& prop, std::nullptr_t)
|
|
{
|
|
this->SetProperty(prop, cmValue{ nullptr });
|
|
}
|
|
void SetProperty(std::string const& prop, std::string const& value)
|
|
{
|
|
this->SetProperty(prop, cmValue(value));
|
|
}
|
|
void AppendProperty(std::string const& prop, std::string const& value,
|
|
bool asString = false);
|
|
cmValue GetProperty(std::string const& prop) const;
|
|
bool GetPropertyAsBool(std::string const& prop) const;
|
|
cmPropertyMap& GetProperties() { return this->Properties; }
|
|
|
|
/** Get the cmMakefile instance that owns this test. */
|
|
cmMakefile* GetMakefile() { return this->Makefile; }
|
|
|
|
/** Get the backtrace of the command that created this test. */
|
|
cmListFileBacktrace const& GetBacktrace() const;
|
|
|
|
/** Get/Set whether this is an old-style test. */
|
|
bool GetOldStyle() const { return this->OldStyle; }
|
|
void SetOldStyle(bool b) { this->OldStyle = b; }
|
|
|
|
/** Get the CMP0158 policy setting */
|
|
cmPolicies::PolicyStatus GetCMP0158() const
|
|
{
|
|
return this->PolicyStatusCMP0158;
|
|
}
|
|
|
|
/** Get/Set the CMP0178 policy setting */
|
|
cmPolicies::PolicyStatus GetCMP0178() const
|
|
{
|
|
return this->PolicyStatusCMP0178;
|
|
}
|
|
void SetCMP0178(cmPolicies::PolicyStatus p)
|
|
{
|
|
this->PolicyStatusCMP0178 = p;
|
|
}
|
|
|
|
/** Get the CMP0224 policy setting */
|
|
cmPolicies::PolicyStatus GetCMP0224() const
|
|
{
|
|
return this->PolicyStatusCMP0224;
|
|
}
|
|
|
|
/** Set/Get whether lists in command lines should be expanded. */
|
|
bool GetCommandExpandLists() const;
|
|
void SetCommandExpandLists(bool b);
|
|
|
|
private:
|
|
cmPropertyMap Properties;
|
|
std::string Name;
|
|
std::vector<std::string> Command;
|
|
std::vector<std::string> BuildDependencies;
|
|
bool CommandExpandLists = false;
|
|
|
|
bool OldStyle;
|
|
|
|
cmMakefile* Makefile;
|
|
cmListFileBacktrace Backtrace;
|
|
cmPolicies::PolicyStatus PolicyStatusCMP0158;
|
|
cmPolicies::PolicyStatus PolicyStatusCMP0178;
|
|
cmPolicies::PolicyStatus PolicyStatusCMP0224;
|
|
};
|