Merge topic 'ctest-dashboard-preset-env'

27f28db197 Help: Clarify CTEST_NO_TESTS_ACTION support and interaction a235560e53 Help: Clarify interaction between CTest parallelism toggles f745dd11ab ctest: Respect test envvars set by CTEST_PRESET 277236a99a cmCTestTestCommand: Factor out preset resolution 81b46d87ba cmCTest: Factor out environment variable handling 4e81913645 Tests: Fix unset(ENV{...}) typos Acked-by: Kitware Robot <kwrobot@kitware.com> Merge-request: !12495
2026-09-25 04:09:36 +03:00 · 2026-09-18 07:59:56 -04:00
parent 08bde1b972 27f28db197
commit ab46a992e5
20 changed files with 421 additions and 82 deletions
@@ -133,6 +133,13 @@ The options are:
    default level of parallelism, or unbounded parallelism, respectively, as
    documented by the :option:`ctest --parallel` option.

+  If given, this argument takes precedence over a test preset's
+  :preset:`testPresets.execution.jobs` field. Both this argument and that
+  field take precedence over the :envvar:`CTEST_PARALLEL_LEVEL` environment
+  variable, and over an explicit :option:`ctest --parallel` given on the
+  command line. See :envvar:`CTEST_PARALLEL_LEVEL` for the full precedence
+  order.
+
 ``RESOURCE_SPEC_FILE <file>``
  .. versionadded:: 3.16

@@ -223,6 +230,13 @@ The options are:
  See also the :variable:`CTEST_TEST_PRESET` and
  :variable:`CTEST_PRESET` variables.

+  In general, only preset fields which have equivalent options to
+  ``ctest_test`` are supported. An exception is the
+  :preset:`testPresets.environment`, which can hold a value for the
+  :envvar:`CTEST_NO_TESTS_ACTION` environment variable and will be respected,
+  despite ``ctest_test`` not having an equivalent option to the
+  :preset:`testPresets.execution.noTestsAction` presets field.
+
 ``PRESETS_FILE <file>``
  .. versionadded:: 4.4

@@ -12,3 +12,18 @@ cases when there are no tests to run. Possible values are: ``error``,
 The :option:`--no-tests=\<action\> <ctest --no-tests>` option to
 :manual:`ctest <ctest(1)>` overrides this environment variable if both
 are given.
+
+When :option:`ctest --preset` is used, this environment variable, if set,
+takes precedence over the :preset:`testPresets.execution.noTestsAction`
+field of the selected test preset. This includes the case where the
+variable is instead set via the preset's own
+:preset:`testPresets.environment` field, which is equivalent to setting it
+in the calling process's environment. An explicit
+:option:`--no-tests=\<action\> <ctest --no-tests>` on the command line
+takes precedence over all of the above.
+
+This environment variable, including when set via a test preset's
+``environment`` field, is also respected when that same preset is used by
+a :command:`ctest_test` command in a :ref:`CTest Script`. However, in that
+case, the preset's ``noTestsAction`` field itself has no effect, because
+:command:`ctest_test` has no equivalent argument to apply it through.
@@ -5,12 +5,12 @@ CTEST_PARALLEL_LEVEL

 Specify the number of tests for CTest to run in parallel.
 For example, if ``CTEST_PARALLEL_LEVEL`` is set to 8, CTest will run
-up to 8 tests concurrently as if ``ctest`` were invoked with the
+up to 8 tests concurrently as if :manual:`ctest(1)` were invoked with the
 :option:`--parallel 8 <ctest --parallel>` option.

 .. versionchanged:: 3.29

-  The value may be empty, or ``0``, to let ctest use a default level of
+  The value may be empty, or ``0``, to let CTest use a default level of
  parallelism, or unbounded parallelism, respectively, as documented by
  the :option:`ctest --parallel` option.

@@ -18,4 +18,23 @@ up to 8 tests concurrently as if ``ctest`` were invoked with the

  In CMake 3.28 and earlier, an empty or ``0`` value was equivalent to ``1``.

+This environment variable is ignored if :option:`ctest --parallel` is given on
+the command line, and is overridden by the :preset:`testPresets.execution.jobs`
+field of a test preset, if that field is set.
+
+A test preset may also set this variable itself, using its own
+:preset:`testPresets.environment` field. Doing so is equivalent to setting
+the variable in the calling process's environment, except that it takes
+effect only for the preset's Test step, and only if ``execution.jobs`` is
+not also set (which would take precedence, as noted above).
+
+When a test preset is used by a :command:`ctest_test` command in a
+:ref:`CTest Script`, that command's own ``PARALLEL_LEVEL`` argument, if given,
+takes precedence over the preset's ``execution.jobs`` field. Together,
+``PARALLEL_LEVEL`` and ``execution.jobs`` take precedence not only over this
+environment variable, but over an explicit :option:`ctest --parallel` on the
+command line as well. This differs from a preset used directly via
+:option:`ctest --preset`, where an explicit ``--parallel`` always wins over the
+preset's ``execution.jobs`` field.
+
 See :manual:`ctest(1)` for more information on parallel test execution.
@@ -192,7 +192,8 @@ The options for running tests are:
    * Otherwise, if the value is ``0``, parallelism is unbounded.

 This option may instead be specified by the :envvar:`CTEST_PARALLEL_LEVEL`
- environment variable.
+ environment variable.  See the documentation of that variable for how it
+ interacts with a test preset's :preset:`testPresets.execution.jobs` field.

 This option can be used with the :prop_test:`PROCESSORS` test property.
 See the `Label and Subproject Summary`_.
@@ -567,7 +568,9 @@ The options for running tests are:
 .. versionadded:: 3.26

 This option can also be set by setting the :envvar:`CTEST_NO_TESTS_ACTION`
- environment variable.
+ environment variable. See the documentation of that variable for how it
+ interacts with a test preset's :preset:`testPresets.execution.noTestsAction`
+ field.

 .. option:: --collect-instrumentation <build>

@@ -20,6 +20,11 @@
  :ctest-option:`--parallel` on the command line. If the value is
  ``0``, it is equivalent to unbounded parallelism.

+  If set, this field takes precedence over the
+  :envvar:`CTEST_PARALLEL_LEVEL` environment variable,
+  including when that variable is instead set via this test
+  preset's own :preset:`testPresets.environment` field.
+
  .. presets-versionchanged:: 11

    This field can also be a string, in which case it must be
@@ -103,6 +108,13 @@
    Equivalent to passing :ctest-option:`--no-tests=ignore`
    on the command line.

+  If set, this field is overridden by the
+  :envvar:`CTEST_NO_TESTS_ACTION` environment variable,
+  including when that variable is instead set via this test
+  preset's own :preset:`testPresets.environment` field.
+  See :envvar:`CTEST_NO_TESTS_ACTION` for the full precedence
+  order.
+
 .. _`CMakePresets.testPresets.execution.testPassthroughArguments`:

 ``testPassthroughArguments``
@@ -1624,6 +1624,11 @@ properties:
                :ctest-option:`--parallel` on the command line. If the value is
                ``0``, it is equivalent to unbounded parallelism.

+                If set, this field takes precedence over the
+                :envvar:`CTEST_PARALLEL_LEVEL` environment variable,
+                including when that variable is instead set via this test
+                preset's own :preset:`testPresets.environment` field.
+
                .. presets-versionchanged:: 11

                  This field can also be a string, in which case it must be
@@ -1765,6 +1770,13 @@ properties:
                ``ignore``
                  Equivalent to passing :ctest-option:`--no-tests=ignore`
                  on the command line.
+
+                If set, this field is overridden by the
+                :envvar:`CTEST_NO_TESTS_ACTION` environment variable,
+                including when that variable is instead set via this test
+                preset's own :preset:`testPresets.environment` field.
+                See :envvar:`CTEST_NO_TESTS_ACTION` for the full precedence
+                order.
            testPassthroughArguments:
              since: 12
              type: array
@@ -2,8 +2,10 @@
   file LICENSE.rst or https://cmake.org/licensing for details.  */
 #include "cmCTestTestCommand.h"

+#include <cassert>
 #include <chrono>
 #include <cstdlib>
+#include <map>
 #include <ratio>
 #include <sstream>
 #include <string>
@@ -28,47 +30,48 @@

 using TestPreset = cmCMakePresetsGraph::TestPreset;

-std::unique_ptr<cmCTestGenericHandler> cmCTestTestCommand::InitializeHandler(
-  HandlerArguments& arguments, cmExecutionStatus& status) const
+cm::optional<cmCTestTestCommand::ResolvedTestPreset>
+cmCTestTestCommand::ResolveTestPreset(cmMakefile& mf,
+                                      std::string const& presetArg,
+                                      std::string const& presetsFileArg,
+                                      cmExecutionStatus& status) const
 {
-  cmMakefile& mf = status.GetMakefile();
-  auto& args = static_cast<TestArguments&>(arguments);
-
-  std::string const sourceDirectory =
-    mf.GetSafeDefinition("CTEST_SOURCE_DIRECTORY");
+  ResolvedTestPreset resolved;
+  resolved.SourceDirectory = mf.GetSafeDefinition("CTEST_SOURCE_DIRECTORY");

  // Presets file is set according to the following priority order:
  // 1) The PRESETS_FILE option to ctest_test()
  // 2) CTEST_PRESETS_FILE script variable
-  std::string const rawPresetsFile = !args.PresetsFile.empty()
-    ? args.PresetsFile
+  std::string const rawPresetsFile = !presetsFileArg.empty()
+    ? presetsFileArg
    : mf.GetSafeDefinition("CTEST_PRESETS_FILE");

-  std::string const presetsFile = rawPresetsFile.empty()
+  resolved.PresetsFile = rawPresetsFile.empty()
    ? ""
-    : cmSystemTools::CollapseFullPath(rawPresetsFile, sourceDirectory);
+    : cmSystemTools::CollapseFullPath(rawPresetsFile,
+                                      resolved.SourceDirectory);

  // Preset name is set according to the following priority order:
  // 1) The PRESET option to ctest_test()
  // 2) CTEST_TEST_PRESET script variable
  // 3) CTEST_PRESET script variable (a warning is emitted if no test preset
  //    exists with this name)
-  std::string effectivePreset = !args.Preset.empty() ? args.Preset
+  resolved.EffectivePreset = !presetArg.empty() ? presetArg
    : cmNonempty(mf.GetDefinition("CTEST_TEST_PRESET"))
    ? *mf.GetDefinition("CTEST_TEST_PRESET")
    : "";
-  if (effectivePreset.empty()) {
+  if (resolved.EffectivePreset.empty()) {
    cmValue v = mf.GetDefinition("CTEST_PRESET");
    if (cmNonempty(v)) {
      std::string presetError;
-      auto presetCheck =
-        TestPresetExists(*v, sourceDirectory, presetsFile, presetError);
+      auto presetCheck = TestPresetExists(*v, resolved.SourceDirectory,
+                                          resolved.PresetsFile, presetError);
      if (presetCheck == PresetCheckResult::ReadError) {
        status.SetError(cmStrCat('\n', presetError));
-        return nullptr;
+        return cm::nullopt;
      }
      if (presetCheck == PresetCheckResult::Found) {
-        effectivePreset = *v;
+        resolved.EffectivePreset = *v;
      } else {
        cmCTestLog(this->CTest, WARNING,
                   "No test preset named \""
@@ -77,30 +80,81 @@ std::unique_ptr<cmCTestGenericHandler> cmCTestTestCommand::InitializeHandler(
    }
  }

-  std::unique_ptr<cmCMakePresetsGraph> presetsGraph;
-  TestPreset const* expandedPreset = nullptr;
-  if (!effectivePreset.empty()) {
-    presetsGraph = cm::make_unique<cmCMakePresetsGraph>();
-    if (!presetsGraph->ReadProjectPresets(sourceDirectory, presetsFile)) {
-      status.SetError(cmStrCat("\n Could not read presets from \"",
-                               sourceDirectory, "\":\n ",
-                               presetsGraph->parseState.GetErrorMessage()));
-      return nullptr;
-    }
-
-    auto resolveResult =
-      presetsGraph->ResolvePreset(effectivePreset, presetsGraph->TestPresets);
-    auto resolveError = cmCMakePresetsGraph::FormatPresetError<TestPreset>(
-      resolveResult.StatusCode, resolveResult.ErrorPresetName,
-      sourceDirectory);
-    if (resolveError) {
-      status.SetError(*resolveError);
-      return nullptr;
-    }
-
-    expandedPreset = resolveResult.Preset;
+  if (resolved.EffectivePreset.empty()) {
+    return cm::optional<ResolvedTestPreset>(std::move(resolved));
  }

+  resolved.PresetsGraph = cm::make_unique<cmCMakePresetsGraph>();
+  if (!resolved.PresetsGraph->ReadProjectPresets(resolved.SourceDirectory,
+                                                 resolved.PresetsFile)) {
+    status.SetError(
+      cmStrCat("\n Could not read presets from \"", resolved.SourceDirectory,
+               "\":\n ", resolved.PresetsGraph->parseState.GetErrorMessage()));
+    return cm::nullopt;
+  }
+
+  auto resolveResult = resolved.PresetsGraph->ResolvePreset(
+    resolved.EffectivePreset, resolved.PresetsGraph->TestPresets);
+  auto resolveError = cmCMakePresetsGraph::FormatPresetError<TestPreset>(
+    resolveResult.StatusCode, resolveResult.ErrorPresetName,
+    resolved.SourceDirectory);
+  if (resolveError) {
+    status.SetError(*resolveError);
+    return cm::nullopt;
+  }
+  resolved.ExpandedPreset = resolveResult.Preset;
+
+  return cm::optional<ResolvedTestPreset>(std::move(resolved));
+}
+
+bool cmCTestTestCommand::ExecuteHandlerCommand(TestArguments& args,
+                                               cmExecutionStatus& status) const
+{
+  cmMakefile& mf = status.GetMakefile();
+
+  // Resolve the preset once here (rather than letting InitializeHandler()
+  // resolve it again) so its Environment can be applied for the duration of
+  // the whole test run, without parsing the presets file (and emitting errors,
+  // etc.) more than once.
+  this->CachedPresetResolution.emplace(
+    ResolveTestPreset(mf, args.Preset, args.PresetsFile, status));
+
+  cmSystemTools::SaveRestoreEnvironment restoreEnv;
+  if (*this->CachedPresetResolution &&
+      (*this->CachedPresetResolution)->ExpandedPreset) {
+    for (auto const& var :
+         (*this->CachedPresetResolution)->ExpandedPreset->Environment) {
+      if (var.second) {
+        cmSystemTools::PutEnv(cmStrCat(var.first, '=', *var.second));
+      }
+    }
+  }
+
+  // Special case for CTest environment variables specified in a preset, which
+  // are handled by cmCTest much earlier than here, and would otherwise be
+  // ignored.
+  if (!this->CTest->UpdateStateFromEnvironment()) {
+    return false;
+  }
+
+  return cmCTestHandlerCommand::ExecuteHandlerCommand(args, status);
+}
+
+std::unique_ptr<cmCTestGenericHandler> cmCTestTestCommand::InitializeHandler(
+  HandlerArguments& arguments, cmExecutionStatus& status) const
+{
+  cmMakefile& mf = status.GetMakefile();
+  auto& args = static_cast<TestArguments&>(arguments);
+
+  assert(this->CachedPresetResolution);
+  cm::optional<ResolvedTestPreset> resolvedPreset =
+    std::move(*this->CachedPresetResolution);
+  this->CachedPresetResolution.reset();
+  if (!resolvedPreset) {
+    return nullptr;
+  }
+  TestPreset const* expandedPreset = resolvedPreset->ExpandedPreset;
+
  cmValue ctestTimeout = mf.GetDefinition("CTEST_TEST_TIMEOUT");

  cmDuration timeout;
@@ -14,16 +14,29 @@

 #include "cmArgumentParser.h"
 #include "cmArgumentParserTypes.h"
+#include "cmCMakePresetsGraph.h"
 #include "cmCTestHandlerCommand.h"

 class cmExecutionStatus;
 class cmCTestTestHandler;
+class cmMakefile;

 class cmCTestTestCommand : public cmCTestHandlerCommand
 {
 public:
  using cmCTestHandlerCommand::cmCTestHandlerCommand;

+  cmCTestTestCommand(cmCTestTestCommand const& other)
+    : cmCTestHandlerCommand(other)
+  {
+  }
+
+  cmCTestTestCommand& operator=(cmCTestTestCommand const& other)
+  {
+    cmCTestHandlerCommand::operator=(other);
+    return *this;
+  }
+
 protected:
  struct TestArguments : HandlerArguments
  {
@@ -53,6 +66,9 @@ protected:
    std::string PresetsFile;
  };

+  bool ExecuteHandlerCommand(TestArguments& args,
+                             cmExecutionStatus& status) const;
+
  template <typename Args>
  static auto MakeTestParser() -> cmArgumentParser<Args>
  {
@@ -87,6 +103,25 @@ protected:
 private:
  std::string GetName() const override { return "ctest_test"; }

+  struct ResolvedTestPreset
+  {
+    std::string SourceDirectory;
+    std::string PresetsFile;
+    std::string EffectivePreset;
+    std::unique_ptr<cmCMakePresetsGraph> PresetsGraph;
+    cmCMakePresetsGraph::TestPreset const* ExpandedPreset = nullptr;
+  };
+
+  cm::optional<ResolvedTestPreset> ResolveTestPreset(
+    cmMakefile& mf, std::string const& presetArg,
+    std::string const& presetsFileArg, cmExecutionStatus& status) const;
+
+  // Set by ExecuteHandlerCommand() (resolves the preset once, up
+  // front) and consumed by InitializeHandler() to avoid parsing the presets
+  // file (and emitting error messages, etc.) more than once.
+  mutable cm::optional<cm::optional<ResolvedTestPreset>>
+    CachedPresetResolution;
+
  virtual std::unique_ptr<cmCTestTestHandler> InitializeActualHandler(
    HandlerArguments& arguments, cmExecutionStatus& status) const;

@@ -374,6 +374,48 @@ void cmCTest::SetParallelLevel(cm::optional<size_t> level)
  this->Impl->ParallelLevel = level;
 }

+bool cmCTest::UpdateStateFromEnvironment()
+{
+  // handle CTEST_PARALLEL_LEVEL environment variable
+  if (!this->Impl->ParallelLevelSetInCli) {
+    if (cm::optional<std::string> parallelEnv =
+          cmSystemTools::GetEnvVar("CTEST_PARALLEL_LEVEL")) {
+      if (parallelEnv->empty() ||
+          parallelEnv->find_first_not_of(" \t") == std::string::npos) {
+        // An empty value tells ctest to choose a default.
+        this->SetParallelLevel(cm::nullopt);
+      } else {
+        // A non-empty value must be a non-negative integer.
+        // Otherwise, ignore it.
+        unsigned long plevel = 0;
+        if (cmStrToULong(*parallelEnv, &plevel)) {
+          this->SetParallelLevel(plevel);
+        }
+      }
+    }
+  }
+
+  // handle CTEST_NO_TESTS_ACTION environment variable
+  if (!this->Impl->NoTestsModeSetInCli) {
+    std::string action;
+    if (cmSystemTools::GetEnv("CTEST_NO_TESTS_ACTION", action) &&
+        !action.empty()) {
+      if (action == "error"_s) {
+        this->Impl->NoTestsMode = cmCTest::NoTests::Error;
+      } else if (action == "ignore"_s) {
+        this->Impl->NoTestsMode = cmCTest::NoTests::Ignore;
+      } else {
+        cmCTestLog(this, ERROR_MESSAGE,
+                   "Unknown value for CTEST_NO_TESTS_ACTION: '" << action
+                                                                << '\'');
+        return false;
+      }
+    }
+  }
+
+  return true;
+}
+
 unsigned long cmCTest::GetTestLoad() const
 {
  return this->Impl->TestLoad;
@@ -2629,41 +2671,8 @@ int cmCTest::Run(std::vector<std::string> const& args)
    }
  }

-  // handle CTEST_PARALLEL_LEVEL environment variable
-  if (!this->Impl->ParallelLevelSetInCli) {
-    if (cm::optional<std::string> parallelEnv =
-          cmSystemTools::GetEnvVar("CTEST_PARALLEL_LEVEL")) {
-      if (parallelEnv->empty() ||
-          parallelEnv->find_first_not_of(" \t") == std::string::npos) {
-        // An empty value tells ctest to choose a default.
-        this->SetParallelLevel(cm::nullopt);
-      } else {
-        // A non-empty value must be a non-negative integer.
-        // Otherwise, ignore it.
-        unsigned long plevel = 0;
-        if (cmStrToULong(*parallelEnv, &plevel)) {
-          this->SetParallelLevel(plevel);
-        }
-      }
-    }
-  }
-
-  // handle CTEST_NO_TESTS_ACTION environment variable
-  if (!this->Impl->NoTestsModeSetInCli) {
-    std::string action;
-    if (cmSystemTools::GetEnv("CTEST_NO_TESTS_ACTION", action) &&
-        !action.empty()) {
-      if (action == "error"_s) {
-        this->Impl->NoTestsMode = cmCTest::NoTests::Error;
-      } else if (action == "ignore"_s) {
-        this->Impl->NoTestsMode = cmCTest::NoTests::Ignore;
-      } else {
-        cmCTestLog(this, ERROR_MESSAGE,
-                   "Unknown value for CTEST_NO_TESTS_ACTION: '" << action
-                                                                << '\'');
-        return 1;
-      }
-    }
+  if (!this->UpdateStateFromEnvironment()) {
+    return 1;
  }

  // Passthrough arguments (after --) are only supported in direct test
@@ -102,6 +102,13 @@ public:
  cm::optional<size_t> GetParallelLevel() const;
  void SetParallelLevel(cm::optional<size_t> level);

+  /**
+   * Check environment variables controlling CTest's behavior and update state
+   * accordingly, when the corresponding command-line toggles are unset.
+   * Returns false if there are errors during parsing.
+   */
+  bool UpdateStateFromEnvironment();
+
  unsigned long GetTestLoad() const;
  void SetTestLoad(unsigned long);

@@ -374,7 +374,7 @@ run_Parallel(env-0)
 set(ENV{CTEST_PARALLEL_LEVEL} 3)
 run_Parallel(env-3)
 unset(ENV{CTEST_PARALLEL_LEVEL})
-unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING)
+unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING})

 function(run_TestLoad name load)
  set(RunCMake_TEST_BINARY_DIR ${RunCMake_BINARY_DIR}/TestLoad)
@@ -916,6 +916,84 @@ block()
    -V)
 endblock()

+# Environment variables from a test preset selected by -D CTEST_PRESET are
+# applied when dashboard mode runs the Test step.
+block()
+  set(src "${RunCMake_BINARY_DIR}/TestPresetCLIVarEnvironment")
+  set(bin "${RunCMake_BINARY_DIR}/TestPresetCLIVarEnvironment-build")
+  file(REMOVE_RECURSE "${src}" "${bin}")
+  file(MAKE_DIRECTORY "${src}" "${bin}")
+  configure_file("${RunCMake_SOURCE_DIR}/TestPresetCLIVarEnvironment-CMakePresets.json.in"
+                 "${src}/CMakePresets.json" @ONLY)
+  file(WRITE "${bin}/DartConfiguration.tcl"
+    "BuildDirectory: ${bin}\n"
+    "SourceDirectory: ${src}\n")
+  file(WRITE "${bin}/CTestTestfile.cmake" "
+add_test(print-env \"${CMAKE_COMMAND}\" -E environment)
+set_tests_properties(print-env PROPERTIES PASS_REGULAR_EXPRESSION \"DASHBOARD_PRESET_ENV=from-preset\")
+")
+  set(RunCMake_TEST_SOURCE_DIR "${src}")
+  set(RunCMake_TEST_BINARY_DIR "${bin}")
+  set(RunCMake_TEST_NO_CLEAN 1)
+  run_cmake_command(TestPresetCLIVarEnvironment
+    ${CMAKE_CTEST_COMMAND}
+    -M Experimental
+    -D "CTEST_PRESET=my-test-preset"
+    -T Test
+    -V)
+endblock()
+
+# CTEST_PARALLEL_LEVEL set by a test preset's environment is respected.
+block()
+  set(src "${RunCMake_BINARY_DIR}/TestPresetParallelLevelEnvironment")
+  set(bin "${RunCMake_BINARY_DIR}/TestPresetParallelLevelEnvironment-build")
+  file(REMOVE_RECURSE "${src}" "${bin}")
+  file(MAKE_DIRECTORY "${src}" "${bin}")
+  configure_file("${RunCMake_SOURCE_DIR}/TestPresetParallelLevelEnvironment-CMakePresets.json.in"
+                 "${src}/CMakePresets.json" @ONLY)
+  file(WRITE "${bin}/DartConfiguration.tcl"
+    "BuildDirectory: ${bin}\n"
+    "SourceDirectory: ${src}\n")
+  file(WRITE "${bin}/CTestTestfile.cmake" "
+foreach(i RANGE 1 6)
+  add_test(test\${i} \"${CMAKE_COMMAND}\" -E true)
+endforeach()
+")
+  set(RunCMake_TEST_SOURCE_DIR "${src}")
+  set(RunCMake_TEST_BINARY_DIR "${bin}")
+  set(RunCMake_TEST_NO_CLEAN 1)
+  # Spoof a number of processors to make these tests predictable.
+  set(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING} 1)
+  run_cmake_command(TestPresetParallelLevelEnvironment
+    ${CMAKE_CTEST_COMMAND}
+    -M Experimental
+    -D "CTEST_PRESET=my-test-preset"
+    -T Test)
+  unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING})
+endblock()
+
+# CTEST_NO_TESTS_ACTION set by a test preset's environment is respected.
+block()
+  set(src "${RunCMake_BINARY_DIR}/TestPresetNoTestsActionEnvironment")
+  set(bin "${RunCMake_BINARY_DIR}/TestPresetNoTestsActionEnvironment-build")
+  file(REMOVE_RECURSE "${src}" "${bin}")
+  file(MAKE_DIRECTORY "${src}" "${bin}")
+  configure_file("${RunCMake_SOURCE_DIR}/TestPresetNoTestsActionEnvironment-CMakePresets.json.in"
+                 "${src}/CMakePresets.json" @ONLY)
+  file(WRITE "${bin}/DartConfiguration.tcl"
+    "BuildDirectory: ${bin}\n"
+    "SourceDirectory: ${src}\n")
+  file(WRITE "${bin}/CTestTestfile.cmake" "")
+  set(RunCMake_TEST_SOURCE_DIR "${src}")
+  set(RunCMake_TEST_BINARY_DIR "${bin}")
+  set(RunCMake_TEST_NO_CLEAN 1)
+  run_cmake_command(TestPresetNoTestsActionEnvironment
+    ${CMAKE_CTEST_COMMAND}
+    -M Experimental
+    -D "CTEST_PRESET=my-test-preset"
+    -T Test)
+endblock()
+
 # Test --output-junit
 function(run_output_junit)
  set(RunCMake_TEST_BINARY_DIR ${RunCMake_BINARY_DIR}/output-junit)
@@ -0,0 +1,24 @@
+{
+  "version": 3,
+  "cmakeMinimumRequired": {
+    "major": 3,
+    "minor": 21,
+    "patch": 0
+  },
+  "configurePresets": [
+    {
+      "name": "my-configure-preset",
+      "generator": "@RunCMake_GENERATOR@",
+      "binaryDir": "${sourceDir}/build"
+    }
+  ],
+  "testPresets": [
+    {
+      "name": "my-test-preset",
+      "configurePreset": "my-configure-preset",
+      "environment": {
+        "DASHBOARD_PRESET_ENV": "from-preset"
+      }
+    }
+  ]
+}
@@ -0,0 +1 @@
+DASHBOARD_PRESET_ENV=from-preset
@@ -0,0 +1,24 @@
+{
+  "version": 3,
+  "cmakeMinimumRequired": {
+    "major": 3,
+    "minor": 21,
+    "patch": 0
+  },
+  "configurePresets": [
+    {
+      "name": "my-configure-preset",
+      "generator": "@RunCMake_GENERATOR@",
+      "binaryDir": "${sourceDir}/build"
+    }
+  ],
+  "testPresets": [
+    {
+      "name": "my-test-preset",
+      "configurePreset": "my-configure-preset",
+      "environment": {
+        "CTEST_NO_TESTS_ACTION": "error"
+      }
+    }
+  ]
+}
@@ -0,0 +1 @@
+8
@@ -0,0 +1 @@
+No tests were found!!!
@@ -0,0 +1,24 @@
+{
+  "version": 3,
+  "cmakeMinimumRequired": {
+    "major": 3,
+    "minor": 21,
+    "patch": 0
+  },
+  "configurePresets": [
+    {
+      "name": "my-configure-preset",
+      "generator": "@RunCMake_GENERATOR@",
+      "binaryDir": "${sourceDir}/build"
+    }
+  ],
+  "testPresets": [
+    {
+      "name": "my-test-preset",
+      "configurePreset": "my-configure-preset",
+      "environment": {
+        "CTEST_PARALLEL_LEVEL": "3"
+      }
+    }
+  ]
+}
@@ -0,0 +1,6 @@
+Test project [^
+]*/Tests/RunCMake/CTestCommandLine/TestPresetParallelLevelEnvironment-build
+    Start [0-9]+: test[0-9]+
+    Start [0-9]+: test[0-9]+
+    Start [0-9]+: test[0-9]+
+1/6 Test #[0-9]+: test[0-9]+ \.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.\.   Passed +[0-9\.]+ sec
@@ -119,7 +119,7 @@ function(run_CTestJobServer)
  run_make_rule(CTestJobServer NoTests 2)
  run_make_rule(CTestJobServer Tests 2)
  run_make_rule(CTestJobServer Tests 3)
-  unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING)
+  unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING})
 endfunction()

 # Jobservers are currently only supported by GNU makes, except MSYS2 make
@@ -126,7 +126,7 @@ set_property(TEST test6 PROPERTY DEPENDS test1)
 ]])
 run_ctest_test(SerialOrder INCLUDE test)
 unset(CASE_CMAKELISTS_SUFFIX_CODE)
-unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING)
+unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING})

 set(CASE_CMAKELISTS_SUFFIX_CODE [[
 add_test(NAME skip COMMAND ${CMAKE_COMMAND} -E true)
@@ -164,7 +164,7 @@ set(ENV{CTEST_PARALLEL_LEVEL} 3)
 run_ctest_test(ParallelEnv3  INCLUDE test)
 unset(ENV{CTEST_PARALLEL_LEVEL})
 unset(CASE_CMAKELISTS_SUFFIX_CODE)
-unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING)
+unset(ENV{__CTEST_FAKE_PROCESSOR_COUNT_FOR_TESTING})

 # Tests for the 'Test Load' feature of ctest
 #