8000 fix: run cleanup and destroy when a verifier fails by jeffcpullen · Pull Request #4678 · ansible/molecule · GitHub
[go: up one dir, main page]

Skip to content

fix: run cleanup and destroy when a verifier fails - #4678

Merged
tanwigeetika1618 merged 1 commit into
ansible:mainfrom
jeffcpullen:fix/verify-failure-runs-destroy
Aug 6, 2026
Merged

tanwigeetika1618 merged 1 commit into
ansible:mainfrom
jeffcpullen:fix/verify-failure-runs-destroy

Conversation

@jeffcpullen
@jeffcpullen jeffcpullen commented Aug 4, 2026 •
Copy link
Copy Markdown
Contributor

Fixes #4677.

Problem

When a verifier fails during molecule test, molecule exits immediately after the verify action and never runs cleanup or destroy — even though --destroy=always is the default. Instances are left running.

This contradicts the documented molecule test behavior, which is to stop on the first failure but still tear down unless --destroy=never (docs/workflow.md). A verify failure was the one path that stopped and skipped teardown.

Root cause

The testinfra verifier reports a non-zero return code by calling util.sysexit_with_message(), which raises SystemExit:

# src/molecule/verifier/testinfra.py (before)
else:
    msg = "Verifier tests failed"
    util.sysexit_with_message(msg, code=result.returncode)

Every other action in a scenario sequence signals failure by raising ScenarioFailureError (e.g. a failed playbook in provisioner/ansible_playbook.py). The --destroy=always teardown in command/base.py::_run_scenarios only catches ScenarioFailureError:

try:
    execute_scenario(scenario, shared_state=scenarios.shared_state)
    scenarios.results.append(scenario.results)
except ScenarioFailureError:
    if command_args.get("destroy") == "always":
        _handle_scenario_failure(...)   # runs cleanup + destroy
    raise

SystemExit is not a ScenarioFailureError (it isn't even an Exception), so it passes straight through the handler and the process exits before cleanup/destroy run. testinfra was the only action in the whole verify/sequence path using sysexit instead of raising.

Fix

Raise ScenarioFailureError with the verifier's return code instead of calling sysexit, and record the failure on the scenario results. A verify failure now flows through the same teardown path as a failed playbook action, so cleanup and destroy run under --destroy=always, and the failure is still surfaced with the same exit code.

# src/molecule/verifier/testinfra.py (after)
else:
    msg = "Verifier tests failed"
    self._config.scenario.results.add_completion(CompletionState.failed(note=msg))
    raise ScenarioFailureError(message=msg, code=result.returncode)

The add_completion(...) call mirrors what the ansible verifier already does (provisioner/ansible_playbook.py). Without it, a molecule test run's recap renders a failed verify as "successful", because an action with no recorded state defaults to successful — so this keeps the recap honest.

Two things worth noting for review:

  • Raising ScenarioFailureError also means a verify failure is now caught by the parallel worker's except Exception handler (worker.py) instead of escaping it as a SystemExit (a BaseException) — so --workers runs report the failed scenario cleanly rather than losing the worker.
  • Because ScenarioFailureError logs its message at CRITICAL on construction, the "Verifier tests failed" line now logs at CRITICAL (and, like every other scenario failure, is echoed once more at ERROR as the run exits). This matches how failed playbook actions already report; it is a small, deliberate consistency change from the previous ERROR-only line.

Reproduction

I used the minimal scenario from #4677 (default driver, create → verify → destroy, an intentionally-failing Testinfra test, and marker files written by the create/destroy playbooks), and ran molecule test with molecule installed from source — once with this change reverted, once with it applied.

Before the fix — molecule test stops at verify:

INFO     [default > create] Executed: Successful
INFO     [default > verify] Executing Testinfra tests found in .../tests/...
FAILED tests/test_default.py::test_verify_fails_intentionally
ERROR    Verifier tests failed

MOLECULE_EXIT=1
create.marker=present
destroy.marker=absent          <-- destroy skipped

After the fix — the failure still exits non-zero, but cleanup and destroy run:

INFO     [default > create] Executed: Successful
INFO     [default > verify] Executing Testinfra tests found in .../tests/...
CRITICAL Verifier tests failed
WARNING  [default > verify] An error occurred during the test sequence action: 'verify'. Cleaning up.
INFO     [default > cleanup] Executing
INFO     [default > destroy] Executing
...
MOLECULE_EXIT=1
create.marker=present
destroy.marker=present         <-- destroy now runs

Tests

tests/unit/verifier/test_testinfra.py had a test named test_testinfra_executes_catches_and_exits_return_code that stubbed run_command to raise SystemExit, so it never actually exercised the non-zero-return-code branch. I replaced it with a test that drives a real non-zero return code and asserts the verifier raises ScenarioFailureError carrying that code — covering the changed branch directly.

I also added tests/unit/test_click_command_ex.py::test_click_command_ex_scenario_failure_exits_with_code, which asserts a ScenarioFailureError carrying a non-1 code exits the process with that same code — so a failed verifier's return code is preserved end to end, not flattened to a generic 1.

The existing --destroy=always handling in tests/unit/command/test_base.py already covers ScenarioFailureError triggering teardown.

Full unit suite green: 866 passed, 6 skipped (tox -e py -- tests/unit/).

Summary by CodeRabbit

  • Bug Fixes
    • Verifier failures now raise a dedicated scenario failure error instead of terminating through a system-exit path.
    • Scenario failures correctly preserve the failed completion status.
    • Command-line execution now exits with the appropriate status code for scenario failures.

When a verifier reported a non-zero return code, molecule exited
immediately after the verify action and skipped cleanup and destroy,
even though --destroy=always is the default. Instances were left
running.

The testinfra verifier signaled failure by calling
util.sysexit_with_message(), which raises SystemExit. Every other
action in a scenario sequence raises ScenarioFailureError on failure,
and the --destroy=always teardown in _run_scenarios() only catches
ScenarioFailureError. SystemExit passed straight through it, so
cleanup and destroy never ran. testinfra was the only action in the
verify/sequence path using sysexit instead of raising.

Raise ScenarioFailureError with the verifier's return code instead, so
a verify failure flows through the same teardown path as a failed
playbook action. Cleanup and destroy now run, and the failure is still
surfaced with the same exit code. Also record the failure on the
scenario results, matching the ansible verifier, so the molecule test
recap reports the failed verify as failed rather than defaulting an
unrecorded action to successful.

Fixes: ansible#4677
Assisted-by: Claude (Anthropic)
Signed-off-by: Jeff Pullen <9343691+jeffcpullen@users.noreply.github.com>
@coderabbitai
coderabbitai Bot commented Aug 4, 2026 •
Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5cfe79e0-85c3-4d6c-9c80-0587e87356d2

📥 Commits

Reviewing files that changed from the base of the PR and between 2733bcf and 4594a2f.

📒 Files selected for processing (3)
  • src/molecule/verifier/testinfra.py
  • tests/unit/test_click_command_ex.py
  • tests/unit/verifier/test_testinfra.py

📝 Walkthrough

Walkthrough

Testinfra verifier failures now record a failed completion and raise ScenarioFailureError with the verifier return code. Unit tests cover both verifier execution and click command exit handling.

Changes

Testinfra failure handling

Layer / File(s) Summary
Raise ScenarioFailureError for verifier failures
src/molecule/verifier/testinfra.py
Testinfra.execute documents and raises ScenarioFailureError after recording a failed verifier completion.
Validate failure and exit-code propagation
tests/unit/verifier/test_testinfra.py, tests/unit/test_click_command_ex.py
Tests verify the exception code and failed verify result. Click command handling passes the exception code to util.sysexit.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: sathyapramod

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: ensuring cleanup and destroy run after verifier failure.
Linked Issues check ✅ Passed The change addresses issue [#4677] by preserving verifier failures while routing execution through cleanup and destroy.
Out of Scope Changes check ✅ Passed The implementation and tests are limited to verifier failure handling, teardown execution, and exit-code preservation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@tanwigeetika1618
tanwigeetika1618 merged commit c7957b1 into ansible:main Aug 6, 2026
20 checks passed
@github-project-automation github-project-automation Bot moved this from Review to Done in 🧰 devtools project board Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Testinfra verification failure skips cleanup and destroy during molecule test

2 participants

0