8000 Remove dependency on sh library by ssbarnea · Pull Request #2917 · ansible/molecule · GitHub
[go: up one dir, main page]

Skip to content

Remove dependency on sh library - #2917

Merged
ssbarnea merged 1 commit into
masterfrom
rm/sh
Oct 27, 2020
Merged

ssbarnea merged 1 commit into
masterfrom
rm/sh

Conversation

@ssbarnea
@ssbarnea ssbarnea commented Oct 27, 2020 •
Copy link
Copy Markdown
Member
  • Main change is replace use of sh library with subprocess-tee while adopting a loose-coupling approach, so we do not rely on implementation details of the subprocess library.
  • This will be part of 3.2.0 major release as it will likely break at least some plugins that are using molecule.util.run_command or the bake().
  • It will initially be tagged as 3.2.0a0 in order to give time for plugin projects to adapt their test suites.
  • This enables us to capture and process the output of the executed commands while still displaying it in real time, still no extra processing is done in this change.

Fixed: #2678
Related: amoffat/sh#450 amoffat/sh#512

@ssbarnea ssbarnea added the major label Oct 27, 2020
@ssbarnea ssbarnea added this to the 3.2.0 milestone Oct 27, 2020
@ssbarnea
ssbarnea force-pushed the rm/sh branch 2 times, most recently from f6acc7d to 8882673 Compare October 27, 2020 14:21
@ssbarnea ssbarnea changed the title WIP: Remove use of sh library Remove dependency on sh library Oct 27, 2020
@ssbarnea
ssbarnea marked this pull request as ready for review October 27, 2020 14:23
@ghost
ghost commented Oct 27, 2020
Copy link
Copy Markdown

I like the simplicity of subprocess-tee.

@ssbarnea
ssbarnea merged commit 1cefe5c into master Oct 27, 2020
@ssbarnea
ssbarnea deleted the rm/sh branch October 27, 2020 16:37
ssbarnea added a commit to ansible-community/molecule-docker that referenced this pull request Oct 28, 2020
ssbarnea added a commit to ansible-community/molecule-docker that referenced this pull request Oct 28, 2020
ssbarnea added a commit to ansible-community/molecule-podman that referenced this pull request Oct 28, 2020
ssbarnea added a commit to ansible-community/molecule-podman that referenced this pull request Oct 28, 2020
ssbarnea added a commit to ansible-community/molecule-podman that referenced this pull request Oct 28, 2020
ssbarnea added a commit to ansible-community/molecule-vagrant that referenced this pull request Oct 28, 2020
ssbarnea added a commit to ansible-community/molecule-vagrant that referenced this pull request Oct 28, 2020
ssbarnea added a commit to ansible-community/molecule-vagrant that referenced this pull request Oct 28, 2020
@decentral1se
Copy link
Copy Markdown
Contributor

👏 👏 👏

shvenkat-rh pushed a commit that referenced this pull request Oct 28, 2026
# fix: pass provisioner verbosity options to ansible

Fixes #3159.

## Problem

The Ansible provisioner's verbosity option is
[documented](https://docs.ansible.com/projects/molecule/configuration/#provisioner-pre-ansible-native)
as a way to raise Ansible verbosity, but it is silently dropped. The
flag never reaches the `ansible-playbook` (or `ansible-navigator`)
command line, so the run stays at verbosity 0. The same drop affects
`molecule --debug`, which raises verbosity the same way internally, so
`--debug` ran Ansible at verbosity 0 as well.

The scenario below sets the documented option and runs converge on it.
On a clean checkout the run shows none of the markers Ansible prints
only at verbosity 1 and above, such as the interpreter banner and the
per-task path lines.

```yaml
# molecule/default/molecule.yml
driver:
  name: default
  options:
    managed: false
platforms:
  - name: instance
provisioner:
  name: ansible
  options:
    vvv: true
  inventory:
    host_vars:
      instance:
        ansible_connection: local
```

#### Molecule converge - missing expected 'task path:' debug output
```console
# molecule converge
<snip>
TASK [Say hello] ***************************************************************
ok: [instance] => {
    "msg": "hello-from-converge"
}
<snip>
```

#### Molecule converge with --debug - missing expected 'task path:'
debug output
```console
# molecule --debug converge -s scenario_debug
<snip>
TASK [Say hello] ***************************************************************
ok: [instance] => {
    "msg": "hello-from-converge"
}
<snip>
```

No verbosity banner, no `task path:`. Neither `-vvv` nor `--debug` was
applied.

## Root cause

In `src/molecule/provisioner/ansible_playbook.py`, `bake()` computes the
flag correctly (`verbose_flag = util.verbose_flag(options)` yields
`["-vvv"]` and strips the `v*` keys from `options`). But
`_build_command()` then emits it through `util.bool2args(verbose_flag)`,
and `util.bool2args` is a stub that unconditionally returns `[]`
regardless of its argument. The finished `["-vvv"]` is computed and
thrown away.

`bool2args` was born a no-op in the 2020 refactor that moved command
building off the `sh` library (#2917). The same change handled the two
sibling call sites (the ansible-galaxy dependency step and the testinfra
verifier) correctly, spreading `*verbose_flag` inline, which is why only
`provisioner.options` verbosity is affected. A later navigator-executor
rewrite (#4457) copied the broken line into the ansible-navigator
branch, so both backends now drop it. `bool2args` has no other callers.
The option worked from 2017 until #2917 and is still the documented way
to set per-scenario verbosity, so this restores it rather than removing
the docs.

The gap stayed quiet for ~5 years because the tests assert the
*computed* value (`verbose_flag()` returns `["-vvv"]`) and the config
object (`options` contains `vvv: True`), but never the built command.
The galaxy path already has a bake-level test asserting `-v` reaches the
command. The provisioner lacked the equivalent.

## Changes

- Spread `*verbose_flag` directly in both backend branches of
`_build_command()` (ansible-playbook and ansible-navigator), matching
the galaxy dependency and testinfra verifier paths that already work.
- Remove `util.bool2args`, whose only two callers were these lines.
- Add bake-level tests asserting the verbosity flag reaches the built
command, parametrized over `v` / `vv` / `vvv` for both backends and
covering the `--debug` path.

With the fix, the same scenario raises verbosity: the `ansible-playbook
[core ...]` banner and `task path:` lines now appear.

```console
# molecule converge
<snip>
TASK [Say hello] ***************************************************************
task path: molecule/default/converge.yml:5
ok: [instance] => {
    "msg": "hello-from-converge"
}
<snip>
```

`molecule --debug` raises verbosity through the same internal path, and
was dropped the same way. Run against the same setup, minus any
`provisioner.options`, so the `--debug` flag is the only variable.
Before the fix it still runs at verbosity 0. With the fix, the banner
and per-task path appear.



```console
# molecule --debug converge -s scenario_debug
<snip>
TASK [Say hello] ***************************************************************
task path: molecule/scenario_debug/converge.yml:5
ok: [instance] => {
    "msg": "hello-from-converge"
}
<snip>
```

Environment for all runs: ansible-core 2.21.4, Python 3.13. Before is a
clean `upstream/main`. After is that same base plus this commit.

The transcripts above exercise the `ansible-playbook` backend. The
`ansible-navigator` backend carries the identical fix and is covered by
the parametrized bake test that asserts the flag reaches its command
line.

## Risk / Impact

None known

## Test plan

- [x] `tox -e lint` green
- [x] `tox -e pkg` green
- [x] `tox -e py` full suite green, same skips as `main` (918 passed / 7
skipped, vs 911 / 7 on `main`)
- [x] New bake tests red on `main`, green on the branch, one per backend
(7 fail on `main`: `v`/`vv`/`vvv` for each backend plus the `--debug`
test)
- [x] Coverage flat vs `main` (90% on both)
- [x] Docs unchanged, the documented `provisioner.options.vvv` example
now behaves as documented

## Related

- The two documented paths that already raise verbosity today: `molecule
-v/-vv/-vvv converge`, and `provisioner.env: {ANSIBLE_VERBOSITY: 3}`.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Verbosity settings are now applied correctly to commands run by
Ansible Playbook and Ansible Navigator. Selecting `v`, `vv`, or `vvv`
produces the corresponding verbosity level, and enabling debug mode
applies maximum verbosity. Command output now reflects the selected
level in both execution modes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jeff Pullen <9343691+jeffcpullen@users.noreply.github.com>
Co-authored-by: Jeff Pullen <9343691+jeffcpullen@users.noreply.github.com>
Co-authored-by: Hrithik Gavankar <hrithikgavankar8582@gmail.com>
Co-authored-by: RAKESH S <124438543+rockygeekz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove sh package as a molecule dependency

2 participants

0