Command Runner (#4476)

* initial commit, passing unit tests

* passing one very silly integration test

* multiple changes:

- updated copyright year
- cmd_runner
  - added fmt_optval
  - created specific exceptions
  - fixed bug in context class where values from module params were not
    being used for resolving cmd arguments
  - changed order of class declaration for readability purpose
- tests
  - minor improvements in integration test code
  - removed some extraneous code in msimple.yml
  - minor improvements in unit tests
  - added few missing cases to unit test

* multiple changes

cmd_runner.py

- renamed InvalidParameterName to MissingArgumentFormat
  - improved exception parameters
- added repr and str to all exceptions
- added unpacking decorator for fmt functions
- CmdRunner
  - improved parameter validation
- _CmdRunnerContext
  - Context runs must now pass named arguments
  - Simplified passing of additional arguments to module.run_command()
  - Provided multiple context variables with info about the run

Integration tests

- rename msimple.py to cmd_echo.py for clarity
- added more test cases

* cmd_runner: env update can be passed to runner

* adding runner context info to output

* added comment on OrderedDict

* wrong variable

* refactored all fmt functions into static methods of a class

Imports should be simpler now, only one object fmt, with attr access to all callables

* added unit tests for CmdRunner

* fixed sanity checks

* fixed mock imports

* added more unit tests for CmdRunner

* terminology consistency

* multiple adjustments:

- remove extraneous imports
- renamed some variables
- added wrapper around arg formatters to handle individual arg ignore_none behaviour

* removed old code commented out in test

* multiple changes:

- ensure fmt functions return list of strings
- renamed fmt parameter from `option` to `args`
- renamed fmt.mapped to fmt.as_map
- simplified fmt.as_map
- added tests for fmt.as_fixed

* more improvements in formats

* fixed sanity

* args_order can be a string (to be split())

and improved integration test

* simplified integration test

* removed overkill str() on values - run_command does that for us

* as_list makes more sense than as_str in that context

* added changelog fragment

* Update plugins/module_utils/cmd_runner.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* adjusted __repr__ output for the exceptions

* added superclass object to classes

* added additional comment on the testcase sample/example

* suggestion from PR

Co-authored-by: Felix Fontein <felix@fontein.de>

tests/integration/targets/cmd_runner/aliases

@@ -0,0 +1 @@
+shippable/posix/group2

tests/integration/targets/cmd_runner/library/cmd_echo.py

@@ -0,0 +1,78 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# (c) 2022, Alexei Znamensky <russoz@gmail.com>
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+import sys
+
+DOCUMENTATION = '''
+module: cmd_echo
+author: "Alexei Znamensky (@russoz)"
+short_description: Simple module for testing
+description:
+  - Simple module test description.
+options:
+  command:
+    description: aaa
+    type: list
+    elements: str
+    required: true
+  arg_formats:
+    description: bbb
+    type: dict
+    required: true
+  arg_order:
+    description: ccc
+    type: raw
+    required: true
+  arg_values:
+    description: ddd
+    type: list
+    required: true
+  aa:
+    description: eee
+    type: raw
+'''
+
+EXAMPLES = ""
+
+RETURN = ""
+
+from ansible.module_utils.basic import AnsibleModule
+from ansible_collections.community.general.plugins.module_utils.cmd_runner import CmdRunner, fmt
+
+
+def main():
+    module = AnsibleModule(
+        argument_spec=dict(
+            arg_formats=dict(type="dict", default={}),
+            arg_order=dict(type="raw", required=True),
+            arg_values=dict(type="dict", default={}),
+            aa=dict(type="raw"),
+        ),
+    )
+    p = module.params
+
+    arg_formats = {}
+    for arg, fmt_spec in p['arg_formats'].items():
+        func = getattr(fmt, fmt_spec['func'])
+        args = fmt_spec.get("args", [])
+
+        arg_formats[arg] = func(*args)
+
+    runner = CmdRunner(module, ['echo', '--'], arg_formats=arg_formats)
+
+    info = None
+    with runner.context(p['arg_order']) as ctx:
+        result = ctx.run(**p['arg_values'])
+        info = ctx.run_info
+    rc, out, err = result
+
+    module.exit_json(rc=rc, out=out, err=err, info=info)
+
+
+if __name__ == '__main__':
+    main()

tests/integration/targets/cmd_runner/tasks/main.yml

@@ -0,0 +1,7 @@
+# (c) 2022, Alexei Znamensky
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+- name: parameterized test cmd_echo
+  ansible.builtin.include_tasks:
+    file: test_cmd_echo.yml
+  loop: "{{ cmd_echo_tests }}"

tests/integration/targets/cmd_runner/tasks/test_cmd_echo.yml

@@ -0,0 +1,13 @@
+---
+- name: test cmd_echo [{{ item.name }}]
+  cmd_echo:
+    arg_formats: "{{ item.arg_formats|default(omit) }}"
+    arg_order: "{{ item.arg_order }}"
+    arg_values: "{{ item.arg_values|default(omit) }}"
+    aa: "{{ item.aa|default(omit) }}"
+  register: test_result
+  ignore_errors: "{{ item.expect_error|default(omit) }}"
+
+- name: check results [{{ item.name }}]
+  assert:
+    that: "{{ item.assertions }}"

tests/integration/targets/cmd_runner/vars/main.yml

@@ -0,0 +1,84 @@
+# -*- coding: utf-8 -*-
+# (c) 2022, Alexei Znamensky
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+cmd_echo_tests:
+  - name: set aa and bb value
+    arg_formats:
+      aa:
+        func: as_opt_eq_val
+        args: [--answer]
+      bb:
+        func: as_bool
+        args: [--bb-here]
+    arg_order: 'aa bb'
+    arg_values:
+      bb: true
+    aa: 11
+    assertions:
+      - test_result.rc == 0
+      - test_result.out == "-- --answer=11 --bb-here\n"
+      - test_result.err == ""
+
+  - name: default aa value
+    arg_formats:
+      aa:
+        func: as_opt_eq_val
+        args: [--answer]
+      bb:
+        func: as_bool
+        args: [--bb-here]
+    arg_order: ['aa', 'bb']
+    arg_values:
+      aa: 43
+      bb: true
+    assertions:
+      - test_result.rc == 0
+      - test_result.out == "-- --answer=43 --bb-here\n"
+      - test_result.err == ""
+
+  - name: implicit aa format
+    arg_formats:
+      bb:
+        func: as_bool
+        args: [--bb-here]
+    arg_order: ['aa', 'bb']
+    arg_values:
+      bb: true
+    aa: 1984
+    assertions:
+      - test_result.rc == 0
+      - test_result.out == "-- --aa 1984 --bb-here\n"
+      - test_result.err == ""
+
+  - name: missing bb format
+    arg_order: ['aa', 'bb']
+    arg_values:
+      bb: true
+    aa: 1984
+    expect_error: true
+    assertions:
+      - test_result is failed
+      - test_result.rc == 1
+      - '"out" not in test_result'
+      - '"err" not in test_result'
+      - >-
+        "MissingArgumentFormat: Cannot find format for parameter bb"
+        in test_result.module_stderr
+
+  - name: missing bb value
+    arg_formats:
+      bb:
+        func: as_bool
+        args: [--bb-here]
+    arg_order: 'aa bb'
+    aa: 1984
+    expect_error: true
+    assertions:
+      - test_result is failed
+      - test_result.rc == 1
+      - '"out" not in test_result'
+      - '"err" not in test_result'
+      - >-
+        "MissingArgumentValue: Cannot find value for parameter bb"
+        in test_result.module_stderr