Simplify and extend from_ini tests (#11534)
Simplify and extend from_ini tests.
(cherry picked from commit e118b23ba0)
Co-authored-by: Felix Fontein <felix@fontein.de>
adds parameter delimiters to from_ini filter (#11512)
* adds parameter delimiters to from_ini filter
fixes issue #11506
* adds changelog fragment
* fixes pylint dangerous-default-value / W0102
* does not assume default delimiters
let that be decided in the super class
* Update plugins/filter/from_ini.py
verbose description
* Update changelogs/fragments/11512-from_ini-delimiters.yaml
* adds input validation
* adss check for delimiters not None
* adds missing import
* removes the negation
* adds suggestions from russoz
* adds ruff format suggestion
---------
(cherry picked from commit aec0e61ba1)
Co-authored-by: Robert Sander <github@gurubert.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
keycloak_user_rolemapping: handle None response for client role lookup (#11471)
* fix(keycloak_user_rolemapping): handle None response for client role lookup
When adding a client role to a user who has no existing roles for that
client, get_client_user_rolemapping_by_id() returns None. The existing
code indexed directly into the result causing a TypeError. Add the same
None check that already existed for realm roles since PR #11256.
Fixes#10960
* fix(tests): use dict format for task vars in keycloak_user_rolemapping tests
Task-level vars requires a YAML mapping, not a sequence. The leading
dash (- roles:) produced a list instead of a dict, which ansible-core
2.20 rejects with "Vars in a Task must be specified as a dictionary".
* Update changelogs/fragments/keycloak-user-rolemapping-client-none-check.yml
---------
(cherry picked from commit 34938ca1ef)
Co-authored-by: Ivan Kokalovic <67540157+koke1997@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
keycloak_realm_key: add full support for all Keycloak key providers (#11468)
* feat(keycloak_realm_key): add support for auto-generated key providers
Add support for Keycloak's auto-generated key providers where Keycloak
manages the key material automatically:
- rsa-generated: Auto-generates RSA signing keys
- hmac-generated: Auto-generates HMAC signing keys
- aes-generated: Auto-generates AES encryption keys
- ecdsa-generated: Auto-generates ECDSA signing keys
New algorithms:
- HMAC: HS256, HS384, HS512
- ECDSA: ES256, ES384, ES512
- AES: AES (no algorithm parameter needed)
New config options:
- secret_size: For HMAC/AES providers (key size in bytes)
- key_size: For RSA-generated provider (key size in bits)
- elliptic_curve: For ECDSA-generated provider (P-256, P-384, P-521)
Changes:
- Make private_key/certificate optional (only required for rsa/rsa-enc)
- Add provider-algorithm validation with clear error messages
- Fix KeyError when managing default realm keys (issue #11459)
- Maintain backward compatibility: RS256 default works for rsa/rsa-generated
Fixes: #11459
* fix: address sanity test failures
- Add 'default: RS256' to algorithm documentation to match spec
- Add no_log=True to secret_size parameter per sanity check
* feat(keycloak_realm_key): extend support for all Keycloak key providers
Add support for remaining auto-generated key providers:
- rsa-enc-generated (RSA encryption keys with RSA1_5, RSA-OAEP, RSA-OAEP-256)
- ecdh-generated (ECDH key exchange with ECDH_ES, ECDH_ES_A128KW/A192KW/A256KW)
- eddsa-generated (EdDSA signing with Ed25519, Ed448 curves)
Changes:
- Add provider-specific elliptic curve config key mapping
(ecdsaEllipticCurveKey, ecdhEllipticCurveKey, eddsaEllipticCurveKey)
- Add PROVIDERS_WITHOUT_ALGORITHM constant for providers that don't need algorithm
- Add elliptic curve validation per provider type
- Update documentation with all supported algorithms and examples
- Add comprehensive integration tests for all new providers
This completes full coverage of all Keycloak key provider types.
* style: apply ruff formatting
* feat(keycloak_realm_key): add java-keystore provider and update_password
Add support for java-keystore provider to import keys from Java
Keystore (JKS or PKCS12) files on the Keycloak server filesystem.
Add update_password parameter to control password handling for
java-keystore provider:
- always (default): Always send passwords to Keycloak
- on_create: Only send passwords when creating, preserve existing
passwords when updating (enables idempotent playbooks)
The on_create mode sends the masked value ("**********") that Keycloak
recognizes as "preserve existing password", matching the behavior when
re-importing an exported realm.
Replace password_checksum with update_password - the checksum approach
was complex and error-prone. The update_password parameter is simpler
and follows the pattern used by ansible.builtin.user module.
Also adds key_info return value containing kid, certificate fingerprint,
status, and expiration for java-keystore keys.
* address PR review feedback
- Remove no_log=True from secret_size (just an int, not sensitive)
- Add version_added: 12.4.0 to new parameters and return values
- Remove "Added in community.general 12.4.0" from description text
- Consolidate changelog entries into 4 focused entries
- Remove bugfix from changelog (now in separate PR #11470)
* address review feedback from russoz and felixfontein
- remove docstrings from module-local helpers
- remove line-by-line comments and unnecessary null guard
- use specific exceptions instead of bare except Exception
- use module.params["key"] instead of .get("key")
- consolidate changelog into single entry
- avoid "complete set" claim, reference Keycloak 26 instead
* address round 2 review feedback
- Extract remove_sensitive_config_keys() helper (DRY refactor)
- Simplify RS256 validation to single code path
- Add TypeError to inner except in compute_certificate_fingerprint()
- Remove redundant comments (L812, L1031)
- Switch .get() to direct dict access for module.params
(cherry picked from commit 80d21f2a0d)
Co-authored-by: Ivan Kokalovic <67540157+koke1997@users.noreply.github.com>
Integration tests: replace ansible_xxx with ansible_facts.xxx (#11479)
Replace ansible_xxx with ansible_facts.xxx.
(cherry picked from commit 476f2bf641)
Co-authored-by: Felix Fontein <felix@fontein.de>
keycloak_client: add valid_post_logout_redirect_uris and backchannel_logout_url (#11473)
* feat(keycloak_client): add valid_post_logout_redirect_uris and backchannel_logout_url
Add two new convenience parameters that map to client attributes:
- valid_post_logout_redirect_uris: sets post.logout.redirect.uris
attribute (list items joined with ##)
- backchannel_logout_url: sets backchannel.logout.url attribute
These fields are not top-level in the Keycloak REST API but are stored
as client attributes. The new parameters provide a user-friendly
interface without requiring users to know the internal attribute names
and ##-separator format.
Fixes#6812, fixes#4892
* consolidate changelog and add PR link per review feedback
(cherry picked from commit df6d6269a6)
Co-authored-by: Ivan Kokalovic <67540157+koke1997@users.noreply.github.com>
keycloak: URL-encode query parameters for usernames with special characters (#11472)
* fix(keycloak): URL-encode query params for usernames with special chars
get_user_by_username() concatenates the username directly into the URL
query string. When the username contains a +, it is interpreted as a
space by the server, returning no match and causing a TypeError.
Use urllib.parse.quote() (already imported) for the username parameter.
Also replace three fragile .replace(' ', '%20') calls in the authz
search methods with proper quote() calls.
Fixes#10305
* Update changelogs/fragments/keycloak-url-encode-query-params.yml
---------
(cherry picked from commit c41de53dbb)
Co-authored-by: Ivan Kokalovic <67540157+koke1997@users.noreply.github.com>
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
Logstash plugin version fix (#11440)
* logstash_plugin: fix argument order when using version parameter
* logstash_plugin: add integration tests
* logstash_plugin: add changelog fragment
(cherry picked from commit 53e1e86bcc)
Co-authored-by: Nicolas Boutet <amd3002@gmail.com>
Add `to_toml` filter (#11423)
* Add to_toml filter
This is based heavily on the to_yaml filter, but
with a pared-down feature set.
* Protect import
* Don't quote datetime as a string
* Use Ansible error types
* Import correct error types
* Don't use AnsibleTypeError
It doesn't seem to be available on older Ansible
core versions.
* Fix antsibull-nox errors
* Install dependencies for to_toml integration test
* Reduce author list to main contributor
* Update version added for to_toml
* Use AnsibleError for missing import
* Use AnsibleFilterError for runtime type check
* Move common code to plugin_utils/_tags.py
* Mark module util as private
* Update BOTMETA for to_toml
* Fix typo
* Correct version number
* Use to_text for to_toml dict key conversions
* Add tomlkit requirement to docs
* Add missing import
* Add aliases for for to_toml integration test
---------
(cherry picked from commit 864695f898)
Co-authored-by: Matt Williams <matt@milliams.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Update RHEL 9.x to 9.7 in CI (#11387)
* Update RHEL 9.x to 9.7 in CI.
* Add skips.
(cherry picked from commit d4089ca29a)
Co-authored-by: Felix Fontein <felix@fontein.de>
Clean up other Python files (#11379)
* Address issues found by ruff check.
* Make mypy happy; remove some Python 2 compat code.
* Also declare port1.
(cherry picked from commit b3dc06a7dd)
Co-authored-by: Felix Fontein <felix@fontein.de>
Add missing integration test aliases files (#11357)
* Add missing aliases files.
* Fix directory name.
* Add another missing aliases file.
* Adjust test to also work with newer jsonpatch versions.
(cherry picked from commit ddf05104f3)
Co-authored-by: Felix Fontein <felix@fontein.de>
sysrc tests: skip FreeBSD 14.2 for ezjail tests (#11276)
Looks like 14.2 no longer works.
(cherry picked from commit a96a5c44a5)
Co-authored-by: Felix Fontein <felix@fontein.de>
monit: investigating tests again - using copilot on this one (#11255)
* add monit version to successful exit
* install the standard monit - if 5.34, then bail out
* add 3sec wait after service restart
- that restart happens exactly before the task receiving the SIGTERM, so maybe, just maybe, it just needs time to get ready for the party
* wait for monit initialisation after restart
* monit tests: check service-specific status in readiness wait
The wait task was checking 'monit status' (general), but the actual
failing command is 'monit status -B httpd_echo' (service-specific).
This causes a race where general status succeeds but service queries
fail. Update to check the exact command format that will be used.
* monit tests: remove 5.34.x version restriction
The version restriction was based on incorrect diagnosis. The actual
issue was the readiness check validating general status instead of
service-specific queries. Now that we check the correct command
format, the tests should work across all monit versions.
* monit tests: add stabilization delay after readiness check
After the readiness check succeeds, add a 1-second pause before
running actual tests. Monit 5.34.x and 5.35 appear to have a
concurrency issue where rapid successive 'monit status -B' calls
can cause hangs even though the first call succeeds.
* monit tests: add retry logic for state changes to handle monit daemon hangs
Monit daemon has an intermittent concurrency bug across versions 5.27-5.35
where 'monit status -B' commands can hang (receiving SIGTERM) even after
the daemon has successfully responded to previous queries. This appears
to be a monit daemon issue, not a timing problem.
Add retry logic with 2-second delays to the state change task to work
around these intermittent hangs. Skip retries if the failure is not
SIGTERM (rc=-15) to avoid masking real errors.
* monit tests: capture and display monit.log for debugging
Add tasks in the always block to capture and display the monit log file.
This will help diagnose the intermittent hanging issues by showing what
monit daemon was doing when 'monit status -B' commands hang.
* monit tests: enable verbose logging (-v flag)
Modify the monit systemd service to start with -v flag for verbose
logging. This should provide more detailed information in the monit
log about what's happening when status commands hang.
* monit: add 0.5s delay after state change command
After extensive testing and analysis with verbose logging enabled, identified
that monit's HTTP interface can become temporarily unresponsive immediately
after processing state change commands (stop, start, restart, etc.).
This manifests as intermittent SIGTERM (rc=-15) failures when the module
calls 'monit status -B <service>' to verify the state change. The issue
affects all monit versions tested (5.27-5.35) and is intermittent, suggesting
a race condition or brief lock in monit's HTTP request handling.
Verbose logging confirmed:
- State change commands complete successfully
- HTTP server reports as 'started'
- But subsequent status checks can hang without any log entry
Adding a 0.5 second sleep after sending state change commands gives the
monit daemon time to fully process the command and become responsive again
before the first status verification check.
This complements the existing readiness check after daemon restart and
the retry logic for SIGTERM failures in the tests.
* tests(monit): remove workarounds after module race condition fix
After 10+ successful CI runs with no SIGTERM failures, removing test-level
workarounds that are now redundant due to the 0.5s delay fix in the module:
- Remove 1-second stabilization pause after daemon restart
The module's built-in 0.5s delay after state changes makes this unnecessary
- Remove retry logic for SIGTERM failures in state change tests
The race condition is now prevented at the module level
- Remove verbose logging setup and log capture
Verbose mode didn't log HTTP requests, so it didn't help diagnose the issue
and adds unnecessary overhead
Kept the readiness check with retries after daemon restart - still needed
to validate daemon is responsive after service restart (different scenario
than the state change race condition).
* restore tasks/main.yml
* monit tests: reduce readiness check retries from 60 to 10
After successful CI runs, observed that monit daemon becomes responsive
within 1-2 seconds after restart. The readiness check typically passes
on the first attempt.
Reducing from 60 retries (30s timeout) to 10 retries (5s timeout) is
more appropriate and allows tests to fail faster if something is
genuinely broken.
* add changelog frag
* Update changelogs/fragments/11255-monit-integrationtests.yml
---------
(cherry picked from commit ac37544c53)
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
lxd_storage_pool_info, lxd_storage_volume_info: new modules (#11198)
* Fix mistaken rebase
* plugins/modules/lxd_storage_: include error codes, clean up notes
* plugins/modules/lxd_storage_: snap_url, ruff fix
* plugins/modules/lxd_storage_volume_info.py: remove checks on expected api returned bits
* plugins/modules/lxd_storage_volume_info.py: required: true
* tests/integration/targets/lxd_storage_volume_info/tasks/main.yaml: add Test fetching specific volume by name
* tests/unit/plugins/modules/test_lxd_storage_: add unit tests
* tests/integration/targets/lxd_storage_pool_info/tasks/main.yaml: add integratio tests
* tests/integration/targets/lxd_storage_: not required
* tests/integration/targets/lxd_storage_: not required perhaps, lxd_project has them
* tests/unit/plugins/modules/test_lxd_storage_volume_info.py: fix python3.8 tests
* tests/unit/plugins/modules/test_lxd_storage_pool_info.py: fix python3.8
* tests/integration/targets/lxd_storage_: correct paths for aliases
* tests/unit/plugins/modules/test_lxd_storage_volume_info.py: remove backticks
* tests/unit/plugins/modules/test_lxd_storage_volume_info.py: remove blank line
* tests/unit/plugins/modules/test_lxd_storage_: python3.8 changes
* tests/unit/plugins/modules/test_lxd_storage_: python3.8 changes
* tests/unit/plugins/lookup/test_github_app_access_token.py: restore
* tests/unit/plugins/connection/test_wsl.py: restore
* plugins/modules/lxd_storage_: use ANSIBLE_LXD_DEFAULT_SNAP_URL and put API version into const
* lxd_storage_volume_info: use recursion to gather all volume details
* tests/integration/targets/lxd_storage_volume_info/tasks/main.yaml: fix silet skipped failures
* tests/integration/targets/lxd_storage_pool_info/tasks/main.yaml: fix silet failures
* lxd_storage_pool_info: update to use recursion to gather all details in one shot
* Remove unnecessary change.
---------
(cherry picked from commit 6365b5a981)
Co-authored-by: Sean McAvoy <seanmcavoy@gmail.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
locale_gen: search for available locales in /usr/local as well (#11046)
* locale_gen: search for available locales in /usr/local as well
* better var name
* add test for /usr/local
* Apply suggestions from code review
* skip /usr/local/ for Archlinux
* improve/update documentation
* add license file for the custom locale
* add changelog frag
* Update plugins/modules/locale_gen.py
* Update changelogs/fragments/11046-locale-gen-usrlocal.yml
---------
(cherry picked from commit 98aca27a8b)
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
kea_command: new module to access an ISC KEA server
This module can be used to access the JSON API of a
KEA DHCP4, DHCP6, DDNS or other services in a generic
way, without having to manually format the JSON, with
response error code checking.
It directly accesses the Unix Domain Socket API so it
needs to execute on the system the server is running,
with superuser privilegues, but without the hassle of
wrapping it into HTTPS and password auth (or client
certificates).
The integration test uses a predefined setup for
convenience, which runs on Debian trixie as well as,
on the CI, Ubuntu noble. It makes assumptions about
the default package configuration and paths and is
therefore tricky to run on other distros/OSes. This
only affects running the KEA server as part of the
tests, not the module.
* Adjust all __future__ imports:
for i in $(grep -REl "__future__.*absolute_import" plugins/ tests/); do
sed -e 's/from __future__ import .*/from __future__ import annotations/g' -i $i;
done
* Remove all UTF-8 encoding specifications for Python source files:
for i in $(grep -REl '[-][*]- coding: utf-8 -[*]-' plugins/ tests/); do
sed -e '/^# -\*- coding: utf-8 -\*-/d' -i $i;
done
* Remove __metaclass__ = type:
for i in $(grep -REl '__metaclass__ = type' plugins/ tests/); do
sed -e '/^__metaclass__ = type/d' -i $i;
done
* Bump version to 12.0.0.
* Remove deprecated modules and plugins.
* state is now required.
* Change default of prepend_hash from auto to never.
* Remove support for force=''.
* Always delegate 'debug'.
* Remove ignore_value_none and ctx_ignore_none parameters.
* Remove parameters on_success and on_failure.
* Update BOTMETA.
* Adjust docs reference.
* Forgot required=True.
* Fix changelog fragment.
* Adjust unit tests.
* Fix changelog.
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
---------
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
* [doc] update requirements for consul_kv module
python-consul has been unmaintained for a while. It uses a legacy way of passing the Consul token when sending requests. This leads to warning messages in Consul log, and will eventually break communication. Using the maintained py-consul library ensures compatibility to newer Consul versions.
* [doc] replace all python-consul occurrences with py-consul
* [fix] tests and possible pip server errors
* [chore] remove referencce to python-consul in comment
---------
Co-authored-by: Sebastian Damm <sebastian.damm@pascom.net>
* Exclude aliases before comparison
* add test
* fragment
* Update changelogs/fragments/10829-fix-keycloak-role-changed-status.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
---------
Co-authored-by: Andre Desrosiers <andre.desrosiers@ssss.gouv.qc.ca>
Co-authored-by: Felix Fontein <felix@fontein.de>
* homebrew: Support old_tokens and oldnames in homebrew package data
Fixes#10804
Since brew info will accept old_tokens (for casks) and oldnames (for formulae) when provided by the homebrew module "name" argument, the module also needs to consider thes old names as valid for the given package. This commit updates _extract_package_name to do that.
All existing package name tests, including existing tests for name aliases and tap prefixing, have been consolidated with new name tests into package_names.yml.
* Added changelog fragment.
* homebrew: replace non-py2 compliant f-string usage
* code formatting lint, and py2 compatibility fixes
* homebrew: added licenses to new files, nox lint
* Update plugins/modules/homebrew.py
use str.format() instead of string addition
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update tests/integration/targets/homebrew/tasks/casks.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update tests/integration/targets/homebrew/tasks/package_names_item.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update tests/integration/targets/homebrew/tasks/formulae.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Fixes for performance concerns on new homebrew tests.
1) tests for alternate package names are commented out in main.yml.
2) the "install via alternate name, uninstall via base name" test
case was deemed duplicative, and has been deleted .
3) minor fixes to use jinja2 "~" for string concat instead of "+"
* Fix nox lint
---------
Co-authored-by: Felix Fontein <felix@fontein.de>
changed the dependency that is used to test the functionality in android_sdk module. The previous dependency was ~100MB, the current one is ~6MB. This should speed up the tests a bit and reduce the traffic.
* random_string: Specify seed while generating random string
* Allow user to specify seed to generate random string
Fixes: #5362
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Apply suggestions from code review
Co-authored-by: Felix Fontein <felix@fontein.de>
---------
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
