Binary attribute support for ldap_attrs and ldap_entry (#11558)

* Binary attribute support for `ldap_attrs` and `ldap_entry`

This commit implements binary attribute support for the `ldap_attrs` and
`ldap_entry` plugins. This used to be "supported" before, because it was
possible to simply load arbitrary binary data into the attributes, but
no longer functions on recent Ansible versions.

In order to support binary attributes, this commit introduces two new
options to both plugins:

  * `binary_attributes`, a list of attribute names which will be
    considered as being binary,
  * `honor_binary_option`, a flag which is true by default and will
    handle all attributes that include the binary option (see RFC 4522)
    as binary automatically.

When an attribute is determined to be binary through either of these
means, the plugin will assume that the attribute's value is in fact
base64-encoded. It will proceed to decode it and handle it accordingly.

While changes to `ldap_entry` are pretty straightforward, more work was
required on `ldap_attrs`.

  * First, because both `present` and `absent` state require checking
    the attribute's current values and normally do that using LDAP search
    queries for each value, a specific path for binary attributes was
    added that loads and caches all values for the attribute and compares
    the values in the Python code.
  * In addition, generating both the modlist and the diff output require
    re-encoding binary attributes' values into base64 so it can be
    transmitted back to Ansible.

* Various fixes on `ldap_attrs`/`ldap_entry` from PR 11558 discussion

* Rename `honor_binary_option` to `honor_binary`

* Add some general documentation about binary attributes

* Fix changelog fragment after renaming one of the new options

* Add examples of `honor_binary` and `binary_attributes`

* Add note that indicates that binary values are supported from 12.5.0+

* Fix punctuation

* Add links to RFC 4522 to `ldap_attrs` and `ldap_entry`

* Catch base64 decoding errors

* Rephrase changelog fragment

* Use f-string to format the encoding error message

changelogs/fragments/11558-binary-ldap-attributes.yml

@@ -0,0 +1,3 @@
+minor_changes:
+  - ldap_attrs - add ``binary_attributes`` and ``honor_binary`` parameters to handle binary attribute values (https://github.com/ansible-collections/community.general/pull/11558).
+  - ldap_entry - add ``binary_attributes`` and ``honor_binary`` parameters to handle creating objects with attributes set to binary values (https://github.com/ansible-collections/community.general/pull/11558).

plugins/modules/ldap_attrs.py

@@ -16,10 +16,15 @@ description:
   - Add or remove multiple LDAP attribute values.
 notes:
   - This only deals with attributes on existing entries. To add or remove whole entries, see M(community.general.ldap_entry).
-  - For O(state=present) and O(state=absent), all value comparisons are performed on the server for maximum accuracy. For
+  - If O(honor_binary=true), an attribute that includes the C(binary) option as per
-    O(state=exact), values have to be compared in Python, which obviously ignores LDAP matching rules. This should work out
+    L(RFC 4522, https://www.rfc-editor.org/rfc/rfc4522.html#section-3) will be considered as binary.  Its contents must be
-    in most cases, but it is theoretically possible to see spurious changes when target and actual values are semantically
+    specified as Base64 and sent to the LDAP after decoding. If an attribute must be handled as binary without including
-    identical but lexically distinct.
+    the C(binary) option, it can be listed in O(binary_attributes).
+  - For O(state=present) and O(state=absent), when handling text attributes, all value comparisons are performed on the
+    server for maximum accuracy. For O(state=exact) or binary attributes, values have to be compared in Python, which
+    obviously ignores LDAP matching rules. This should work out in most cases, but it is theoretically possible to see
+    spurious changes when target and actual values are semantically identical but lexically distinct.
+  - Support for binary values was added in community.general 12.5.0.
 version_added: '0.2.0'
 author:
   - Jiri Tyr (@jtyr)
@@ -42,6 +47,21 @@ options:
       - The state of the attribute values. If V(present), all given attribute values are added if they are missing. If V(absent),
         all given attribute values are removed if present. If V(exact), the set of attribute values is forced to exactly those
         provided and no others. If O(state=exact) and the attribute value is empty, all values for this attribute are removed.
+  binary_attributes:
+    description:
+      - If O(state=present), attributes whose values must be handled as raw sequences of bytes must be listed here.
+      - The values provided for the attributes will be converted from Base64.
+    type: list
+    elements: str
+    default: []
+    version_added: 12.5.0
+  honor_binary:
+    description:
+      - If O(state=present) and this option is V(true), attributes whose name include the V(binary) option
+        will be treated as Base64-encoded byte sequences automatically, even if they are not listed in O(binary_attributes).
+    type: bool
+    default: false
+    version_added: 12.5.0
   attributes:
     required: true
     type: dict
@@ -125,6 +145,17 @@ EXAMPLES = r"""
       olcRootPW: "{SSHA}tabyipcHzhwESzRaGA7oQ/SDoBZQOGND"
     state: exact
 
+- name: Replace a CA certificate
+  community.general.ldap_attrs:
+    dn: cn=ISRG Root X1,ou=ca,ou=certificates,dc=example,dc=org
+    honor_binary: true
+    state: exact
+    attributes:
+      cACertificate;binary: >-
+        MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
+        TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+        # ...
+
 - name: Remove an attribute with a specific value
   community.general.ldap_attrs:
     dn: uid=jdoe,ou=people,dc=example,dc=com
@@ -156,6 +187,8 @@ modlist:
     - [2, "olcRootDN", ["cn=root,dc=example,dc=com"]]
 """
 
+import base64
+import binascii
 import re
 import traceback
 
@@ -187,6 +220,11 @@ class LdapAttrs(LdapGeneric):
         self.attrs = self.module.params["attributes"]
         self.state = self.module.params["state"]
         self.ordered = self.module.params["ordered"]
+        self.binary = set(attr.lower() for attr in self.module.params["binary_attributes"])
+        self.honor_binary = self.module.params["honor_binary"]
+
+        # Cached attribute values
+        self._cached_values = {}
 
     def _order_values(self, values):
         """Prepend X-ORDERED index numbers to attribute's values."""
@@ -199,25 +237,37 @@ class LdapAttrs(LdapGeneric):
 
         return ordered_values
 
-    def _normalize_values(self, values):
+    def _is_binary(self, attr_name):
+        """Check if an attribute must be considered binary."""
+        lc_name = attr_name.lower()
+        return (self.honor_binary and "binary" in lc_name.split(";")) or lc_name in self.binary
+
+    def _normalize_values(self, values, is_binary):
         """Normalize attribute's values."""
-        norm_values = []
+        if is_binary:
-
+            converter = base64.b64decode
-        if isinstance(values, list):
-            if self.ordered:
-                norm_values = list(map(to_bytes, self._order_values(list(map(str, values)))))
         else:
-                norm_values = list(map(to_bytes, values))
+            converter = to_bytes
-        else:
-            norm_values = [to_bytes(str(values))]
 
-        return norm_values
+        if not isinstance(values, list):
+            values = [values]
+        elif self.ordered and not is_binary:
+            values = self._order_values([str(value) for value in values])
+
+        try:
+            return [converter(value) for value in values]
+        except binascii.Error:
+            return None
 
     def add(self):
         modlist = []
         new_attrs = {}
+        bad_bin_attrs = []
         for name, values in self.module.params["attributes"].items():
-            norm_values = self._normalize_values(values)
+            norm_values = self._normalize_values(values, self._is_binary(name))
+            if norm_values is None:
+                bad_bin_attrs.append(name)
+                continue
             added_values = []
             for value in norm_values:
                 if self._is_value_absent(name, value):
@@ -225,14 +275,18 @@ class LdapAttrs(LdapGeneric):
                     added_values.append(value)
             if added_values:
                 new_attrs[name] = norm_values
-        return modlist, {}, new_attrs
+        return modlist, {}, new_attrs, bad_bin_attrs
 
     def delete(self):
         modlist = []
         old_attrs = {}
         new_attrs = {}
+        bad_bin_attrs = []
         for name, values in self.module.params["attributes"].items():
-            norm_values = self._normalize_values(values)
+            norm_values = self._normalize_values(values, self._is_binary(name))
+            if norm_values is None:
+                bad_bin_attrs.append(name)
+                continue
             removed_values = []
             for value in norm_values:
                 if self._is_value_present(name, value):
@@ -241,20 +295,19 @@ class LdapAttrs(LdapGeneric):
             if removed_values:
                 old_attrs[name] = norm_values
                 new_attrs[name] = [value for value in norm_values if value not in removed_values]
-        return modlist, old_attrs, new_attrs
+        return modlist, old_attrs, new_attrs, bad_bin_attrs
 
     def exact(self):
         modlist = []
         old_attrs = {}
         new_attrs = {}
+        bad_bin_attrs = []
         for name, values in self.module.params["attributes"].items():
-            norm_values = self._normalize_values(values)
+            norm_values = self._normalize_values(values, self._is_binary(name))
-            try:
+            if norm_values is None:
-                results = self.connection.search_s(self.dn, ldap.SCOPE_BASE, attrlist=[name])
+                bad_bin_attrs.append(name)
-            except ldap.LDAPError as e:
+                continue
-                self.fail(f"Cannot search for attribute {name}", e)
+            current = self._get_all_values_of(name)
-
-            current = results[0][1].get(name, [])
 
             if frozenset(norm_values) != frozenset(current):
                 if len(current) == 0:
@@ -269,10 +322,13 @@ class LdapAttrs(LdapGeneric):
                     old_attrs[name] = current[0]
                     new_attrs[name] = norm_values[0]
 
-        return modlist, old_attrs, new_attrs
+        return modlist, old_attrs, new_attrs, bad_bin_attrs
 
     def _is_value_present(self, name, value):
         """True if the target attribute has the given value."""
+        if self._is_binary(name):
+            return value in self._get_all_values_of(name)
+
         try:
             escaped_value = ldap.filter.escape_filter_chars(to_text(value))
             filterstr = f"({name}={escaped_value})"
@@ -283,15 +339,52 @@ class LdapAttrs(LdapGeneric):
 
         return is_present
 
+    def _get_all_values_of(self, name):
+        """Return all values of an attribute."""
+        lc_name = name.lower()
+        if lc_name not in self._cached_values:
+            try:
+                results = self.connection.search_s(self.dn, ldap.SCOPE_BASE, attrlist=[name])
+            except ldap.LDAPError as e:
+                self.fail(f"Cannot search for attribute {name}", e)
+            self._cached_values[lc_name] = results[0][1].get(name, [])
+        return self._cached_values[lc_name]
+
     def _is_value_absent(self, name, value):
         """True if the target attribute doesn't have the given value."""
         return not self._is_value_present(name, value)
 
+    def _reencode_modlist(self, modlist):
+        """Re-encode binary attribute values in the modlist into Base64 in
+        order to avoid crashing the plugin when returning the modlist to
+        Ansible."""
+        output = []
+        for mod_op, attr, values in modlist:
+            if self._is_binary(attr) and values is not None:
+                values = [base64.b64encode(value) for value in values]
+            output.append((mod_op, attr, values))
+        return output
+
+    def _reencode_attributes(self, attributes):
+        """Re-encode binary attribute values in an attribute dict into Base64 in
+        order to avoid crashing the plugin when returning the dict to Ansible."""
+        output = {}
+        for name, values in attributes.items():
+            if self._is_binary(name):
+                if isinstance(values, list):
+                    values = [base64.b64encode(value) for value in values]
+                else:
+                    values = base64.b64encode(values)
+            output[name] = values
+        return output
+
 
 def main():
     module = AnsibleModule(
         argument_spec=gen_specs(
             attributes=dict(type="dict", required=True),
+            binary_attributes=dict(default=[], type="list", elements="str"),
+            honor_binary=dict(default=False, type="bool"),
             ordered=dict(type="bool", default=False),
             state=dict(type="str", default="present", choices=["absent", "exact", "present"]),
         ),
@@ -306,16 +399,21 @@ def main():
     ldap = LdapAttrs(module)
     old_attrs = None
     new_attrs = None
+    modlist = []
 
     state = module.params["state"]
 
     # Perform action
     if state == "present":
-        modlist, old_attrs, new_attrs = ldap.add()
+        modlist, old_attrs, new_attrs, bad_attrs = ldap.add()
     elif state == "absent":
-        modlist, old_attrs, new_attrs = ldap.delete()
+        modlist, old_attrs, new_attrs, bad_attrs = ldap.delete()
     elif state == "exact":
-        modlist, old_attrs, new_attrs = ldap.exact()
+        modlist, old_attrs, new_attrs, bad_attrs = ldap.exact()
+
+    if bad_attrs:
+        s_bad_attrs = ", ".join(bad_attrs)
+        module.fail_json(msg=f"Invalid Base64-encoded attribute values for {s_bad_attrs}")
 
     changed = False
 
@@ -328,6 +426,12 @@ def main():
             except Exception as e:
                 module.fail_json(msg="Attribute action failed.", details=f"{e}")
 
+    # If the data contain binary attributes/changes, we need to re-encode them
+    # using Base64.
+    modlist = ldap._reencode_modlist(modlist)
+    old_attrs = ldap._reencode_attributes(old_attrs)
+    new_attrs = ldap._reencode_attributes(new_attrs)
+
     module.exit_json(changed=changed, modlist=modlist, diff={"before": old_attrs, "after": new_attrs})
 
 

plugins/modules/ldap_entry.py

@@ -33,8 +33,28 @@ options:
         by using YAML block modifiers as seen in the examples for this module.
       - Note that when using values that YAML/ansible-core interprets as other types, like V(yes), V(no) (booleans), or V(2.10)
         (float), make sure to quote them if these are meant to be strings. Otherwise the wrong values may be sent to LDAP.
+      - If O(honor_binary=true), an attribute that includes the C(binary) option as per
+        L(RFC 4522, https://www.rfc-editor.org/rfc/rfc4522.html#section-3) will be considered as binary. Its contents must be
+        specified as Base64 and sent to the LDAP after decoding. If an attribute must be handled as binary without including
+        the C(binary) option, it can be listed in O(binary_attributes).
+        Support for binary values was added in community.general 12.5.0.
     type: dict
     default: {}
+  binary_attributes:
+    description:
+      - If O(state=present), attributes whose values must be handled as raw sequences of bytes must be listed here.
+      - The values provided for the attributes will be converted from Base64.
+    type: list
+    elements: str
+    default: []
+    version_added: 12.5.0
+  honor_binary:
+    description:
+      - If O(state=present) and this option is V(true), attributes whose name include the V(binary) option
+        will be treated as Base64-encoded byte sequences automatically, even if they are not listed in O(binary_attributes).
+    type: bool
+    default: false
+    version_added: 12.5.0
   objectClass:
     description:
       - If O(state=present), value or list of values to use when creating the entry. It can either be a string or an actual
@@ -71,9 +91,30 @@ EXAMPLES = r"""
     objectClass:
       - simpleSecurityObject
       - organizationalRole
+      - myPhotoObject
+    binary_attributes:
+      - myPhoto
     attributes:
       description: An LDAP administrator
       userPassword: "{SSHA}tabyipcHzhwESzRaGA7oQ/SDoBZQOGND"
+      myPhoto: >-
+        /9j/4AAQSkZJRgABAQAAAQABAAD/4gIcSUNDX1BST0ZJTEUAAQEAAAIMbGNt
+        cwIQAABtbnRyUkdCIFhZWiAH3AABABkAAwApADlhY3NwQVBQTAAAAAAAAAAA
+        # ...
+
+- name: Make sure a CA certificate is present
+  community.general.ldap_attrs:
+    dn: cn=ISRG Root X1,ou=ca,ou=certificates,dc=example,dc=org
+    honor_binary: true
+    objectClass:
+      - pkiCA
+      - applicationProcess
+    attributes:
+      cn: ISRG Root X1
+      cACertificate;binary: >-
+        MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
+        TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+        # ...
 
 - name: Set possible values for attributes elements
   community.general.ldap_entry:
@@ -128,6 +169,8 @@ RETURN = r"""
 # Default return values
 """
 
+import base64
+import binascii
 import traceback
 
 from ansible.module_utils.basic import AnsibleModule, missing_required_lib
@@ -157,25 +200,44 @@ class LdapEntry(LdapGeneric):
         # Shortcuts
         self.state = self.module.params["state"]
         self.recursive = self.module.params["recursive"]
+        self.binary = set(attr.lower() for attr in self.module.params["binary_attributes"])
+        self.honor_binary = self.module.params["honor_binary"]
 
         # Add the objectClass into the list of attributes
         self.module.params["attributes"]["objectClass"] = self.module.params["objectClass"]
 
         # Load attributes
         if self.state == "present":
-            self.attrs = self._load_attrs()
+            self.attrs, bad_attrs = self._load_attrs()
+            if bad_attrs:
+                s_bad_attrs = ", ".join(bad_attrs)
+                self.module.fail_json(msg=f"Invalid Base64-encoded attribute values for {s_bad_attrs}")
+
+    def _is_binary(self, attr_name):
+        """Check if an attribute must be considered binary."""
+        lc_name = attr_name.lower()
+        return (self.honor_binary and "binary" in lc_name.split(";")) or lc_name in self.binary
 
     def _load_attrs(self):
-        """Turn attribute's value to array."""
+        """Turn attribute's value to array. Attribute values are converted to
+        raw bytes, either by encoding the string itself, or by decoding it from
+        base 64, depending on the binary attributes settings."""
         attrs = {}
+        bad_attrs = []
 
         for name, value in self.module.params["attributes"].items():
-            if isinstance(value, list):
+            if self._is_binary(name):
-                attrs[name] = list(map(to_bytes, value))
+                converter = base64.b64decode
             else:
-                attrs[name] = [to_bytes(value)]
+                converter = to_bytes
+            if not isinstance(value, list):
+                value = [value]
+            try:
+                attrs[name] = [converter(v) for v in value]
+            except binascii.Error:
+                bad_attrs.append(name)
 
-        return attrs
+        return attrs, bad_attrs
 
     def add(self):
         """If self.dn does not exist, returns a callable that will add it."""
@@ -236,6 +298,8 @@ def main():
     module = AnsibleModule(
         argument_spec=gen_specs(
             attributes=dict(default={}, type="dict"),
+            binary_attributes=dict(default=[], type="list", elements="str"),
+            honor_binary=dict(default=False, type="bool"),
             objectClass=dict(type="list", elements="str"),
             state=dict(default="present", choices=["present", "absent"]),
             recursive=dict(default=False, type="bool"),