All notable changes to Hermes will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- client plugin
usersgroups_ldap: fixed missing var initialization introduced in previous version that broke the whole plugin
- client plugin
usersgroups_ldap: the client no longer connects to the LDAP server at startup, in order to avoid an ldap.SERVER_DOWN error which occurred after a restart of the LDAP server when no data had been processed by the LDAP client since its startup
- Bumped python dependencies to their latest version:
- kafka-python (used by plugins/messagebus_producers/kafka and plugins/messagebus_consumers/kafka)
- oracledb (used by plugins/datasources/oracle)
- psycopg (used by plugins/datasources/postgresql)
- python-ldap (used by plugins/datasources/ldap and plugins/clients/usersgroups_ldap)
- client plugin
usersgroups_ldap: Now disconnecting once event batch is processed to free resources and avoid ldap.SERVER_DOWN error that occurred after an LDAP server restart
- Jinja rendering is now processed in a sandboxed environment
- Fixed data alteration when rendering Jinja template (#11). This fix also significantly improves type handling in Jinja templates. IMPORTANT NOTE As Jinja templates rendering has been significantly modified, it's possible that your Jinja expressions will be interpreted differently with this release. You really should revalidate the behavior of your Jinja expressions in a test environment with this new version before updating it in your production environment.
- client plugin
usersgroups_bsspartage: account password is now properly updated on target (#12) - clients: objects are now properly deleted when processing the deletion of a data type from the datamodel (#13), and any error met during this processing now terminate the app, but the deletion will resume upon restart (#14)
- Bumped python dependencies to their latest version:
- kafka-python (used by plugins/messagebus_producers/kafka and plugins/messagebus_consumers/kafka)
- pypsrp (used by plugins/clients/usersgroups_adpypsrp)
- [usergroups clients] Handle cases where group membership is provided in a single entry, with new dedicated types
MembersOfGroupsandMembersOfGroupsSendersto use instead ofGroupsMembersandGroupsSenders(#5)
- Bumped python dependencies to their latest version:
- oracledb (used by plugins/datasources/oracle)
- psycopg (used by plugins/datasources/postgresql)
- pypsrp (used by plugins/clients/usersgroups_adpypsrp)
- client plugin
usersgroups_adpypsrp: added the brand newno_profile=Trueoption of pypsrp which avoids creating an unnecessary user profile on the Windows host
- client plugin
usersgroups_adpypsrp: set a default value for optional config settingotherAttributes - clients are now able to use the same
hermesTypemore than once in datamodel (#9) - [clients] fixed invalid purge of an object's events from error queue when this object is removed (#10)
- client now displays an explicit error message when its datamodel is missing a linked server data type that provides the foreign keys required by a client data type
regex_searchattribute plugin: removed a forgottenraisestatement that was always called whenregex_searchwas called with arbitrary arguments (*argsnot empty)- now failing when ran from an incompatible Python version (#3)
- client no longer saves its configuration cache file until a first event has been properly processed, thus preventing it from getting permanently stuck when it is started with a configuration containing Jinja syntax errors (#4)
- Added support for Python 3.14
- Bumped python dependencies to their latest version:
- Cerberus
- oracledb (used by plugins/datasources/oracle)
- kafka-python (used by plugins/messagebus_producers/kafka and plugins/messagebus_consumers/kafka)
- Frozen the version of psycopg to guarantee consistency
- Bumped python dependencies to their latest version:
- PyYAML
- oracledb (used by plugins/datasources/oracle)
- python-ldap (used by plugins/datasources/ldap and plugins/clients/usersgroups_ldap)
- [Clients] Fixed Jinja expressions relying on multiple attributes not rendered properly when one of them was removed (#1).
IMPORTANT NOTE This bug only affects clients that contain Jinja expressions that rely on multiple attributes from the server. If your clients are likely to be impacted, here are the steps to fix invalid values (proceed on each client):
- Edit the client's config file, and in each data-type declared and maybe impacted, insert a non-significant space into the Jinja expression : e.g. replace
{{ attr1 ~ attr2 }}by{{ attr1 ~ attr2 }}. This simple change will trigger a datamodel update on the next client startup, which will recalculate the values of all Jinja expressions and propagate the new (good) values if they differ from the previous ones. - Restart your client.
- Edit the client's config file, and in each data-type declared and maybe impacted, insert a non-significant space into the Jinja expression : e.g. replace
- Fixed the lockfile name that prevented multiple instances of the same plugin from being started from the same source code. Previously, the lockfile was named with the concatenation of the path of the folder containing the source code and the name of the executed plugin. Now the lockfile is named with the concatenation of the current working dir and the plugin name.
- Bumped python dependencies to their latest version:
- oracledb (used by plugins/datasources/oracle)
- kafka-python-ng isn't maintained anymore: replaced by kafka-python that is maintained again (used by plugins/messagebus_consumers kafka and plugins/messagebus_producers/kafka)
- New
regex_searchattribute plugin that allows to search in a string to extract the part that matches the specified regular expression. See https://hermes.insa-strasbourg.fr/en/setup/configuration/plugins/attributes/regex_search/ for details.
- Fixed improper handling of a plugin's empty configuration schema file
- Bumped python dependencies to their latest version:
- oracledb (used by plugins/datasources/oracle)
- pycryptodomex (used by plugins/attributes/crypto_RSA_OAEP)
- Client plugins
usersgroups_adpypsrp: added a new optional typeSubGroupsMembersthat allows to handle nested groups
- Bumped python dependencies to their latest version:
- oracledb (used by plugins/datasources/oracle)
- pycryptodomex (used by plugins/attributes/crypto_RSA_OAEP)
- Client plugins
usersgroups_adpypsrp: added a new optional settingnegotiate_servicethat allows to override the service part of the calculated SPN used when authenticating the Kerberos server
- Client plugins
usersgroups_adpypsrp: added installation of Kerberos authentication dependencies
- Removed client plugin
usersgroups_adwinrmas it has poor performances and was redundant withusersgroups_adpypsrp
- Client plugins
usersgroups_adpypsrp: added a new optional settingUsers_mandatory_groupsthat allows to force each new user to be added to the specified group list
- Bumped python dependencies to their latest version:
- Cerberus
- cryptography (used by plugins/clients/usersgroups_adwinrm)
- Jinja2
- oracledb (used by plugins/datasources/oracle)
- Bumped python dependencies to their latest version:
- cryptography (used by plugins/clients/usersgroups_adwinrm)
- Added a new client plugin
usersgroups_bsspartageto manage Users, UserPasswords, Groups, GroupsMembers, GroupsSenders and Ressources on a RENATER's PARTAGE instance. - Added the
ldaphashhelper that replace the passlib dependency inplugins.attributes.ldapPasswordHashandplugins.clients.usersgroups_bsspartage - Added Python 3.13 to compatibility list
- The
plugins.attributes.ldapPasswordHashplugin now provides the hash list in the same order as the different algorithms were specified to it - Ensure the timezone info of datetime instances is discarded during serialization
plugins.datasources.ldap: when converting LDAP datetime string to datetime instance, discard the timezone info
- Moved helpers classes from
clients.helperstohelpers, as helpers can be useful to other types of plugins
- Removed passlib dependency, which is incompatible with Python 3.13
- Bumped python dependencies to their latest version:
- cryptography (used by plugins/clients/usersgroups_adwinrm)
- oracledb (used by plugins/datasources/oracle)
- Added
isAnErrorRetryread-only attribute that can let client plugin handler know if the current event is being processed as part of an error retry. This can be useful for example to perform additional checks when a library happens to throw exceptions even though it has correctly processed the requested changes, as python-ldap sometimes does. - Usergroups_LDAP client plugin: Significantly improved reliability in case of error recovery, by adding specific checks to determine what may have already been processed in
plugins.clients.usergroups_ldap
- Fixed a bug in autoremediation: two modified events were previously merged incorrectly. Tests have been updated to ensure this does not happen again
- Usergroups_LDAP client plugin: internal primary key attributes are now automatically added to the
attributesToIgnoreinplugins.clients.usergroups_ldap - Usergroups_LDAP client plugin: fixed missing escape of a value in LDAP search filter in
plugins.clients.usergroups_ldap - SQLite messagebus consumer plugin
plugins.messagebus_consumers.sqlite: fixed an error raised when the message bus was empty, e.g. when all events it contained had been purged because they had exceeded their retention duration. - Improved the way logs are handled in unit tests: now using a NullHandler instead of calls to logging.disable(). This fixes new functional tests that monitor logs, which were failing from github actions.
- Force a save of
_hermesconfig.jsonand_dataschema.jsonclients cache files at exit, to ensure cache files version update is saved, and avoid version migrations at each restart as those files aren't expected to be updated often.
- Bumped python dependencies to their latest version :
- kafka-python-ng (used by plugins/messagebus_consumers kafka and plugins/messagebus_producers/kafka)
- pycryptodomex (used by plugins/attributes/crypto_RSA_OAEP)
- Added foreign keys support (documentation).
- Added a
hermes-server.datamodel.*objtype*.foreignkeysconfiguration directive on server allowing to declare foreign keys, to improve error handling on client. The foreign keys will be propagated to clients. - Added a
hermes-client.foreignkeys_policyconfiguration directive on clients allowing to chose how to handle events on "parent objects" of objects with errors.
- Added a
- Fixed a KeyError exception that could only occur on client primary keys update, if a removed event was in error queue.
- kafka-python-ng set a default hard limit of 1MB for each message (event) sent to Kafka message bus. This limit can now be overriden by setting the new optional
plugins.messagebus.kafka.settings.max_request_sizeconfiguration directive forplugins.messagebus_producers.kafkaused only byhermes-server.
- Kafka messagebus consumer and producer plugins: fixed occasional
kafka.errors.NoBrokersAvailable: NoBrokersAvailableerrors caused by a timeout while detecting the Kafka broker API version. Now the broker API version is only detected at application startup and reported in the logs, allowing to declare it using a new optionalplugins.messagebus.kafka.settings.api_versionconfiguration directive, which will disable the broker API version detection.
- Bumped python dependencies to their latest version :
- PyYAML
- cryptography (used by plugins/clients/usersgroups_adwinrm)
- pywinrm (used by plugins/clients/usersgroups_adwinrm)
- oracledb (used by plugins/datasources/oracle)
- Added a directory
clients/helpers/to store some helpers modules that can be used by client plugins. - Added
clients.helper.commandhelper, to run local commands on client's host. - Added
clients.helper.randompasswordhelper, to generate random passwords with specific constraints. - Client plugins
usersgroups_adpypsrpandusersgroups_adwinrm: replaced the random generation of non-configurable passwords with theclients.helper.randompasswordhelper, and added the possibility of configuring it using the client plugin configuration file. - Added a new client plugin
usersgroups_kadmin_heimdalto manage users and their passwords in an Heimdal Kerberos server.
- Added a setting
hermes.logs.long_string_limitto avoid to fill the logs with big strings content. If a string attribute content is greater than this limit, it will be truncated to this limit and marked as a LONG_STRING in logs. - Added the support of the python type
bytesfrom datasources. It works, but should be used only for small binary content. - Added the BLOB support in the Oracle datasource plugin (plugins/datasources/oracle/), now that
bytesare supported.
- Improved autoremediation. A new attribute
isPartiallyProcessedhas been added to clients, and should be used in most clients plugins. It must be set toTrueas soon as the slightest modification has been propagated to the target. It allows to merge events whosecurrentStepis different from 0 but whose previous steps have not modified anything on the target. See clients plugins error handling documentation for details. - Added the ability to fetch dictionnaries values for attributes : the dictionnaries can contain any supported value type : int, float, str, datetime, list, dict. No filtering of dict values is made : None, empty sub-dict, or empty list are left as provided.
- Added a new setting
hermes.cli_socket.dont_manage_sockfilethat allow to delegate the CLI server sockfile creation to SystemD. - Added a facultative configuration file named APPNAME-cli-config.yml for CLI tools to allow certain users to use the CLI without granting them read access to the configuration file.
- Added a new setting
hermes.umaskallowing to set up the default umask for each file or directory created by the application : cache dirs, cache files and log files. - Now using tox to validate code with black and flake8, run the test suite on each supported major version of Python, and print coverage results
- Added Python 3.11 and 3.12 to compatibility list
- Fixed an improper validation of the
from_raw_dictandfrom_json_dictarguments of the Dataschema constructor, which treated an empty dictionary as if the argument had not been defined. Thanks to Julien Houchard for reporting and fixing this bug. - Removed unnecessary cache and logs directories checks in CLI tools that could have caused failure in case of missing directory or missing write permissions.
- Fixed a bug which prevented the Kafka consumer plugin (plugins/messagebus_consumers/kafka/) from running due to an improper call to define the timeout.
- Fixed regex escape sequences that was generating SyntaxWarning on Python >= 3.12
- Updated the code to make it compatible with flake8, in addition to black
- Added missing dependency of
requests-credsspto client pluginusersgroups_adpypsrp - Clients stopped whenever the message bus was unavailable. Now they wait 60 seconds before retrying to contact it
- Server kept trying to contact the message bus when it was unavailable, causing excessive CPU load and log file filling. Now the server waits 60 seconds before retrying to contact it
- Bumped python dependencies to their latest version :
- Jinja2
- cryptography (used by plugins/clients/usersgroups_adwinrm)
- oracledb (used by plugins/datasources/oracle)
- kafka-python isn't maintained anymore : replaced by kafka-python-ng (used by plugins/messagebus_consumers kafka and plugins/messagebus_producers/kafka)
- First release, by @Boris Lechner. The code is stable enough to start testing it.