diff --git a/ChangeLog.md b/ChangeLog.md index e35dc0119f..4b904d2c19 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -1,4703 +1,5176 @@ +# Pacemaker-3.0.0-rc1 (14 Nov 2024) +* 1938 commits with 685 files changed, 26363 insertions(+), 33503 deletions(-) + +## Features added since Pacemaker-2.1.9 + +* For more details, especially about build option changes, see + https://projects.clusterlabs.org/w/projects/pacemaker/pacemaker_3.0_changes/ +* **upgrades:** drop support for rolling upgrades from versions less than 2.0.0 +* **Pacemaker Remote:** drop support for Pacemaker 1 cluster nodes connecting + to Pacemaker Remote 3 or later nodes or bundles, and Pacemaker 3 or later + cluster nodes connecting to Pacemaker Remote 1.1.14 or earlier nodes +* **build:** drop creation of deprecated `pacemaker_remoted` link to + `pacemaker-remoted` +* **environment:** drop support for deprecated `HA_cib_timeout`, + `HA_shutdown_delay`, `PCMK_cib_timeout`, `PCMK_dh_min_bits`, and + `PCMK_shutdown_delay` variables +* **environment:** `PCMK_panic_action` values are case-sensitive +* **CIB:** XML syntax must be correctly formed (no errors will be ignored) +* **CIB:** `validate-with` must be set, case-sensitive, and not `pacemaker-0.6`, + `pacemaker-0.7`, `pacemaker-1.1`, `pacemaker-next`, `transitional-0.6`, or + unknown +* **CIB:** deprecate `concurrent-fencing` cluster option, which now defaults to + true +* **CIB:** deprecate `record-pending` operation option +* **CIB:** drop support for deprecated cluster options + `crmd-finalization-timeout`, `crmd-integration-timeout`, and + `crmd-transition-delay`, and `remove-after-stop` +* **CIB:** drop support for deprecated `poweroff` value of `stonith-action` + cluster option +* **CIB:** drop support for nodes with `type` set to deprecated value `ping` or + an invalid value +* **CIB:** drop support for deprecated `nagios` and `upstart` resource classes +* **CIB:** drop support for deprecated `master` resources +* **CIB:** drop support for deprecated `masters` bundle option +* **CIB:** drop support for deprecated bundles using `rkt` containers +* **CIB:** drop support for `instance_attributes` in `rsc_defaults` +* **CIB:** drop support for deprecated `restart-type` resource option +* **CIB:** drop support for deprecated `can-fail`, `can_fail`, and + `role_after_failure` operation options +* **CIB:** drop support for deprecated `rsc-instance`, `with-rsc-instance`, + `first-instance`, and `then-instance` constraint options +* **CIB:** drop support for deprecated `lifetime` elements in constraints +* **CIB:** drop support for multiple top-level rules in location constraints or + name/value blocks +* **CIB:** drop support for `name` attribute when `id-ref` attribute is present +* **CIB:** drop support for deprecated `moon` in `date_spec` elements of rules +* **CIB:** `globally-unique` clone option defaults to true if `clone-node-max` + is greater than 1 +* **CIB:** location constraints with an invalid `score`, `boolean-op`, or + `role`, colocation constraints with an invalid `score`, `rsc-role`, or + `with-rsc-role`, and ticket constraints with an invalid `rsc-role` are now + ignored (previously, default values would be used) +* **CIB:** ignore node expressions for meta-attributes +* **CIB:** treat misconfigured rules and rule expressions as not passing +* **CIB:** treat negative `migration-threshold` as invalid and use default +* **CIB:** invalid fencing level indexes are rejected by schema +* **agents:** drop ocf:pacemaker:o2cb resource agent +* **agents:** do not pass `HA_mcp`, `HA_quorum_type`, `PCMK_mcp`, or + `PCMK_quorum_type`, environment variables to agents +* **libcrmcommon:** Drop environment variable +* **alerts:** don't send deprecated alert environment variables to agents +* **fencer:** drop support for non-clustered mode in fencer +* **fencing:** default `pcmk_host_argument` to `none` if `port` not advertised +* **liblrmd,libstonithd:** use standard default timeout (20s) for meta-data + actions +* **pacemaker-based:** reject remote users if PAM not available +* **tools:** `crm_shadow --reset` now requires `--force` +* **tools:** define behavior of `attrd_updater -Q` without `-N` +* **tools:** deprecate `cibadmin --local` option +* **tools:** drop `--text-fancy` command-line option from all tools +* **tools:** drop deprecated `cibadmin --host` and `--no-bcast` options +* **tools:** drop deprecated `crm_resource --get-property`, `--set-property`, + and `--xml-file` options +* **tools:** `cibadmin --md5-sum-versioned` no longer prints feature set +* **tools:** `crm_resource` rejects invalid timeouts +* **tools:** `crm_resource --delete` does not accept clone instances +* **tools:** `crm_resource --delete` without `-t opt` exits with a usage error +* **tools:** `crm_resource --delete` now succeeds if resource doesn't exist +* **tools:** `crm_resource --option` throws usage error if appropriate +* **tools:** drop deprecated `crm_mon` options `--as-html`, + `--disable-ncurses`, `--simple-status`, and `--web-cgi` + +## Fixes since Pacemaker-2.1.9 + +* **executor:** avoid use-after-free during shutdown +* **libcrmcommon:** rule expressions with the empty string as value now pass + when the corresponding node attribute is the empty string +* **libstonithd:** avoid use-after-free when retrieving metadata of Linux-HA + fence agents +* **libstonithd:** free escaped metadata descriptions correctly +* **scheduler:** apply promotion priority to cloned group instances +* **scheduler:** correctly retrieve any existing fail-count for increment +* **scheduler:** don't apply colocations twice for promotion priority +* **scheduler:** ignore `nvpair` with invalid `id-ref` +* **scheduler:** ignore `value` in `nvpair` with `id-ref` +* **scheduler:** use first action meta-attribute if block has duplicates +* **scheduler:** consider group locations when member is explicit colocation + dependent +* **tools:** don't trigger an assertion if stdout is closed +* **tools:** CIB clients retry signon if first try fails +* **tools:** don't double-free XML in `crm_verify` after schema update + +## Public API changes since Pacemaker-2.1.9 + +* **libcib:** drop `util_compat.h` header +* **libcib:** drop enum values `cib_database`, `cib_inhibit_bcast`, + `cib_mixed_update`, `cib_quorum_override`, and `cib_zero_copy` +* **libcib:** drop `cib_api_operations_t` members `delete_absolute()`, + inputfd(), `is_master()`, `quit()`, `set_master()`, `set_op_callback()`, + `set_slave()`, `set_slave_all()`, `signon_raw()`, `update()` +* **libcib:** drop `cib_t` member `op_callback()` +* **libcib:** drop functions `cib_get_generation()`, `cib_metadata()`, + `cib_pref()`, `get_object_parent()`, `get_object_path()`, + `get_object_root()`, `query_node_uname()`, and `set_standby()` +* **libcrmcluster:** add `pcmk_cluster_t` member `priv` +* **libcrmcluster:** drop enums (including all their values) + `crm_ais_msg_class`, `crm_ais_msg_types`, `crm_get_peer_flags`, + `crm_join_phase`, `crm_node_flags`, and `crm_status_type` +* **libcrmcluster:** drop type alias `crm_node_t` +* **libcrmcluster:** drop struct `crm_peer_node_s` +* **libcrmcluster:** drop `pcmk_cluster_t` members `cpg_handle`, + `group`, `nodeid`, `uname`, and `uuid` +* **libcrmcluster:** drop global variables `crm_have_quorum`, `crm_peer_cache`, + `crm_peer_seq`, `crm_remote_peer_cache` +* **libcrmcluster:** drop enum value `crm_join_nack_quiet` +* **libcrmcluster:** drop string constants `CRM_NODE_LOST`, `CRM_NODE_MEMBER` +* **libcrmcluster:** drop functions `cluster_connect_cpg()`, + `cluster_disconnect_cpg()`, `crm_active_peers()`, `crm_cluster_disconnect()`, + `crm_get_peer()`, `crm_get_peer_full()`, `crm_is_corosync_peer_active()`, + `crm_is_peer_active()`, `crm_join_phase_str()`, `crm_peer_destroy()`, + `crm_peer_init()`, `crm_peer_uname()`, `crm_peer_uuid()`, + `crm_remote_peer_cache_refresh()`, `crm_remote_peer_cache_remove()`, + `crm_remote_peer_cache_size()`, `crm_remote_peer_get()`, + `crm_set_autoreap()`, `crm_set_status_callback()`, `crm_terminate_member()`, + `crm_terminate_member_no_mainloop()`, `get_local_nodeid()`, + `get_local_node_name()`, `get_node_name()`, `is_corosync_cluster()`, + `pcmk_cpg_membership()`, `pcmk_message_common_cs()`, `reap_crm_member()`, + `send_cluster_message()`, `send_cluster_text()`, `set_uuid()`, and + `text2msg_type()` +* **libcrmcluster:** rename struct `crm_cluster_s` to `pcmk__cluster` +* **libcrmcommon:** add enum `pcmk_ipc_server` value `pcmk_ipc_unknown` +* **libcrmcommon:** rename structs `pe_action_s` to `pcmk__action`, `pe_node_s` + to `pcmk__scored_node`, `pe_node_shared_s` to `pcmk__node_details`, + `pe_resource_s` to `pcmk__resource`, and `pe_working_set_s` to + `pcmk__scheduler` +* **libcrmcommon:** add `pcmk_node_t` members `assign` and `private` +* **libcrmcommon:** add `pcmk_resource_t` member `private` +* **libcrmcommon:** change `pcmk_scheduler_t` member `flags` type to `uint64_t` +* **libcrmcommon:** add defined constants `PCMK_OCF_ROOT`, + `PCMK_SCHEDULER_INPUT_DIR`, `PCMK_SCHEMA_DIR`, `PCMK_VALUE_CRASH`, + `PCMK_VALUE_OFF`, and `PCMK_VALUE_REBOOT` +* **libcrmcommon:** deprecate defined constants `CIB_OPTIONS_FIRST`, + `CRM_SCHEMA_DIRECTORY`, `CRM_SYSTEM_STONITHD`, `CRM_XS`, `OCF_ROOT_DIR`, and + `PE_STATE_DIR` +* **libcrmcommon:** deprecate functions `calculate_on_disk_digest()`, + `calculate_operation_digest()`, `calculate_xml_versioned_digest()`, + `char2score()`, `crm_extended_logging()`, `crm_ipc_connect()`, + `crm_is_daemon_name()`, `crm_xml_cleanup()`, `crm_xml_init()`, + `crm_xml_sanitize_id()`, `crm_xml_set_id()`, `expand_idref()`, `free_xml()`, + `hash2nvpair()`, `pcmk_free_xml_subtree()`, `pcmk_nvpairs2xml_attrs()`, + `pcmk_sort_nvpairs()`, `pcmk_xml_attrs2nvpairs()`, and `sorted_xml()` +* **libcrmcommon:** drop headers `agents_compat.h`, `compatibility.h`, + `logging_compat.h`, `mainloop_compat.h`, `results_compat.h`, + `scores_compat.h`, `tags.h`, `tickets.h`, and `xml_io_compat.h` +* **libcrmcommon:** drop global variables `crm_config_error`, + `crm_config_warning`, `resource_class_functions[]`, `was_processing_error`, + and `was_processing_warning` +* **libcrmcommon:** drop enums (including all their values) + `action_fail_response`, `action_tasks`, `node_type`, `pcmk_rsc_flags`, + `pcmk_sched_flags`, `pe_action_flags`, `pe_discover_e`, `pe_link_state`, + `pe_obj_types`, `pe_ordering`, `pe_print_options`, `pe_restart`, + `rsc_recovery_type`, `rsc_start_requirement`, and `xml_log_options` +* **libcrmcommon:** drop enum `crm_ipc_flags` value `crm_ipc_server_error`, + and enum `crm_ipc_flags` value `crm_ipc_server_info` +* **libcrmcommon:** drop enum `ocf_exitcode` values `PCMK_OCF_CANCELLED`, + `PCMK_OCF_DEGRADED_MASTER`, `PCMK_OCF_EXEC_ERROR`, `PCMK_OCF_FAILED_MASTER`, + `PCMK_OCF_NOT_SUPPORTED`, `PCMK_OCF_OTHER_ERROR`, `PCMK_OCF_PENDING`, + `PCMK_OCF_RUNNING_MASTER`, `PCMK_OCF_SIGNAL`, and `PCMK_OCF_TIMEOUT` +* **libcrmcommon:** drop enum `pe_find` values `pe_find_anon`, `pe_find_any`, + `pe_find_clone`, `pe_find_current`, `pe_find_inactive`, and `pe_find_renamed` +* **libcrmcommon:** drop types `pcmk_assignment_methods_t`, + `pcmk_rsc_methods_t`, `pcmk_tag_t`, `pcmk_ticket_t`, + `resource_alloc_functions_t`, and `resource_object_functions_t` +* **libcrmcommon:** drop structs `pe_action_wrapper_s`, `pe_tag_s`, + `pe_ticket_s`, `resource_alloc_functions_s`, and + `resource_object_functions_s` +* **libcrmcommon:** drop `pcmk_scheduler_t` members `actions`, `action_id`, + `blocked_resources`, `colocation_constraints`, `config_hash`, `dc_uuid`, + `disabled_resources`, `failed`, `graph`, `localhost`, `max_valid_nodes`, + `ninstances`, `node_pending_timeout`, `now`, `num_synapse`, `op_defaults`, + `ordering_constraints`, `param_check`, `placement_constraints`, + `placement_strategy`, `priority_fencing_delay`, `recheck_by`, `resources`, + `rsc_defaults`, `shutdown_lock`, `singletons`, `stonith_action`, + `stonith_timeout`, `stop_needed`, `tags`, `template_rsc_sets`, `tickets`, and + `ticket_constraints` +* **libcrmcommon:** drop direct access to all members of `pcmk_action_t` +* **libcrmcommon:** drop `pcmk_node_t` members `count`, `fixed`, + `rsc_discover_mode`, and `weight` +* **libcrmcommon:** drop `pcmk_resource_t` members `actions`, `allocated_to`, + `allowed_nodes`, `children`, `clone_name`, `cluster`, `cmds`, `container`, + `dangling_migrations`, `exclusive_discover`, `failure_timeout`, `fillers`, + `fns`, `is_remote_node`, `known_on`, `lock_node`, `lock_time`, `meta`, + `migration_threshold`, `next_role`, `ops_xml`, `orig_xml`, `parameters`, + `parameter_cache`, `parent`, `partial_migration_source`, + `partial_migration_target`, `pending_node`, `pending_task`, `priority`, + `recovery_type`, `remote_reconnect_ms`, `restart_type`, `role`, `rsc_cons`, + `rsc_cons_lhs`, `rsc_location`, `rsc_tickets`, `running_on`, `sort_index`, + `stickiness`, `utilization`, `variant`, `variant_opaque`, and `xml` +* **libcrmcommon:** drop struct `pe_node_shared_s` members `allocated_rsc`, + `attrs`, `digest_cache`, `expected_up`, `id`, `is_dc`, `num_resources`, + `priority`, `remote_maintenance`, `remote_requires_reset`, `remote_rsc`, + `remote_was_fenced`, `rsc_discovery_enabled`, `scheduler`, `standby`, + `standby_onfail`, `type`, `uname`, `unpacked`, `unseen`, and `utilization` +* **libcrmcommon:** drop defined constants `CRMD_ACTION_CANCEL`, + `CRMD_ACTION_DELETE`, `CRMD_ACTION_DEMOTE`, `CRMD_ACTION_DEMOTED`, + `CRMD_ACTION_METADATA`, `CRMD_ACTION_MIGRATE`, `CRMD_ACTION_MIGRATED`, + `CRMD_ACTION_NOTIFIED`, `CRMD_ACTION_NOTIFY`, `CRMD_ACTION_PROMOTE`, + `CRMD_ACTION_PROMOTED`, `CRMD_ACTION_RELOAD`, `CRMD_ACTION_RELOAD_AGENT`, + `CRMD_ACTION_START`, `CRMD_ACTION_STARTED`, `CRMD_ACTION_STATUS`, + `CRMD_ACTION_STOP`, `CRMD_ACTION_STOPPED`, `CRMD_METADATA`, + `CRM_ATTR_RA_VERSION`, `CRM_DEFAULT_OP_TIMEOUT_S`, `CRM_INFINITY_S`, + `CRM_MINUS_INFINITY_S`, `CRM_OP_FENCE`, `CRM_OP_LOCAL_SHUTDOWN`, + `CRM_OP_LRM_QUERY`, `CRM_OP_LRM_REFRESH`, `CRM_OP_RELAXED_CLONE`, + `CRM_OP_RELAXED_SET`, `CRM_PLUS_INFINITY_S`, `EOS`, `F_CLIENTNAME`, + `F_CRM_DATA`, `F_CRM_DC_LEAVING`, `F_CRM_ELECTION_AGE_S`, + `F_CRM_ELECTION_AGE_US`, `F_CRM_ELECTION_ID`, `F_CRM_ELECTION_OWNER`, + `F_CRM_HOST_FROM`, `F_CRM_HOST_TO`, `F_CRM_JOIN_ID`, `F_CRM_MSG_TYPE`, + `F_CRM_ORIGIN`, `F_CRM_REFERENCE`, `F_CRM_SYS_FROM`, `F_CRM_SYS_TO`, + `F_CRM_TASK`, `F_CRM_TGRAPH`, `F_CRM_TGRAPH_INPUT`, `F_CRM_THROTTLE_MAX`, + `F_CRM_THROTTLE_MODE`, `F_CRM_USER`, `F_CRM_VERSION`, `F_ORIG`, `F_SEQ`, + `F_SUBTYPE`, `F_TYPE`, `F_XML_TAGNAME`, `INFINITY`, `INFINITY_S`, + `MAX_IPC_DELAY`, `MINUS_INFINITY_S`, `OFFLINESTATUS`, `ONLINESTATUS`, + `PCMK_DEFAULT_METADATA_TIMEOUT_MS`, `PCMK_RESOURCE_CLASS_NAGIOS`, + `PCMK_RESOURCE_CLASS_UPSTART`, `PCMK_XA_PROMOTED_MAX_LEGACY`, + `PCMK_XA_PROMOTED_NODE_MAX_LEGACY`, `PCMK_XE_PROMOTABLE_LEGACY`, + `PCMK_XE_PROMOTED_MAX_LEGACY`, `PCMK_XE_PROMOTED_NODE_MAX_LEGACY`, + `RSC_CANCEL`, `RSC_DELETE`, `RSC_DEMOTE`, `RSC_DEMOTED`, `RSC_METADATA`, + `RSC_MIGRATE`, `RSC_MIGRATED`, `RSC_NOTIFIED`, `RSC_NOTIFY`, `RSC_PROMOTE`, + `RSC_PROMOTED`, `RSC_ROLE_MASTER`, `RSC_ROLE_PROMOTED`, + `RSC_ROLE_PROMOTED_S`, `RSC_ROLE_SLAVE`, `RSC_ROLE_STARTED`, + `RSC_ROLE_STOPPED`, `RSC_ROLE_UNKNOWN`, `RSC_ROLE_UNPROMOTED`, `RSC_START`, + `RSC_STARTED`, `RSC_STATUS`, `RSC_STOP`, `RSC_STOPPED`, `SUPPORT_UPSTART`, + `T_ATTRD`, `T_CRM`, `XML_ACL_ATTR_ATTRIBUTE`, `XML_ACL_ATTR_KIND`, + `XML_ACL_ATTR_REF`, `XML_ACL_ATTR_REFv1`, `XML_ACL_ATTR_TAG`, + `XML_ACL_ATTR_TAGv1`, `XML_ACL_ATTR_XPATH`, `XML_ACL_TAG_DENY`, + `XML_ACL_TAG_GROUP`, `XML_ACL_TAG_PERMISSION`, `XML_ACL_TAG_READ`, + `XML_ACL_TAG_ROLE`, `XML_ACL_TAG_ROLE_REF`, `XML_ACL_TAG_ROLE_REFv1`, + `XML_ACL_TAG_USER`, `XML_ACL_TAG_USERv1`, `XML_ACL_TAG_WRITE`, + `XML_AGENT_ATTR_CLASS`, `XML_AGENT_ATTR_PROVIDER`, `XML_ALERT_ATTR_PATH`, + `XML_ALERT_ATTR_REC_VALUE`, `XML_ALERT_ATTR_TIMEOUT`, + `XML_ALERT_ATTR_TSTAMP_FORMAT`, `XML_ATTR_CRM_VERSION`, `XML_ATTR_DC_UUID`, + `XML_ATTR_DESC`, `XML_ATTR_DIGEST`, `XML_ATTR_GENERATION`, + `XML_ATTR_GENERATION_ADMIN`, `XML_ATTR_HAVE_QUORUM`, + `XML_ATTR_HAVE_WATCHDOG`, `XML_ATTR_ID`, `XML_ATTR_IDREF`, + `XML_ATTR_ID_LONG`, `XML_ATTR_NAME`, `XML_ATTR_NUMUPDATES`, `XML_ATTR_OP`, + `XML_ATTR_ORIGIN`, `XML_ATTR_QUORUM_PANIC`, `XML_ATTR_RA_VERSION`, + `XML_ATTR_REFERENCE`, `XML_ATTR_REQUEST`, `XML_ATTR_RESPONSE`, + `XML_ATTR_STONITH_DEVICES`, `XML_ATTR_STONITH_INDEX`, + `XML_ATTR_STONITH_TARGET`, `XML_ATTR_STONITH_TARGET_ATTRIBUTE`, + `XML_ATTR_STONITH_TARGET_PATTERN`, `XML_ATTR_STONITH_TARGET_VALUE`, + `XML_ATTR_TE_NOWAIT`, `XML_ATTR_TE_TARGET_RC`, `XML_ATTR_TIMEOUT`, + `XML_ATTR_TRANSITION_KEY`, `XML_ATTR_TRANSITION_MAGIC`, `XML_ATTR_TSTAMP`, + `XML_ATTR_TYPE`, `XML_ATTR_UNAME`, `XML_ATTR_UPDATE_CLIENT`, + `XML_ATTR_UPDATE_ORIG`, `XML_ATTR_UPDATE_USER`, `XML_ATTR_UUID`, + `XML_ATTR_VALIDATION`, `XML_ATTR_VERBOSE`, `XML_ATTR_VERSION`, + `XML_BOOLEAN_FALSE`, `XML_BOOLEAN_NO`, `XML_BOOLEAN_TRUE`, `XML_BOOLEAN_YES`, + `XML_CIB_ATTR_PRIORITY`, `XML_CIB_ATTR_REPLACE`, `XML_CIB_ATTR_SHUTDOWN`, + `XML_CIB_ATTR_SOURCE`, `XML_CIB_ATTR_WRITTEN`, `XML_CIB_TAG_ACLS`, + `XML_CIB_TAG_ALERT`, `XML_CIB_TAG_ALERTS`, `XML_CIB_TAG_ALERT_ATTR`, + `XML_CIB_TAG_ALERT_ATTRIBUTES`, `XML_CIB_TAG_ALERT_FENCING`, + `XML_CIB_TAG_ALERT_NODES`, `XML_CIB_TAG_ALERT_RECIPIENT`, + `XML_CIB_TAG_ALERT_RESOURCES`, `XML_CIB_TAG_ALERT_SELECT`, + `XML_CIB_TAG_CONFIGURATION`, `XML_CIB_TAG_CONSTRAINTS`, + `XML_CIB_TAG_CONTAINER`, `XML_CIB_TAG_CRMCONFIG`, `XML_CIB_TAG_DOMAINS`, + `XML_CIB_TAG_GENERATION_TUPPLE`, `XML_CIB_TAG_GROUP`, + `XML_CIB_TAG_INCARNATION`, `XML_CIB_TAG_LRM`, `XML_CIB_TAG_MASTER`, + `XML_CIB_TAG_NODE`, `XML_CIB_TAG_NODES`, `XML_CIB_TAG_NVPAIR`, + `XML_CIB_TAG_OBJ_REF`, `XML_CIB_TAG_OPCONFIG`, `XML_CIB_TAG_PROPSET`, + `XML_CIB_TAG_RESOURCE`, `XML_CIB_TAG_RESOURCES`, `XML_CIB_TAG_RSCCONFIG`, + `XML_CIB_TAG_RSC_TEMPLATE`, `XML_CIB_TAG_SECTION_ALL`, `XML_CIB_TAG_STATE`, + `XML_CIB_TAG_STATUS`, `XML_CIB_TAG_TAG`, `XML_CIB_TAG_TAGS`, + `XML_CIB_TAG_TICKETS`, `XML_CIB_TAG_TICKET_STATE`, `XML_COLOC_ATTR_INFLUENCE`, + `XML_COLOC_ATTR_NODE_ATTR`, `XML_COLOC_ATTR_SOURCE`, + `XML_COLOC_ATTR_SOURCE_INSTANCE`, `XML_COLOC_ATTR_SOURCE_ROLE`, + `XML_COLOC_ATTR_TARGET`, `XML_COLOC_ATTR_TARGET_INSTANCE`, + `XML_COLOC_ATTR_TARGET_ROLE`, `XML_CONFIG_ATTR_DC_DEADTIME`, + `XML_CONFIG_ATTR_ELECTION_FAIL`, `XML_CONFIG_ATTR_FENCE_REACTION`, + `XML_CONFIG_ATTR_FORCE_QUIT`, `XML_CONFIG_ATTR_NODE_PENDING_TIMEOUT`, + `XML_CONFIG_ATTR_PRIORITY_FENCING_DELAY`, `XML_CONFIG_ATTR_RECHECK`, + `XML_CONFIG_ATTR_SHUTDOWN_LOCK`, `XML_CONFIG_ATTR_SHUTDOWN_LOCK_LIMIT`, + `XML_CONS_ATTR_SYMMETRICAL`, `XML_CONS_TAG_RSC_DEPEND`, + `XML_CONS_TAG_RSC_LOCATION`, `XML_CONS_TAG_RSC_ORDER`, `XML_CONS_TAG_RSC_SET`, + `XML_CONS_TAG_RSC_TICKET`, `XML_CRM_TAG_PING`, `XML_DIFF_ATTR`, + `XML_DIFF_CHANGE`, `XML_DIFF_LIST`, `XML_DIFF_MARKER`, `XML_DIFF_OP`, + `XML_DIFF_PATH`, `XML_DIFF_POSITION`, `XML_DIFF_RESULT`, `XML_DIFF_VERSION`, + `XML_DIFF_VSOURCE`, `XML_DIFF_VTARGET`, `XML_EXPR_ATTR_ATTRIBUTE`, + `XML_EXPR_ATTR_OPERATION`, `XML_EXPR_ATTR_TYPE`, `XML_EXPR_ATTR_VALUE`, + `XML_EXPR_ATTR_VALUE_SOURCE`, `XML_FAILCIB_ATTR_ID`, + `XML_FAILCIB_ATTR_OBJTYPE`, `XML_FAILCIB_ATTR_OP`, `XML_FAILCIB_ATTR_REASON`, + `XML_FAIL_TAG_CIB`, `XML_GRAPH_TAG_CRM_EVENT`, `XML_GRAPH_TAG_DOWNED`, + `XML_GRAPH_TAG_MAINTENANCE`, `XML_GRAPH_TAG_PSEUDO_EVENT`, + `XML_GRAPH_TAG_RSC_OP`, `XML_LOCATION_ATTR_DISCOVERY`, `XML_LOC_ATTR_SOURCE`, + `XML_LOC_ATTR_SOURCE_PATTERN`, `XML_LRM_ATTR_CALLID`, + `XML_LRM_ATTR_EXIT_REASON`, `XML_LRM_ATTR_INTERVAL`, + `XML_LRM_ATTR_INTERVAL_MS`, `XML_LRM_ATTR_MIGRATE_SOURCE`, + `XML_LRM_ATTR_MIGRATE_TARGET`, `XML_LRM_ATTR_OPSTATUS`, + `XML_LRM_ATTR_OP_DIGEST`, `XML_LRM_ATTR_OP_RESTART`, `XML_LRM_ATTR_OP_SECURE`, + `XML_LRM_ATTR_RC`, `XML_LRM_ATTR_RESTART_DIGEST`, `XML_LRM_ATTR_ROUTER_NODE`, + `XML_LRM_ATTR_RSCID`, `XML_LRM_ATTR_SECURE_DIGEST`, `XML_LRM_ATTR_TARGET`, + `XML_LRM_ATTR_TARGET_UUID`, `XML_LRM_ATTR_TASK`, `XML_LRM_ATTR_TASK_KEY`, + `XML_LRM_TAG_RESOURCE`, `XML_LRM_TAG_RESOURCES`, `XML_LRM_TAG_RSC_OP`, + `XML_NODE_ATTR_RSC_DISCOVERY`, `XML_NODE_EXPECTED`, `XML_NODE_IN_CLUSTER`, + `XML_NODE_IS_FENCED`, `XML_NODE_IS_MAINTENANCE`, `XML_NODE_IS_PEER`, + `XML_NODE_IS_REMOTE`, `XML_NODE_JOIN_STATE`, `XML_NVPAIR_ATTR_NAME`, + `XML_NVPAIR_ATTR_VALUE`, `XML_OP_ATTR_ALLOW_MIGRATE`, + `XML_OP_ATTR_DIGESTS_ALL`, `XML_OP_ATTR_DIGESTS_SECURE`, + `XML_OP_ATTR_ON_FAIL`, `XML_OP_ATTR_ORIGIN`, `XML_OP_ATTR_PENDING`, + `XML_OP_ATTR_START_DELAY`, `XML_ORDER_ATTR_FIRST`, + `XML_ORDER_ATTR_FIRST_ACTION`, `XML_ORDER_ATTR_FIRST_INSTANCE`, + `XML_ORDER_ATTR_KIND`, `XML_ORDER_ATTR_THEN`, `XML_ORDER_ATTR_THEN_ACTION`, + `XML_ORDER_ATTR_THEN_INSTANCE`, `XML_PARANOIA_CHECKS`, + `XML_PING_ATTR_CRMDSTATE`, `XML_PING_ATTR_PACEMAKERDSTATE`, + `XML_PING_ATTR_PACEMAKERDSTATE_INIT`, + `XML_PING_ATTR_PACEMAKERDSTATE_REMOTE`, + `XML_PING_ATTR_PACEMAKERDSTATE_RUNNING`, + `XML_PING_ATTR_PACEMAKERDSTATE_SHUTDOWNCOMPLETE` + `XML_PING_ATTR_PACEMAKERDSTATE_SHUTTINGDOWN` + `XML_PING_ATTR_PACEMAKERDSTATE_STARTINGDAEMONS` + `XML_PING_ATTR_PACEMAKERDSTATE_WAITPING`, `XML_PING_ATTR_STATUS`, + `XML_PING_ATTR_SYSFROM`, `XML_REMOTE_ATTR_RECONNECT_INTERVAL`, + `XML_RSC_ATTR_CLEAR_INTERVAL`, `XML_RSC_ATTR_CLEAR_OP`, + `XML_RSC_ATTR_CONTAINER`, `XML_RSC_ATTR_CRITICAL`, + `XML_RSC_ATTR_FAIL_STICKINESS`, `XML_RSC_ATTR_FAIL_TIMEOUT`, + `XML_RSC_ATTR_INCARNATION`, `XML_RSC_ATTR_INCARNATION_MAX`, + `XML_RSC_ATTR_INCARNATION_MIN`, `XML_RSC_ATTR_INCARNATION_NODEMAX`, + `XML_RSC_ATTR_INTERLEAVE`, `XML_RSC_ATTR_INTERNAL_RSC`, + `XML_RSC_ATTR_MAINTENANCE`, `XML_RSC_ATTR_MANAGED`, `XML_RSC_ATTR_MASTER_MAX`, + `XML_RSC_ATTR_MASTER_NODEMAX`, `XML_RSC_ATTR_MULTIPLE`, `XML_RSC_ATTR_NOTIFY`, + `XML_RSC_ATTR_ORDERED`, `XML_RSC_ATTR_PROMOTABLE`, + `XML_RSC_ATTR_PROMOTED_MAX`, `XML_RSC_ATTR_PROMOTED_NODEMAX`, + `XML_RSC_ATTR_PROVIDES`, `XML_RSC_ATTR_REMOTE_NODE`, + `XML_RSC_ATTR_REMOTE_RA_ADDR`, `XML_RSC_ATTR_REMOTE_RA_PORT`, + `XML_RSC_ATTR_REMOTE_RA_SERVER`, `XML_RSC_ATTR_REQUIRES`, + `XML_RSC_ATTR_RESTART`, `XML_RSC_ATTR_STICKINESS`, `XML_RSC_ATTR_TARGET`, + `XML_RSC_ATTR_TARGET_ROLE`, `XML_RSC_ATTR_UNIQUE`, `XML_RSC_OP_LAST_CHANGE`, + `XML_RSC_OP_LAST_RUN`, `XML_RSC_OP_T_EXEC`, `XML_RSC_OP_T_QUEUE`, + `XML_RULE_ATTR_BOOLEAN_OP`, `XML_RULE_ATTR_ROLE`, `XML_RULE_ATTR_SCORE`, + `XML_RULE_ATTR_SCORE_ATTRIBUTE`, `XML_TAG_ATTRS`, `XML_TAG_ATTR_SETS`, + `XML_TAG_DIFF`, `XML_TAG_DIFF_ADDED`, `XML_TAG_DIFF_REMOVED`, + `XML_TAG_EXPRESSION`, `XML_TAG_FAILED`, `XML_TAG_FENCING_LEVEL`, + `XML_TAG_FENCING_TOPOLOGY`, `XML_TAG_FRAGMENT`, `XML_TAG_GRAPH`, + `XML_TAG_META_SETS`, `XML_TAG_OPTIONS`, `XML_TAG_OP_VER_ATTRS`, + `XML_TAG_OP_VER_META`, `XML_TAG_PARAM`, `XML_TAG_PARAMS`, + `XML_TAG_RESOURCE_REF`, `XML_TAG_RSC_VER_ATTRS`, `XML_TAG_RULE`, + `XML_TAG_TRANSIENT_NODEATTRS`, `XML_TAG_UTILIZATION`, + `XML_TICKET_ATTR_LOSS_POLICY`, and `XML_TICKET_ATTR_TICKET` +* **libcrmcommon:** drop support for patchset format 1 in + `xml_apply_patchset()`, `xml_create_patchset()`, and `xml_patch_versions()` +* **libcrmcommon:** drop functions/macros `add_message_xml()`, + `add_node_copy()`, `add_node_nocopy()`, `apply_xml_diff()`, `bz2_strerror()`, + `can_prune_leaf()`, `copy_in_properties()` `create_hello_message()`, + `create_reply()`, `create_reply_adv()`, `create_request()`, + `create_request_adv()`, `create_xml_node()`, `crm_action_str()`, + `crm_add_logfile()`, `crm_atoi()`, `crm_build_path()`, `crm_destroy_xml()`, + `crm_errno2exit()`, `crm_ftoa()`, `crm_get_interval()`, + `crm_hash_table_size()`, `crm_itoa()`, `crm_itoa_stack()`, + `crm_log_cli_init()`, `crm_map_element_name()`, `crm_next_same_xml()`, + `crm_parse_int()`, `crm_parse_interval_spec()`, `crm_parse_ll()`, + `crm_provider_required()`, `crm_signal()`, `crm_str()`, + `crm_strcase_equal()`, `crm_strcase_hash()`, `crm_strcase_table_new()`, + `crm_strip_trailing_newline()`, `crm_str_eq()`, `crm_str_hash()`, + `crm_str_table_dup()`, `crm_str_table_new()`, `crm_ttoa()`, + `crm_xml_add_boolean()`, `crm_xml_escape()`, `crm_xml_replace()`, + `diff_xml_object()`, `do_crm_log_always()`, `dump_xml_formatted()`, + `dump_xml_formatted_with_text()`, `dump_xml_unformatted()`, + `expand_plus_plus()`, `filename2xml()`, `find_entity()`, + `find_xml_children()`, `find_xml_node()`, `first_named_child()`, + `fix_plus_plus_recursive()`, `getDocPtr()`, `get_message_xml()`, + `get_schema_name()`, `get_schema_version()`, `get_xpath_object_relative()`, + `g_str_hash_traditional()`, `ID()`, `is_not_set()`, `is_set_any()`, + `log_data_element()`, `pcmk_action_text()`, `pcmk_create_html_node()`, + `pcmk_create_xml_text_node()`, `pcmk_format_named_time()`, + `pcmk_format_nvpair()`, `pcmk_hostname()`, `pcmk_log_xml_impl()`, + `pcmk_numeric_strcasecmp()`, `pcmk_on_fail_text()`, `pcmk_parse_action()`, + `pcmk_scan_nvpair()`, `purge_diff_markers()`, `replace_xml_child()`, + `safe_str_eq()`, `safe_str_neq()`, `score2char()`, `score2char_stack()`, + `stdin2xml()`, `string2xml()` `subtract_xml_object()`, `TYPE()`, + `update_validation()`, `update_xml_child()`, `validate_xml()`, + `validate_xml_verbose()`, `write_xml_fd()`, `write_xml_file()`, + `xml_get_path()`, `xml_has_children()`, `xml_latest_schema()`, + `xml_log_changes()`, `xml_log_patchset()`, `xml_remove_prop()`, `__likely()`, + and `__unlikely()` +* **libcrmservice:** drop header `services_compat.h` +* **libcrmservice:** deprecate function `services_ocf_exitcode_str()` +* **libcrmservice:** drop enums (including all their values) `nagios_exitcode` + and `op_status` +* **libcrmservice:** drop defined constants `F_LRMD_ALERT`, `F_LRMD_ALERT_ID`, + `F_LRMD_ALERT_PATH`, `F_LRMD_CALLBACK_TOKEN`, `F_LRMD_CALLDATA`, + `F_LRMD_CALLID`, `F_LRMD_CALLOPTS`, `F_LRMD_CLASS`, `F_LRMD_CLIENTID`, + `F_LRMD_CLIENTNAME`, `F_LRMD_EXEC_RC`, `F_LRMD_IPC_CLIENT`, + `F_LRMD_IPC_IPC_SERVER`, `F_LRMD_IPC_MSG`, `F_LRMD_IPC_MSG_FLAGS`, + `F_LRMD_IPC_MSG_ID`, `F_LRMD_IPC_OP`, `F_LRMD_IPC_SESSION`, `F_LRMD_IPC_USER`, + `F_LRMD_IS_IPC_PROVIDER`, `F_LRMD_OPERATION`, `F_LRMD_OP_STATUS`, + `F_LRMD_ORIGIN`, `F_LRMD_PROTOCOL_VERSION`, `F_LRMD_PROVIDER`, `F_LRMD_RC`, + `F_LRMD_REMOTE_MSG_ID`, `F_LRMD_REMOTE_MSG_TYPE`, `F_LRMD_RSC`, + `F_LRMD_RSC_ACTION`, `F_LRMD_RSC_DELETED`, `F_LRMD_RSC_EXEC_TIME`, + `F_LRMD_RSC_EXIT_REASON`, `F_LRMD_RSC_ID`, `F_LRMD_RSC_INTERVAL`, + `F_LRMD_RSC_OUTPUT`, `F_LRMD_RSC_QUEUE_TIME`, `F_LRMD_RSC_RCCHANGE_TIME`, + `F_LRMD_RSC_RUN_TIME`, `F_LRMD_RSC_START_DELAY`, `F_LRMD_RSC_USERDATA_STR`, + `F_LRMD_TIMEOUT`, `F_LRMD_TYPE`, `F_LRMD_WATCHDOG`, `T_LRMD`, + `T_LRMD_IPC_PROXY`, `T_LRMD_NOTIFY`, `T_LRMD_REPLY`, and `T_LRMD_RSC_OP` +* **libcrmservice:** drop functions `services_action_create()`, + `services_get_ocf_exitcode()`, `services_list()`, and + `services_lrm_status_str()` +* **liblrmd:** drop header `lrmd_compat.h` +* **liblrmd:** redefine `lrmd_opt_drop_recurring` enum value +* **liblrmd:** change type of `lrmd_event_data_t` members `t_rcchange` and + `t_run` to `time_t` +* **liblrmd:** drop defined constants `ALT_REMOTE_KEY_LOCATION` and + `LRMD_MIN_PROTOCOL_VERSION` +* **libpacemaker:** add high-level API equivalents of `stonith_admin` options + including `pcmk_request_fencing()`, `pcmk_fence_history()`, + `pcmk_fence_installed()`, `pcmk_fence_last()`, `pcmk_fence_list_targets()`, + `pcmk_fence_metadata()`, `pcmk_fence_registered()`, + `pcmk_fence_register_level()`, `pcmk_fence_unregister_level()`, and + `pcmk_fence_validate()` +* **libpe_rules:** drop functions `find_expression_type()`, + `pe_evaluate_rules()`, `pe_eval_expr()`, `pe_eval_rules()`, + `pe_eval_subexpr()`, `pe_expand_re_matches()`, `pe_test_expression()`, + `pe_test_expression_full()`, `pe_test_expression_re()`, `pe_test_rule()`, + `pe_test_rule_full()`, `pe_test_rule_re()`, `test_expression()`, + `test_ruleset()`, and `unpack_instance_attributes()` +* **libpe_status,libpe_rules:** drop types `action_t`, `action_wrapper_t`, + `no_quorum_policy_t`, `pe_action_t`, `pe_action_wrapper_t`, `pe_node_t`, + `pe_resource_t`, `pe_tag_t`, `pe_ticket_t`, `tag_t`, and `ticket_t` +* **libpe_status,libpe_rules:** drop enums (including all their values) + `pe_check_parameters` and `pe_graph_flags` +* **libpe_status,libpe_rules:** drop defined constants `pe_flag_check_config`, + `pe_flag_concurrent_fencing`, `pe_flag_enable_unfencing`, + `pe_flag_have_remote_nodes`, `pe_flag_have_status`, + `pe_flag_maintenance_mode`, `pe_flag_no_compat`, `pe_flag_no_counts`, + `pe_flag_quick_location`, `pe_flag_remove_after_stop`, `pe_flag_sanitized`, + `pe_flag_show_scores`, `pe_flag_show_utilization`, `pe_flag_shutdown_lock`, + `pe_flag_startup_fencing`, `pe_flag_startup_probes`, + `pe_flag_start_failure_fatal`, `pe_flag_stdout`, `pe_flag_stonith_enabled`, + `pe_flag_stop_action_orphans`, `pe_flag_stop_everything`, + `pe_flag_stop_rsc_orphans`, `pe_flag_symmetric_cluster`, `pe_rsc_allocating`, + `pe_rsc_allow_migrate`, `pe_rsc_allow_remote_remotes`, `pe_rsc_block`, + `pe_rsc_critical`, `pe_rsc_detect_loop`, `pe_rsc_failed`, + `pe_rsc_failure_ignored`, `pe_rsc_fence_device`, `pe_rsc_is_container`, + `pe_rsc_maintenance`, `pe_rsc_merging`, `pe_rsc_needs_fencing`, + `pe_rsc_needs_quorum`, `pe_rsc_needs_unfencing`, `pe_rsc_notify`, + `pe_rsc_orphan`, `pe_rsc_orphan_container_filler`, `pe_rsc_promotable`, + `pe_rsc_provisional`, `pe_rsc_reload`, `pe_rsc_replica_container`, + `pe_rsc_restarting`, `pe_rsc_runnable`, `pe_rsc_starting`, + `pe_rsc_start_pending`, `pe_rsc_stop`, `pe_rsc_stopping`, + `pe_rsc_stop_unexpected`, `pe_rsc_unique`, `RSC_ROLE_MASTER_S`, + `RSC_ROLE_MAX`, `RSC_ROLE_PROMOTED_LEGACY_S`, `RSC_ROLE_SLAVE_S`, + `RSC_ROLE_STARTED_S`, `RSC_ROLE_STOPPED_S`, `RSC_ROLE_UNKNOWN_S`, + `RSC_ROLE_UNPROMOTED_LEGACY_S`, and `RSC_ROLE_UNPROMOTED_S` +* **libpe_status,libpe_rules:** drop functions fail2text(), `pe_pref()`, + `recovery2text()`, `role2text()`, `task2text()`, `text2role()`, and + `text2task()` +* **libpe_status:** drop functions/macros `pe_rsc_is_anon_clone()`, + `pe_rsc_is_bundled()`, `pe_rsc_is_clone()`, `pe_rsc_is_unique_clone()`, and + `status_print()` +* **libstonithd:** drop `stonith_t` member `call_timeout` +* **libstonithd:** drop `stonith_event_t` members `message` and `type` +* **libstonithd:** drop defined constants `T_STONITH_NOTIFY_DISCONNECT`, + `T_STONITH_NOTIFY_FENCE`, `T_STONITH_NOTIFY_HISTORY`, and + `T_STONITH_NOTIFY_HISTORY_SYNCED` +* **libstonithd:** drop function `get_stonith_provider()` + # Pacemaker-2.1.9 (31 Oct 2024) - 169 commits with 252 files changed, 4498 insertions(+), 2259 deletions(-) ## Features added since Pacemaker-2.1.8 + **build:** support building with libxml2 2.13.0 or newer + **CIB:** new no-quorum-policy value "fence" replaces now-deprecated "suicide" + **tools:** iso8601 supports standard `--output-as/--output-to` arguments ## Fixes since Pacemaker-2.1.8 + **tools:** restore crmadmin default timeout to 30 seconds instead of none *(regression introduced in 2.1.5)* + **tools:** `crm_resource` did not return error if schema upgrade failed *(regression introduced in 2.1.8)* + **CIB:** detect newly created alerts section *(regression introduced in 2.1.7)* + **CIB:** treat empty environment variables (`CIB_file` etc.) same as unset + **CIB:** remote CIB administration now cannot block server + **executor:** don't block during TLS handshakes + **executor:** discard any agent output after about 10MiB + **scheduler:** avoid memory leak when checking for unfencing-capable devices + **libcrmcommon:** check for overflow when parsing and manipulating date/times + **tools:** properly handle resources removed from configuration when displaying node history in `crm_mon` + **tools:** crmadmin -D/--dc_lookup no longer hangs when there is no DC + **tools:** don't assert if stdout or stderr is closed by caller ## Public API changes since Pacemaker-2.1.8 + **libcrmcommon:** add enum `pcmk_ra_caps` value `pcmk_ra_cap_cli_exec` + **libcrmcommon:** add `pcmk_cib_node_shutdown()` + **libcrmcommon:** add `pcmk_parse_score()` + **libcrmcommon:** deprecate `CRM_ASSERT()` + **libcrmcommon:** deprecate `PCMK_VALUE_FENCE_LEGACY` defined constant + **libstonithd:** add enum `stonith_call_options` value `st_opt_allow_self_fencing` + **libstonithd:** deprecate enum `stonith_call_options` value `st_opt_allow_suicide` + **libstonithd:** deprecate enum `stonith_call_options` value `st_opt_scope_local` + **libstonithd:** deprecate enum `stonith_call_options` value `st_opt_verbose` # Pacemaker-2.1.8 (08 Aug 2024) - 2559 commits with 511 files changed, 46898 insertions(+), 23417 deletions(-) ## Features added since Pacemaker-2.1.7 + **local options:** support `PCMK_panic_action="off"` or "sync-off" + **local options:** deprecate `PCMK_dh_min_bits` environment variable + **CIB:** deprecate omitting validate-with from the CIB or setting it to "none" or an unknown schema + **CIB:** deprecate "default" and "#default" as explicit meta-attribute values + **CIB:** deprecate resource-discovery-enabled node attribute + **CIB:** deprecate support for multiple top-level rules within a location constraint (a single rule may still contain multiple sub-rules) + **CIB:** deprecate support for node attribute expressions in rules beneath op, `op_defaults`, or fence device meta-attributes + **CIB:** deprecate support for rkt in bundles + **CIB:** drop support for (nonworking) rules based on the #role node attribute (role-based location constraints may still contain rules) + **CIB manager,controller,fencer,scheduler:** deprecate "metadata" command-line option (instead, use `crm_attribute` `--list-options` mentioned below) + **pacemaker-remoted:** newer schema files are now downloaded from the cluster, allowing more command-line tools to work when the Pacemaker Remote node has an older Pacemaker version than the cluster + **agents:** deprecate the ocf:pacemaker:o2cb resource agent + **tools:** deprecate `--text-fancy` command-line option in all tools + **tools:** `crm_attribute` `--list-options` lists all possible cluster options + **tools:** `crm_resource` `--list-options` lists all possible primitive meta-attributes or special fence device parameters + **tools:** `crm_verify` now reports invalid fence topology levels + **tools:** new `--score` option for cibadmin `--modify` and `crm_attribute` --update enables expansion of "++" and "+=" in attribute values without a warning (using such expansions without `--score` is now deprecated) + **tools:** `crm_ticket` supports standard `--output-as/--output-to` arguments ## Fixes since Pacemaker-2.1.7 + **tools:** restore the (deprecated) ability to automatically correct malformed XML passed via standard input *(regression introduced in 2.1.7)* + **tools:** `crm_verify` distinguishes configuration warnings and errors *(regression introduced in 2.1.7)* + **tools:** `crm_node` -i must initialize nodeid before passing pointer *(regression introduced in 2.1.7)* + **CIB manager:** avoid memory leak from asynchronous client requests *(regression introduced in 2.1.7)* + **scheduler:** don't apply colocations twice for promotion priority *(regression introduced in 2.1.7)* + **CIB:** restore the (deprecated) ability to use validate-with="pacemaker-next" *(regression introduced in 2.1.6)* + **controller:** avoid zombie children when asynchronous actions exit while a synchronous meta-data action is in progress *(regression introduced in 2.1.5)* + **libcrmcommon:** avoid file descriptor leak in asynchronous IPC clients *(regression introduced in 2.1.3)* + **scheduler:** avoid crash when logging an invalid utilization attribute value *(regression introduced in 2.1.3)* + **tools:** `crm_mon` no longer crashes on some platforms when the fencer connection is lost *(regression introduced in 2.1.0)* + **attribute manager:** write Pacemaker Remote node attributes even if node is not cached + **attribute manager:** avoid use-after-free when remote node in cluster node cache + **attribute manager:** correctly propagate utilization attributes to peers to avoid the possibility of later being written out as regular node attributes + **fencer:** correctly parse action-specific timeouts with units other than seconds + **fencer:** avoid unnecessary timeouts when the watchdog timeout is greater than a query timeout, per-device fencing timeout, or stonith-timeout + **libcrmcommon:** avoid possible buffer overflows when parsing and formatting date/times + **libcrmcommon:** don't assume next schema will validate when not transforming + **libcrmcommon:** when displaying XML, don't show "" for empty attribute values, and properly escape special characters + **libstonithd:** avoid double free when invalid replies are received + **scheduler:** if the user specifies a timeout of 0, use the default 20s as documented + **scheduler:** consider group's location constraints when a member is an explicit dependent in a colocation constraint + **scheduler:** sort promotable cloned group instances properly for promotion + **agents:** ocf:pacemaker:SysInfo respects `attrd_updater` dampening + **agents:** ocf:pacemaker:HealthSMART properly handles SMART data missing temperature + **tools:** cibadmin, `crm_attribute`, `crm_node`, `crm_resource`, `crm_shadow`, and `crm_ticket` now retry CIB connections after transient failures + **tools:** cibadmin `--replace` now leaves "++" and "+=" unexpanded in XML attribute values rather than wrongly treat them as 0 + **tools:** cibsecret avoids possible truncation issue in process listing + **tools:** `crm_attribute` `--node` localhost or `--node` auto works + **tools:** `crm_resource` ignores resource meta-attribute node expressions for consistency with how the cluster works + **tools:** `crm_resource` honors rules when getting utilization attributes + **tools:** `crm_verify` `--output-as=xml` includes detailed messages + **tools:** `crm_mon` exits upon loss of an attached pseudo-terminal to avoid possibility of 100% CPU usage (seen when run via sudo with `use_pty` configured) ## Public API changes since Pacemaker-2.1.7 + **libcib:** add `cib_score_update` `cib_call_options` value + **libcib:** deprecate functions `cib_get_generation()`, `cib_metadata()`, `cib_pref()`, `query_node_uname()`, and `set_standby()` + **libcib:** deprecate `T_CIB_DIFF_NOTIFY` + **libcib:** deprecate `` element in CIB create reply + **libcrmcluster:** add enum `pcmk_cluster_layer` + **libcrmcluster:** add functions `pcmk_cluster_connect()`, `pcmk_cluster_disconnect()`, `pcmk_cluster_layer_text()`, `pcmk_cluster_set_destroy_fn()`, `pcmk_cpg_set_confchg_fn()`, `pcmk_cpg_set_deliver_fn()`, and `pcmk_get_cluster_layer()` + **libcrmcluster:** add type `pcmk_cluster_t` + **libcrmcluster:** deprecate functions `cluster_connect_cpg()`, `cluster_disconnect_cpg()`, `crm_active_peers()`, `crm_cluster_connect()`, `crm_cluster_disconnect()`, `crm_get_peer()`, `crm_get_peer_full()`, `crm_is_corosync_peer_active()`, `crm_is_peer_active()`, `crm_join_phase_str()`, `crm_peer_destroy()`, `crm_peer_init()`, `crm_peer_uname()`, `crm_peer_uuid()`, `crm_remote_node_cache_size()`, `crm_remote_peer_cache_refresh()`, `crm_remote_peer_cache_remove()`, `crm_remote_peer_get()`, `crm_set_autoreap()`, `crm_set_status_callback()`, `get_cluster_type()`, `get_local_nodeid()`, `get_local_node_name()`, `get_node_name()`, `is_corosync_cluster()`, `name_for_cluster_type()`, `pcmk_cpg_membership()`, `pcmk_message_common_cs()`, `reap_crm_member()`, `send_cluster_message()`, `send_cluster_text()`, and `text2msg_type()` + **libcrmcluster:** deprecate enums `crm_ais_msg_types`, `crm_status_type`, `cluster_type_e`, `crm_ais_msg_class`, `crm_get_peer_flags`, `crm_join_phase`, and `crm_node_flags`, including all their values + **libcrmcluster:** deprecate global variables `crm_have_quorum`, `crm_peer_cache`, `crm_peer_seq`, and `crm_remote_peer_cache` + **libcrmcluster:** deprecate `crm_cluster_t` and struct `crm_cluster_s`, including all its members + **libcrmcluster:** deprecate `crm_node_t` and struct `crm_peer_node_s`, including all its members + **libcrmcluster:** deprecate constants `CRM_NODE_LOST` and `CRM_NODE_MEMBER` + **libcrmcommon:** add constants `PCMK_ACTION_METADATA`, `PCMK_META_ALLOW_MIGRATE`, `PCMK_META_ALLOW_UNHEALTHY_NODES`, `PCMK_META_CONTAINER_ATTRIBUTE_TARGET`, `PCMK_META_CRITICAL`, `PCMK_META_GLOBALLY_UNIQUE`, `PCMK_META_INTERLEAVE`, `PCMK_META_INTERVAL`, `PCMK_META_INTERVAL_ORIGIN`, `PCMK_META_IS_MANAGED`, `PCMK_META_MAINTENANCE`, `PCMK_META_MULTIPLE_ACTIVE`, `PCMK_META_NOTIFY`, `PCMK_META_ON_FAIL`, `PCMK_META_ORDERED`, `PCMK_META_PRIORITY`, `PCMK_META_PROMOTABLE`, `PCMK_META_RECORD_PENDING`, `PCMK_META_REMOTE_ADDR`, `PCMK_META_REMOTE_ALLOW_MIGRATE`, `PCMK_META_REMOTE_CONNECT_TIMEOUT`, `PCMK_META_REMOTE_NODE`, `PCMK_META_REMOTE_PORT`, `PCMK_META_REQUIRES`, `PCMK_META_RESOURCE_STICKINESS`, `PCMK_META_START_DELAY`, `PCMK_META_TARGET_ROLE`, `PCMK_META_TIMEOUT`, `PCMK_META_TIMESTAMP_FORMAT`, `PCMK_NODE_ATTR_MAINTENANCE`, `PCMK_NODE_ATTR_STANDBY`, `PCMK_OPT_BATCH_LIMIT`, `PCMK_OPT_CLUSTER_DELAY`, `PCMK_OPT_CLUSTER_INFRASTRUCTURE`, `PCMK_OPT_CLUSTER_IPC_LIMIT`, `PCMK_OPT_CLUSTER_NAME`, `PCMK_OPT_CLUSTER_RECHECK_INTERVAL`, `PCMK_OPT_CONCURRENT_FENCING`, `PCMK_OPT_DC_DEADTIME`, `PCMK_OPT_DC_VERSION`, `PCMK_OPT_ELECTION_TIMEOUT`, `PCMK_OPT_ENABLE_ACL`, `PCMK_OPT_ENABLE_STARTUP_PROBES`, `PCMK_OPT_FENCE_REACTION`, `PCMK_OPT_HAVE_WATCHDOG`, `PCMK_OPT_JOIN_FINALIZATION_TIMEOUT`, `PCMK_OPT_JOIN_INTEGRATION_TIMEOUT`, `PCMK_OPT_LOAD_THRESHOLD`, `PCMK_OPT_MAINTENANCE_MODE`, `PCMK_OPT_MIGRATION_LIMIT`, `PCMK_OPT_NODE_ACTION_LIMIT`, `PCMK_OPT_NODE_HEALTH_BASE`, `PCMK_OPT_NODE_HEALTH_GREEN`, `PCMK_OPT_NODE_HEALTH_RED`, `PCMK_OPT_NODE_HEALTH_STRATEGY`, `PCMK_OPT_NODE_HEALTH_YELLOW`, `PCMK_OPT_NODE_PENDING_TIMEOUT`, `PCMK_OPT_NO_QUORUM_POLICY`, `PCMK_OPT_PE_ERROR_SERIES_MAX`, `PCMK_OPT_PE_INPUT_SERIES_MAX`, `PCMK_OPT_PE_WARN_SERIES_MAX`, `PCMK_OPT_PLACEMENT_STRATEGY`, `PCMK_OPT_PRIORITY_FENCING_DELAY`, `PCMK_OPT_SHUTDOWN_ESCALATION`, `PCMK_OPT_SHUTDOWN_LOCK`, `PCMK_OPT_SHUTDOWN_LOCK_LIMIT`, `PCMK_OPT_STARTUP_FENCING`, `PCMK_OPT_START_FAILURE_IS_FATAL`, `PCMK_OPT_STONITH_ACTION`, `PCMK_OPT_STONITH_ENABLED`, `PCMK_OPT_STONITH_MAX_ATTEMPTS`, `PCMK_OPT_STONITH_TIMEOUT`, `PCMK_OPT_STONITH_WATCHDOG_TIMEOUT`, `PCMK_OPT_STOP_ALL_RESOURCES`, `PCMK_OPT_STOP_ORPHAN_ACTIONS`, `PCMK_OPT_STOP_ORPHAN_RESOURCES`, `PCMK_OPT_SYMMETRIC_CLUSTER`, `PCMK_OPT_TRANSITION_DELAY`, `PCMK_REMOTE_RA_ADDR`, `PCMK_REMOTE_RA_PORT`, `PCMK_REMOTE_RA_RECONNECT_INTERVAL`, `PCMK_REMOTE_RA_SERVER`, `PCMK_ROLE_PROMOTED`, `PCMK_ROLE_STARTED`, `PCMK_ROLE_STOPPED`, `PCMK_ROLE_UNPROMOTED`, `PCMK_SCORE_INFINITY`, `PCMK_VALUE_ALWAYS`, `PCMK_VALUE_AND`, `PCMK_VALUE_BALANCED`, `PCMK_VALUE_BLOCK`, `PCMK_VALUE_BOOLEAN`, `PCMK_VALUE_CIB_BOOTSTRAP_OPTIONS`, `PCMK_VALUE_COROSYNC`, `PCMK_VALUE_CREATE`, `PCMK_VALUE_CUSTOM`, `PCMK_VALUE_DATE_SPEC`, `PCMK_VALUE_DEFAULT`, `PCMK_VALUE_DEFINED`, `PCMK_VALUE_DELETE`, `PCMK_VALUE_DEMOTE`, `PCMK_VALUE_DENY`, `PCMK_VALUE_DURATION`, `PCMK_VALUE_DYNAMIC_LIST`, `PCMK_VALUE_EQ`, `PCMK_VALUE_EXCLUSIVE`, `PCMK_VALUE_FAILED`, `PCMK_VALUE_FALSE`, `PCMK_VALUE_FENCE`, `PCMK_VALUE_FENCE_LEGACY`, `PCMK_VALUE_FENCING`, `PCMK_VALUE_FREEZE`, `PCMK_VALUE_GRANTED`, `PCMK_VALUE_GREEN`, `PCMK_VALUE_GT`, `PCMK_VALUE_GTE`, `PCMK_VALUE_HOST`, `PCMK_VALUE_IGNORE`, `PCMK_VALUE_IN_RANGE`, `PCMK_VALUE_INFINITY`, `PCMK_VALUE_INTEGER`, `PCMK_VALUE_LITERAL`, `PCMK_VALUE_LT`, `PCMK_VALUE_LTE`, `PCMK_VALUE_MANDATORY`, `PCMK_VALUE_MEMBER`, `PCMK_VALUE_META`, `PCMK_VALUE_MIGRATE_ON_RED`, `PCMK_VALUE_MINIMAL`, `PCMK_VALUE_MINUS_INFINITY`, `PCMK_VALUE_MODIFY`, `PCMK_VALUE_MOVE`, `PCMK_VALUE_NE`, `PCMK_VALUE_NEVER`, `PCMK_VALUE_NONE`, `PCMK_VALUE_NONNEGATIVE_INTEGER`, `PCMK_VALUE_NOTHING`, `PCMK_VALUE_NOT_DEFINED`, `PCMK_VALUE_NUMBER`, `PCMK_VALUE_OFFLINE`, `PCMK_VALUE_ONLINE`, `PCMK_VALUE_ONLY_GREEN`, `PCMK_VALUE_OPTIONAL`, `PCMK_VALUE_OR`, `PCMK_VALUE_PANIC`, `PCMK_VALUE_PARAM`, `PCMK_VALUE_PENDING`, `PCMK_VALUE_PERCENTAGE`, `PCMK_VALUE_PLUS_INFINITY`, `PCMK_VALUE_PORT`, `PCMK_VALUE_PROGRESSIVE`, `PCMK_VALUE_QUORUM`, `PCMK_VALUE_READ`, `PCMK_VALUE_RED`, `PCMK_VALUE_REMOTE`, `PCMK_VALUE_RESTART`, `PCMK_VALUE_RESTART_CONTAINER`, `PCMK_VALUE_REVOKED`, `PCMK_VALUE_SCORE`, `PCMK_VALUE_SELECT`, `PCMK_VALUE_SERIALIZE`, `PCMK_VALUE_STANDBY`, `PCMK_VALUE_STATIC_LIST`, `PCMK_VALUE_STATUS`, `PCMK_VALUE_STOP`, `PCMK_VALUE_STOP_ONLY`, `PCMK_VALUE_STOP_START`, `PCMK_VALUE_STOP_UNEXPECTED`, `PCMK_VALUE_STRING`, `PCMK_VALUE_SUCCESS`, `PCMK_VALUE_TIMEOUT`, `PCMK_VALUE_TRUE`, `PCMK_VALUE_UNFENCING`, `PCMK_VALUE_UNKNOWN`, `PCMK_VALUE_UTILIZATION`, `PCMK_VALUE_VERSION`, `PCMK_VALUE_WRITE`, `PCMK_VALUE_YELLOW`, `PCMK_XA_ACTION`, `PCMK_XA_ACTIVE`, `PCMK_XA_ADD_HOST`, `PCMK_XA_ADMIN_EPOCH`, `PCMK_XA_ADVANCED`, `PCMK_XA_AGENT`, `PCMK_XA_API_VERSION`, `PCMK_XA_ATTRIBUTE`, `PCMK_XA_AUTHOR`, `PCMK_XA_AUTOMATIC`, `PCMK_XA_BLOCKED`, `PCMK_XA_BOOLEAN_OP`, `PCMK_XA_BUILD`, `PCMK_XA_CACHED`, `PCMK_XA_CALL`, `PCMK_XA_CIB_LAST_WRITTEN`, `PCMK_XA_CIB_NODE`, `PCMK_XA_CLASS`, `PCMK_XA_CLIENT`, `PCMK_XA_CODE`, `PCMK_XA_COMMENT`, `PCMK_XA_COMPLETED`, `PCMK_XA_CONTROL_PORT`, `PCMK_XA_COUNT`, `PCMK_XA_CRMD`, `PCMK_XA_CRM_DEBUG_ORIGIN`, `PCMK_XA_CRM_FEATURE_SET`, `PCMK_XA_CRM_TIMESTAMP`, `PCMK_XA_DAYS`, `PCMK_XA_DC_UUID`, `PCMK_XA_DEFAULT`, `PCMK_XA_DELEGATE`, `PCMK_XA_DESCRIPTION`, `PCMK_XA_DEST`, `PCMK_XA_DEVICE`, `PCMK_XA_DEVICES`, `PCMK_XA_DISABLED`, `PCMK_XA_DURATION`, `PCMK_XA_END`, `PCMK_XA_EPOCH`, `PCMK_XA_EXEC`, `PCMK_XA_EXECUTION_CODE`, `PCMK_XA_EXECUTION_DATE`, `PCMK_XA_EXECUTION_MESSAGE`, `PCMK_XA_EXEC_TIME`, `PCMK_XA_EXITCODE`, `PCMK_XA_EXITREASON`, `PCMK_XA_EXITSTATUS`, `PCMK_XA_EXIT_REASON`, `PCMK_XA_EXPECTED`, `PCMK_XA_EXPECTED_UP`, `PCMK_XA_EXTENDED_STATUS`, `PCMK_XA_FAILED`, `PCMK_XA_FAILURE_IGNORED`, `PCMK_XA_FAIL_COUNT`, `PCMK_XA_FEATURES`, `PCMK_XA_FEATURE_SET`, `PCMK_XA_FILE`, `PCMK_XA_FIRST`, `PCMK_XA_FIRST_ACTION`, `PCMK_XA_FOR`, `PCMK_XA_FUNCTION`, `PCMK_XA_GENERATED`, `PCMK_XA_HASH`, `PCMK_XA_HAVE_QUORUM`, `PCMK_XA_HEALTH`, `PCMK_XA_HOST`, `PCMK_XA_HOST_INTERFACE`, `PCMK_XA_HOST_NETMASK`, `PCMK_XA_HOURS`, `PCMK_XA_ID`, `PCMK_XA_ID_AS_RESOURCE`, `PCMK_XA_ID_REF`, `PCMK_XA_IMAGE`, `PCMK_XA_INDEX`, `PCMK_XA_INFLUENCE`, `PCMK_XA_INSTANCE`, `PCMK_XA_INTERNAL_PORT`, `PCMK_XA_INTERVAL`, `PCMK_XA_IP_RANGE_START`, `PCMK_XA_IS_DC`, `PCMK_XA_KIND`, `PCMK_XA_LANG`, `PCMK_XA_LAST_FAILURE`, `PCMK_XA_LAST_GRANTED`, `PCMK_XA_LAST_RC_CHANGE`, `PCMK_XA_LAST_UPDATED`, `PCMK_XA_LOCKED_TO`, `PCMK_XA_LOCKED_TO_HYPHEN`, `PCMK_XA_LOSS_POLICY`, `PCMK_XA_MAINTENANCE`, `PCMK_XA_MAINTENANCE_MODE`, `PCMK_XA_MANAGED`, `PCMK_XA_MESSAGE`, `PCMK_XA_MINUTES`, `PCMK_XA_MIXED_VERSION`, `PCMK_XA_MONTHDAYS`, `PCMK_XA_MONTHS`, `PCMK_XA_MULTI_STATE`, `PCMK_XA_NAME`, `PCMK_XA_NETWORK`, `PCMK_XA_NEXT_ROLE`, `PCMK_XA_NODE`, `PCMK_XA_NODEID`, `PCMK_XA_NODES_RUNNING_ON`, `PCMK_XA_NODE_ATTRIBUTE`, `PCMK_XA_NODE_NAME`, `PCMK_XA_NODE_PATH`, `PCMK_XA_NO_QUORUM_PANIC`, `PCMK_XA_NO_QUORUM_POLICY`, `PCMK_XA_NUMBER`, `PCMK_XA_NUMBER_RESOURCES`, `PCMK_XA_NUM_UPDATES`, `PCMK_XA_OBJECT_TYPE`, `PCMK_XA_ONLINE`, `PCMK_XA_ON_TARGET`, `PCMK_XA_OP`, `PCMK_XA_OPERATION`, `PCMK_XA_OPTIONS`, `PCMK_XA_OP_KEY`, `PCMK_XA_ORIGIN`, `PCMK_XA_ORPHAN`, `PCMK_XA_ORPHANED`, `PCMK_XA_PACEMAKERD_STATE`, `PCMK_XA_PATH`, `PCMK_XA_PENDING`, `PCMK_XA_PORT`, `PCMK_XA_PRESENT`, `PCMK_XA_PRIORITY_FENCING_DELAY_MS`, `PCMK_XA_PROGRAM`, `PCMK_XA_PROMOTABLE`, `PCMK_XA_PROMOTED_MAX`, `PCMK_XA_PROMOTED_ONLY`, `PCMK_XA_PROVIDER`, `PCMK_XA_QUEUED`, `PCMK_XA_QUEUE_TIME`, `PCMK_XA_QUORUM`, `PCMK_XA_RANGE`, `PCMK_XA_RC`, `PCMK_XA_RC_TEXT`, `PCMK_XA_REASON`, `PCMK_XA_REFERENCE`, `PCMK_XA_RELOADABLE`, `PCMK_XA_REMAIN_STOPPED`, `PCMK_XA_REMOTE_CLEAR_PORT`, `PCMK_XA_REMOTE_NODE`, `PCMK_XA_REMOTE_TLS_PORT`, `PCMK_XA_REPLICAS`, `PCMK_XA_REPLICAS_PER_HOST`, `PCMK_XA_REQUEST`, `PCMK_XA_REQUIRE_ALL`, `PCMK_XA_RESOURCE`, `PCMK_XA_RESOURCES_RUNNING`, `PCMK_XA_RESOURCE_AGENT`, `PCMK_XA_RESOURCE_DISCOVERY`, `PCMK_XA_RESULT`, `PCMK_XA_ROLE`, `PCMK_XA_RSC`, `PCMK_XA_RSC_PATTERN`, `PCMK_XA_RSC_ROLE`, `PCMK_XA_RULE_ID`, `PCMK_XA_RUNNING`, `PCMK_XA_RUNNING_ON`, `PCMK_XA_RUN_COMMAND`, `PCMK_XA_SCOPE`, `PCMK_XA_SCORE`, `PCMK_XA_SCORE_ATTRIBUTE`, `PCMK_XA_SECONDS`, `PCMK_XA_SEQUENTIAL`, `PCMK_XA_SHUTDOWN`, `PCMK_XA_SOURCE`, `PCMK_XA_SOURCE_DIR`, `PCMK_XA_SOURCE_DIR_ROOT`, `PCMK_XA_SPEC`, `PCMK_XA_STANDARD`, `PCMK_XA_STANDBY`, `PCMK_XA_STANDBY_ONFAIL`, `PCMK_XA_START`, `PCMK_XA_STATE`, `PCMK_XA_STATUS`, `PCMK_XA_STONITH_ENABLED`, `PCMK_XA_STONITH_TIMEOUT_MS`, `PCMK_XA_STOP_ALL_RESOURCES`, `PCMK_XA_SYMMETRICAL`, `PCMK_XA_SYMMETRIC_CLUSTER`, `PCMK_XA_SYS_FROM`, `PCMK_XA_TAG`, `PCMK_XA_TARGET`, `PCMK_XA_TARGET_ATTRIBUTE`, `PCMK_XA_TARGET_DIR`, `PCMK_XA_TARGET_PATTERN`, `PCMK_XA_TARGET_ROLE`, `PCMK_XA_TARGET_VALUE`, `PCMK_XA_TASK`, `PCMK_XA_TEMPLATE`, `PCMK_XA_THEN`, `PCMK_XA_THEN_ACTION`, `PCMK_XA_TICKET`, `PCMK_XA_TIME`, `PCMK_XA_TYPE`, `PCMK_XA_UNAME`, `PCMK_XA_UNCLEAN`, `PCMK_XA_UNHEALTHY`, `PCMK_XA_UNIQUE`, `PCMK_XA_UNMANAGED`, `PCMK_XA_UPDATE_CLIENT`, `PCMK_XA_UPDATE_ORIGIN`, `PCMK_XA_UPDATE_USER`, `PCMK_XA_USER`, `PCMK_XA_VALID`, `PCMK_XA_VALIDATE_WITH`, `PCMK_XA_VALUE`, `PCMK_XA_VALUE_SOURCE`, `PCMK_XA_VERSION`, `PCMK_XA_WATCHDOG`, `PCMK_XA_WEEKDAYS`, `PCMK_XA_WEEKS`, `PCMK_XA_WEEKYEARS`, `PCMK_XA_WEIGHT`, `PCMK_XA_WHEN`, `PCMK_XA_WITH_QUORUM`, `PCMK_XA_WITH_RSC`, `PCMK_XA_WITH_RSC_ROLE`, `PCMK_XA_XPATH`, `PCMK_XA_YEARDAYS`, `PCMK_XA_YEARS`, `PCMK_XE_ACLS`, `PCMK_XE_ACL_GROUP`, `PCMK_XE_ACL_PERMISSION`, `PCMK_XE_ACL_ROLE`, `PCMK_XE_ACL_TARGET`, `PCMK_XE_ACTION`, `PCMK_XE_ACTIONS`, `PCMK_XE_AGENT`, `PCMK_XE_AGENTS`, `PCMK_XE_AGENT_STATUS`, `PCMK_XE_ALERT`, `PCMK_XE_ALERTS`, `PCMK_XE_ALLOCATIONS`, `PCMK_XE_ALLOCATIONS_UTILIZATIONS`, `PCMK_XE_ATTRIBUTE`, `PCMK_XE_BAN`, `PCMK_XE_BANS`, `PCMK_XE_BUNDLE`, `PCMK_XE_CAPACITY`, `PCMK_XE_CHANGE`, `PCMK_XE_CHANGE_ATTR`, `PCMK_XE_CHANGE_LIST`, `PCMK_XE_CHANGE_RESULT`, `PCMK_XE_CHECK`, `PCMK_XE_CIB`, `PCMK_XE_CLONE`, `PCMK_XE_CLUSTER_ACTION`, `PCMK_XE_CLUSTER_INFO`, `PCMK_XE_CLUSTER_OPTIONS`, `PCMK_XE_CLUSTER_PROPERTY_SET`, `PCMK_XE_CLUSTER_STATUS`, `PCMK_XE_COMMAND`, `PCMK_XE_CONFIGURATION`, `PCMK_XE_CONSTRAINTS`, `PCMK_XE_CONTENT`, `PCMK_XE_CRM_CONFIG`, `PCMK_XE_CRM_MON`, `PCMK_XE_CRM_MON_DISCONNECTED`, `PCMK_XE_CURRENT_DC`, `PCMK_XE_DATE_SPEC`, `PCMK_XE_DC`, `PCMK_XE_DEPRECATED`, `PCMK_XE_DIFF`, `PCMK_XE_DIGEST`, `PCMK_XE_DIGESTS`, `PCMK_XE_DOCKER`, `PCMK_XE_DURATION`, `PCMK_XE_ERROR`, `PCMK_XE_ERRORS`, `PCMK_XE_EXPRESSION`, `PCMK_XE_FAILURE`, `PCMK_XE_FAILURES`, `PCMK_XE_FEATURE`, `PCMK_XE_FEATURES`, `PCMK_XE_FENCE_EVENT`, `PCMK_XE_FENCE_HISTORY`, `PCMK_XE_FENCING_ACTION`, `PCMK_XE_FENCING_LEVEL`, `PCMK_XE_FENCING_TOPOLOGY`, `PCMK_XE_GROUP`, `PCMK_XE_INJECT_ATTR`, `PCMK_XE_INJECT_SPEC`, `PCMK_XE_INSTANCE_ATTRIBUTES`, `PCMK_XE_INSTRUCTION`, `PCMK_XE_ITEM`, `PCMK_XE_LAST_CHANGE`, `PCMK_XE_LAST_FENCED`, `PCMK_XE_LAST_UPDATE`, `PCMK_XE_LIST`, `PCMK_XE_LONGDESC`, `PCMK_XE_METADATA`, `PCMK_XE_META_ATTRIBUTES`, `PCMK_XE_MODIFICATIONS`, `PCMK_XE_MODIFY_NODE`, `PCMK_XE_MODIFY_TICKET`, `PCMK_XE_NETWORK`, `PCMK_XE_NODE`, `PCMK_XE_NODES`, `PCMK_XE_NODES_CONFIGURED`, `PCMK_XE_NODE_ACTION`, `PCMK_XE_NODE_ATTRIBUTES`, `PCMK_XE_NODE_HISTORY`, `PCMK_XE_NODE_INFO`, `PCMK_XE_NODE_WEIGHT`, `PCMK_XE_NVPAIR`, `PCMK_XE_OBJ_REF`, `PCMK_XE_OP`, `PCMK_XE_OPERATION`, `PCMK_XE_OPERATIONS`, `PCMK_XE_OPERATION_HISTORY`, `PCMK_XE_OPTION`, `PCMK_XE_OP_DEFAULTS`, `PCMK_XE_OUTPUT`, `PCMK_XE_OVERRIDE`, `PCMK_XE_OVERRIDES`, `PCMK_XE_PACEMAKERD`, `PCMK_XE_PACEMAKER_RESULT`, `PCMK_XE_PARAMETER`, `PCMK_XE_PARAMETERS`, `PCMK_XE_PODMAN`, `PCMK_XE_PORT_MAPPING`, `PCMK_XE_POSITION`, `PCMK_XE_PRIMITIVE`, `PCMK_XE_PROMOTION_SCORE`, `PCMK_XE_PROVIDER`, `PCMK_XE_PROVIDERS`, `PCMK_XE_PSEUDO_ACTION`, `PCMK_XE_REASON`, `PCMK_XE_RECIPIENT`, `PCMK_XE_REPLICA`, `PCMK_XE_RESOURCE`, `PCMK_XE_RESOURCES`, `PCMK_XE_RESOURCES_CONFIGURED`, `PCMK_XE_RESOURCE_AGENT`, `PCMK_XE_RESOURCE_AGENT_ACTION`, `PCMK_XE_RESOURCE_CONFIG`, `PCMK_XE_RESOURCE_HISTORY`, `PCMK_XE_RESOURCE_REF`, `PCMK_XE_RESOURCE_SET`, `PCMK_XE_RESULT_CODE`, `PCMK_XE_REVISED_CLUSTER_STATUS`, `PCMK_XE_ROLE`, `PCMK_XE_RSC_ACTION`, `PCMK_XE_RSC_COLOCATION`, `PCMK_XE_RSC_DEFAULTS`, `PCMK_XE_RSC_LOCATION`, `PCMK_XE_RSC_ORDER`, `PCMK_XE_RSC_TICKET`, `PCMK_XE_RULE`, `PCMK_XE_RULE_CHECK`, `PCMK_XE_SELECT`, `PCMK_XE_SELECT_ATTRIBUTES`, `PCMK_XE_SELECT_FENCING`, `PCMK_XE_SELECT_NODES`, `PCMK_XE_SELECT_RESOURCES`, `PCMK_XE_SHADOW`, `PCMK_XE_SHORTDESC`, `PCMK_XE_SOURCE`, `PCMK_XE_SPECIAL`, `PCMK_XE_STACK`, `PCMK_XE_STATUS`, `PCMK_XE_STORAGE`, `PCMK_XE_STORAGE_MAPPING`, `PCMK_XE_SUMMARY`, `PCMK_XE_TAG`, `PCMK_XE_TAGS`, `PCMK_XE_TARGET`, `PCMK_XE_TEMPLATE`, `PCMK_XE_TICKET`, `PCMK_XE_TICKETS`, `PCMK_XE_TIMING`, `PCMK_XE_TIMINGS`, `PCMK_XE_TRANSITION`, `PCMK_XE_UTILIZATION`, `PCMK_XE_UTILIZATIONS`, `PCMK_XE_VALIDATE`, `PCMK_XE_VERSION`, `PCMK_XE_XML`, and `PCMK_XE_XML_PATCHSET` + **libcrmcommon:** add functions `pcmk_action_text()`, `pcmk_find_node()`, `pcmk_foreach_active_resource()`, `pcmk_get_dc()`, `pcmk_parse_interval_spec()`, `pcmk_get_no_quorum_policy()`, `pcmk_has_quorum()`, `pcmk_node_is_clean()`, `pcmk_update_configured_schema()`, `pcmk_node_is_in_maintenance()`, `pcmk_node_is_online()`, `pcmk_node_is_pending()`, `pcmk_node_is_shutting_down()`, `pcmk_on_fail_text()`, `pcmk_parse_action()`, `pcmk_parse_role()`, `pcmk_resource_id()`, `pcmk_resource_is_managed()`, `pcmk_role_text()`, and `pcmk_set_scheduler_cib()` + **libcrmcommon:** add type `pcmk_rule_input_t` + **libcrmcommon:** deprecate globals `crm_log_level`, `crm_trace_nonlog`, `was_processing_error`, and `was_processing_warning` + **libcrmcommon:** deprecate functions `add_message_xml()`, `add_node_copy()`, `can_prune_leaf()`, `cli_config_update()`, `copy_in_properties()`, `copy_xml()`, `create_hello_message()`, `pcmk_parse_action()`, `create_reply()`, `create_reply_adv()`, `create_request()`, `create_request_adv()`, `create_xml_node()`, `crm_map_element_name()`, `crm_next_same_xml()`, `crm_parse_interval_spec()`, `crm_xml_escape()`, `diff_xml_object()`, `dump_xml_formatted()`, `dump_xml_formatted_with_text()`, `dump_xml_unformatted()`, `expand_plus_plus()`, `filename2xml()`, `find_xml_children()`, `find_xml_node()`, `first_named_child()`, `fix_plus_plus_recursive()`, `get_message_xml()`, `get_schema_name()`, `get_schema_version()`, `get_xpath_object_relative()`, `ID()`, `pcmk_action_text()`, `pcmk_create_html_node()`, `pcmk_create_xml_text_node()`, `pcmk_hostname()`, `pcmk_on_fail_text()`, `purge_diff_markers()`, `replace_xml_child()`, `stdin2xml()`, `string2xml()`, `subtract_xml_object()`, `update_validation()`, `update_xml_child()`, `validate_xml()`, `validate_xml_verbose()`, `write_xml_fd()`, `write_xml_file()`, `xml_latest_schema()`, and `xml_remove_prop()` + **libcrmcommon:** deprecate constants `CIB_OPTIONS_FIRST`, `CRM_INFINITY_S`, `CRM_MINUS_INFINITY_S`, `CRM_OP_LOCAL_SHUTDOWN`, `CRM_PLUS_INFINITY_S`, `CRM_SCORE_INFINITY`, `F_CLIENTNAME`, `F_CRM_DATA`, `F_CRM_DC_LEAVING`, `F_CRM_ELECTION_AGE_S`, `F_CRM_ELECTION_AGE_US`, `F_CRM_ELECTION_ID`, `F_CRM_ELECTION_OWNER`, `F_CRM_HOST_FROM`, `F_CRM_HOST_TO`, `F_CRM_JOIN_ID`, `F_CRM_MSG_TYPE`, `F_CRM_ORIGIN`, `F_CRM_REFERENCE`, `F_CRM_SYS_FROM`, `F_CRM_SYS_TO`, `F_CRM_TASK`, `F_CRM_TGRAPH`, `F_CRM_TGRAPH_INPUT`, `F_CRM_THROTTLE_MAX`, `F_CRM_THROTTLE_MODE`, `F_CRM_USER`, `F_CRM_VERSION`, `F_ORIG`, `F_SEQ`, `F_SUBTYPE`, `F_TYPE`, `F_XML_TAGNAME`, `INFINITY`, `INFINITY_S`, `MINUS_INFINITY_S`, `OFFLINESTATUS`, `ONLINESTATUS`, `PCMK_XA_PROMOTED_MAX_LEGACY`, `PCMK_XA_PROMOTED_NODE_MAX_LEGACY`, `PCMK_XA_TARGET_PATTERN`, `PCMK_XA_UNAME`, `PCMK_XE_PROMOTABLE_LEGACY`, `SUPPORT_UPSTART`, `T_ATTRD`, `T_CRM`, `XML_ACL_ATTR_ATTRIBUTE`, `XML_ACL_ATTR_KIND`, `XML_ACL_ATTR_REF`, `XML_ACL_ATTR_REFv1`, `XML_ACL_ATTR_TAG`, `XML_ACL_ATTR_TAGv1`, `XML_ACL_ATTR_XPATH`, `XML_ACL_TAG_DENY`, `XML_ACL_TAG_GROUP`, `XML_ACL_TAG_PERMISSION`, `XML_ACL_TAG_READ`, `XML_ACL_TAG_ROLE`, `XML_ACL_TAG_ROLE_REF`, `XML_ACL_TAG_ROLE_REFv1`, `XML_ACL_TAG_USER`, `XML_ACL_TAG_USERv1`, `XML_ACL_TAG_WRITE`, `XML_AGENT_ATTR_CLASS`, `XML_AGENT_ATTR_PROVIDER`, `XML_ALERT_ATTR_PATH`, `XML_ALERT_ATTR_REC_VALUE`, `XML_ALERT_ATTR_TIMEOUT`, `XML_ALERT_ATTR_TSTAMP_FORMAT`, `XML_ATTR_CRM_VERSION`, `XML_ATTR_DC_UUID`, `XML_ATTR_DESC`, `XML_ATTR_DIGEST`, `XML_ATTR_GENERATION`, `XML_ATTR_GENERATION_ADMIN`, `XML_ATTR_HAVE_QUORUM`, `XML_ATTR_HAVE_WATCHDOG`, `XML_ATTR_ID`, `XML_ATTR_IDREF`, `XML_ATTR_ID_LONG`, `XML_ATTR_NAME`, `XML_ATTR_NUMUPDATES`, `XML_ATTR_OP`, `XML_ATTR_ORIGIN`, `XML_ATTR_QUORUM_PANIC`, `XML_ATTR_REFERENCE`, `XML_ATTR_REQUEST`, `XML_ATTR_RESPONSE`, `XML_ATTR_STONITH_DEVICES`, `XML_ATTR_STONITH_INDEX`, `XML_ATTR_STONITH_TARGET`, `XML_ATTR_STONITH_TARGET_ATTRIBUTE`, `XML_ATTR_STONITH_TARGET_VALUE`, `XML_ATTR_TE_NOWAIT`, `XML_ATTR_TE_TARGET_RC`, `XML_ATTR_TIMEOUT`, `XML_ATTR_TRANSITION_KEY`, `XML_ATTR_TRANSITION_MAGIC`, `XML_ATTR_TSTAMP`, `XML_ATTR_TYPE`, `XML_ATTR_UPDATE_CLIENT`, `XML_ATTR_UPDATE_ORIGIN`, `XML_ATTR_UPDATE_USER`, `XML_ATTR_VALIDATION`, `XML_ATTR_VERSION`, `XML_BOOLEAN_FALSE`, `XML_BOOLEAN_NO`, `XML_BOOLEAN_TRUE`, `XML_BOOLEAN_YES`, `XML_CIB_ATTR_PRIORITY`, `XML_CIB_ATTR_SHUTDOWN`, `XML_CIB_ATTR_WRITTEN`, `XML_CIB_TAG_ACLS`, `XML_CIB_TAG_ALERT`, `XML_CIB_TAG_ALERTS`, `XML_CIB_TAG_ALERT_ATTR`, `XML_CIB_TAG_ALERT_ATTRIBUTES`, `XML_CIB_TAG_ALERT_FENCING`, `XML_CIB_TAG_ALERT_NODES`, `XML_CIB_TAG_ALERT_RECIPIENT`, `XML_CIB_TAG_ALERT_RESOURCES`, `XML_CIB_TAG_ALERT_SELECT`, `XML_CIB_TAG_CONFIGURATION`, `XML_CIB_TAG_CONSTRAINTS`, `XML_CIB_TAG_CONTAINER`, `XML_CIB_TAG_CRMCONFIG`, `XML_CIB_TAG_GENERATION_TUPPLE`, `XML_CIB_TAG_GROUP`, `XML_CIB_TAG_INCARNATION`, `XML_CIB_TAG_LRM`, `XML_CIB_TAG_NODE`, `XML_CIB_TAG_NODES`, `XML_CIB_TAG_NVPAIR`, `XML_CIB_TAG_OBJ_REF`, `XML_CIB_TAG_OPCONFIG`, `XML_CIB_TAG_PROPSET`, `XML_CIB_TAG_RESOURCE`, `XML_CIB_TAG_RESOURCES`, `XML_CIB_TAG_RSCCONFIG`, `XML_CIB_TAG_RSC_TEMPLATE`, `XML_CIB_TAG_SECTION_ALL`, `XML_CIB_TAG_STATE`, `XML_CIB_TAG_STATUS`, `XML_CIB_TAG_TAG`, `XML_CIB_TAG_TAGS`, `XML_CIB_TAG_TICKETS`, `XML_CIB_TAG_TICKET_STATE`, `XML_COLOC_ATTR_INFLUENCE`, `XML_COLOC_ATTR_NODE_ATTR`, `XML_COLOC_ATTR_SOURCE`, `XML_COLOC_ATTR_SOURCE_INSTANCE`, `XML_COLOC_ATTR_SOURCE_ROLE`, `XML_COLOC_ATTR_TARGET`, `XML_COLOC_ATTR_TARGET_INSTANCE`, `XML_COLOC_ATTR_TARGET_ROLE`, `XML_CONFIG_ATTR_DC_DEADTIME`, `XML_CONFIG_ATTR_ELECTION_FAIL`, `XML_CONFIG_ATTR_FENCE_REACTION`, `XML_CONFIG_ATTR_FORCE_QUIT`, `XML_CONFIG_ATTR_NODE_PENDING_TIMEOUT`, `XML_CONFIG_ATTR_PRIORITY_FENCING_DELAY`, `XML_CONFIG_ATTR_RECHECK`, `XML_CONFIG_ATTR_SHUTDOWN_LOCK`, `XML_CONFIG_ATTR_SHUTDOWN_LOCK_LIMIT`, `XML_CONS_ATTR_SYMMETRICAL`, `XML_CONS_TAG_RSC_DEPEND`, `XML_CONS_TAG_RSC_LOCATION`, `XML_CONS_TAG_RSC_ORDER`, `XML_CONS_TAG_RSC_SET`, `XML_CONS_TAG_RSC_TICKET`, `XML_CRM_TAG_PING`, `XML_DIFF_ATTR`, `XML_DIFF_CHANGE`, `XML_DIFF_LIST`, `XML_DIFF_MARKER`, `XML_DIFF_OP`, `XML_DIFF_PATH`, `XML_DIFF_POSITION`, `XML_DIFF_RESULT`, `XML_DIFF_VERSION`, `XML_DIFF_VSOURCE`, `XML_DIFF_VTARGET`, `XML_EXPR_ATTR_ATTRIBUTE`, `XML_EXPR_ATTR_OPERATION`, `XML_EXPR_ATTR_TYPE`, `XML_EXPR_ATTR_VALUE`, `XML_EXPR_ATTR_VALUE_SOURCE`, `XML_FAILCIB_ATTR_ID`, `XML_FAILCIB_ATTR_OBJTYPE`, `XML_FAILCIB_ATTR_OP`, `XML_FAILCIB_ATTR_REASON`, `XML_FAIL_TAG_CIB`, `XML_GRAPH_TAG_CRM_EVENT`, `XML_GRAPH_TAG_DOWNED`, `XML_GRAPH_TAG_MAINTENANCE`, `XML_GRAPH_TAG_PSEUDO_EVENT`, `XML_GRAPH_TAG_RSC_OP`, `XML_LOCATION_ATTR_DISCOVERY`, `XML_LOC_ATTR_SOURCE`, `XML_LOC_ATTR_SOURCE_PATTERN`, `XML_LRM_ATTR_CALLID`, `XML_LRM_ATTR_EXIT_REASON`, `XML_LRM_ATTR_INTERVAL`, `XML_LRM_ATTR_INTERVAL_MS`, `XML_LRM_ATTR_MIGRATE_SOURCE`, `XML_LRM_ATTR_MIGRATE_TARGET`, `XML_LRM_ATTR_OPSTATUS`, `XML_LRM_ATTR_OP_DIGEST`, `XML_LRM_ATTR_OP_RESTART`, `XML_LRM_ATTR_OP_SECURE`, `XML_LRM_ATTR_RC`, `XML_LRM_ATTR_RESTART_DIGEST`, `XML_LRM_ATTR_ROUTER_NODE`, `XML_LRM_ATTR_RSCID`, `XML_LRM_ATTR_SECURE_DIGEST`, `XML_LRM_ATTR_TARGET`, `XML_LRM_ATTR_TARGET_UUID`, `XML_LRM_ATTR_TASK`, `XML_LRM_ATTR_TASK_KEY`, `XML_LRM_TAG_RESOURCE`, `XML_LRM_TAG_RESOURCES`, `XML_LRM_TAG_RSC_OP`, `XML_NODE_ATTR_RSC_DISCOVERY`, `XML_NODE_IS_FENCED`, `XML_NODE_IS_MAINTENANCE`, `XML_NODE_IS_REMOTE`, `XML_NVPAIR_ATTR_NAME`, `XML_NVPAIR_ATTR_VALUE`, `XML_OP_ATTR_ALLOW_MIGRATE`, `XML_OP_ATTR_DIGESTS_ALL`, `XML_OP_ATTR_DIGESTS_SECURE`, `XML_OP_ATTR_INTERVAL_ORIGIN`, `XML_OP_ATTR_ON_FAIL`, `XML_OP_ATTR_PENDING`, `XML_OP_ATTR_START_DELAY`, `XML_ORDER_ATTR_FIRST`, `XML_ORDER_ATTR_FIRST_ACTION`, `XML_ORDER_ATTR_KIND`, `XML_ORDER_ATTR_THEN`, `XML_ORDER_ATTR_THEN_ACTION`, `XML_PING_ATTR_CRMDSTATE`, `XML_PING_ATTR_PACEMAKERDSTATE`, `XML_PING_ATTR_PACEMAKERDSTATE_INIT`, `XML_PING_ATTR_PACEMAKERDSTATE_REMOTE`, `XML_PING_ATTR_PACEMAKERDSTATE_RUNNING`, `XML_PING_ATTR_PACEMAKERDSTATE_SHUTDOWNCOMPLETE`, `XML_PING_ATTR_PACEMAKERDSTATE_SHUTTINGDOWN`, `XML_PING_ATTR_PACEMAKERDSTATE_STARTINGDAEMONS`, `XML_PING_ATTR_PACEMAKERDSTATE_WAITPING`, `XML_PING_ATTR_STATUS`, `XML_PING_ATTR_SYSFROM`, `XML_REMOTE_ATTR_RECONNECT_INTERVAL`, `XML_RSC_ATTR_CLEAR_INTERVAL`, `XML_RSC_ATTR_CLEAR_OP`, `XML_RSC_ATTR_CONTAINER`, `XML_RSC_ATTR_CRITICAL`, `XML_RSC_ATTR_INCARNATION`, `XML_RSC_ATTR_INTERLEAVE`, `XML_RSC_ATTR_INTERNAL_RSC`, `XML_RSC_ATTR_MAINTENANCE`, `XML_RSC_ATTR_MANAGED`, `XML_RSC_ATTR_MULTIPLE`, `XML_RSC_ATTR_NOTIFY`, `XML_RSC_ATTR_ORDERED`, `XML_RSC_ATTR_PROMOTABLE`, `XML_RSC_ATTR_REMOTE_NODE`, `XML_RSC_ATTR_REMOTE_RA_ADDR`, `XML_RSC_ATTR_REMOTE_RA_PORT`, `XML_RSC_ATTR_REMOTE_RA_SERVER`, `XML_RSC_ATTR_REQUIRES`, `XML_RSC_ATTR_RESTART`, `XML_RSC_ATTR_STICKINESS`, `XML_RSC_ATTR_TARGET`, `XML_RSC_ATTR_TARGET_ROLE`, `XML_RSC_ATTR_UNIQUE`, `XML_RSC_OP_LAST_CHANGE`, `XML_RSC_OP_T_EXEC`, `XML_RSC_OP_T_QUEUE`, `XML_RULE_ATTR_BOOLEAN_OP`, `XML_RULE_ATTR_ROLE`, `XML_RULE_ATTR_SCORE`, `XML_RULE_ATTR_SCORE_ATTRIBUTE`, `XML_TAG_ATTRS`, `XML_TAG_ATTR_SETS`, `XML_TAG_CIB`, `XML_TAG_DIFF`, `XML_TAG_EXPRESSION`, `XML_TAG_FAILED`, `XML_TAG_FENCING_LEVEL`, `XML_TAG_FENCING_TOPOLOGY`, `XML_TAG_GRAPH`, `XML_TAG_META_SETS`, `XML_TAG_OPTIONS`, `XML_TAG_PARAM`, `XML_TAG_PARAMS`, `XML_TAG_RESOURCE_REF`, `XML_TAG_RULE`, `XML_TAG_TRANSIENT_NODEATTRS`, `XML_TAG_UTILIZATION`, `XML_TICKET_ATTR_LOSS_POLICY`, and `XML_TICKET_ATTR_TICKET` + **libcrmcommon:** deprecate direct access to all members of `pcmk_scheduler_t`, `pcmk_tag_t`, and `pcmk_ticket_t` + **libcrmcommon:** deprecate `pcmk_rsc_methods_t`, `pcmk_assignment_methods_t`, struct `pe_action_s`, struct `pe_resource_s`, struct `resource_alloc_functions_s`, struct `resource_object_functions_s`, struct `pe_node_s`, and struct `pe_node_shared_s`, including all their members + **libcrmcommon:** deprecate enums `action_fail_response`, `action_tasks`, `expression_type`, `node_type`, `pcmk_rsc_flags`, `pcmk_scheduler_flags`, `pe_action_flags`, `pe_discover_e`, `pe_obj_types`, `rsc_recovery_type`, and `rsc_start_requirement`, including all their values + **liblrmd:** deprecate constants `F_LRMD_ALERT`, `F_LRMD_ALERT_ID`, `F_LRMD_ALERT_PATH`, `F_LRMD_CALLBACK_TOKEN`, `F_LRMD_CALLDATA`, `F_LRMD_CALLID`, `F_LRMD_CALLOPTS`, `F_LRMD_CLASS`, `F_LRMD_CLIENTID`, `F_LRMD_CLIENTNAME`, `F_LRMD_EXEC_RC`, `F_LRMD_IPC_CLIENT`, `F_LRMD_IPC_IPC_SERVER`, `F_LRMD_IPC_MSG`, `F_LRMD_IPC_MSG_FLAGS`, `F_LRMD_IPC_MSG_ID`, `F_LRMD_IPC_OP`, `F_LRMD_IPC_USER`, `F_LRMD_IS_IPC_PROVIDER`, `F_LRMD_OPERATION`, `F_LRMD_OP_STATUS`, `F_LRMD_ORIGIN`, `F_LRMD_PROTOCOL_VERSION`, `F_LRMD_PROVIDER`, `F_LRMD_RC`, `F_LRMD_REMOTE_MSG_ID`, `F_LRMD_REMOTE_MSG_TYPE`, `F_LRMD_RSC`, `F_LRMD_RSC_ACTION`, `F_LRMD_RSC_DELETED`, `F_LRMD_RSC_EXEC_TIME`, `F_LRMD_RSC_EXIT_REASON`, `F_LRMD_RSC_ID`, `F_LRMD_RSC_INTERVAL`, `F_LRMD_RSC_OUTPUT`, `F_LRMD_RSC_QUEUE_TIME`, `F_LRMD_RSC_RCCHANGE_TIME`, `F_LRMD_RSC_RUN_TIME`, `F_LRMD_RSC_START_DELAY`, `F_LRMD_RSC_USERDATA_STR`, `F_LRMD_TIMEOUT`, `F_LRMD_TYPE`, `F_LRMD_WATCHDOG`, `T_LRMD`, `T_LRMD_IPC_PROXY`, `T_LRMD_NOTIFY`, `T_LRMD_REPLY`, and `T_LRMD_RSC_OP` + **libpacemaker:** distribute pacemaker.h header to allow high-level API usage + **libpe_rules:** deprecate functions `find_expression_type()`, `pe_evaluate_rules()`, `pe_eval_expr()`, `pe_eval_rules()`, `pe_eval_subexpr()`, `pe_expand_re_matches()`, `pe_test_expression()`, and `pe_test_rule()` + **libpe_rules,libpe_status:** move enum `expression_type` and globals `was_processing_error` and `was_processing_warning` to libcrmcommon + **libpe_rules,libpe_status:** deprecate role member of `pe_op_eval_data` + **libpe_rules,libpe_status:** deprecate functions `text2task()`, `fail2text()`, `recovery2text()`, `role2text()`, `task2text()`, and `text2role()` + **libpe_status:** deprecate functions `pe_find_node()`, `pe_pref()`, `pe_rsc_is_anon_clone()`, `pe_rsc_is_bundled()`, `pe_rsc_is_clone()`, and `pe_rsc_is_unique_clone()` + **libpe_status:** deprecate global `resource_class_functions` + **libstonithd:** deprecate constants `T_STONITH_NOTIFY_DISCONNECT`, `T_STONITH_NOTIFY_FENCE`, `T_STONITH_NOTIFY_HISTORY`, and `T_STONITH_NOTIFY_HISTORY_SYNCED` # Pacemaker-2.1.7 (19 Dec 2023) - 1388 commits with 358 files changed, 23771 insertions(+), 17219 deletions(-) ## Features added since Pacemaker-2.1.6 + **build:** allow building with libxml2 2.12.0 and greater + **CIB:** deprecate "ordering" attribute of "resource_set" + **CIB:** new cluster option "node-pending-timeout" (defaulting to 0, meaning no timeout, to preserve existing behavior) allows fencing of nodes that do not join Pacemaker's controller group within this much time after joining the cluster + **controller:** `PCMK_node_start_state` now works with Pacemaker Remote nodes + **tools:** `crm_verify` now supports `--quiet` option (currently same as default behavior, but in the future, verbose behavior might become the default, so script writers are recommended to explicitly add `--quiet` if they do not want output) + **tools:** `crm_node` supports standard `--output-as/--output-to` arguments + **tests:** CTSlab.py was renamed to cts-lab ## Fixes since Pacemaker-2.1.6 + **logging:** restore ability to enable XML trace logs by file and function *(regression introduced in 2.1.6)* + **scheduler:** avoid double free with disabled recurring actions *(regression introduced in 2.1.5)* + **tools:** consider dampening argument when setting values with `attrd_updater` *(regression introduced in 2.1.5)* + **tools:** wait for reply from `crm_node -R` *(regression introduced in 2.0.5)* + **agents:** handle dampening parameter consistently and correctly + **CIB:** be more strict about ignoring colocation elements without an ID + **controller:** do not check whether watchdog fencing is enabled if "stonith-watchdog-timeout" is not configured + **controller:** don't try to execute agent action at shutdown + **controller:** avoid race condition when updating node state during join + **controller:** correctly determine state of a fenced node without a name + **controller:** wait a second between fencer connection attempts + **libpacemaker:** avoid shuffling clone instances unnecessarily + **libpacemaker:** get bundle container's promotion score from correct node + **libpacemaker:** role-based colocations now work with bundles + **libpacemaker:** clone-node-max now works with cloned groups + **scheduler:** compare anti-colocation dependent negative preferences against stickiness + **scheduler:** consider explicit colocations with group members + **scheduler:** avoid fencing a pending node without a name + **scheduler:** properly evaluate rules in action meta-attributes + **scheduler:** properly sort rule-based blocks when overwriting values + **tools:** `crm_resource` `--wait` will now wait if any actions are pending (previously it would wait only if new actions were planned) + **tools:** `crm_verify` `--output-as=xml` now includes detailed messages + **tools:** avoid showing pending nodes as having "<3.15.1" feature set in `crm_mon` + **tools:** fix display of clone descriptions + **tools:** `crm_resource` now reports an error rather than timing out when trying to restart an unmanaged resource + **tools:** `crm_resource` now properly detects which promoted role name to use in ban and move constraints ## Public API changes since Pacemaker-2.1.6 (all API/ABI backward-compatible) + **libcib:** `cib_t` now supports transactions via new `cib_api_operations_t` methods, new `cib_transaction` value in enum `cib_call_options`, and new `cib_t` transaction and user members + **libcib:** `cib_t` now supports setting the ACL user for methods via new `cib_api_operations_t` `set_user()` method + **libcib:** deprecate `cib_api_operations_t` methods `inputfd()`, `noop()`, `quit()`, `set_op_callback()`, and `signon_raw()` + **libcib:** deprecate `cib_call_options` values `cib_mixed_update`, `cib_scope_local`, and `cib_zero_copy` + **libcib:** deprecate `cib_t` `op_callback` member + **libcrmcluster:** deprecate `set_uuid()` + **libcrmcluster:** `send_cluster_message()'s` data argument is const + **libcrmcommon:** add enum `pcmk_rc_e` values `pcmk_rc_compression`, `pcmk_rc_ns_resolution`, and `pcmk_rc_no_transaction` + **libcrmcommon,libpe_rules,libpe_status:** many APIs have been moved from `libpe_rules` and `libpe_status` to libcrmcommon, sometimes with new names (deprecating the old ones), as described below + **libcrmcommon:** add (and deprecate) `PCMK_DEFAULT_METADATA_TIMEOUT_MS` defined constant + **libcrmcommon:** add enum `pcmk_rsc_flags` + **libcrmcommon:** add enum `pcmk_scheduler_flags` + **libcrmcommon:** add `pcmk_action_added_to_graph` + **libcrmcommon:** add `pcmk_action_always_in_graph` + **libcrmcommon:** add `pcmk_action_attrs_evaluated` + **libcrmcommon:** add `PCMK_ACTION_CANCEL` string constant + **libcrmcommon:** add `PCMK_ACTION_CLEAR_FAILCOUNT` string constant + **libcrmcommon:** add `PCMK_ACTION_CLONE_ONE_OR_MORE` string constant + **libcrmcommon:** add `PCMK_ACTION_DELETE` string constant + **libcrmcommon:** add `PCMK_ACTION_DEMOTE` string constant + **libcrmcommon:** add `pcmk_action_demote` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_DEMOTED` string constant + **libcrmcommon:** add `pcmk_action_demoted` to enum `action_tasks` + **libcrmcommon:** add `pcmk_action_detect_loop` + **libcrmcommon:** add `PCMK_ACTION_DO_SHUTDOWN` string constant + **libcrmcommon:** add `pcmk_action_fence` to enum `action_tasks` + **libcrmcommon:** add `pcmk_action_inputs_deduplicated` + **libcrmcommon:** add `PCMK_ACTION_LIST` string constant + **libcrmcommon:** add `PCMK_ACTION_LOAD_STOPPED` string constant + **libcrmcommon:** add `PCMK_ACTION_LRM_DELETE` string constant + **libcrmcommon:** add `PCMK_ACTION_MAINTENANCE_NODES` string constant + **libcrmcommon:** add `PCMK_ACTION_META_DATA` string constant + **libcrmcommon:** add `pcmk_action_migratable` + **libcrmcommon:** add `PCMK_ACTION_MIGRATE_FROM` string constant + **libcrmcommon:** add `PCMK_ACTION_MIGRATE_TO` string constant + **libcrmcommon:** add `pcmk_action_migration_abort` + **libcrmcommon:** add `pcmk_action_min_runnable` + **libcrmcommon:** add `PCMK_ACTION_MONITOR` string constant + **libcrmcommon:** add `pcmk_action_monitor` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_NOTIFIED` string constant + **libcrmcommon:** add `pcmk_action_notified` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_NOTIFY` string constant + **libcrmcommon:** add `pcmk_action_notify` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_OFF` string constant + **libcrmcommon:** add `PCMK_ACTION_ON` string constant + **libcrmcommon:** add `PCMK_ACTION_ONE_OR_MORE` string constant + **libcrmcommon:** add `pcmk_action_on_dc` + **libcrmcommon:** add `pcmk_action_optional` + **libcrmcommon:** add `PCMK_ACTION_PROMOTE` string constant + **libcrmcommon:** add `pcmk_action_promote` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_PROMOTED` string constant + **libcrmcommon:** add `pcmk_action_promoted` to enum `action_tasks` + **libcrmcommon:** add `pcmk_action_pseudo` + **libcrmcommon:** add `PCMK_ACTION_REBOOT` string constant + **libcrmcommon:** add `PCMK_ACTION_RELOAD` string constant + **libcrmcommon:** add `PCMK_ACTION_RELOAD_AGENT` string constant + **libcrmcommon:** add `pcmk_action_reschedule` + **libcrmcommon:** add `pcmk_action_runnable` + **libcrmcommon:** add `PCMK_ACTION_RUNNING` string constant + **libcrmcommon:** add `pcmk_action_shutdown` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_START` string constant + **libcrmcommon:** add `pcmk_action_start` to enum `action_tasks` + **libcrmcommon:** add `pcmk_action_started` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_STATUS` string constant + **libcrmcommon:** add `PCMK_ACTION_STONITH` string constant + **libcrmcommon:** add `PCMK_ACTION_STOP` string constant + **libcrmcommon:** add `pcmk_action_stop` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_STOPPED` string constant + **libcrmcommon:** add `pcmk_action_stopped` to enum `action_tasks` + **libcrmcommon:** add `pcmk_action_t` type + **libcrmcommon:** add `pcmk_action_unspecified` to enum `action_tasks` + **libcrmcommon:** add `PCMK_ACTION_VALIDATE_ALL` string constant + **libcrmcommon:** add `pcmk_assignment_methods_t` type + **libcrmcommon:** add `PCMK_DEFAULT_ACTION_TIMEOUT_MS` defined constant + **libcrmcommon:** add `pcmk_log_xml_as()` + **libcrmcommon:** add `PCMK_META_CLONE_MAX` string constant + **libcrmcommon:** add `PCMK_META_CLONE_MIN` string constant + **libcrmcommon:** add `PCMK_META_CLONE_NODE_MAX` string constant + **libcrmcommon:** add `PCMK_META_FAILURE_TIMEOUT` string constant + **libcrmcommon:** add `PCMK_META_MIGRATION_THRESHOLD` string constant + **libcrmcommon:** add `PCMK_META_PROMOTED_MAX` string constant + **libcrmcommon:** add `PCMK_META_PROMOTED_NODE_MAX` string constant + **libcrmcommon:** add `pcmk_multiply_active_block` to enum `rsc_recovery_type` + **libcrmcommon:** add `pcmk_multiply_active_restart` to enum `rsc_recovery_type` + **libcrmcommon:** add `pcmk_multiply_active_stop` to enum `rsc_recovery_type` + **libcrmcommon:** add `pcmk_multiply_active_unexpected` to enum `rsc_recovery_type` + **libcrmcommon:** add `PCMK_NODE_ATTR_TERMINATE` string constant + **libcrmcommon:** add `pcmk_node_t` type + **libcrmcommon:** add `pcmk_node_variant_cluster` + **libcrmcommon:** add `pcmk_node_variant_remote` + **libcrmcommon:** add `pcmk_no_action_flags` + **libcrmcommon:** add `pcmk_no_quorum_demote` + **libcrmcommon:** add `pcmk_no_quorum_fence` + **libcrmcommon:** add `pcmk_no_quorum_freeze` + **libcrmcommon:** add `pcmk_no_quorum_ignore` + **libcrmcommon:** add `pcmk_no_quorum_stop` + **libcrmcommon:** add `pcmk_on_fail_ban` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_block` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_demote` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_fence_node` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_ignore` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_reset_remote` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_restart` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_restart_container` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_standby_node` to `action_fail_response` + **libcrmcommon:** add `pcmk_on_fail_stop` to enum `action_fail_response` + **libcrmcommon:** add `pcmk_probe_always` to enum `pe_discover_e` + **libcrmcommon:** add `pcmk_probe_exclusive` to enum `pe_discover_e` + **libcrmcommon:** add `pcmk_probe_never` to enum `pe_discover_e` + **libcrmcommon:** add `pcmk_requires_fencing` to enum `rsc_start_requirement` + **libcrmcommon:** add `pcmk_requires_nothing` to enum `rsc_start_requirement` + **libcrmcommon:** add `pcmk_requires_quorum` to enum `rsc_start_requirement` + **libcrmcommon:** add `pcmk_resource_t` type + **libcrmcommon:** add `pcmk_role_promoted` to enum `rsc_role_e` + **libcrmcommon:** add `pcmk_role_started` to enum `rsc_role_e` + **libcrmcommon:** add `pcmk_role_stopped` to enum `rsc_role_e` + **libcrmcommon:** add `pcmk_role_unknown` to enum `rsc_role_e` + **libcrmcommon:** add `pcmk_role_unpromoted` to enum `rsc_role_e` + **libcrmcommon:** add `pcmk_rsc_match_anon_basename` + **libcrmcommon:** add `pcmk_rsc_match_basename` + **libcrmcommon:** add `pcmk_rsc_match_clone_only` + **libcrmcommon:** add `pcmk_rsc_match_current_node` + **libcrmcommon:** add `pcmk_rsc_match_history` + **libcrmcommon:** add `pcmk_rsc_methods_t` type + **libcrmcommon:** add `pcmk_rsc_variant_bundle` + **libcrmcommon:** add `pcmk_rsc_variant_clone` + **libcrmcommon:** add `pcmk_rsc_variant_group` + **libcrmcommon:** add `pcmk_rsc_variant_primitive` + **libcrmcommon:** add `pcmk_rsc_variant_unknown` + **libcrmcommon:** add `pcmk_scheduler_t` type + **libcrmcommon:** add `pcmk_tag_t` type + **libcrmcommon:** add `pcmk_ticket_t` type + **libcrmcommon:** add `PCMK_XA_FORMAT` string constant + **libcrmcommon:** `crm_ipc_send()'s` message argument is now const + **libcrmcommon:** deprecate `action_demote` in enum `action_tasks` + **libcrmcommon:** deprecate `action_demoted` in enum `action_tasks` + **libcrmcommon:** deprecate `action_fail_block` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_demote` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_fence` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_ignore` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_migrate` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_recover` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_reset_remote` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_standby` in enum `action_fail_response` + **libcrmcommon:** deprecate `action_fail_stop` in `action_fail_response` + **libcrmcommon:** deprecate `action_notified` in enum `action_tasks` + **libcrmcommon:** deprecate `action_notify` in enum `action_tasks` + **libcrmcommon:** deprecate `action_promote` in enum `action_tasks` + **libcrmcommon:** deprecate `action_promoted` in enum `action_tasks` + **libcrmcommon:** deprecate `action_restart_container` in enum `action_fail_response` + **libcrmcommon:** deprecate `CRMD_ACTION_CANCEL` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_DELETE` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_DEMOTE` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_DEMOTED` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_METADATA` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_MIGRATE` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_MIGRATED` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_NOTIFIED` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_NOTIFY` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_PROMOTE` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_PROMOTED` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_RELOAD` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_RELOAD_AGENT` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_START` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_STARTED` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_STATUS` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_STOP` string constant + **libcrmcommon:** deprecate `CRMD_ACTION_STOPPED` string constant + **libcrmcommon:** deprecate `CRMD_METADATA_CALL_TIMEOUT` defined constant + **libcrmcommon:** deprecate `crm_action_str()` + **libcrmcommon:** deprecate `CRM_DEFAULT_OP_TIMEOUT_S` string constant + **libcrmcommon:** deprecate `crm_element_name()` + **libcrmcommon:** deprecate `CRM_OP_FENCE` string constant + **libcrmcommon:** deprecate `CRM_OP_RELAXED_CLONE` string constant + **libcrmcommon:** deprecate `CRM_OP_RELAXED_SET` string constant + **libcrmcommon:** deprecate `crm_xml_replace()` + **libcrmcommon:** deprecate enum `pe_link_state` + **libcrmcommon:** deprecate `getDocPtr()` + **libcrmcommon:** deprecate `monitor_rsc` in enum `action_tasks` + **libcrmcommon:** deprecate `node_member` + **libcrmcommon:** deprecate `node_remote` + **libcrmcommon:** deprecate `no_action` in enum `action_tasks` + **libcrmcommon:** deprecate `no_quorum_demote` + **libcrmcommon:** deprecate `no_quorum_freeze` + **libcrmcommon:** deprecate `no_quorum_ignore` + **libcrmcommon:** deprecate `no_quorum_stop` + **libcrmcommon:** deprecate `no_quorum_suicide` + **libcrmcommon:** deprecate `pcmk_log_xml_impl()` + **libcrmcommon:** deprecate `pcmk_scheduler_t` localhost member + **libcrmcommon:** deprecate `pe_action_dangle` + **libcrmcommon:** deprecate `pe_action_dc` + **libcrmcommon:** deprecate `pe_action_dedup` + **libcrmcommon:** deprecate `pe_action_dumped` + **libcrmcommon:** deprecate `pe_action_have_node_attrs` + **libcrmcommon:** deprecate `pe_action_implied_by_stonith` + **libcrmcommon:** deprecate `pe_action_migrate_runnable` + **libcrmcommon:** deprecate `pe_action_optional` + **libcrmcommon:** deprecate `pe_action_print_always` + **libcrmcommon:** deprecate `pe_action_processed` + **libcrmcommon:** deprecate `pe_action_pseudo` + **libcrmcommon:** deprecate `pe_action_requires_any` + **libcrmcommon:** deprecate `pe_action_reschedule` + **libcrmcommon:** deprecate `pe_action_runnable` + **libcrmcommon:** deprecate `pe_action_tracking` + **libcrmcommon:** deprecate `pe_clone` + **libcrmcommon:** deprecate `pe_container` + **libcrmcommon:** deprecate `pe_discover_always` in enum `pe_discover_e` + **libcrmcommon:** deprecate `pe_discover_exclusive` in enum `pe_discover_e` + **libcrmcommon:** deprecate `pe_discover_never` in enum `pe_discover_e` + **libcrmcommon:** deprecate `pe_find_anon` + **libcrmcommon:** deprecate `pe_find_any` + **libcrmcommon:** deprecate `pe_find_clone` + **libcrmcommon:** deprecate `pe_find_current` + **libcrmcommon:** deprecate `pe_find_inactive` + **libcrmcommon:** deprecate `pe_find_renamed` + **libcrmcommon:** deprecate `pe_group` + **libcrmcommon:** deprecate `pe_native` + **libcrmcommon:** deprecate `pe_unknown` + **libcrmcommon:** deprecate `recovery_block` in enum `rsc_recovery_type` + **libcrmcommon:** deprecate `recovery_stop_only` in enum `rsc_recovery_type` + **libcrmcommon:** deprecate `recovery_stop_start` in enum `rsc_recovery_type` + **libcrmcommon:** deprecate `recovery_stop_unexpected` in enum `rsc_recovery_type` + **libcrmcommon:** deprecate `RSC_CANCEL` string constant + **libcrmcommon:** deprecate `RSC_DELETE` string constant + **libcrmcommon:** deprecate `RSC_DEMOTE` string constant + **libcrmcommon:** deprecate `RSC_DEMOTED` string constant + **libcrmcommon:** deprecate `RSC_METADATA` string constant + **libcrmcommon:** deprecate `RSC_MIGRATE` string constant + **libcrmcommon:** deprecate `RSC_MIGRATED` string constant + **libcrmcommon:** deprecate `RSC_NOTIFIED` string constant + **libcrmcommon:** deprecate `RSC_NOTIFY` string constant + **libcrmcommon:** deprecate `RSC_PROMOTE` string constant + **libcrmcommon:** deprecate `RSC_PROMOTED` string constant + **libcrmcommon:** deprecate `rsc_req_nothing` in enum `rsc_start_requirement` + **libcrmcommon:** deprecate `rsc_req_quorum` in enum `rsc_start_requirement` + **libcrmcommon:** deprecate `rsc_req_stonith` in enum `rsc_start_requirement` + **libcrmcommon:** deprecate `RSC_ROLE_PROMOTED` in enum `rsc_role_e` + **libcrmcommon:** deprecate `RSC_ROLE_STARTED` in enum `rsc_role_e` + **libcrmcommon:** deprecate `RSC_ROLE_STOPPED` in enum `rsc_role_e` + **libcrmcommon:** deprecate `RSC_ROLE_UNKNOWN` in enum `rsc_role_e` + **libcrmcommon:** deprecate `RSC_ROLE_UNPROMOTED` + **libcrmcommon:** deprecate `RSC_START` string constant + **libcrmcommon:** deprecate `RSC_STARTED` string constant + **libcrmcommon:** deprecate `RSC_STATUS` string constant + **libcrmcommon:** deprecate `RSC_STOP` string constant + **libcrmcommon:** deprecate `RSC_STOPPED` string constant + **libcrmcommon:** deprecate `shutdown_crm` in enum `action_tasks` + **libcrmcommon:** deprecate `started_rsc` in enum `action_tasks` + **libcrmcommon:** deprecate `start_rsc` in enum `action_tasks` + **libcrmcommon:** deprecate `stonith_node` in enum `action_tasks` + **libcrmcommon:** deprecate `stopped_rsc` in enum `action_tasks` + **libcrmcommon:** deprecate `stop_rsc` in enum `action_tasks` + **libcrmcommon:** deprecate `TYPE()` macro + **libcrmcommon:** deprecate `XML_ATTR_VERBOSE` string constant + **libcrmcommon:** deprecate `XML_CIB_ATTR_SOURCE` string constant + **libcrmcommon:** deprecate `XML_CIB_TAG_DOMAINS` string constant + **libcrmcommon:** deprecate `xml_has_children()` + **libcrmcommon:** deprecate `XML_NODE_EXPECTED` string constant + **libcrmcommon:** deprecate `XML_NODE_IN_CLUSTER` string constant + **libcrmcommon:** deprecate `XML_NODE_IS_PEER` string constant + **libcrmcommon:** deprecate `XML_NODE_JOIN_STATE` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_FAIL_STICKINESS` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_FAIL_TIMEOUT` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_INCARNATION_MAX` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_INCARNATION_MIN` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_INCARNATION_NODEMAX` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_PROMOTED_MAX` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_PROMOTED_NODEMAX` string constant + **libcrmcommon:** deprecate `XML_TAG_DIFF_ADDED` string constant + **libcrmcommon:** deprecate `XML_TAG_DIFF_REMOVED` string constant + **libcrmcommon:** deprecate `XML_TAG_FRAGMENT` + **libcrmcommon:** `dump_xml_formatted()'s` argument is now const + **libcrmcommon:** `dump_xml_formatted_with_text()'s` argument is const + **libcrmcommon:** `dump_xml_unformatted()'s` argument is now const + **libcrmcommon:** `save_xml_to_file()'s` xml argument is now const + **libcrmcommon:** `validate_xml_verbose()'s` `xml_blob` argument is const + **libcrmcommon:** `write_xml_fd()'s` xml argument is now const + **libcrmcommon:** `write_xml_file()'s` xml argument is now const + **libcrmcommon:** `xml_top` argument of `xpath_search()` is now const + **libcrmcommon,libpe_rules,libpe_status:** move enum `pe_ordering`, struct `pe_action_wrapper_s`, struct `pe_tag_s`, struct `pe_ticket_s`, struct `resource_object_functions_s`, enum `node_type`, enum `pe_action_flags`, enum `pe_discover_e`, enum `pe_find`, enum `pe_link_state`, enum `pe_obj_types`, enum `pe_quorum_policy`, enum `pe_restart`, struct `pe_action_s`, struct `pe_node_s`, struct `pe_node_shared_s`, struct `pe_resource_s`, struct `pe_working_set_s`, enum `action_fail_response`, enum `action_tasks`, enum `pe_print_options`, enum `rsc_recovery_type`, enum `rsc_role_e`, and enum `rsc_start_requirement` to libcrmcommon + **libpacemaker,libpe_rules,libpe_status:** use `pcmk_action_t` instead of `pe_action_t`, `pcmk_node_t` instead of `pe_node_t`, `pcmk_resource_t` instead of `pe_resource_t`, and `pcmk_scheduler_t` instead of `pe_working_set_t` in all API structs and functions + **libpacemaker:** add `pcmk_list_alternatives()`, `pcmk_list_providers()`, `pcmk_list_standards()`, and `pcmk_list_agents()` for functionality equivalent to `crm_resource` `--list-ocf-alternatives,` `--list-ocf-providers`, `--list-standards,` and `--list-agents` + **libpe_rules,libpe_status:** deprecate `pe_action_t` type + **libpe_rules,libpe_status:** deprecate `pe_action_wrapper_t` + **libpe_rules,libpe_status:** deprecate `pe_node_t` type + **libpe_rules,libpe_status:** deprecate `pe_resource_t` type + **libpe_rules,libpe_status:** deprecate `pe_tag_t` + **libpe_rules,libpe_status:** deprecate `pe_ticket_t` + **libpe_rules,libpe_status:** deprecate `pe_working_set_t` type + **libpe_rules,libpe_status:** deprecate `resource_alloc_functions_t` type + **libpe_rules,libpe_status:** deprecate `resource_object_functions_t` + **libpe_status,libpe_rules:** deprecate enum `pe_ordering` and all its values + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_MAX` + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_PROMOTED_LEGACY_S` string constant + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_PROMOTED_S` string constant + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_STARTED_S` string constant + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_STOPPED_S` string constant + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_UNKNOWN_S` + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_UNPROMOTED_LEGACY_S` string constant + **libpe_status,libpe_rules:** deprecate `RSC_ROLE_UNPROMOTED_S` string constant + **libpe_status:** deprecate enum `pe_check_parameters` + **libpe_status:** deprecate `pe_flag_check_config` + **libpe_status:** deprecate `pe_flag_concurrent_fencing` + **libpe_status:** deprecate `pe_flag_enable_unfencing` + **libpe_status:** deprecate `pe_flag_have_quorum` + **libpe_status:** deprecate `pe_flag_have_remote_nodes` + **libpe_status:** deprecate `pe_flag_have_status` + **libpe_status:** deprecate `pe_flag_have_stonith_resource` + **libpe_status:** deprecate `pe_flag_maintenance_mode` + **libpe_status:** deprecate `pe_flag_no_compat` + **libpe_status:** deprecate `pe_flag_no_counts` + **libpe_status:** deprecate `pe_flag_quick_location` + **libpe_status:** deprecate `pe_flag_sanitized` + **libpe_status:** deprecate `pe_flag_show_scores` + **libpe_status:** deprecate `pe_flag_show_utilization` + **libpe_status:** deprecate `pe_flag_shutdown_lock` + **libpe_status:** deprecate `pe_flag_startup_fencing` + **libpe_status:** deprecate `pe_flag_startup_probes` + **libpe_status:** deprecate `pe_flag_start_failure_fatal` + **libpe_status:** deprecate `pe_flag_stonith_enabled` + **libpe_status:** deprecate `pe_flag_stop_action_orphans` + **libpe_status:** deprecate `pe_flag_stop_everything` + **libpe_status:** deprecate `pe_flag_stop_rsc_orphans` + **libpe_status:** deprecate `pe_flag_symmetric_cluster` + **libpe_status:** deprecate `pe_rsc_allow_migrate` + **libpe_status:** deprecate `pe_rsc_allow_remote_remotes` + **libpe_status:** deprecate `pe_rsc_assigning` + **libpe_status:** deprecate `pe_rsc_block` + **libpe_status:** deprecate `pe_rsc_critical` + **libpe_status:** deprecate `pe_rsc_detect_loop` + **libpe_status:** deprecate `pe_rsc_failed` + **libpe_status:** deprecate `pe_rsc_failure_ignored` + **libpe_status:** deprecate `pe_rsc_fence_device` + **libpe_status:** deprecate `pe_rsc_is_container` + **libpe_status:** deprecate `pe_rsc_maintenance` + **libpe_status:** deprecate `pe_rsc_managed` + **libpe_status:** deprecate `pe_rsc_merging` + **libpe_status:** deprecate `pe_rsc_needs_fencing` + **libpe_status:** deprecate `pe_rsc_needs_quorum` + **libpe_status:** deprecate `pe_rsc_needs_unfencing` + **libpe_status:** deprecate `pe_rsc_notify` + **libpe_status:** deprecate `pe_rsc_orphan` + **libpe_status:** deprecate `pe_rsc_orphan_container_filler` + **libpe_status:** deprecate `pe_rsc_promotable` + **libpe_status:** deprecate `pe_rsc_provisional` + **libpe_status:** deprecate `pe_rsc_reload` + **libpe_status:** deprecate `pe_rsc_replica_container` + **libpe_status:** deprecate `pe_rsc_restarting` + **libpe_status:** deprecate `pe_rsc_runnable` + **libpe_status:** deprecate `pe_rsc_start_pending` + **libpe_status:** deprecate `pe_rsc_stop` + **libpe_status:** deprecate `pe_rsc_stop_unexpected` + **libpe_status:** deprecate `pe_rsc_unique` # Pacemaker-2.1.6 (24 May 2023) - 1124 commits with 402 files changed, 25220 insertions(+), 14751 deletions(-) ## Features added since Pacemaker-2.1.5 + **CIB:** deprecate "moon" in `date_spec` elements in rules + **CIB:** deprecate support for Nagios resources + **CIB:** utilization attributes may be set as transient + **CIB:** alerts and alert recipients support an "enabled" meta-attribute + All daemons support `--output-as/--output-to` options including XML output with interactive options + **tools:** `attrd_updater` supports `--wait` parameter that can be set to "no" (return immediately after submitting request, which is the default and previous behavior), "local" (return after the new value has taken effect on the local node), or "cluster" (return after new value has taken effect on all nodes) + **tools:** `attrd_updater` supports `-z/--utilization` to modify utilization attributes + **tools:** `attrd_updater` supports `--pattern` to affect all attributes matching a given pattern + **tools:** `crm_attribute` supports `--pattern` with permanent node attributes (in addition to previous support for transient attributes) + **tools:** `crm_mon` displays resource descriptions if `--show-description` or `--show-detail` is given + **tools:** `crm_mon` shows maintenance mode when enabled per-resource + **tools:** `crm_mon` `--interval` can be used to update XML and text output (in addition to previous support for HTML) + **tools:** `crm_mon` fencing history includes microseconds in timestamps + **tools:** `crm_mon` shows which node it was run on + **tools:** `crm_mon` shows whether Pacemaker or Pacemaker Remote is running or shutting down + **tools:** deprecate `crm_mon --simple-status` + **tools:** `crm_resource` `--element` option can be used with `--get-parameter`, `--set-parameter,` and `--delete-parameter` to modify resource properties such as class, provider, type, and description + **tools:** `crm_resource` `--list` shows resource descriptions better (including when `--output-as=xml` is used) + **tools:** `crm_shadow` supports standard `--output-as/--output-to` arguments ## Fixes since Pacemaker-2.1.5 + **pacemakerd:** -S should wait for cluster to shut down before returning *(regression introduced in 2.1.1)* + **Pacemaker Remote:** remote nodes wait for all-clear from cluster before shutting down *(regression introduced in 2.1.5)* + **tools:** `attrd_updater` `--query` without `--node` shows attributes from all nodes instead of local node *(regression introduced in 2.1.5)* + **pacemaker-attrd:** Preserve a Pacemaker Remote node's transient attributes if its connection to the cluster is lost but reconnects + **CIB manager:** successful CIB schema upgrade always forces a write + **controller:** avoid election storm when joining node has CIB newer than DC can accept + **controller:** avoid election storm due to incompatible CIB + **controller:** avoid use-after-free when disconnecting proxy IPCs during shutdown + **controller:** avoid double-incrementing failcount for simulated failures + **controller:** avoid reprobing remote nodes when target is a cluster node + **controller:** delay join finalization if a transition is in progress + **controller:** initial fencing timeout includes any priority-fencing-delay + **controller:** shutdown gracefully if scheduler connection is interrupted during shutdown + **fencer:** avoid crash during shutdown when action is pending + **fencer:** calculate fencing timeout correctly when watchdog is used with topology + **fencer:** apply priority-fencing-delay only to first device tried + **fencer:** total and per-device fencing timeouts include any priority-fencing-delay and `pcmk_delay_base` + **scheduler:** fix a number of corner cases with colocations, including preventing a resource from starting if it has a mandatory colocation with a group whose start is blocked, preventing optional anti-colocation from overriding infinite stickiness, and correctly considering a group's colocation dependents when the group is itself colocated with a clone + **scheduler:** honor as many mandatory colocations as possible before considering any optional ones + **scheduler:** ensure earlier group member starts occur after later member stops + **scheduler:** handle orderings with bundles more correctly + **scheduler:** ensure expired results never affect resource state + **scheduler:** handle cleaned `migrate_from` history correctly + **scheduler:** prevent pending monitor of one clone instance from causing unexpected stop of other instances + **scheduler:** prevent inactive clone instances from starting if probe is unrunnable on any node + **agents:** SysInfo calculates `cpu_load` correctly + **tools:** `cibadmin --scope` accepts status + **tools:** `crm_attribute -p ""` works same as `-p` when called from resource agent + **tools:** `crm_attribute` recognizes "-INFINITY" as value instead of options + **tools:** `crm_mon` avoids displaying recurring monitors as pending if first attempt fails + **tools:** `crm_mon` `--daemonize` shows disconnected message when CIB connection is lost instead of continuing to display last known state + **tools:** `crm_mon` avoids crash when built without curses library support + **tools:** `crm_mon` supports `--output-as=none` correctly + **tools:** `crm_resource` `--ban` or `--move` works with single-replica bundles + **tools:** `crm_shadow` `--commit` now works with `CIB_file` + **tools:** `crm_simulate` failure injection avoids crash if node name is unknown ## Public API changes since Pacemaker-2.1.5 + **Python:** New "pacemaker" Python module (packaged as python3-pacemaker in RPMs built with "make rpm") contains supported public API with BuildOptions and ExitStatus classes + **libcib:** add `client_id()` method member to `cib_api_operations_t` + **libcib:** deprecate `cib_database` + **libcib:** deprecate `cib_quorum_override` + **libcib:** deprecate the `cib_api_operations_t:update()` method + **libcrmcluster:** add `pcmk_cluster_new()` + **libcrmcluster:** add `crm_join_nack_quiet` + **libcrmcluster:** add `pcmk_cluster_free()` + **libcrmcluster:** node argument to `send_cluster_message()` is now const + **libcrmcluster:** node argument to `send_cluster_text()` is now const + **libcrmcommon:** add `crm_time_usecs` + **libcrmcommon:** add `PCMK_META_ENABLED` + **libcrmcommon:** add `pcmk_pacemakerd_state_remote` + **libcrmcommon:** add `pcmk_rc_bad_xml_patch` + **libcrmcommon:** add `pcmk_rc_bad_input` + **libcrmcommon:** add `pcmk_rc_disabled` + **libcrmcommon:** deprecate `add_xml_nocopy()` + **libcrmcommon:** deprecate `log_data_element()` + **libcrmcommon:** deprecate `PCMK_RESOURCE_CLASS_NAGIOS` + **libcrmcommon:** deprecate `PCMK_RESOURCE_CLASS_UPSTART` + **libcrmcommon:** deprecate `XML_ATTR_UUID` + **libcrmcommon:** deprecate `XML_CIB_ATTR_REPLACE` + **libcrmcommon:** deprecate `xml_log_changes()` + **libcrmcommon:** deprecate `xml_log_options` enum + **libcrmcommon:** deprecate `xml_log_patchset()` + **libcrmcommon:** argument to `pcmk_xe_is_probe()` is now const + **libcrmcommon:** argument to `pcmk_xe_mask_probe_failure()` is now const + **libcrmcommon:** patchset argument of `xml_log_patchset()` is now const + **libcrmcommon:** `rsc_op_expected_rc()` argument is now const + **libcrmcommon:** second argument to `copy_in_properties()` is now const + **libcrmcommon:** xml argument of `xml_log_changes()` is now const + **libcrmservice:** deprecate enum `nagios_exitcode` + **libpacemaker:** add `pcmk_query_node_info()` + **libpacemaker:** add `pcmk_query_node_name()` + **libpacemaker:** multiple arguments to `pcmk_simulate()` are now const + **libpacemaker:** node argument to `pcmk_resource_digests()` is now const + **libpacemaker:** `node_types` argument to `pcmk_list_nodes()` is now const + **libpacemaker:** `pcmk_controller_status()` node name arg is now const + **libpe_rules:** last argument to `pe_expand_re_matches()` is now const + **libpe_rules:** `rule_data` argument to `pe_eval_nvpairs()` is now const + **libpe_rules:** second argument to `pe_eval_expr()` and `pe_eval_subexpr()` is now const + **libpe_rules:** second argument to `pe_eval_rules()` is now const + **libpe_rules:** second argument to `pe_unpack_nvpairs()` is now const + **libpe_status:** add `pe_rsc_detect_loop` + **libpe_status:** add `pe_rsc_replica_container` + **libpe_status:** deprecate fixed member of `pe_node_t` + **libpe_status:** argument to `pe_rsc_is_bundled()` is now const + **libpe_status:** argument to `rsc_printable_id()` is now const + **libpe_status:** first argument to `calculate_active_ops()` is now const + **libpe_status:** first argument to `pe_find_node()` is now const + **libpe_status:** first argument to `pe_find_node_any()` is now const + **libpe_status:** first argument to `pe_find_node_id()` is now const + **libpe_status:** first argument to `resource_object_functions_t:is_filtered()` is now const # Pacemaker-2.1.5 (7 Dec 2022) - 1287 commits with 447 files changed, 33546 insertions(+), 21518 deletions(-) ## Features added since Pacemaker-2.1.4 + **CIB:** access control lists (ACLs) for groups are supported + **CIB:** ACL target and group XML supports "name" attribute to specify a name that is not a unique XML ID + **CIB:** deprecate pacemaker-next schema + **CIB:** deprecate first-instance and then-instance in ordering constraints and rsc-instance and with-rsc-instance in colocation constraints (only usable with pacemaker-next schema) + **CIB:** deprecate "collocated" and "ordered" meta-attributes for groups (true is default, and resource sets should be used instead of false) + **build:** support building with -D_TIME_BITS=64 + **build:** support building with compilers that are strict about void + **build:** allow building RPMs from source distribution + **fencer:** deprecate stand-alone mode + **agents:** ClusterMon, controld, HealthCPU, ifspeed, and SysInfo agents support OCF 1.1 standard + **agents:** non-functional SystemHealth agent has been removed + **tools:** non-functional ipmiservicelogd and notifyServicelogEvent tools have been removed + **tools:** `crm_attribute` supports querying all attributes on a given node + **tools:** `crm_attribute` `--query,` --delete, and `--update` support regular expressions + **tools:** `crm_error` supports standard `--output-as/--output-to` arguments + **tools:** `crm_error` lists all return codes if none are specified + **tools:** `crm_mon` `--show-detail` displays the CRM feature set of each node and makes display of fencing actions more technical + **tools:** `crm_resource` `--why` checks node health + **tools:** `crm_resource` `--constraints` now accepts `--recursive` (equivalent to `--stack)` and `--force` (to show constraints for a group member instead of the group) ## Fixes since Pacemaker-2.1.4 + **tools:** `crm_error` does not print spurious output when given a negative return code argument *(regression introduced in 2.0.4)* + **tools:** avoid crash if `crm_resource` is given extraneous arguments *(regression introduced in 2.0.5)* + **tools:** `stonith_admin` `--validate` XML output shows correct validation status and errors *(regressions introduced in 2.0.5 and 2.1.2)* + **tools:** `crm_resource` `--list-operations` shows pending operations as pending instead of complete *(regression introduced in 2.1.0)* + **controller:** move resources if appropriate after they are reordered in CIB *(regression introduced in 2.1.3)* + **fencing:** allow fence devices to be registered if local node is not in CIB *(regression introduced in 2.1.3)* + **tools:** `crm_mon` `--one-shot` should succeed even if pacemaker is shutting down *(regression introduced in 2.1.3)* + **tools:** avoid memory leak in `crm_mon` *(regression introduced in 2.1.3)* + **tools:** `crm_attribute` `--quiet` outputs nothing instead of "(null)" if attribute has no value *(regression introduced in 2.1.3)* + **tools:** accept deprecated and unused `attrd_updater` `--quiet` option *(regression introduced in 2.1.3)* + **CIB:** avoid crashes when XML IDs grow very long + **controller:** pre-load agent metadata asynchronously to avoid timeout when agent's metadata action runs `crm_node` + **controller:** avoid timing issue that increments resource fail count twice + **fencing:** unfence all nodes after device configuration changes + **fencing:** ignore node that executed action when checking whether actions are equivalent + **scheduler,controller:** calculate secure digest consistently + **scheduler:** consider roles when blocking colocation dependents + **scheduler:** prioritize group colocations properly + **scheduler:** properly consider effect of "with group" colocations + **scheduler:** handle corner cases in live migrations + **scheduler:** avoid perpetual moving of bundle containers in certain situations + **scheduler:** properly calculate resource parameter digests without history + **scheduler:** do not enforce stop if newer monitor indicates resource was not running on target of failed `migrate_to` + **scheduler:** do not enforce stop on rejoined node after failed `migrate_to` + **scheduler:** don't demote on expected node when multiple-active is set to `stop_unexpected` + **scheduler:** prevent resources running on multiple nodes after partial live migration + **scheduler:** restart resource instead of reload if extra parameters in operation change + **schemas:** Consider days, minutes, seconds, and yeardays in date expressions valid + **schemas:** Consider `in_range` with an end and duration valid + **schemas:** Consider score and score-attribute optional in rules + **tools:** `crm_resource` `--digests` uses most recent operation history entry + **tools:** if multiple return code options are given to `crm_error`, use the last one + **tools:** `crm_resource` correctly detects if a resource is unmanaged or disabled even if there are multiple settings using rules ## Public API changes since Pacemaker-2.1.4 + **libcib:** add `cib_api_operations_t:set_primary` + **libcib:** add `cib_api_operations_t:set_secondary` + **libcib:** deprecate `cib_api_operations_t:delete_absolute()` + **libcib:** deprecate `cib_api_operations_t:is_master` + **libcib:** deprecate `cib_api_operations_t:set_master` + **libcib:** deprecate `cib_api_operations_t:set_slave` + **libcib:** deprecate `cib_api_operations_t:set_slave_all` + **libcrmcommon:** deprecated `XML_CIB_TAG_MASTER` constant is usable again *(regression introduced in 2.1.0)* + **libcrmcommon:** `pcmk_ipc_api_t` supports attribute manager IPC + **libcrmcommon:** add `pcmk_rc_unpack_error` + **libcrmcommon:** add `CRM_EX_FAILED_PROMOTED` + **libcrmcommon:** add `CRM_EX_NONE` + **libcrmcommon:** add `CRM_EX_PROMOTED` + **libcrmcommon:** add `pcmk_readable_score()` + **libcrmcommon:** add `PCMK_XA_PROMOTED_MAX_LEGACY` string constant + **libcrmcommon:** add `PCMK_XA_PROMOTED_NODE_MAX_LEGACY` string constant + **libcrmcommon:** argument to `crm_time_check()` is now const + **libcrmcommon:** argument to `pcmk_controld_api_replies_expected()` is now const + **libcrmcommon:** argument to `pcmk_xml_attrs2nvpairs()` is now const + **libcrmcommon:** argument to `xml2list()` is now const + **libcrmcommon:** argument to `xml_acl_denied()` is now const + **libcrmcommon:** argument to `xml_acl_enabled()` is now const + **libcrmcommon:** argument to `xml_get_path()` is now const + **libcrmcommon:** arguments to `crm_time_add()` are now const + **libcrmcommon:** arguments to `crm_time_compare()` are now const + **libcrmcommon:** arguments to `crm_time_subtract()` are now const + **libcrmcommon:** argv argument to `crm_log_preinit()` is now char *const * + **libcrmcommon:** `crm_time_calculate_duration()` arguments are now const + **libcrmcommon:** deprecate `CRM_ATTR_RA_VERSION` + **libcrmcommon:** deprecate `crm_destroy_xml()` + **libcrmcommon:** deprecate `crm_ipc_server_error` + **libcrmcommon:** deprecate `crm_ipc_server_info` + **libcrmcommon:** deprecate `CRM_OP_LRM_QUERY` + **libcrmcommon:** deprecate `crm_str()` + **libcrmcommon:** deprecate `PCMK_XE_PROMOTED_MAX_LEGACY` string constant + **libcrmcommon:** deprecate `PCMK_XE_PROMOTED_NODE_MAX_LEGACY` constant + **libcrmcommon:** deprecate `score2char()` + **libcrmcommon:** deprecate `score2char_stack()` + **libcrmcommon:** deprecate `XML_ATTR_RA_VERSION` + **libcrmcommon:** deprecate `xml_get_path()` + **libcrmcommon:** deprecate `XML_PARANOIA_CHECKS` + **libcrmcommon:** deprecate `XML_TAG_OP_VER_ATTRS` + **libcrmcommon:** deprecate `XML_TAG_OP_VER_META` + **libcrmcommon:** deprecate `XML_TAG_RSC_VER_ATTRS` + **libcrmcommon:** dt argument of `crm_time_get_gregorian()` is now const + **libcrmcommon:** dt argument of `crm_time_get_isoweek()` is now const + **libcrmcommon:** dt argument of `crm_time_get_ordinal()` is now const + **libcrmcommon:** dt argument of `crm_time_get_timeofday()` is now const + **libcrmcommon:** dt argument of `crm_time_get_timezone()` is now const + **libcrmcommon:** first argument to `create_reply()` is now const + **libcrmcommon:** first argument to `crm_copy_xml_element()` is now const + **libcrmcommon:** first argument to `find_xml_node()` is now const + **libcrmcommon:** first argument to `get_message_xml()` is now const + **libcrmcommon:** first argument to `pcmk_ipc_name()` is now const + **libcrmcommon:** first argument to `xml_patch_versions()` is now const + **libcrmcommon:** last argument to `crm_write_blackbox()` is now const + **libcrmcommon:** add `pcmk_rc_duplicate_id` + **libcrmcommon:** add `pcmk_result_get_strings()` + **libcrmcommon:** add `pcmk_result_type` enum + **libcrmcommon:** add `PCMK_XE_DATE_EXPRESSION` constant + **libcrmcommon:** add `PCMK_XE_OP_EXPRESSION` constant + **libcrmcommon:** add `PCMK_XE_RSC_EXPRESSION` constant + **libcrmcommon:** first argument to `crm_time_as_string()` is now const + **libcrmcommon:** `crm_time_t` argument to `crm_time_log_alias()` is now const + **libcrmcommon:** argument to `crm_time_get_seconds()` is now const + **libcrmcommon:** argument to `crm_time_get_seconds_since_epoch()` is now const + **libcrmcommon:** sixth argument to `log_data_element()` is now const + **libcrmcommon:** source argument to `crm_time_set()` is now const + **libcrmcommon:** source argument to `crm_time_set_timet()` is now const + **libcrmcommon:** source argument to `pcmk_copy_time()` is now const + **libpacemaker:** add `pcmk_check_rule()` + **libpacemaker:** add `pcmk_check_rules()` + **libpacemaker:** add `pcmk_show_result_code()` + **libpacemaker:** add `pcmk_list_result_codes()` + **libpacemaker:** add `pcmk_rc_disp_flags` enum + **libpacemaker:** `ipc_name` argument to `pcmk_pacemakerd_status()` is now const + **libpe_rules:** deprecate `version_expr` enum value + **libpe_rules:** second argument to `pe_eval_nvpairs()` is now const + **libpe_status:** argument to `pe_rsc_is_anon_clone()` is now const + **libpe_status:** argument to `pe_rsc_is_unique_clone()` is now const + **libpe_status:** deprecate enum `pe_graph_flags` + **libpe_status:** first argument to `pe_rsc_is_clone()` is now const + **libpe_status:** two arguments to `get_rsc_attributes()` are now const + **libpe_status:** second argument to `pe_eval_versioned_attributes()` is now const + **libpe_status:** second argument to `pe_rsc_params()` is now const + **libstonithd:** deprecate `stonith_event_t:message` + **libstonithd:** deprecate `stonith_event_t:type` + **libstonithd:** last argument to `stonith_api_operations_t:register_device()` is now const + **libstonithd:** last argument to `stonith_api_operations_t:register_level()` is now const + **libstonithd:** last argument to `stonith_api_operations_t:register_level_full()` is now const + **libstonithd:** params argument to `stonith_api_operations_t:validate()` is now const # Pacemaker-2.1.4 (13 Jun 2022) - 17 commits with 9 files changed, 52 insertions(+), 18 deletions(-) ## Fixes since Pacemaker-2.1.3 + **fencing:** get target-by-attribute working again *(regression in 2.1.3)* + **fencing:** avoid use-after-free when processing self-fencing requests with topology *(regression in 2.1.3)* + **resources:** typo in HealthSMART meta-data *(regression in 2.1.3)* + **fencing:** avoid memory leaks when processing topology requests + **fencing:** delegate shouldn't matter when checking equivalent fencing + **tools:** fix CSS syntax error in `crm_mon --output-as=html` # Pacemaker-2.1.3 (1 Jun 2022) - 814 commits with 332 files changed, 23435 insertions(+), 12137 deletions(-) ## Features added since Pacemaker-2.1.2 + Internal failures of resource actions (such as an OCF agent not being found) are shown with a detailed exit reason in logs, `crm_mon` output, etc. + Support for CIB entries with type="ping" is deprecated (this was an undocumented means of defining a quorum-only node) + **build:** configure script supports experimental `--enable-nls` option to enable native language translations (currently only Chinese translations of certain help text are available) + **rpm:** `crm_attribute` is now part of the pacemaker-cli package instead of the pacemaker package + **CIB:** resources support allow-unhealthy-nodes meta-attribute to exempt the resource from bans due to node health checks (particularly useful for health monitoring resources themselves) + **CIB:** multiple-active cluster property can be set to "stop_unexpected" to leave the expected instance running and stop only any unexpected ones + **CIB:** bundles support resource utilization + **pacemakerd:** regularly check that subdaemons are active and accepting IPC connections so sbd can self-fence a node with a malfunctioning subdaemon + **pacemaker-schedulerd:** support `--output-as/--output-to` options including XML output with interactive options + **tools:** cibadmin `--show-access` option to show CIB colorized for ACLs + **tools:** `crm_attribute` supports standard `--output-as/--output-to` options + **tools:** `crm_mon` output indicates if a node's health is yellow or red + **tools:** for probes that failed because the service is not installed or locally configured, `crm_mon` displays the resource as stopped rather than failed + **tools:** `crm_rule` supports standard `--output-as/--output-to` options, allows passing multiple -r options, and is no longer experimental + **tools:** `stonith_admin` fencing commands display reasons for failures + **resource agents:** HealthSMART supports OCF 1.1 standard and new `OCF_RESKEY_dampen` and `OCF_RESKEY_smartctl` parameters ## Fixes since Pacemaker-2.1.2 + **build:** avoid circular library dependency *(regression introduced in 2.1.0)* + **systemd:** if pacemakerd exits immediately after starting, wait 1 second before trying to respawn, and allow 5 attempts + **fencer:** get fencing completion time correctly *(regression introduced in 2.1.2)* + **fencer:** avoid memory leak when broadcasting history differences *(regression introduced in 2.1.0)* + **controller:** correctly match "node down" events so remote nodes don't get fenced when connection is stopped *(regression introduced in 2.1.2)* + **executor:** avoid possible double free during notify operation *(regression introduced in 2.1.1)* + **tools:** get `stonith_admin` -T/--tag option working again *(regression introduced in 2.0.3)* + **resources:** use correct syntax in Stateful meta-data *(regression introduced in 2.1.0)* + **corosync:** repeat `corosync_cfg_trackstart` if first attempt fails + **libcrmcommon:** retry IPC requests after EAGAIN errors + **executor,scheduler:** treat "no secrets" fence results as a hard error + **fencing:** handle dynamic target query failures better + **fencing:** don't set stonith action to pending if fork fails + **pacemakerd:** avoid race condition when subdaemaon is checked while exiting + **scheduler:** avoid memory leak when displaying clones in certain conditions + **scheduler:** properly set data set flags when scheduling actions + **tools:** support command-line `crm_attribute` calls on Pacemaker remote nodes whose node name in the cluster differs from their local hostname + **tools:** prevent possible `crm_resource` crashes if multiple commands specified ## Public API changes since Pacemaker-2.1.2 + **libcrmcommon:** `pcmk_ipc_api_t` supports scheduler IPC + **libpacemaker:** add `pcmk_status()` (equivalent to `crm_mon)` + **libcib:** deprecate `get_object_parent()` + **libcib:** deprecate `get_object_path()` + **libcib:** deprecate `get_object_root()` + **libcrmcommon:** add `pcmk_cib_parent_name_for()` + **libcrmcommon:** add `pcmk_cib_xpath_for()` + **libcrmcommon:** add `pcmk_find_cib_element()` + **libcrmcommon:** deprecate `crm_xml_add_boolean()` + **libpe_status:** add `pe_flag_check_config` + **libpe_status:** add `pe_node_shared_s:data_set` + **libpe_status:** add `pe_rsc_restarting` flag + **libpe_status:** add `pe_rsc_stop_unexpected` flag + **libpe_status:** add `recovery_stop_unexpected` to enum `rsc_recovery_type` + **libpe_status:** deprecate `node_ping` + **libpe_status:** deprecate `pe_order_stonith_stop` + **libpe_status:** deprecate `pe_rsc_starting` and `pe_rsc_stopping` + **libstonithd:** add `exit_reason` member to `stonith_history_t` + **libstonithd:** deprecate `stonith_t:call_timeout` + **libstonithd:** `stonith_api_del_notification()` with NULL second argument removes all notifications # Pacemaker-2.1.2 (23 Nov 2021) - 462 commits with 223 files changed, 16518 insertions(+), 11743 deletions(-) ## Features added since Pacemaker-2.1.1 + **build:** when built with `--with-initdir,` Pacemaker uses the value to find LSB resources (in addition to being where Pacemaker's own init scripts are installed) + **build:** cmocka is new dependency for unit tests ("make check") + **rpm:** `fence_watchdog` now comes with pacemaker package (not pacemaker-cli) + **daemons:** metadata for cluster options supports OCF 1.1 standard + **executor:** nagios warning results now map to OCF "degraded" result code + **fencing:** `pcmk_delay_base` can optionally specify different delays per node + **fencing:** `pcmk_host_map` supports escaped characters such as spaces in values + **resources:** HealthIOWait agent supports OCF 1.1 standard, and validate works + **tools:** `crm_mon` shows exit reasons for actions failed due to internal errors + **tools:** `crm_mon` failed action display is more human-friendly by default + **tools:** `crm_resource` `--force-*` now outputs exit reasons if available ## Fixes since Pacemaker-2.1.1 + **pkg-config:** return correct value for ocfdir *(regression introduced in 2.1.0)* + **tools:** fix `crm_mon` `--hide-headers` and related options *(regression introduced in 2.0.4)* + **attrd:** check election status upon node loss to avoid election timeout + **controller:** improved handling of executor connection failures + **executor:** properly detect systemd unit existence + **pacemakerd:** recover properly from Corosync crash + **fencing:** fencing results are now sorted with sub-second resolution + **fencing:** fix `fence_watchdog` version output, metadata output, and man page + **fencing:** mark state as done if remapped "on" times out + **tools:** map LSB status to OCF correctly with `crm_resource` --force-check ## Public API changes since Pacemaker-2.1.1 + **libcrmcommon:** deprecate `PCMK_OCF_EXEC_ERROR` + **libcrmcommon:** deprecate `PCMK_OCF_PENDING` + **libcrmcommon:** deprecate `PCMK_OCF_SIGNAL` + **libcrmcommon:** add `CRM_EX_DEGRADED` and `CRM_EX_DEGRADED_PROMOTED` + **libcrmcommon:** add enum `pcmk_exec_status` + **libcrmcommon:** add `PCMK_EXEC_MAX` + **libcrmcommon:** add `PCMK_EXEC_NO_FENCE_DEVICE` + **libcrmcommon:** add `PCMK_EXEC_NO_SECRETS` + **libcrmcommon:** add `pcmk_exec_status_str()` + **libcrmcommon:** add `pcmk_rc2ocf()` + **libcrmcommon:** deprecate `PCMK_OCF_TIMEOUT` + **libcrmservice:** add `services_result2ocf()` + **libcrmservice:** deprecate enum `op_status` + **libcrmservice:** deprecate `LSB_ROOT_DIR` + **libcrmservice:** deprecate `NAGIOS_NOT_INSTALLED` + **libcrmservice:** deprecate `NAGIOS_STATE_DEPENDENT` + **libcrmservice:** deprecate `services_get_ocf_exitcode()` + **libcrmservice:** deprecate `services_list()` and `services_action_create()` + **libcrmservice:** deprecate `services_lrm_status_str()` + **libpacemaker:** add enum `pcmk_sim_flags` + **libpacemaker:** add `pcmk_injections_t` + **libpacemaker:** add `pcmk_free_injections()` + **libpacemaker:** add `pcmk_simulate()` + **libstonithd:** add opaque member to `stonith_event_t` + **libstonithd:** add opaque member to `stonith_callback_data_t` # Pacemaker-2.1.1 (09 Sep 2021) - 231 commits with 102 files changed, 4912 insertions(+), 3428 deletions(-) ## Features added since Pacemaker-2.1.0 + enhanced support for OCF Resource Agent API 1.1 standard + **agents:** ocf:pacemaker:attribute and ocf:pacemaker:ping agents now support 1.1 + **tools:** `crm_resource` passes output format to agents so they will use it if supported + **tools:** `crm_resource` `--validate` and `--force-check` options accept optional check level to pass to agent + **tools:** `crm_mon` XML output includes stonith-timeout-ms and priority-fencing-delay-ms cluster properties + **pacemakerd:** support `--output-as/--output-to` options including XML output with interactive options such as --features ## Fixes since Pacemaker-2.1.0 + **pacemaker-attrd:** avoid repeated unfencing of remote nodes when DC joined cluster after remote was up + **controller:** ensure newly joining node learns the node names of non-DCs + **controller:** ensure lost node's transient attributes are cleared without DC + **scheduler:** avoid invalid transition when group member is unmanaged (CLBZ#5423) + **scheduler:** don't schedule probes of unmanaged resources on pending nodes + **executor:** avoid crash after TLS connection errors *(regression introduced in 2.0.4)* + **fencing:** avoid repeated attempts to get (nonexistent) meta-data for watchdog device *(regression introduced in 2.1.0)* + **fencing:** select correct device when `pcmk_host_check="dynamic-list"` and `pcmk_host_map` are both specified (CLBZ#5474) + **tools:** `crm_attribute` supports node attribute values starting with a '-' again *(regression introduced in 2.1.0)* + **tools:** `crm_attribute` deprecated `--get-value` option does not require an argument *(regression introduced in 2.1.0)* + **tools:** avoid `crm_mon` memory leaks when filtering by resource or node *(regressions introduced in 2.0.4 and 2.0.5)* + **tools:** exit with status 0 (not 64) for `--version` argument to `crm_rule`, `crm_error`, `crm_diff`, and `crm_node` *(regression introduced in 2.0.4)* and `crm_attribute` *(regression introduced in 2.1.0)* + **tools:** `crm_mon` should show active unmanaged resources on offline nodes without requiring `-r/--inactive` + **tools:** better `crm_resource` error messages for unsupported resource types + **tools:** `crm_simulate` `--show-failcounts` includes all relevant information + **tools:** `crm_mon` should not show inactive group members without `--inactive` + **tools:** `crm_mon` XML output should show members of cloned groups + **libcrmcommon:** correctly handle case-sensitive XML IDs ## Public API changes since Pacemaker-2.1.0 + **libcrmcommon:** add `pcmk_section_e` type + **libcrmcommon:** add `pcmk_show_opt_e` type + **libcrmcommon:** add `pcmk_pacemakerd_api_shutdown()` + **libpe_status:** deprecate enum `pe_print_options` # Pacemaker-2.1.0 (08 Jun 2021) - 849 commits with 327 files changed, 22089 insertions(+), 12633 deletions(-) ## Features added since Pacemaker-2.0.5 + support for OCF Resource Agent API 1.1 standard - allow Promoted and Unpromoted role names in CIB (in addition to Master and Slave, which are deprecated), and use new role names in output, logs, and constraints created by `crm_resource --ban` - advertise 1.1 support to agents, and provide `notify_promoted_*` and `notify_unpromoted_*` environment variables to agents with notify actions - support `reloadable` parameter attribute and `reload-agent` action in agents that advertise 1.1 support - support 1.1 standard in ocf:pacemaker:Dummy, ocf:pacemaker:remote, and ocf:pacemaker:Stateful resource agents - add "promoted-only" (in addition to "master-only", which is deprecated) in `crm_mon` XML output for bans + support for noncritical resources - colocation constraints accept an "influence" attribute that determines whether dependent influences main resource's location (the default of "true" preserves the previous behavior, while "false" makes the dependent stop if it reaches its migration-threshold in failures rather than cause both resources to move to another node) - resources accept a "critical" meta-attribute that serves as default for all colocation constraints involving the resource as the dependent, as well as groups involving the resource + detail log uses millisecond-resolution timestamps when Pacemaker is built with libqb 2.0 or later + **CIB:** deprecate the remove-after-stop cluster property, `can_fail` action meta-attribute, and support for Upstart-based resources + **controller:** the `PCMK_panic_action` environment variable may be set to sync-crash or sync-reboot to attempt to synchronize local disks before crashing or rebooting, which can be helpful to record cached log messages but runs the risk of the sync hanging and leaving the host running after a critical error + **tools:** `CIB_file="-"` can be used to get the CIB from standard input + **tools:** crmadmin, `crm_resource`, `crm_simulate`, and `crm_verify` support standard `--output-as/--output-to` options (including XML output, intended for parsing by scripts and higher-level tools) + **tools:** `crm_attribute` accepts `-p/--promotion` option to operate on promotion score (replacing `crm_master`, which is deprecated) + **tools:** `crm_resource` accepts `--promoted` option (replacing `--master,` which is deprecated) + **tools:** `crm_resource` accepts `--digests` advanced option + **tools:** `crm_simulate` accepts `--show-attrs` and `--show-failcounts` options - Build process changes since Pacemaker-2.0.5 + Pacemaker requires newer versions of certain dependencies, including Python 3.2 or later (support for Python 2 has been dropped), glib 2.32.0 or later, libqb 0.17.0 or later, GnuTLS 2.12.0 or later (to enable Pacemaker Remote support), rpm 4.11.0 (if building RPMs), and a C library that provides setenv() and unsetenv() + **configure:** `--enable-legacy-links` (which is deprecated) defaults to "no", meaning that symbolic links will not be created for the Pacemaker 1 daemon names + **configure:** `--enable-compat-2.0` prevents certain output changes (most significantly role names) to maintain compatibility with older tools, scripts, and resource agents that rely on previous output + **configure:** `--with-resource-stickiness-default` sets a `resource-stickiness` default in newly created CIBs + **configure:** `--with-concurrent-fencing-default` specifies default for `concurrent-fencing` cluster property + **configure:** `--with-sbd-sync-default` specifies default for syncing start-up with sbd + **configure:** `--with-ocfrapath` specifies resource agent directories to search + **configure:** `--with-ocfrainstalldir` specifies where to install ocf:pacemaker agents + **configure:** `--with-gnutls="no"` explicitly disables support for Pacemaker Remote and the `remote-tls-port` cluster property + **configure:** `--with-acl` has been removed (support for ACLs is always built) + **configure:** deprecated `--with-pkgname,` `--with-pkg-name`, `--with-brand`, `--enable-ansi,` and `--enable-no-stack` options have been removed + environment variables file (typically /etc/sysconfig/pacemaker or /etc/default/pacemaker) will be installed when `make install` is run + documentation has dependency on python3-sphinx instead of publican, and is generated beneath doc/sphinx ## Fixes since Pacemaker-2.0.5 + **controller:** always refresh agent meta-data after start, in case agent was updated *(regression introduced in 1.1.18)* + **tools:** avoid crash when running `crm_mon` in daemonized mode with CGI output *(regression introduced in 2.0.3)* + **tools:** correctly treat unspecified node as all nodes instead of local node when `attrd_updater` `--query` or `crm_resource` `--cleanup` is run on a Pacemaker Remote node *(regressions introduced in 1.1.14 and 1.1.17)* + **tools:** exit with status 0 (not 64) for `--version` argument to `crm_simulate` *(regression introduced in 2.0.4)* and `crm_resource` and crmadmin *(regression introduced in 2.0.5)* + **pacemaker-attrd:** avoid race condition where transient attributes for a leaving node could be reinstated when the node rejoins, potentially causing a node that was just rebooted to exit the cluster immediately after rejoining + **controller,scheduler:** fix year 2038 issues affecting shutdowns, remote node fencing, last-rc-change, and ticket last-granted dates + **controller:** retry scheduler connection after failure, to avoid cluster stopping on a node without stopping resources (clbz#5445) + **fencing:** avoid pending fencing entry getting "stuck" in history if originating node was unreachable when result was received + **fencing:** retry getting agent meta-data if initial attempt fails + **fencing:** detect when devices have been removed from configuration + **scheduler:** constrain clone-min, clone-max, clone-node-max, promoted-max, and promoted-node-max options to non-negative values + **scheduler:** constrain resource priorities and node-health-base to score range + **scheduler:** treat invalid duration fields in time-based rules as 0, not -1 + **scheduler:** node attribute rule expressions with a value-source of "param" or "meta" work when rsc or rsc-pattern with an inverted match is given, as well as when rsc-pattern with a regular match is given + **scheduler:** node attribute rule expressions with a value-source of "param" work with a resource parameter that itself is determined by a node attribute rule expression + **scheduler:** avoid remote connection shutdown hanging when connection history for node hosting connection is not last in CIB status + **scheduler:** route monitor cancellations behind moving remote connections correctly + **libcrmcommon:** avoid potential integer overflow when adding seconds to times + **tools:** cibsecret syncs to remote nodes and guest nodes as well as cluster nodes + **tools:** show other available cluster information in `crm_mon` even if fence history is not available + **tools:** retry failed fencer and CIB connections in `crm_mon` + **tools:** `crm_mon` reports if Pacemaker is waiting for sbd at start-up + **tools:** respect rules when showing node attributes in `crm_mon` + **tools:** improve messages when `crm_mon` is run on disconnected remote node + **tools:** constrain node IDs to non-negative values for `crm_node -N` + **tools:** `crm_node` -l on restarted node works even when Corosync 2 is used without node names specified in Corosync configuration + **tools:** fix issues in calculation of non-sensitive resource parameter digests that made `crm_simulate` wrongly think configuration changed - C API changes since Pacemaker-2.0.5 + **all:** new `PCMK_ALLOW_DEPRECATED` constant controls API availability + **libcrmcluster:** deprecate `crm_terminate_member()` + **libcrmcluster:** deprecate `crm_terminate_member_no_mainloop()` + **libcrmcommon:** add `CRMD_ACTION_RELOAD_AGENT` string constant + **libcrmcommon:** add `PCMK_OCF_MAJOR_VERSION` string constant + **libcrmcommon:** add `PCMK_OCF_MINOR_VERSION` string constant + **libcrmcommon:** add `PCMK_OCF_RUNNING_PROMOTED` enum value + **libcrmcommon:** add `PCMK_OCF_VERSION` string constant + **libcrmcommon:** add `PCMK_XE_PROMOTABLE_LEGACY` string constant + **libcrmcommon:** add `PCMK_XE_PROMOTED_MAX_LEGACY` string constant + **libcrmcommon:** add `PCMK_XE_PROMOTED_NODE_MAX_LEGACY` string constant + **libcrmcommon:** add enum `ocf_exitcode` (moved from libcrmservice) + **libcrmcommon:** deprecate `__builtin_expect()` + **libcrmcommon:** deprecate `__likely()` + **libcrmcommon:** deprecate `__unlikely()` + **libcrmcommon:** deprecate `crm_atoi()` + **libcrmcommon:** deprecate `crm_build_path()` + **libcrmcommon:** deprecate `crm_config_error` global variable + **libcrmcommon:** deprecate `crm_config_warning` global variable + **libcrmcommon:** deprecate `crm_ftoa()` + **libcrmcommon:** deprecate `crm_hash_table_size()` + **libcrmcommon:** deprecate `crm_itoa()` + **libcrmcommon:** deprecate `crm_itoa_stack()` + **libcrmcommon:** deprecate `crm_log_cli_init()` + **libcrmcommon:** deprecate `crm_parse_int()` + **libcrmcommon:** deprecate `crm_parse_ll()` + **libcrmcommon:** deprecate `crm_str_hash()` + **libcrmcommon:** deprecate `crm_str_table_dup()` + **libcrmcommon:** deprecate `crm_str_table_new()` + **libcrmcommon:** deprecate `crm_strcase_equal()` + **libcrmcommon:** deprecate `crm_strcase_hash()` + **libcrmcommon:** deprecate `crm_strcase_table_new()` + **libcrmcommon:** deprecate `crm_strip_trailing_newline()` + **libcrmcommon:** deprecate `crm_ttoa()` + **libcrmcommon:** deprecate `EOS` constant + **libcrmcommon:** deprecate `GListPtr` type + **libcrmcommon:** deprecate `g_str_hash_traditional()` + **libcrmcommon:** deprecate `MAX_IPC_DELAY` constant + **libcrmcommon:** deprecate `pcmk_format_named_time()` + **libcrmcommon:** deprecate `pcmk_format_nvpair()` + **libcrmcommon:** deprecate `pcmk_numeric_strcasecmp()` + **libcrmcommon:** deprecate `PCMK_OCF_DEGRADED_MASTER` enum value + **libcrmcommon:** deprecate `PCMK_OCF_FAILED_MASTER` enum value + **libcrmcommon:** deprecate `PCMK_OCF_RUNNING_MASTER` enum value + **libcrmcommon:** deprecate `pcmk_scan_nvpair()` + **libcrmcommon:** deprecate `XML_CIB_TAG_MASTER` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_MASTER_MAX` string constant + **libcrmcommon:** deprecate `XML_RSC_ATTR_MASTER_NODEMAX` string constant + **libcrmservice:** enum `ocf_exitcode` is obtained from libcrmcommon + **libpacemaker:** add `pcmk_controller_status()` function + **libpacemaker:** add `pcmk_designated_controller()` function + **libpacemaker:** add `pcmk_list_nodes()` function + **libpacemaker:** add `pcmk_pacemakerd_status()` function + **libpacemaker:** add `pcmk_resource_digests()` function + **libpe_status:** add `parameter_cache` member to `pe_resource_t` + **libpe_status:** add `pe_order_promoted_implies_first` enum value + **libpe_status:** add `pe_rsc_params()` + **libpe_status:** add `RSC_ROLE_PROMOTED` enum value + **libpe_status:** add `RSC_ROLE_PROMOTED_LEGACY_S` string constant + **libpe_status:** add `RSC_ROLE_PROMOTED_S` string constant + **libpe_status:** add `RSC_ROLE_UNPROMOTED` enum value + **libpe_status:** add `RSC_ROLE_UNPROMOTED_LEGACY_S` string constant + **libpe_status:** add `RSC_ROLE_UNPROMOTED_S` string constant + **libpe_status:** add priv member to `pcmk_working_set_t`, for Pacemaker use only + **libpe_status:** deprecate `pe_order_implies_first_master` enum value + **libpe_status:** deprecate `pe_print_details` enum value + **libpe_status:** deprecate `pe_print_dev` enum value + **libpe_status:** deprecate `pe_print_html` enum value + **libpe_status:** deprecate `pe_print_log` enum value + **libpe_status:** deprecate `pe_print_max_details` enum value + **libpe_status:** deprecate `pe_print_ncurses` enum value + **libpe_status:** deprecate `pe_print_xml` enum value + **libpe_status:** deprecate `pe_resource_t` parameters member + **libpe_status:** deprecate `RSC_ROLE_MASTER` enum value + **libpe_status:** deprecate `RSC_ROLE_MASTER_S` string constant + **libpe_status:** deprecate `RSC_ROLE_SLAVE` enum value + **libpe_status:** deprecate `RSC_ROLE_SLAVE_S` string constant + **libpe_status:** ignore `->parameter()` resource object function's create argument # Pacemaker-2.0.5 (02 Dec 2020) - 534 commits with 286 files changed, 23133 insertions(+), 14626 deletions(-) ## Features added since Pacemaker-2.0.4 + **configuration:** Add type="integer" to rule elements, allowing for specifying 64-bit integers and specifying double-precision floating point numbers when type="number". + **daemons:** Recognize new OCF agent status codes 190 (degraded) and 191 (degraded master) to be treated as success but displayed as errors. + **sbd-integration:** support `SBD_SYNC_RESOURCE_STARTUP` environment variable to better synchronize Pacemaker start-up and shutdown with SBD + **scheduler:** Add rule-based tests to `op_defaults` and `rsc_defaults.` + **scheduler:** Add on-fail=demote and no-quorum-policy=demote recovery policies for promoted resources. + **tools:** Add `--resource=` to filter `crm_mon` output for a resource. + **tools:** Add -P to crmadmin to show pacemakerd status. + **tools:** In cibsecret, read value to set from input (or stdin) if not specified. ## Fixes for regressions introduced in Pacemaker-2.0.4 + **tools:** Add the node name back to bundle instances in `crm_mon.` + **tools:** get `stonith_admin` `--list-targets` working again ## Fixes for regressions introduced in Pacemaker-2.0.3 + **tools:** Fix adding HTTP headers to `crm_mon` in daemon mode. + **tools:** Show expected score of ping attributes in `crm_mon` XML output ## Fixes for regressions introduced in Pacemaker-2.0.1 + **scheduler:** require pre-/post-start notifications correctly. ## Changes since Pacemaker-2.0.4 + Prevent the bypassing of ACLs by direct IPC (CVE-2020-25654) + **build:** Fix a build issue on Fedora s390x. + **build:** Fix python2 vs. python3 packaging problems on openSUSE Tumbleweed + **build:** Update pkgconfig files for CI testing + **controller:** avoid recovery delay when shutdown locks expire + **controller:** Log correct timeout for timed-out stonith monitor + **fencer:** avoid infinite loop if device is removed during operation + **fencer:** avoid possible use-of-NULL when parsing metadata + **libfencing:** add `port` or `plug` parameter according to metadata on `validate` if no `pcmk_host_argument` specified + **libfencing:** respect `pcmk_host_argument=none` on `validate` + **scheduler:** disallow on-fail=stop for stop operations + **scheduler:** don't schedule a dangling migration stop if one already occurred + **scheduler:** don't select instance to be promoted on a guest that can't run + **scheduler:** fix build when `DEFAULT_CONCURRENT_FENCING_TRUE` is set + **scheduler:** Remove `pe_print_expanded_xml` print option. + **scheduler:** Use `pcmk_monitor_timeout` as stonith start timeout + **tools:** Add management status to `crm_mon`'s group output. + **tools:** Add "No active resources" to one case in `crm_mon` output. + **tools:** Allow tags and alerts in `cibadmin --scope` + **tools:** Avoid `crm_node` on Pacemaker Remote nodes failing when other executor actions are occurring. + **tools:** Avoid pending fence actions getting stuck in `crm_mon` display + **tools:** "Connectivity is lost" may not be displayed even if the ping communication is lost. + **tools:** Display stop-all-resources in `crm_mon's` cluster options. + **tools:** don't use pssh -q option in cibsecret unless supported + **tools:** Fix adding the http-equiv header to `crm_mon` in daemon mode. + **tools:** If a clone or group is disabled, display that in `crm_mon` as part of the resource's header. + **tools:** `crm_node` -l and -p now work from Pacemaker Remote nodes + **tools:** Don't display `crm_resource` error messages twice. + **tools:** Print inactive resources by default with `crm_mon` xml output. + **tools:** properly detect local node name in cibsecret + **tools:** Revert some `crm_resource` string-related checks on variables that aren't strings + **tools:** Use bash in cibsecret + **xml:** Mark new `crm_mon` attributes as optional in schema # Pacemaker-2.0.4 (15 Jun 2020) - 515 commits with 269 files changed, 22560 insertions(+), 13726 deletions(-) ## Features added since Pacemaker-2.0.3 + **build:** Add support for glib-based unit tests. + **controller:** add new feature 'shutdown-locks' Optionally allow a gracefully shutdown node to have the resources locked to it for a configurable time. So that it can be restarted with exactly the same resources running as before. + **controller/fencing/scheduler:** add new feature 'priority-fencing-delay' Optionally derive the priority of a node from the resource-priorities of the resources it is running. In a fencing-race the node with the highest priority has a certain advantage over the others as fencing requests for that node are executed with an additional delay. controlled via cluster option priority-fencing-delay (default = 0) + **tools:** `stonith_admin`: add `--delay` option (default = 0) to support enforced fencing delay + **tools:** Add `--include=` and `--exclude=` to `crm_mon.` + **tools:** Add `--node=` to filter `crm_mon` output for a node (or tag). + **tools:** Allow more fine grained filtering of fence-history in `crm_mon.` + **tools:** Allow `crm_rule` to check some `date_specs.` ## Fixes for regressions introduced in Pacemaker-2.0.0 + **tools:** ensure that getting the local node name using `crm_node` always works from Pacemaker Remote node command line ## Changes since Pacemaker-2.0.3 + **build:** improve checking headers for C++ fitness + **build:** restore build on latest OpenSUSE + **fencing:** Report an error getting history in output instead of empty history + **fencing:** Improve parameter handling for RHCS-style fence-agents - make parameter `action` shown as not required - add `port` or `plug` parameter according to metadata - `plug` parameter shown as non-required in the metadata + **controller:** clear leaving node's transient attributes even if there is no DC + **controller:** don't ack same request twice + **iso8601:** Fix `crm_time_parse_offset()` to parse offset with plus sign. + **libcrmcommon, libpe:** validate interval specs correctly + **libcrmcommon:** Add `pcmk_str_is_infinity`, `pcmk_str_is_minus_infinity` and `pcmk__unregister_formats.` + **libcrmcommon:** Extend what `pcmk__split_range` can do. + **libfencing:** Export formatted output functions. + **libpe_status:** Add node list arg to output messages preventing weird behavior + potential segfaults + **libpe_status:** Update the maint mode message for HTML output. + **fencing, tools:** Fix arguments to stonith-event. + **scheduler:** don't incorporate dependencies' colocations multiple times + **scheduler:** ensure attenuated scores still have some effect + **scheduler:** ignore colocation constraints with 0 score + **scheduler:** make sure cluster-wide maintenance-mode=true overrides per-resource settings + **scheduler:** properly detect whether guest node is fenceable + **scheduler:** do not differentiate case where all the nodes have equal priority + **tests:** Add tests for `pe_cron_range_satisfied.` + **tests:** Add tests for the current behavior of `pcmk__split_range.` + **tools, lib:** Use standard pacemaker return codes in `crm_rule.` + **tools:** Correct stonith-event arguments in `crm_mon.` + **tools:** Fix man pages for `crm_mon` & `crm_diff.` + **tools:** Make `crm_mon` `--interval` understand more formats. + **tools:** Fix `--html-title=` in `crm_mon.` + **tools:** Print errors to stderr for `crm_mon's` legacy xml. + **tools:** use return codes consistently in `stonith_admin` + **tools:** Use glib for cmdline handling in `crm_diff`, `crm_simulate` & `crm_error` + **xml:** Add a new version of the tags schema. + **based:** populate `cib_nodes` when `cibadmin -R --scope=configuration` + **cibsecret:** don't use pssh -q option unless supported + **fencing:** avoid infinite loop if device is removed during operation + **fencing:** avoid possible use-of-NULL when parsing metadata + **fencing:** Remove dangling 'pending' for completed DC fencing. (CLBZ#5401) # Pacemaker-2.0.3 (25 Nov 2019) - 601 commits with 227 files changed, 17862 insertions(+), 10471 deletions(-) ## Features added since Pacemaker-2.0.2 + **controller:** new 'fence-reaction' cluster option specifies whether local node should 'stop' or 'panic' if notified of own fencing + **controller:** more cluster properties support ISO 8601 time specifications + **controller:** calculate cluster recheck interval dynamically when possible + **Pacemaker Remote:** allow file for environment variables when used in bundle + **Pacemaker Remote:** allow configurable listen address and TLS priorities + **tools:** `crm_mon` now supports standard `--output-as/--output-to` options + **tools:** `crm_mon` HTML output supports user-defined CSS stylesheet + **tools:** `stonith_admin` supports HTML output in addition to text and XML + **tools:** `crm_simulate` supports `--repeat` option to repeat profiling tests + **tools:** new `pcmk_simtimes` tool compares `crm_simulate` profiling output + **agents:** SysInfo supports K, T, and P units in addition to Kb and G ## Changes since Pacemaker-2.0.2 + **fencer:** do not block concurrent fencing actions on a device *(regression since 2.0.2)* + **all:** avoid Year 2038 issues + **all:** allow ISO 8601 strings of form "T