diff --git a/ChangeLog.md b/ChangeLog.md
index e35dc0119f..695d3ceba4 100644
--- a/ChangeLog.md
+++ b/ChangeLog.md
@@ -1,4703 +1,5175 @@
+# Pacemaker-3.0.0-rc1 (14 Nov 2024)
+* 1938 commits with 685 files changed, 26363 insertions(+), 33503 deletions(-)
+
+## Features added since Pacemaker-2.1.9
+
+* For more details, especially about build option changes, see
+ https://projects.clusterlabs.org/w/projects/pacemaker/pacemaker_3.0_changes/
+* **upgrades:** drop support for rolling upgrades from versions less than 2.0.0
+* **Pacemaker Remote:** drop support for Pacemaker 1 cluster nodes connecting
+ to Pacemaker Remote 3 or later nodes or bundles, and Pacemaker 3 or later
+ cluster nodes connecting to Pacemaker Remote 1.1.14 or earlier nodes
+* **build:** drop creation of deprecated `pacemaker_remoted` link to
+ `pacemaker-remoted`
+* **environment:** drop support for deprecated `HA_cib_timeout`,
+ `HA_shutdown_delay`, `PCMK_cib_timeout`, `PCMK_dh_min_bits`, and
+ `PCMK_shutdown_delay` variables
+* **environment:** `PCMK_panic_action` values are case-sensitive
+* **CIB:** XML syntax must be correctly formed (no errors will be ignored)
+* **CIB:** `validate-with` must be set, case-sensitive, and not `pacemaker-0.6`,
+ `pacemaker-0.7`, `pacemaker-1.1`, `pacemaker-next`, `transitional-0.6`, or
+ unknown
+* **CIB:** deprecate `concurrent-fencing` cluster option, which now defaults to
+ true
+* **CIB:** deprecate `record-pending` operation option
+* **CIB:** drop support for deprecated cluster options
+ `crmd-finalization-timeout`, `crmd-integration-timeout`, and
+ `crmd-transition-delay`, and `remove-after-stop`
+* **CIB:** drop support for deprecated `poweroff` value of `stonith-action`
+ cluster option
+* **CIB:** drop support for nodes with `type` set to deprecated value `ping` or
+ an invalid value
+* **CIB:** drop support for deprecated `nagios` and `upstart` resource classes
+* **CIB:** drop support for deprecated `master` resources
+* **CIB:** drop support for deprecated `masters` bundle option
+* **CIB:** drop support for deprecated bundles using `rkt` containers
+* **CIB:** drop support for `instance_attributes` in `rsc_defaults`
+* **CIB:** drop support for deprecated `restart-type` resource option
+* **CIB:** drop support for deprecated `can-fail`, `can_fail`, and
+ `role_after_failure` operation options
+* **CIB:** drop support for deprecated `rsc-instance`, `with-rsc-instance`,
+ `first-instance`, and `then-instance` constraint options
+* **CIB:** drop support for deprecated `lifetime` elements in constraints
+* **CIB:** drop support for multiple top-level rules in location constraints or
+ name/value blocks
+* **CIB:** drop support for `name` attribute when `id-ref` attribute is present
+* **CIB:** drop support for deprecated `moon` in `date_spec` elements of rules
+* **CIB:** `globally-unique` clone option defaults to true if `clone-node-max`
+ is greater than 1
+* **CIB:** location constraints with an invalid `score`, `boolean-op`, or
+ `role`, colocation constraints with an invalid `score`, `rsc-role`, or
+ `with-rsc-role`, and ticket constraints with an invalid `rsc-role` are now
+ ignored (previously, default values would be used)
+* **CIB:** ignore node expressions for meta-attributes
+* **CIB:** treat misconfigured rules and rule expressions as not passing
+* **CIB:** treat negative `migration-threshold` as invalid and use default
+* **CIB:** invalid fencing level indexes are rejected by schema
+* **agents:** drop ocf:pacemaker:o2cb resource agent
+* **agents:** do not pass `HA_mcp`, `HA_quorum_type`, `PCMK_mcp`, or
+ `PCMK_quorum_type`, environment variables to agents
+* **alerts:** don't send deprecated alert environment variables to agents
+* **fencer:** drop support for non-clustered mode in fencer
+* **fencing:** default `pcmk_host_argument` to `none` if `port` not advertised
+* **liblrmd,libstonithd:** use standard default timeout (20s) for meta-data
+ actions
+* **pacemaker-based:** reject remote users if PAM not available
+* **tools:** `crm_shadow --reset` now requires `--force`
+* **tools:** define behavior of `attrd_updater -Q` without `-N`
+* **tools:** deprecate `cibadmin --local` option
+* **tools:** drop `--text-fancy` command-line option from all tools
+* **tools:** drop deprecated `cibadmin --host` and `--no-bcast` options
+* **tools:** drop deprecated `crm_resource --get-property`, `--set-property`,
+ and `--xml-file` options
+* **tools:** `cibadmin --md5-sum-versioned` no longer prints feature set
+* **tools:** `crm_resource` rejects invalid timeouts
+* **tools:** `crm_resource --delete` does not accept clone instances
+* **tools:** `crm_resource --delete` without `-t opt` exits with a usage error
+* **tools:** `crm_resource --delete` now succeeds if resource doesn't exist
+* **tools:** `crm_resource --option` throws usage error if appropriate
+* **tools:** drop deprecated `crm_mon` options `--as-html`,
+ `--disable-ncurses`, `--simple-status`, and `--web-cgi`
+
+## Fixes since Pacemaker-2.1.9
+
+* **executor:** avoid use-after-free during shutdown
+* **libcrmcommon:** rule expressions with the empty string as value now pass
+ when the corresponding node attribute is the empty string
+* **libstonithd:** avoid use-after-free when retrieving metadata of Linux-HA
+ fence agents
+* **libstonithd:** free escaped metadata descriptions correctly
+* **scheduler:** apply promotion priority to cloned group instances
+* **scheduler:** correctly retrieve any existing fail-count for increment
+* **scheduler:** don't apply colocations twice for promotion priority
+* **scheduler:** ignore `nvpair` with invalid `id-ref`
+* **scheduler:** ignore `value` in `nvpair` with `id-ref`
+* **scheduler:** use first action meta-attribute if block has duplicates
+* **scheduler:** consider group locations when member is explicit colocation
+ dependent
+* **tools:** don't trigger an assertion if stdout is closed
+* **tools:** CIB clients retry signon if first try fails
+* **tools:** don't double-free XML in `crm_verify` after schema update
+
+## Public API changes since Pacemaker-2.1.9
+
+* **libcib:** drop `util_compat.h` header
+* **libcib:** drop enum values `cib_database`, `cib_inhibit_bcast`,
+ `cib_mixed_update`, `cib_quorum_override`, and `cib_zero_copy`
+* **libcib:** drop `cib_api_operations_t` members `delete_absolute()`,
+ `inputfd()`, `is_master()`, `quit()`, `set_master()`, `set_op_callback()`,
+ `set_slave()`, `set_slave_all()`, `signon_raw()`, `update()`
+* **libcib:** drop `cib_t` member `op_callback()`
+* **libcib:** drop functions `cib_get_generation()`, `cib_metadata()`,
+ `cib_pref()`, `get_object_parent()`, `get_object_path()`,
+ `get_object_root()`, `query_node_uname()`, and `set_standby()`
+* **libcrmcluster:** add `pcmk_cluster_t` member `priv`
+* **libcrmcluster:** drop enums (including all their values)
+ `crm_ais_msg_class`, `crm_ais_msg_types`, `crm_get_peer_flags`,
+ `crm_join_phase`, `crm_node_flags`, and `crm_status_type`
+* **libcrmcluster:** drop type alias `crm_node_t`
+* **libcrmcluster:** drop struct `crm_peer_node_s`
+* **libcrmcluster:** drop `pcmk_cluster_t` members `cpg_handle`,
+ `group`, `nodeid`, `uname`, and `uuid`
+* **libcrmcluster:** drop global variables `crm_have_quorum`, `crm_peer_cache`,
+ `crm_peer_seq`, `crm_remote_peer_cache`
+* **libcrmcluster:** drop enum value `crm_join_nack_quiet`
+* **libcrmcluster:** drop string constants `CRM_NODE_LOST`, `CRM_NODE_MEMBER`
+* **libcrmcluster:** drop functions `cluster_connect_cpg()`,
+ `cluster_disconnect_cpg()`, `crm_active_peers()`, `crm_cluster_disconnect()`,
+ `crm_get_peer()`, `crm_get_peer_full()`, `crm_is_corosync_peer_active()`,
+ `crm_is_peer_active()`, `crm_join_phase_str()`, `crm_peer_destroy()`,
+ `crm_peer_init()`, `crm_peer_uname()`, `crm_peer_uuid()`,
+ `crm_remote_peer_cache_refresh()`, `crm_remote_peer_cache_remove()`,
+ `crm_remote_peer_cache_size()`, `crm_remote_peer_get()`,
+ `crm_set_autoreap()`, `crm_set_status_callback()`, `crm_terminate_member()`,
+ `crm_terminate_member_no_mainloop()`, `get_local_nodeid()`,
+ `get_local_node_name()`, `get_node_name()`, `is_corosync_cluster()`,
+ `pcmk_cpg_membership()`, `pcmk_message_common_cs()`, `reap_crm_member()`,
+ `send_cluster_message()`, `send_cluster_text()`, `set_uuid()`, and
+ `text2msg_type()`
+* **libcrmcluster:** rename struct `crm_cluster_s` to `pcmk__cluster`
+* **libcrmcommon:** add enum `pcmk_ipc_server` value `pcmk_ipc_unknown`
+* **libcrmcommon:** rename structs `pe_action_s` to `pcmk__action`, `pe_node_s`
+ to `pcmk__scored_node`, `pe_node_shared_s` to `pcmk__node_details`,
+ `pe_resource_s` to `pcmk__resource`, and `pe_working_set_s` to
+ `pcmk__scheduler`
+* **libcrmcommon:** add `pcmk_node_t` members `assign` and `private`
+* **libcrmcommon:** add `pcmk_resource_t` member `private`
+* **libcrmcommon:** change `pcmk_scheduler_t` member `flags` type to `uint64_t`
+* **libcrmcommon:** add defined constants `PCMK_OCF_ROOT`,
+ `PCMK_SCHEDULER_INPUT_DIR`, `PCMK_SCHEMA_DIR`, `PCMK_VALUE_CRASH`,
+ `PCMK_VALUE_OFF`, and `PCMK_VALUE_REBOOT`
+* **libcrmcommon:** deprecate defined constants `CIB_OPTIONS_FIRST`,
+ `CRM_SCHEMA_DIRECTORY`, `CRM_SYSTEM_STONITHD`, `CRM_XS`, `OCF_ROOT_DIR`, and
+ `PE_STATE_DIR`
+* **libcrmcommon:** deprecate functions `calculate_on_disk_digest()`,
+ `calculate_operation_digest()`, `calculate_xml_versioned_digest()`,
+ `char2score()`, `crm_extended_logging()`, `crm_ipc_connect()`,
+ `crm_is_daemon_name()`, `crm_xml_cleanup()`, `crm_xml_init()`,
+ `crm_xml_sanitize_id()`, `crm_xml_set_id()`, `expand_idref()`, `free_xml()`,
+ `hash2nvpair()`, `pcmk_free_xml_subtree()`, `pcmk_nvpairs2xml_attrs()`,
+ `pcmk_sort_nvpairs()`, `pcmk_xml_attrs2nvpairs()`, and `sorted_xml()`
+* **libcrmcommon:** drop headers `agents_compat.h`, `compatibility.h`,
+ `logging_compat.h`, `mainloop_compat.h`, `results_compat.h`,
+ `scores_compat.h`, `tags.h`, `tickets.h`, and `xml_io_compat.h`
+* **libcrmcommon:** drop global variables `crm_config_error`,
+ `crm_config_warning`, `resource_class_functions[]`, `was_processing_error`,
+ and `was_processing_warning`
+* **libcrmcommon:** drop enums (including all their values)
+ `action_fail_response`, `action_tasks`, `node_type`, `pcmk_rsc_flags`,
+ `pcmk_sched_flags`, `pe_action_flags`, `pe_discover_e`, `pe_link_state`,
+ `pe_obj_types`, `pe_ordering`, `pe_print_options`, `pe_restart`,
+ `rsc_recovery_type`, `rsc_start_requirement`, and `xml_log_options`
+* **libcrmcommon:** drop enum `crm_ipc_flags` value `crm_ipc_server_error`,
+ and enum `crm_ipc_flags` value `crm_ipc_server_info`
+* **libcrmcommon:** drop enum `ocf_exitcode` values `PCMK_OCF_CANCELLED`,
+ `PCMK_OCF_DEGRADED_MASTER`, `PCMK_OCF_EXEC_ERROR`, `PCMK_OCF_FAILED_MASTER`,
+ `PCMK_OCF_NOT_SUPPORTED`, `PCMK_OCF_OTHER_ERROR`, `PCMK_OCF_PENDING`,
+ `PCMK_OCF_RUNNING_MASTER`, `PCMK_OCF_SIGNAL`, and `PCMK_OCF_TIMEOUT`
+* **libcrmcommon:** drop enum `pe_find` values `pe_find_anon`, `pe_find_any`,
+ `pe_find_clone`, `pe_find_current`, `pe_find_inactive`, and `pe_find_renamed`
+* **libcrmcommon:** drop types `pcmk_assignment_methods_t`,
+ `pcmk_rsc_methods_t`, `pcmk_tag_t`, `pcmk_ticket_t`,
+ `resource_alloc_functions_t`, and `resource_object_functions_t`
+* **libcrmcommon:** drop structs `pe_action_wrapper_s`, `pe_tag_s`,
+ `pe_ticket_s`, `resource_alloc_functions_s`, and
+ `resource_object_functions_s`
+* **libcrmcommon:** drop `pcmk_scheduler_t` members `actions`, `action_id`,
+ `blocked_resources`, `colocation_constraints`, `config_hash`, `dc_uuid`,
+ `disabled_resources`, `failed`, `graph`, `localhost`, `max_valid_nodes`,
+ `ninstances`, `node_pending_timeout`, `now`, `num_synapse`, `op_defaults`,
+ `ordering_constraints`, `param_check`, `placement_constraints`,
+ `placement_strategy`, `priority_fencing_delay`, `recheck_by`, `resources`,
+ `rsc_defaults`, `shutdown_lock`, `singletons`, `stonith_action`,
+ `stonith_timeout`, `stop_needed`, `tags`, `template_rsc_sets`, `tickets`, and
+ `ticket_constraints`
+* **libcrmcommon:** drop direct access to all members of `pcmk_action_t`
+* **libcrmcommon:** drop `pcmk_node_t` members `count`, `fixed`,
+ `rsc_discover_mode`, and `weight`
+* **libcrmcommon:** drop `pcmk_resource_t` members `actions`, `allocated_to`,
+ `allowed_nodes`, `children`, `clone_name`, `cluster`, `cmds`, `container`,
+ `dangling_migrations`, `exclusive_discover`, `failure_timeout`, `fillers`,
+ `fns`, `is_remote_node`, `known_on`, `lock_node`, `lock_time`, `meta`,
+ `migration_threshold`, `next_role`, `ops_xml`, `orig_xml`, `parameters`,
+ `parameter_cache`, `parent`, `partial_migration_source`,
+ `partial_migration_target`, `pending_node`, `pending_task`, `priority`,
+ `recovery_type`, `remote_reconnect_ms`, `restart_type`, `role`, `rsc_cons`,
+ `rsc_cons_lhs`, `rsc_location`, `rsc_tickets`, `running_on`, `sort_index`,
+ `stickiness`, `utilization`, `variant`, `variant_opaque`, and `xml`
+* **libcrmcommon:** drop struct `pe_node_shared_s` members `allocated_rsc`,
+ `attrs`, `digest_cache`, `expected_up`, `id`, `is_dc`, `num_resources`,
+ `priority`, `remote_maintenance`, `remote_requires_reset`, `remote_rsc`,
+ `remote_was_fenced`, `rsc_discovery_enabled`, `scheduler`, `standby`,
+ `standby_onfail`, `type`, `uname`, `unpacked`, `unseen`, and `utilization`
+* **libcrmcommon:** drop defined constants `CRMD_ACTION_CANCEL`,
+ `CRMD_ACTION_DELETE`, `CRMD_ACTION_DEMOTE`, `CRMD_ACTION_DEMOTED`,
+ `CRMD_ACTION_METADATA`, `CRMD_ACTION_MIGRATE`, `CRMD_ACTION_MIGRATED`,
+ `CRMD_ACTION_NOTIFIED`, `CRMD_ACTION_NOTIFY`, `CRMD_ACTION_PROMOTE`,
+ `CRMD_ACTION_PROMOTED`, `CRMD_ACTION_RELOAD`, `CRMD_ACTION_RELOAD_AGENT`,
+ `CRMD_ACTION_START`, `CRMD_ACTION_STARTED`, `CRMD_ACTION_STATUS`,
+ `CRMD_ACTION_STOP`, `CRMD_ACTION_STOPPED`, `CRMD_METADATA`,
+ `CRM_ATTR_RA_VERSION`, `CRM_DEFAULT_OP_TIMEOUT_S`, `CRM_INFINITY_S`,
+ `CRM_MINUS_INFINITY_S`, `CRM_OP_FENCE`, `CRM_OP_LOCAL_SHUTDOWN`,
+ `CRM_OP_LRM_QUERY`, `CRM_OP_LRM_REFRESH`, `CRM_OP_RELAXED_CLONE`,
+ `CRM_OP_RELAXED_SET`, `CRM_PLUS_INFINITY_S`, `EOS`, `F_CLIENTNAME`,
+ `F_CRM_DATA`, `F_CRM_DC_LEAVING`, `F_CRM_ELECTION_AGE_S`,
+ `F_CRM_ELECTION_AGE_US`, `F_CRM_ELECTION_ID`, `F_CRM_ELECTION_OWNER`,
+ `F_CRM_HOST_FROM`, `F_CRM_HOST_TO`, `F_CRM_JOIN_ID`, `F_CRM_MSG_TYPE`,
+ `F_CRM_ORIGIN`, `F_CRM_REFERENCE`, `F_CRM_SYS_FROM`, `F_CRM_SYS_TO`,
+ `F_CRM_TASK`, `F_CRM_TGRAPH`, `F_CRM_TGRAPH_INPUT`, `F_CRM_THROTTLE_MAX`,
+ `F_CRM_THROTTLE_MODE`, `F_CRM_USER`, `F_CRM_VERSION`, `F_ORIG`, `F_SEQ`,
+ `F_SUBTYPE`, `F_TYPE`, `F_XML_TAGNAME`, `INFINITY`, `INFINITY_S`,
+ `MAX_IPC_DELAY`, `MINUS_INFINITY_S`, `OFFLINESTATUS`, `ONLINESTATUS`,
+ `PCMK_DEFAULT_METADATA_TIMEOUT_MS`, `PCMK_RESOURCE_CLASS_NAGIOS`,
+ `PCMK_RESOURCE_CLASS_UPSTART`, `PCMK_XA_PROMOTED_MAX_LEGACY`,
+ `PCMK_XA_PROMOTED_NODE_MAX_LEGACY`, `PCMK_XE_PROMOTABLE_LEGACY`,
+ `PCMK_XE_PROMOTED_MAX_LEGACY`, `PCMK_XE_PROMOTED_NODE_MAX_LEGACY`,
+ `RSC_CANCEL`, `RSC_DELETE`, `RSC_DEMOTE`, `RSC_DEMOTED`, `RSC_METADATA`,
+ `RSC_MIGRATE`, `RSC_MIGRATED`, `RSC_NOTIFIED`, `RSC_NOTIFY`, `RSC_PROMOTE`,
+ `RSC_PROMOTED`, `RSC_ROLE_MASTER`, `RSC_ROLE_PROMOTED`,
+ `RSC_ROLE_PROMOTED_S`, `RSC_ROLE_SLAVE`, `RSC_ROLE_STARTED`,
+ `RSC_ROLE_STOPPED`, `RSC_ROLE_UNKNOWN`, `RSC_ROLE_UNPROMOTED`, `RSC_START`,
+ `RSC_STARTED`, `RSC_STATUS`, `RSC_STOP`, `RSC_STOPPED`, `SUPPORT_UPSTART`,
+ `T_ATTRD`, `T_CRM`, `XML_ACL_ATTR_ATTRIBUTE`, `XML_ACL_ATTR_KIND`,
+ `XML_ACL_ATTR_REF`, `XML_ACL_ATTR_REFv1`, `XML_ACL_ATTR_TAG`,
+ `XML_ACL_ATTR_TAGv1`, `XML_ACL_ATTR_XPATH`, `XML_ACL_TAG_DENY`,
+ `XML_ACL_TAG_GROUP`, `XML_ACL_TAG_PERMISSION`, `XML_ACL_TAG_READ`,
+ `XML_ACL_TAG_ROLE`, `XML_ACL_TAG_ROLE_REF`, `XML_ACL_TAG_ROLE_REFv1`,
+ `XML_ACL_TAG_USER`, `XML_ACL_TAG_USERv1`, `XML_ACL_TAG_WRITE`,
+ `XML_AGENT_ATTR_CLASS`, `XML_AGENT_ATTR_PROVIDER`, `XML_ALERT_ATTR_PATH`,
+ `XML_ALERT_ATTR_REC_VALUE`, `XML_ALERT_ATTR_TIMEOUT`,
+ `XML_ALERT_ATTR_TSTAMP_FORMAT`, `XML_ATTR_CRM_VERSION`, `XML_ATTR_DC_UUID`,
+ `XML_ATTR_DESC`, `XML_ATTR_DIGEST`, `XML_ATTR_GENERATION`,
+ `XML_ATTR_GENERATION_ADMIN`, `XML_ATTR_HAVE_QUORUM`,
+ `XML_ATTR_HAVE_WATCHDOG`, `XML_ATTR_ID`, `XML_ATTR_IDREF`,
+ `XML_ATTR_ID_LONG`, `XML_ATTR_NAME`, `XML_ATTR_NUMUPDATES`, `XML_ATTR_OP`,
+ `XML_ATTR_ORIGIN`, `XML_ATTR_QUORUM_PANIC`, `XML_ATTR_RA_VERSION`,
+ `XML_ATTR_REFERENCE`, `XML_ATTR_REQUEST`, `XML_ATTR_RESPONSE`,
+ `XML_ATTR_STONITH_DEVICES`, `XML_ATTR_STONITH_INDEX`,
+ `XML_ATTR_STONITH_TARGET`, `XML_ATTR_STONITH_TARGET_ATTRIBUTE`,
+ `XML_ATTR_STONITH_TARGET_PATTERN`, `XML_ATTR_STONITH_TARGET_VALUE`,
+ `XML_ATTR_TE_NOWAIT`, `XML_ATTR_TE_TARGET_RC`, `XML_ATTR_TIMEOUT`,
+ `XML_ATTR_TRANSITION_KEY`, `XML_ATTR_TRANSITION_MAGIC`, `XML_ATTR_TSTAMP`,
+ `XML_ATTR_TYPE`, `XML_ATTR_UNAME`, `XML_ATTR_UPDATE_CLIENT`,
+ `XML_ATTR_UPDATE_ORIG`, `XML_ATTR_UPDATE_USER`, `XML_ATTR_UUID`,
+ `XML_ATTR_VALIDATION`, `XML_ATTR_VERBOSE`, `XML_ATTR_VERSION`,
+ `XML_BOOLEAN_FALSE`, `XML_BOOLEAN_NO`, `XML_BOOLEAN_TRUE`, `XML_BOOLEAN_YES`,
+ `XML_CIB_ATTR_PRIORITY`, `XML_CIB_ATTR_REPLACE`, `XML_CIB_ATTR_SHUTDOWN`,
+ `XML_CIB_ATTR_SOURCE`, `XML_CIB_ATTR_WRITTEN`, `XML_CIB_TAG_ACLS`,
+ `XML_CIB_TAG_ALERT`, `XML_CIB_TAG_ALERTS`, `XML_CIB_TAG_ALERT_ATTR`,
+ `XML_CIB_TAG_ALERT_ATTRIBUTES`, `XML_CIB_TAG_ALERT_FENCING`,
+ `XML_CIB_TAG_ALERT_NODES`, `XML_CIB_TAG_ALERT_RECIPIENT`,
+ `XML_CIB_TAG_ALERT_RESOURCES`, `XML_CIB_TAG_ALERT_SELECT`,
+ `XML_CIB_TAG_CONFIGURATION`, `XML_CIB_TAG_CONSTRAINTS`,
+ `XML_CIB_TAG_CONTAINER`, `XML_CIB_TAG_CRMCONFIG`, `XML_CIB_TAG_DOMAINS`,
+ `XML_CIB_TAG_GENERATION_TUPPLE`, `XML_CIB_TAG_GROUP`,
+ `XML_CIB_TAG_INCARNATION`, `XML_CIB_TAG_LRM`, `XML_CIB_TAG_MASTER`,
+ `XML_CIB_TAG_NODE`, `XML_CIB_TAG_NODES`, `XML_CIB_TAG_NVPAIR`,
+ `XML_CIB_TAG_OBJ_REF`, `XML_CIB_TAG_OPCONFIG`, `XML_CIB_TAG_PROPSET`,
+ `XML_CIB_TAG_RESOURCE`, `XML_CIB_TAG_RESOURCES`, `XML_CIB_TAG_RSCCONFIG`,
+ `XML_CIB_TAG_RSC_TEMPLATE`, `XML_CIB_TAG_SECTION_ALL`, `XML_CIB_TAG_STATE`,
+ `XML_CIB_TAG_STATUS`, `XML_CIB_TAG_TAG`, `XML_CIB_TAG_TAGS`,
+ `XML_CIB_TAG_TICKETS`, `XML_CIB_TAG_TICKET_STATE`, `XML_COLOC_ATTR_INFLUENCE`,
+ `XML_COLOC_ATTR_NODE_ATTR`, `XML_COLOC_ATTR_SOURCE`,
+ `XML_COLOC_ATTR_SOURCE_INSTANCE`, `XML_COLOC_ATTR_SOURCE_ROLE`,
+ `XML_COLOC_ATTR_TARGET`, `XML_COLOC_ATTR_TARGET_INSTANCE`,
+ `XML_COLOC_ATTR_TARGET_ROLE`, `XML_CONFIG_ATTR_DC_DEADTIME`,
+ `XML_CONFIG_ATTR_ELECTION_FAIL`, `XML_CONFIG_ATTR_FENCE_REACTION`,
+ `XML_CONFIG_ATTR_FORCE_QUIT`, `XML_CONFIG_ATTR_NODE_PENDING_TIMEOUT`,
+ `XML_CONFIG_ATTR_PRIORITY_FENCING_DELAY`, `XML_CONFIG_ATTR_RECHECK`,
+ `XML_CONFIG_ATTR_SHUTDOWN_LOCK`, `XML_CONFIG_ATTR_SHUTDOWN_LOCK_LIMIT`,
+ `XML_CONS_ATTR_SYMMETRICAL`, `XML_CONS_TAG_RSC_DEPEND`,
+ `XML_CONS_TAG_RSC_LOCATION`, `XML_CONS_TAG_RSC_ORDER`, `XML_CONS_TAG_RSC_SET`,
+ `XML_CONS_TAG_RSC_TICKET`, `XML_CRM_TAG_PING`, `XML_DIFF_ATTR`,
+ `XML_DIFF_CHANGE`, `XML_DIFF_LIST`, `XML_DIFF_MARKER`, `XML_DIFF_OP`,
+ `XML_DIFF_PATH`, `XML_DIFF_POSITION`, `XML_DIFF_RESULT`, `XML_DIFF_VERSION`,
+ `XML_DIFF_VSOURCE`, `XML_DIFF_VTARGET`, `XML_EXPR_ATTR_ATTRIBUTE`,
+ `XML_EXPR_ATTR_OPERATION`, `XML_EXPR_ATTR_TYPE`, `XML_EXPR_ATTR_VALUE`,
+ `XML_EXPR_ATTR_VALUE_SOURCE`, `XML_FAILCIB_ATTR_ID`,
+ `XML_FAILCIB_ATTR_OBJTYPE`, `XML_FAILCIB_ATTR_OP`, `XML_FAILCIB_ATTR_REASON`,
+ `XML_FAIL_TAG_CIB`, `XML_GRAPH_TAG_CRM_EVENT`, `XML_GRAPH_TAG_DOWNED`,
+ `XML_GRAPH_TAG_MAINTENANCE`, `XML_GRAPH_TAG_PSEUDO_EVENT`,
+ `XML_GRAPH_TAG_RSC_OP`, `XML_LOCATION_ATTR_DISCOVERY`, `XML_LOC_ATTR_SOURCE`,
+ `XML_LOC_ATTR_SOURCE_PATTERN`, `XML_LRM_ATTR_CALLID`,
+ `XML_LRM_ATTR_EXIT_REASON`, `XML_LRM_ATTR_INTERVAL`,
+ `XML_LRM_ATTR_INTERVAL_MS`, `XML_LRM_ATTR_MIGRATE_SOURCE`,
+ `XML_LRM_ATTR_MIGRATE_TARGET`, `XML_LRM_ATTR_OPSTATUS`,
+ `XML_LRM_ATTR_OP_DIGEST`, `XML_LRM_ATTR_OP_RESTART`, `XML_LRM_ATTR_OP_SECURE`,
+ `XML_LRM_ATTR_RC`, `XML_LRM_ATTR_RESTART_DIGEST`, `XML_LRM_ATTR_ROUTER_NODE`,
+ `XML_LRM_ATTR_RSCID`, `XML_LRM_ATTR_SECURE_DIGEST`, `XML_LRM_ATTR_TARGET`,
+ `XML_LRM_ATTR_TARGET_UUID`, `XML_LRM_ATTR_TASK`, `XML_LRM_ATTR_TASK_KEY`,
+ `XML_LRM_TAG_RESOURCE`, `XML_LRM_TAG_RESOURCES`, `XML_LRM_TAG_RSC_OP`,
+ `XML_NODE_ATTR_RSC_DISCOVERY`, `XML_NODE_EXPECTED`, `XML_NODE_IN_CLUSTER`,
+ `XML_NODE_IS_FENCED`, `XML_NODE_IS_MAINTENANCE`, `XML_NODE_IS_PEER`,
+ `XML_NODE_IS_REMOTE`, `XML_NODE_JOIN_STATE`, `XML_NVPAIR_ATTR_NAME`,
+ `XML_NVPAIR_ATTR_VALUE`, `XML_OP_ATTR_ALLOW_MIGRATE`,
+ `XML_OP_ATTR_DIGESTS_ALL`, `XML_OP_ATTR_DIGESTS_SECURE`,
+ `XML_OP_ATTR_ON_FAIL`, `XML_OP_ATTR_ORIGIN`, `XML_OP_ATTR_PENDING`,
+ `XML_OP_ATTR_START_DELAY`, `XML_ORDER_ATTR_FIRST`,
+ `XML_ORDER_ATTR_FIRST_ACTION`, `XML_ORDER_ATTR_FIRST_INSTANCE`,
+ `XML_ORDER_ATTR_KIND`, `XML_ORDER_ATTR_THEN`, `XML_ORDER_ATTR_THEN_ACTION`,
+ `XML_ORDER_ATTR_THEN_INSTANCE`, `XML_PARANOIA_CHECKS`,
+ `XML_PING_ATTR_CRMDSTATE`, `XML_PING_ATTR_PACEMAKERDSTATE`,
+ `XML_PING_ATTR_PACEMAKERDSTATE_INIT`,
+ `XML_PING_ATTR_PACEMAKERDSTATE_REMOTE`,
+ `XML_PING_ATTR_PACEMAKERDSTATE_RUNNING`,
+ `XML_PING_ATTR_PACEMAKERDSTATE_SHUTDOWNCOMPLETE`
+ `XML_PING_ATTR_PACEMAKERDSTATE_SHUTTINGDOWN`
+ `XML_PING_ATTR_PACEMAKERDSTATE_STARTINGDAEMONS`
+ `XML_PING_ATTR_PACEMAKERDSTATE_WAITPING`, `XML_PING_ATTR_STATUS`,
+ `XML_PING_ATTR_SYSFROM`, `XML_REMOTE_ATTR_RECONNECT_INTERVAL`,
+ `XML_RSC_ATTR_CLEAR_INTERVAL`, `XML_RSC_ATTR_CLEAR_OP`,
+ `XML_RSC_ATTR_CONTAINER`, `XML_RSC_ATTR_CRITICAL`,
+ `XML_RSC_ATTR_FAIL_STICKINESS`, `XML_RSC_ATTR_FAIL_TIMEOUT`,
+ `XML_RSC_ATTR_INCARNATION`, `XML_RSC_ATTR_INCARNATION_MAX`,
+ `XML_RSC_ATTR_INCARNATION_MIN`, `XML_RSC_ATTR_INCARNATION_NODEMAX`,
+ `XML_RSC_ATTR_INTERLEAVE`, `XML_RSC_ATTR_INTERNAL_RSC`,
+ `XML_RSC_ATTR_MAINTENANCE`, `XML_RSC_ATTR_MANAGED`, `XML_RSC_ATTR_MASTER_MAX`,
+ `XML_RSC_ATTR_MASTER_NODEMAX`, `XML_RSC_ATTR_MULTIPLE`, `XML_RSC_ATTR_NOTIFY`,
+ `XML_RSC_ATTR_ORDERED`, `XML_RSC_ATTR_PROMOTABLE`,
+ `XML_RSC_ATTR_PROMOTED_MAX`, `XML_RSC_ATTR_PROMOTED_NODEMAX`,
+ `XML_RSC_ATTR_PROVIDES`, `XML_RSC_ATTR_REMOTE_NODE`,
+ `XML_RSC_ATTR_REMOTE_RA_ADDR`, `XML_RSC_ATTR_REMOTE_RA_PORT`,
+ `XML_RSC_ATTR_REMOTE_RA_SERVER`, `XML_RSC_ATTR_REQUIRES`,
+ `XML_RSC_ATTR_RESTART`, `XML_RSC_ATTR_STICKINESS`, `XML_RSC_ATTR_TARGET`,
+ `XML_RSC_ATTR_TARGET_ROLE`, `XML_RSC_ATTR_UNIQUE`, `XML_RSC_OP_LAST_CHANGE`,
+ `XML_RSC_OP_LAST_RUN`, `XML_RSC_OP_T_EXEC`, `XML_RSC_OP_T_QUEUE`,
+ `XML_RULE_ATTR_BOOLEAN_OP`, `XML_RULE_ATTR_ROLE`, `XML_RULE_ATTR_SCORE`,
+ `XML_RULE_ATTR_SCORE_ATTRIBUTE`, `XML_TAG_ATTRS`, `XML_TAG_ATTR_SETS`,
+ `XML_TAG_DIFF`, `XML_TAG_DIFF_ADDED`, `XML_TAG_DIFF_REMOVED`,
+ `XML_TAG_EXPRESSION`, `XML_TAG_FAILED`, `XML_TAG_FENCING_LEVEL`,
+ `XML_TAG_FENCING_TOPOLOGY`, `XML_TAG_FRAGMENT`, `XML_TAG_GRAPH`,
+ `XML_TAG_META_SETS`, `XML_TAG_OPTIONS`, `XML_TAG_OP_VER_ATTRS`,
+ `XML_TAG_OP_VER_META`, `XML_TAG_PARAM`, `XML_TAG_PARAMS`,
+ `XML_TAG_RESOURCE_REF`, `XML_TAG_RSC_VER_ATTRS`, `XML_TAG_RULE`,
+ `XML_TAG_TRANSIENT_NODEATTRS`, `XML_TAG_UTILIZATION`,
+ `XML_TICKET_ATTR_LOSS_POLICY`, and `XML_TICKET_ATTR_TICKET`
+* **libcrmcommon:** drop support for patchset format 1 in
+ `xml_apply_patchset()`, `xml_create_patchset()`, and `xml_patch_versions()`
+* **libcrmcommon:** drop functions/macros `add_message_xml()`,
+ `add_node_copy()`, `add_node_nocopy()`, `apply_xml_diff()`, `bz2_strerror()`,
+ `can_prune_leaf()`, `copy_in_properties()` `create_hello_message()`,
+ `create_reply()`, `create_reply_adv()`, `create_request()`,
+ `create_request_adv()`, `create_xml_node()`, `crm_action_str()`,
+ `crm_add_logfile()`, `crm_atoi()`, `crm_build_path()`, `crm_destroy_xml()`,
+ `crm_errno2exit()`, `crm_ftoa()`, `crm_get_interval()`,
+ `crm_hash_table_size()`, `crm_itoa()`, `crm_itoa_stack()`,
+ `crm_log_cli_init()`, `crm_map_element_name()`, `crm_next_same_xml()`,
+ `crm_parse_int()`, `crm_parse_interval_spec()`, `crm_parse_ll()`,
+ `crm_provider_required()`, `crm_signal()`, `crm_str()`,
+ `crm_strcase_equal()`, `crm_strcase_hash()`, `crm_strcase_table_new()`,
+ `crm_strip_trailing_newline()`, `crm_str_eq()`, `crm_str_hash()`,
+ `crm_str_table_dup()`, `crm_str_table_new()`, `crm_ttoa()`,
+ `crm_xml_add_boolean()`, `crm_xml_escape()`, `crm_xml_replace()`,
+ `diff_xml_object()`, `do_crm_log_always()`, `dump_xml_formatted()`,
+ `dump_xml_formatted_with_text()`, `dump_xml_unformatted()`,
+ `expand_plus_plus()`, `filename2xml()`, `find_entity()`,
+ `find_xml_children()`, `find_xml_node()`, `first_named_child()`,
+ `fix_plus_plus_recursive()`, `getDocPtr()`, `get_message_xml()`,
+ `get_schema_name()`, `get_schema_version()`, `get_xpath_object_relative()`,
+ `g_str_hash_traditional()`, `ID()`, `is_not_set()`, `is_set_any()`,
+ `log_data_element()`, `pcmk_action_text()`, `pcmk_create_html_node()`,
+ `pcmk_create_xml_text_node()`, `pcmk_format_named_time()`,
+ `pcmk_format_nvpair()`, `pcmk_hostname()`, `pcmk_log_xml_impl()`,
+ `pcmk_numeric_strcasecmp()`, `pcmk_on_fail_text()`, `pcmk_parse_action()`,
+ `pcmk_scan_nvpair()`, `purge_diff_markers()`, `replace_xml_child()`,
+ `safe_str_eq()`, `safe_str_neq()`, `score2char()`, `score2char_stack()`,
+ `stdin2xml()`, `string2xml()` `subtract_xml_object()`, `TYPE()`,
+ `update_validation()`, `update_xml_child()`, `validate_xml()`,
+ `validate_xml_verbose()`, `write_xml_fd()`, `write_xml_file()`,
+ `xml_get_path()`, `xml_has_children()`, `xml_latest_schema()`,
+ `xml_log_changes()`, `xml_log_patchset()`, `xml_remove_prop()`, `__likely()`,
+ and `__unlikely()`
+* **libcrmservice:** drop header `services_compat.h`
+* **libcrmservice:** deprecate function `services_ocf_exitcode_str()`
+* **libcrmservice:** drop enums (including all their values) `nagios_exitcode`
+ and `op_status`
+* **libcrmservice:** drop defined constants `F_LRMD_ALERT`, `F_LRMD_ALERT_ID`,
+ `F_LRMD_ALERT_PATH`, `F_LRMD_CALLBACK_TOKEN`, `F_LRMD_CALLDATA`,
+ `F_LRMD_CALLID`, `F_LRMD_CALLOPTS`, `F_LRMD_CLASS`, `F_LRMD_CLIENTID`,
+ `F_LRMD_CLIENTNAME`, `F_LRMD_EXEC_RC`, `F_LRMD_IPC_CLIENT`,
+ `F_LRMD_IPC_IPC_SERVER`, `F_LRMD_IPC_MSG`, `F_LRMD_IPC_MSG_FLAGS`,
+ `F_LRMD_IPC_MSG_ID`, `F_LRMD_IPC_OP`, `F_LRMD_IPC_SESSION`, `F_LRMD_IPC_USER`,
+ `F_LRMD_IS_IPC_PROVIDER`, `F_LRMD_OPERATION`, `F_LRMD_OP_STATUS`,
+ `F_LRMD_ORIGIN`, `F_LRMD_PROTOCOL_VERSION`, `F_LRMD_PROVIDER`, `F_LRMD_RC`,
+ `F_LRMD_REMOTE_MSG_ID`, `F_LRMD_REMOTE_MSG_TYPE`, `F_LRMD_RSC`,
+ `F_LRMD_RSC_ACTION`, `F_LRMD_RSC_DELETED`, `F_LRMD_RSC_EXEC_TIME`,
+ `F_LRMD_RSC_EXIT_REASON`, `F_LRMD_RSC_ID`, `F_LRMD_RSC_INTERVAL`,
+ `F_LRMD_RSC_OUTPUT`, `F_LRMD_RSC_QUEUE_TIME`, `F_LRMD_RSC_RCCHANGE_TIME`,
+ `F_LRMD_RSC_RUN_TIME`, `F_LRMD_RSC_START_DELAY`, `F_LRMD_RSC_USERDATA_STR`,
+ `F_LRMD_TIMEOUT`, `F_LRMD_TYPE`, `F_LRMD_WATCHDOG`, `T_LRMD`,
+ `T_LRMD_IPC_PROXY`, `T_LRMD_NOTIFY`, `T_LRMD_REPLY`, and `T_LRMD_RSC_OP`
+* **libcrmservice:** drop functions `services_action_create()`,
+ `services_get_ocf_exitcode()`, `services_list()`, and
+ `services_lrm_status_str()`
+* **liblrmd:** drop header `lrmd_compat.h`
+* **liblrmd:** redefine `lrmd_opt_drop_recurring` enum value
+* **liblrmd:** change type of `lrmd_event_data_t` members `t_rcchange` and
+ `t_run` to `time_t`
+* **liblrmd:** drop defined constants `ALT_REMOTE_KEY_LOCATION` and
+ `LRMD_MIN_PROTOCOL_VERSION`
+* **libpacemaker:** add high-level API equivalents of `stonith_admin` options
+ including `pcmk_request_fencing()`, `pcmk_fence_history()`,
+ `pcmk_fence_installed()`, `pcmk_fence_last()`, `pcmk_fence_list_targets()`,
+ `pcmk_fence_metadata()`, `pcmk_fence_registered()`,
+ `pcmk_fence_register_level()`, `pcmk_fence_unregister_level()`, and
+ `pcmk_fence_validate()`
+* **libpe_rules:** drop functions `find_expression_type()`,
+ `pe_evaluate_rules()`, `pe_eval_expr()`, `pe_eval_rules()`,
+ `pe_eval_subexpr()`, `pe_expand_re_matches()`, `pe_test_expression()`,
+ `pe_test_expression_full()`, `pe_test_expression_re()`, `pe_test_rule()`,
+ `pe_test_rule_full()`, `pe_test_rule_re()`, `test_expression()`,
+ `test_ruleset()`, and `unpack_instance_attributes()`
+* **libpe_status,libpe_rules:** drop types `action_t`, `action_wrapper_t`,
+ `no_quorum_policy_t`, `pe_action_t`, `pe_action_wrapper_t`, `pe_node_t`,
+ `pe_resource_t`, `pe_tag_t`, `pe_ticket_t`, `tag_t`, and `ticket_t`
+* **libpe_status,libpe_rules:** drop enums (including all their values)
+ `pe_check_parameters` and `pe_graph_flags`
+* **libpe_status,libpe_rules:** drop defined constants `pe_flag_check_config`,
+ `pe_flag_concurrent_fencing`, `pe_flag_enable_unfencing`,
+ `pe_flag_have_remote_nodes`, `pe_flag_have_status`,
+ `pe_flag_maintenance_mode`, `pe_flag_no_compat`, `pe_flag_no_counts`,
+ `pe_flag_quick_location`, `pe_flag_remove_after_stop`, `pe_flag_sanitized`,
+ `pe_flag_show_scores`, `pe_flag_show_utilization`, `pe_flag_shutdown_lock`,
+ `pe_flag_startup_fencing`, `pe_flag_startup_probes`,
+ `pe_flag_start_failure_fatal`, `pe_flag_stdout`, `pe_flag_stonith_enabled`,
+ `pe_flag_stop_action_orphans`, `pe_flag_stop_everything`,
+ `pe_flag_stop_rsc_orphans`, `pe_flag_symmetric_cluster`, `pe_rsc_allocating`,
+ `pe_rsc_allow_migrate`, `pe_rsc_allow_remote_remotes`, `pe_rsc_block`,
+ `pe_rsc_critical`, `pe_rsc_detect_loop`, `pe_rsc_failed`,
+ `pe_rsc_failure_ignored`, `pe_rsc_fence_device`, `pe_rsc_is_container`,
+ `pe_rsc_maintenance`, `pe_rsc_merging`, `pe_rsc_needs_fencing`,
+ `pe_rsc_needs_quorum`, `pe_rsc_needs_unfencing`, `pe_rsc_notify`,
+ `pe_rsc_orphan`, `pe_rsc_orphan_container_filler`, `pe_rsc_promotable`,
+ `pe_rsc_provisional`, `pe_rsc_reload`, `pe_rsc_replica_container`,
+ `pe_rsc_restarting`, `pe_rsc_runnable`, `pe_rsc_starting`,
+ `pe_rsc_start_pending`, `pe_rsc_stop`, `pe_rsc_stopping`,
+ `pe_rsc_stop_unexpected`, `pe_rsc_unique`, `RSC_ROLE_MASTER_S`,
+ `RSC_ROLE_MAX`, `RSC_ROLE_PROMOTED_LEGACY_S`, `RSC_ROLE_SLAVE_S`,
+ `RSC_ROLE_STARTED_S`, `RSC_ROLE_STOPPED_S`, `RSC_ROLE_UNKNOWN_S`,
+ `RSC_ROLE_UNPROMOTED_LEGACY_S`, and `RSC_ROLE_UNPROMOTED_S`
+* **libpe_status,libpe_rules:** drop functions `fail2text(),` `pe_pref()`,
+ `recovery2text()`, `role2text()`, `task2text()`, `text2role()`, and
+ `text2task()`
+* **libpe_status:** drop functions/macros `pe_rsc_is_anon_clone()`,
+ `pe_rsc_is_bundled()`, `pe_rsc_is_clone()`, `pe_rsc_is_unique_clone()`, and
+ `status_print()`
+* **libstonithd:** drop `stonith_t` member `call_timeout`
+* **libstonithd:** drop `stonith_event_t` members `message` and `type`
+* **libstonithd:** drop defined constants `T_STONITH_NOTIFY_DISCONNECT`,
+ `T_STONITH_NOTIFY_FENCE`, `T_STONITH_NOTIFY_HISTORY`, and
+ `T_STONITH_NOTIFY_HISTORY_SYNCED`
+* **libstonithd:** drop function `get_stonith_provider()`
+
# Pacemaker-2.1.9 (31 Oct 2024)
- 169 commits with 252 files changed, 4498 insertions(+), 2259 deletions(-)
## Features added since Pacemaker-2.1.8
+ **build:** support building with libxml2 2.13.0 or newer
+ **CIB:** new no-quorum-policy value "fence" replaces now-deprecated "suicide"
+ **tools:** iso8601 supports standard `--output-as/--output-to` arguments
## Fixes since Pacemaker-2.1.8
+ **tools:** restore crmadmin default timeout to 30 seconds instead of none
*(regression introduced in 2.1.5)*
+ **tools:** `crm_resource` did not return error if schema upgrade failed
*(regression introduced in 2.1.8)*
+ **CIB:** detect newly created alerts section *(regression introduced in 2.1.7)*
+ **CIB:** treat empty environment variables (`CIB_file` etc.) same as unset
+ **CIB:** remote CIB administration now cannot block server
+ **executor:** don't block during TLS handshakes
+ **executor:** discard any agent output after about 10MiB
+ **scheduler:** avoid memory leak when checking for unfencing-capable devices
+ **libcrmcommon:** check for overflow when parsing and manipulating date/times
+ **tools:** properly handle resources removed from configuration when
displaying node history in `crm_mon`
+ **tools:** crmadmin -D/--dc_lookup no longer hangs when there is no DC
+ **tools:** don't assert if stdout or stderr is closed by caller
## Public API changes since Pacemaker-2.1.8
+ **libcrmcommon:** add enum `pcmk_ra_caps` value `pcmk_ra_cap_cli_exec`
+ **libcrmcommon:** add `pcmk_cib_node_shutdown()`
+ **libcrmcommon:** add `pcmk_parse_score()`
+ **libcrmcommon:** deprecate `CRM_ASSERT()`
+ **libcrmcommon:** deprecate `PCMK_VALUE_FENCE_LEGACY` defined constant
+ **libstonithd:** add enum `stonith_call_options` value `st_opt_allow_self_fencing`
+ **libstonithd:** deprecate enum `stonith_call_options` value `st_opt_allow_suicide`
+ **libstonithd:** deprecate enum `stonith_call_options` value `st_opt_scope_local`
+ **libstonithd:** deprecate enum `stonith_call_options` value `st_opt_verbose`
# Pacemaker-2.1.8 (08 Aug 2024)
- 2559 commits with 511 files changed, 46898 insertions(+), 23417 deletions(-)
## Features added since Pacemaker-2.1.7
+ **local options:** support `PCMK_panic_action="off"` or "sync-off"
+ **local options:** deprecate `PCMK_dh_min_bits` environment variable
+ **CIB:** deprecate omitting validate-with from the CIB or setting it to "none"
or an unknown schema
+ **CIB:** deprecate "default" and "#default" as explicit meta-attribute values
+ **CIB:** deprecate resource-discovery-enabled node attribute
+ **CIB:** deprecate support for multiple top-level rules within a location
constraint (a single rule may still contain multiple sub-rules)
+ **CIB:** deprecate support for node attribute expressions in rules beneath op,
`op_defaults`, or fence device meta-attributes
+ **CIB:** deprecate support for rkt in bundles
+ **CIB:** drop support for (nonworking) rules based on the #role node attribute
(role-based location constraints may still contain rules)
+ **CIB manager,controller,fencer,scheduler:** deprecate "metadata" command-line
option (instead, use `crm_attribute` `--list-options` mentioned below)
+ **pacemaker-remoted:** newer schema files are now downloaded from the cluster,
allowing more command-line tools to work when the Pacemaker Remote node
has an older Pacemaker version than the cluster
+ **agents:** deprecate the ocf:pacemaker:o2cb resource agent
+ **tools:** deprecate `--text-fancy` command-line option in all tools
+ **tools:** `crm_attribute` `--list-options` lists all possible cluster options
+ **tools:** `crm_resource` `--list-options` lists all possible primitive
meta-attributes or special fence device parameters
+ **tools:** `crm_verify` now reports invalid fence topology levels
+ **tools:** new `--score` option for cibadmin `--modify` and `crm_attribute` --update
enables expansion of "++" and "+=" in attribute values without a warning
(using such expansions without `--score` is now deprecated)
+ **tools:** `crm_ticket` supports standard `--output-as/--output-to` arguments
## Fixes since Pacemaker-2.1.7
+ **tools:** restore the (deprecated) ability to automatically correct malformed
XML passed via standard input *(regression introduced in 2.1.7)*
+ **tools:** `crm_verify` distinguishes configuration warnings and errors
*(regression introduced in 2.1.7)*
+ **tools:** `crm_node` -i must initialize nodeid before passing pointer
*(regression introduced in 2.1.7)*
+ **CIB manager:** avoid memory leak from asynchronous client requests
*(regression introduced in 2.1.7)*
+ **scheduler:** don't apply colocations twice for promotion priority
*(regression introduced in 2.1.7)*
+ **CIB:** restore the (deprecated) ability to use validate-with="pacemaker-next"
*(regression introduced in 2.1.6)*
+ **controller:** avoid zombie children when asynchronous actions exit while a
synchronous meta-data action is in progress
*(regression introduced in 2.1.5)*
+ **libcrmcommon:** avoid file descriptor leak in asynchronous IPC clients
*(regression introduced in 2.1.3)*
+ **scheduler:** avoid crash when logging an invalid utilization attribute value
*(regression introduced in 2.1.3)*
+ **tools:** `crm_mon` no longer crashes on some platforms when the fencer
connection is lost *(regression introduced in 2.1.0)*
+ **attribute manager:** write Pacemaker Remote node attributes even if node is
not cached
+ **attribute manager:** avoid use-after-free when remote node in cluster node
cache
+ **attribute manager:** correctly propagate utilization attributes to peers
to avoid the possibility of later being written out as regular node
attributes
+ **fencer:** correctly parse action-specific timeouts with units other than
seconds
+ **fencer:** avoid unnecessary timeouts when the watchdog timeout is greater
than a query timeout, per-device fencing timeout, or stonith-timeout
+ **libcrmcommon:** avoid possible buffer overflows when parsing and formatting
date/times
+ **libcrmcommon:** don't assume next schema will validate when not transforming
+ **libcrmcommon:** when displaying XML, don't show "<null>" for empty attribute
values, and properly escape special characters
+ **libstonithd:** avoid double free when invalid replies are received
+ **scheduler:** if the user specifies a timeout of 0, use the default 20s as
documented
+ **scheduler:** consider group's location constraints when a member is an
explicit dependent in a colocation constraint
+ **scheduler:** sort promotable cloned group instances properly for promotion
+ **agents:** ocf:pacemaker:SysInfo respects `attrd_updater` dampening
+ **agents:** ocf:pacemaker:HealthSMART properly handles SMART data missing
temperature
+ **tools:** cibadmin, `crm_attribute`, `crm_node`, `crm_resource`, `crm_shadow`, and
`crm_ticket` now retry CIB connections after transient failures
+ **tools:** cibadmin `--replace` now leaves "++" and "+=" unexpanded in XML
attribute values rather than wrongly treat them as 0
+ **tools:** cibsecret avoids possible truncation issue in process listing
+ **tools:** `crm_attribute` `--node` localhost or `--node` auto works
+ **tools:** `crm_resource` ignores resource meta-attribute node expressions
for consistency with how the cluster works
+ **tools:** `crm_resource` honors rules when getting utilization attributes
+ **tools:** `crm_verify` `--output-as=xml` includes detailed messages
+ **tools:** `crm_mon` exits upon loss of an attached pseudo-terminal to avoid
possibility of 100% CPU usage (seen when run via sudo with `use_pty`
configured)
## Public API changes since Pacemaker-2.1.7
+ **libcib:** add `cib_score_update` `cib_call_options` value
+ **libcib:** deprecate functions `cib_get_generation()`, `cib_metadata()`,
`cib_pref()`, `query_node_uname()`, and `set_standby()`
+ **libcib:** deprecate `T_CIB_DIFF_NOTIFY`
+ **libcib:** deprecate `<failed>` element in CIB create reply
+ **libcrmcluster:** add enum `pcmk_cluster_layer`
+ **libcrmcluster:** add functions `pcmk_cluster_connect()`,
`pcmk_cluster_disconnect()`, `pcmk_cluster_layer_text()`,
`pcmk_cluster_set_destroy_fn()`, `pcmk_cpg_set_confchg_fn()`,
`pcmk_cpg_set_deliver_fn()`, and `pcmk_get_cluster_layer()`
+ **libcrmcluster:** add type `pcmk_cluster_t`
+ **libcrmcluster:** deprecate functions `cluster_connect_cpg()`,
`cluster_disconnect_cpg()`, `crm_active_peers()`, `crm_cluster_connect()`,
`crm_cluster_disconnect()`, `crm_get_peer()`, `crm_get_peer_full()`,
`crm_is_corosync_peer_active()`, `crm_is_peer_active()`, `crm_join_phase_str()`,
`crm_peer_destroy()`, `crm_peer_init()`, `crm_peer_uname()`,
`crm_peer_uuid()`, `crm_remote_node_cache_size()`,
`crm_remote_peer_cache_refresh()`, `crm_remote_peer_cache_remove()`,
`crm_remote_peer_get()`, `crm_set_autoreap()`, `crm_set_status_callback()`,
`get_cluster_type()`, `get_local_nodeid()`, `get_local_node_name()`,
`get_node_name()`, `is_corosync_cluster()`, `name_for_cluster_type()`,
`pcmk_cpg_membership()`, `pcmk_message_common_cs()`, `reap_crm_member()`,
`send_cluster_message()`, `send_cluster_text()`, and `text2msg_type()`
+ **libcrmcluster:** deprecate enums `crm_ais_msg_types`, `crm_status_type`,
`cluster_type_e`, `crm_ais_msg_class`, `crm_get_peer_flags`, `crm_join_phase`,
and `crm_node_flags`, including all their values
+ **libcrmcluster:** deprecate global variables `crm_have_quorum`, `crm_peer_cache`,
`crm_peer_seq`, and `crm_remote_peer_cache`
+ **libcrmcluster:** deprecate `crm_cluster_t` and struct `crm_cluster_s`, including
all its members
+ **libcrmcluster:** deprecate `crm_node_t` and struct `crm_peer_node_s`, including
all its members
+ **libcrmcluster:** deprecate constants `CRM_NODE_LOST` and `CRM_NODE_MEMBER`
+ **libcrmcommon:** add constants `PCMK_ACTION_METADATA`, `PCMK_META_ALLOW_MIGRATE`,
`PCMK_META_ALLOW_UNHEALTHY_NODES`, `PCMK_META_CONTAINER_ATTRIBUTE_TARGET`,
`PCMK_META_CRITICAL`, `PCMK_META_GLOBALLY_UNIQUE`, `PCMK_META_INTERLEAVE`,
`PCMK_META_INTERVAL`, `PCMK_META_INTERVAL_ORIGIN`, `PCMK_META_IS_MANAGED`,
`PCMK_META_MAINTENANCE`, `PCMK_META_MULTIPLE_ACTIVE`, `PCMK_META_NOTIFY`,
`PCMK_META_ON_FAIL`, `PCMK_META_ORDERED`, `PCMK_META_PRIORITY`,
`PCMK_META_PROMOTABLE`, `PCMK_META_RECORD_PENDING`, `PCMK_META_REMOTE_ADDR`,
`PCMK_META_REMOTE_ALLOW_MIGRATE`, `PCMK_META_REMOTE_CONNECT_TIMEOUT`,
`PCMK_META_REMOTE_NODE`, `PCMK_META_REMOTE_PORT`, `PCMK_META_REQUIRES`,
`PCMK_META_RESOURCE_STICKINESS`, `PCMK_META_START_DELAY`,
`PCMK_META_TARGET_ROLE`, `PCMK_META_TIMEOUT`, `PCMK_META_TIMESTAMP_FORMAT`,
`PCMK_NODE_ATTR_MAINTENANCE`, `PCMK_NODE_ATTR_STANDBY`, `PCMK_OPT_BATCH_LIMIT`,
`PCMK_OPT_CLUSTER_DELAY`, `PCMK_OPT_CLUSTER_INFRASTRUCTURE`,
`PCMK_OPT_CLUSTER_IPC_LIMIT`, `PCMK_OPT_CLUSTER_NAME`,
`PCMK_OPT_CLUSTER_RECHECK_INTERVAL`, `PCMK_OPT_CONCURRENT_FENCING`,
`PCMK_OPT_DC_DEADTIME`, `PCMK_OPT_DC_VERSION`, `PCMK_OPT_ELECTION_TIMEOUT`,
`PCMK_OPT_ENABLE_ACL`, `PCMK_OPT_ENABLE_STARTUP_PROBES`,
`PCMK_OPT_FENCE_REACTION`, `PCMK_OPT_HAVE_WATCHDOG`,
`PCMK_OPT_JOIN_FINALIZATION_TIMEOUT`, `PCMK_OPT_JOIN_INTEGRATION_TIMEOUT`,
`PCMK_OPT_LOAD_THRESHOLD`, `PCMK_OPT_MAINTENANCE_MODE`,
`PCMK_OPT_MIGRATION_LIMIT`, `PCMK_OPT_NODE_ACTION_LIMIT`,
`PCMK_OPT_NODE_HEALTH_BASE`, `PCMK_OPT_NODE_HEALTH_GREEN`,
`PCMK_OPT_NODE_HEALTH_RED`, `PCMK_OPT_NODE_HEALTH_STRATEGY`,
`PCMK_OPT_NODE_HEALTH_YELLOW`, `PCMK_OPT_NODE_PENDING_TIMEOUT`,
`PCMK_OPT_NO_QUORUM_POLICY`, `PCMK_OPT_PE_ERROR_SERIES_MAX`,
`PCMK_OPT_PE_INPUT_SERIES_MAX`, `PCMK_OPT_PE_WARN_SERIES_MAX`,
`PCMK_OPT_PLACEMENT_STRATEGY`, `PCMK_OPT_PRIORITY_FENCING_DELAY`,
`PCMK_OPT_SHUTDOWN_ESCALATION`, `PCMK_OPT_SHUTDOWN_LOCK`,
`PCMK_OPT_SHUTDOWN_LOCK_LIMIT`, `PCMK_OPT_STARTUP_FENCING`,
`PCMK_OPT_START_FAILURE_IS_FATAL`, `PCMK_OPT_STONITH_ACTION`,
`PCMK_OPT_STONITH_ENABLED`, `PCMK_OPT_STONITH_MAX_ATTEMPTS`,
`PCMK_OPT_STONITH_TIMEOUT`, `PCMK_OPT_STONITH_WATCHDOG_TIMEOUT`,
`PCMK_OPT_STOP_ALL_RESOURCES`, `PCMK_OPT_STOP_ORPHAN_ACTIONS`,
`PCMK_OPT_STOP_ORPHAN_RESOURCES`, `PCMK_OPT_SYMMETRIC_CLUSTER`,
`PCMK_OPT_TRANSITION_DELAY`, `PCMK_REMOTE_RA_ADDR`, `PCMK_REMOTE_RA_PORT`,
`PCMK_REMOTE_RA_RECONNECT_INTERVAL`, `PCMK_REMOTE_RA_SERVER`,
`PCMK_ROLE_PROMOTED`, `PCMK_ROLE_STARTED`, `PCMK_ROLE_STOPPED`,
`PCMK_ROLE_UNPROMOTED`, `PCMK_SCORE_INFINITY`, `PCMK_VALUE_ALWAYS`,
`PCMK_VALUE_AND`, `PCMK_VALUE_BALANCED`, `PCMK_VALUE_BLOCK`, `PCMK_VALUE_BOOLEAN`,
`PCMK_VALUE_CIB_BOOTSTRAP_OPTIONS`, `PCMK_VALUE_COROSYNC`, `PCMK_VALUE_CREATE`,
`PCMK_VALUE_CUSTOM`, `PCMK_VALUE_DATE_SPEC`, `PCMK_VALUE_DEFAULT`,
`PCMK_VALUE_DEFINED`, `PCMK_VALUE_DELETE`, `PCMK_VALUE_DEMOTE`, `PCMK_VALUE_DENY`,
`PCMK_VALUE_DURATION`, `PCMK_VALUE_DYNAMIC_LIST`, `PCMK_VALUE_EQ`,
`PCMK_VALUE_EXCLUSIVE`, `PCMK_VALUE_FAILED`, `PCMK_VALUE_FALSE`,
`PCMK_VALUE_FENCE`, `PCMK_VALUE_FENCE_LEGACY`, `PCMK_VALUE_FENCING`,
`PCMK_VALUE_FREEZE`, `PCMK_VALUE_GRANTED`, `PCMK_VALUE_GREEN`, `PCMK_VALUE_GT`,
`PCMK_VALUE_GTE`, `PCMK_VALUE_HOST`, `PCMK_VALUE_IGNORE`, `PCMK_VALUE_IN_RANGE`,
`PCMK_VALUE_INFINITY`, `PCMK_VALUE_INTEGER`, `PCMK_VALUE_LITERAL`,
`PCMK_VALUE_LT`, `PCMK_VALUE_LTE`, `PCMK_VALUE_MANDATORY`, `PCMK_VALUE_MEMBER`,
`PCMK_VALUE_META`, `PCMK_VALUE_MIGRATE_ON_RED`, `PCMK_VALUE_MINIMAL`,
`PCMK_VALUE_MINUS_INFINITY`, `PCMK_VALUE_MODIFY`, `PCMK_VALUE_MOVE`,
`PCMK_VALUE_NE`, `PCMK_VALUE_NEVER`, `PCMK_VALUE_NONE`,
`PCMK_VALUE_NONNEGATIVE_INTEGER`, `PCMK_VALUE_NOTHING`,
`PCMK_VALUE_NOT_DEFINED`, `PCMK_VALUE_NUMBER`, `PCMK_VALUE_OFFLINE`,
`PCMK_VALUE_ONLINE`, `PCMK_VALUE_ONLY_GREEN`, `PCMK_VALUE_OPTIONAL`,
`PCMK_VALUE_OR`, `PCMK_VALUE_PANIC`, `PCMK_VALUE_PARAM`, `PCMK_VALUE_PENDING`,
`PCMK_VALUE_PERCENTAGE`, `PCMK_VALUE_PLUS_INFINITY`, `PCMK_VALUE_PORT`,
`PCMK_VALUE_PROGRESSIVE`, `PCMK_VALUE_QUORUM`, `PCMK_VALUE_READ`,
`PCMK_VALUE_RED`, `PCMK_VALUE_REMOTE`, `PCMK_VALUE_RESTART`,
`PCMK_VALUE_RESTART_CONTAINER`, `PCMK_VALUE_REVOKED`, `PCMK_VALUE_SCORE`,
`PCMK_VALUE_SELECT`, `PCMK_VALUE_SERIALIZE`, `PCMK_VALUE_STANDBY`,
`PCMK_VALUE_STATIC_LIST`, `PCMK_VALUE_STATUS`, `PCMK_VALUE_STOP`,
`PCMK_VALUE_STOP_ONLY`, `PCMK_VALUE_STOP_START`, `PCMK_VALUE_STOP_UNEXPECTED`,
`PCMK_VALUE_STRING`, `PCMK_VALUE_SUCCESS`, `PCMK_VALUE_TIMEOUT`,
`PCMK_VALUE_TRUE`, `PCMK_VALUE_UNFENCING`, `PCMK_VALUE_UNKNOWN`,
`PCMK_VALUE_UTILIZATION`, `PCMK_VALUE_VERSION`, `PCMK_VALUE_WRITE`,
`PCMK_VALUE_YELLOW`, `PCMK_XA_ACTION`, `PCMK_XA_ACTIVE`, `PCMK_XA_ADD_HOST`,
`PCMK_XA_ADMIN_EPOCH`, `PCMK_XA_ADVANCED`, `PCMK_XA_AGENT`, `PCMK_XA_API_VERSION`,
`PCMK_XA_ATTRIBUTE`, `PCMK_XA_AUTHOR`, `PCMK_XA_AUTOMATIC`, `PCMK_XA_BLOCKED`,
`PCMK_XA_BOOLEAN_OP`, `PCMK_XA_BUILD`, `PCMK_XA_CACHED`, `PCMK_XA_CALL`,
`PCMK_XA_CIB_LAST_WRITTEN`, `PCMK_XA_CIB_NODE`, `PCMK_XA_CLASS`, `PCMK_XA_CLIENT`,
`PCMK_XA_CODE`, `PCMK_XA_COMMENT`, `PCMK_XA_COMPLETED`, `PCMK_XA_CONTROL_PORT`,
`PCMK_XA_COUNT`, `PCMK_XA_CRMD`, `PCMK_XA_CRM_DEBUG_ORIGIN`,
`PCMK_XA_CRM_FEATURE_SET`, `PCMK_XA_CRM_TIMESTAMP`, `PCMK_XA_DAYS`,
`PCMK_XA_DC_UUID`, `PCMK_XA_DEFAULT`, `PCMK_XA_DELEGATE`, `PCMK_XA_DESCRIPTION`,
`PCMK_XA_DEST`, `PCMK_XA_DEVICE`, `PCMK_XA_DEVICES`, `PCMK_XA_DISABLED`,
`PCMK_XA_DURATION`, `PCMK_XA_END`, `PCMK_XA_EPOCH`, `PCMK_XA_EXEC`,
`PCMK_XA_EXECUTION_CODE`, `PCMK_XA_EXECUTION_DATE`, `PCMK_XA_EXECUTION_MESSAGE`,
`PCMK_XA_EXEC_TIME`, `PCMK_XA_EXITCODE`, `PCMK_XA_EXITREASON`,
`PCMK_XA_EXITSTATUS`, `PCMK_XA_EXIT_REASON`, `PCMK_XA_EXPECTED`,
`PCMK_XA_EXPECTED_UP`, `PCMK_XA_EXTENDED_STATUS`, `PCMK_XA_FAILED`,
`PCMK_XA_FAILURE_IGNORED`, `PCMK_XA_FAIL_COUNT`, `PCMK_XA_FEATURES`,
`PCMK_XA_FEATURE_SET`, `PCMK_XA_FILE`, `PCMK_XA_FIRST`, `PCMK_XA_FIRST_ACTION`,
`PCMK_XA_FOR`, `PCMK_XA_FUNCTION`, `PCMK_XA_GENERATED`, `PCMK_XA_HASH`,
`PCMK_XA_HAVE_QUORUM`, `PCMK_XA_HEALTH`, `PCMK_XA_HOST`, `PCMK_XA_HOST_INTERFACE`,
`PCMK_XA_HOST_NETMASK`, `PCMK_XA_HOURS`, `PCMK_XA_ID`, `PCMK_XA_ID_AS_RESOURCE`,
`PCMK_XA_ID_REF`, `PCMK_XA_IMAGE`, `PCMK_XA_INDEX`, `PCMK_XA_INFLUENCE`,
`PCMK_XA_INSTANCE`, `PCMK_XA_INTERNAL_PORT`, `PCMK_XA_INTERVAL`,
`PCMK_XA_IP_RANGE_START`, `PCMK_XA_IS_DC`, `PCMK_XA_KIND`, `PCMK_XA_LANG`,
`PCMK_XA_LAST_FAILURE`, `PCMK_XA_LAST_GRANTED`, `PCMK_XA_LAST_RC_CHANGE`,
`PCMK_XA_LAST_UPDATED`, `PCMK_XA_LOCKED_TO`, `PCMK_XA_LOCKED_TO_HYPHEN`,
`PCMK_XA_LOSS_POLICY`, `PCMK_XA_MAINTENANCE`, `PCMK_XA_MAINTENANCE_MODE`,
`PCMK_XA_MANAGED`, `PCMK_XA_MESSAGE`, `PCMK_XA_MINUTES`, `PCMK_XA_MIXED_VERSION`,
`PCMK_XA_MONTHDAYS`, `PCMK_XA_MONTHS`, `PCMK_XA_MULTI_STATE`, `PCMK_XA_NAME`,
`PCMK_XA_NETWORK`, `PCMK_XA_NEXT_ROLE`, `PCMK_XA_NODE`, `PCMK_XA_NODEID`,
`PCMK_XA_NODES_RUNNING_ON`, `PCMK_XA_NODE_ATTRIBUTE`, `PCMK_XA_NODE_NAME`,
`PCMK_XA_NODE_PATH`, `PCMK_XA_NO_QUORUM_PANIC`, `PCMK_XA_NO_QUORUM_POLICY`,
`PCMK_XA_NUMBER`, `PCMK_XA_NUMBER_RESOURCES`, `PCMK_XA_NUM_UPDATES`,
`PCMK_XA_OBJECT_TYPE`, `PCMK_XA_ONLINE`, `PCMK_XA_ON_TARGET`, `PCMK_XA_OP`,
`PCMK_XA_OPERATION`, `PCMK_XA_OPTIONS`, `PCMK_XA_OP_KEY`, `PCMK_XA_ORIGIN`,
`PCMK_XA_ORPHAN`, `PCMK_XA_ORPHANED`, `PCMK_XA_PACEMAKERD_STATE`, `PCMK_XA_PATH`,
`PCMK_XA_PENDING`, `PCMK_XA_PORT`, `PCMK_XA_PRESENT`,
`PCMK_XA_PRIORITY_FENCING_DELAY_MS`, `PCMK_XA_PROGRAM`, `PCMK_XA_PROMOTABLE`,
`PCMK_XA_PROMOTED_MAX`, `PCMK_XA_PROMOTED_ONLY`, `PCMK_XA_PROVIDER`,
`PCMK_XA_QUEUED`, `PCMK_XA_QUEUE_TIME`, `PCMK_XA_QUORUM`, `PCMK_XA_RANGE`,
`PCMK_XA_RC`, `PCMK_XA_RC_TEXT`, `PCMK_XA_REASON`, `PCMK_XA_REFERENCE`,
`PCMK_XA_RELOADABLE`, `PCMK_XA_REMAIN_STOPPED`, `PCMK_XA_REMOTE_CLEAR_PORT`,
`PCMK_XA_REMOTE_NODE`, `PCMK_XA_REMOTE_TLS_PORT`, `PCMK_XA_REPLICAS`,
`PCMK_XA_REPLICAS_PER_HOST`, `PCMK_XA_REQUEST`, `PCMK_XA_REQUIRE_ALL`,
`PCMK_XA_RESOURCE`, `PCMK_XA_RESOURCES_RUNNING`, `PCMK_XA_RESOURCE_AGENT`,
`PCMK_XA_RESOURCE_DISCOVERY`, `PCMK_XA_RESULT`, `PCMK_XA_ROLE`, `PCMK_XA_RSC`,
`PCMK_XA_RSC_PATTERN`, `PCMK_XA_RSC_ROLE`, `PCMK_XA_RULE_ID`, `PCMK_XA_RUNNING`,
`PCMK_XA_RUNNING_ON`, `PCMK_XA_RUN_COMMAND`, `PCMK_XA_SCOPE`, `PCMK_XA_SCORE`,
`PCMK_XA_SCORE_ATTRIBUTE`, `PCMK_XA_SECONDS`, `PCMK_XA_SEQUENTIAL`,
`PCMK_XA_SHUTDOWN`, `PCMK_XA_SOURCE`, `PCMK_XA_SOURCE_DIR`,
`PCMK_XA_SOURCE_DIR_ROOT`, `PCMK_XA_SPEC`, `PCMK_XA_STANDARD`, `PCMK_XA_STANDBY`,
`PCMK_XA_STANDBY_ONFAIL`, `PCMK_XA_START`, `PCMK_XA_STATE`, `PCMK_XA_STATUS`,
`PCMK_XA_STONITH_ENABLED`, `PCMK_XA_STONITH_TIMEOUT_MS`,
`PCMK_XA_STOP_ALL_RESOURCES`, `PCMK_XA_SYMMETRICAL`,
`PCMK_XA_SYMMETRIC_CLUSTER`, `PCMK_XA_SYS_FROM`, `PCMK_XA_TAG`, `PCMK_XA_TARGET`,
`PCMK_XA_TARGET_ATTRIBUTE`, `PCMK_XA_TARGET_DIR`, `PCMK_XA_TARGET_PATTERN`,
`PCMK_XA_TARGET_ROLE`, `PCMK_XA_TARGET_VALUE`, `PCMK_XA_TASK`, `PCMK_XA_TEMPLATE`,
`PCMK_XA_THEN`, `PCMK_XA_THEN_ACTION`, `PCMK_XA_TICKET`, `PCMK_XA_TIME`,
`PCMK_XA_TYPE`, `PCMK_XA_UNAME`, `PCMK_XA_UNCLEAN`, `PCMK_XA_UNHEALTHY`,
`PCMK_XA_UNIQUE`, `PCMK_XA_UNMANAGED`, `PCMK_XA_UPDATE_CLIENT`,
`PCMK_XA_UPDATE_ORIGIN`, `PCMK_XA_UPDATE_USER`, `PCMK_XA_USER`, `PCMK_XA_VALID`,
`PCMK_XA_VALIDATE_WITH`, `PCMK_XA_VALUE`, `PCMK_XA_VALUE_SOURCE`,
`PCMK_XA_VERSION`, `PCMK_XA_WATCHDOG`, `PCMK_XA_WEEKDAYS`, `PCMK_XA_WEEKS`,
`PCMK_XA_WEEKYEARS`, `PCMK_XA_WEIGHT`, `PCMK_XA_WHEN`, `PCMK_XA_WITH_QUORUM`,
`PCMK_XA_WITH_RSC`, `PCMK_XA_WITH_RSC_ROLE`, `PCMK_XA_XPATH`, `PCMK_XA_YEARDAYS`,
`PCMK_XA_YEARS`, `PCMK_XE_ACLS`, `PCMK_XE_ACL_GROUP`, `PCMK_XE_ACL_PERMISSION`,
`PCMK_XE_ACL_ROLE`, `PCMK_XE_ACL_TARGET`, `PCMK_XE_ACTION`, `PCMK_XE_ACTIONS`,
`PCMK_XE_AGENT`, `PCMK_XE_AGENTS`, `PCMK_XE_AGENT_STATUS`, `PCMK_XE_ALERT`,
`PCMK_XE_ALERTS`, `PCMK_XE_ALLOCATIONS`, `PCMK_XE_ALLOCATIONS_UTILIZATIONS`,
`PCMK_XE_ATTRIBUTE`, `PCMK_XE_BAN`, `PCMK_XE_BANS`, `PCMK_XE_BUNDLE`,
`PCMK_XE_CAPACITY`, `PCMK_XE_CHANGE`, `PCMK_XE_CHANGE_ATTR`,
`PCMK_XE_CHANGE_LIST`, `PCMK_XE_CHANGE_RESULT`, `PCMK_XE_CHECK`, `PCMK_XE_CIB`,
`PCMK_XE_CLONE`, `PCMK_XE_CLUSTER_ACTION`, `PCMK_XE_CLUSTER_INFO`,
`PCMK_XE_CLUSTER_OPTIONS`, `PCMK_XE_CLUSTER_PROPERTY_SET`,
`PCMK_XE_CLUSTER_STATUS`, `PCMK_XE_COMMAND`, `PCMK_XE_CONFIGURATION`,
`PCMK_XE_CONSTRAINTS`, `PCMK_XE_CONTENT`, `PCMK_XE_CRM_CONFIG`, `PCMK_XE_CRM_MON`,
`PCMK_XE_CRM_MON_DISCONNECTED`, `PCMK_XE_CURRENT_DC`, `PCMK_XE_DATE_SPEC`,
`PCMK_XE_DC`, `PCMK_XE_DEPRECATED`, `PCMK_XE_DIFF`, `PCMK_XE_DIGEST`,
`PCMK_XE_DIGESTS`, `PCMK_XE_DOCKER`, `PCMK_XE_DURATION`, `PCMK_XE_ERROR`,
`PCMK_XE_ERRORS`, `PCMK_XE_EXPRESSION`, `PCMK_XE_FAILURE`, `PCMK_XE_FAILURES`,
`PCMK_XE_FEATURE`, `PCMK_XE_FEATURES`, `PCMK_XE_FENCE_EVENT`,
`PCMK_XE_FENCE_HISTORY`, `PCMK_XE_FENCING_ACTION`, `PCMK_XE_FENCING_LEVEL`,
`PCMK_XE_FENCING_TOPOLOGY`, `PCMK_XE_GROUP`, `PCMK_XE_INJECT_ATTR`,
`PCMK_XE_INJECT_SPEC`, `PCMK_XE_INSTANCE_ATTRIBUTES`, `PCMK_XE_INSTRUCTION`,
`PCMK_XE_ITEM`, `PCMK_XE_LAST_CHANGE`, `PCMK_XE_LAST_FENCED`,
`PCMK_XE_LAST_UPDATE`, `PCMK_XE_LIST`, `PCMK_XE_LONGDESC`, `PCMK_XE_METADATA`,
`PCMK_XE_META_ATTRIBUTES`, `PCMK_XE_MODIFICATIONS`, `PCMK_XE_MODIFY_NODE`,
`PCMK_XE_MODIFY_TICKET`, `PCMK_XE_NETWORK`, `PCMK_XE_NODE`, `PCMK_XE_NODES`,
`PCMK_XE_NODES_CONFIGURED`, `PCMK_XE_NODE_ACTION`, `PCMK_XE_NODE_ATTRIBUTES`,
`PCMK_XE_NODE_HISTORY`, `PCMK_XE_NODE_INFO`, `PCMK_XE_NODE_WEIGHT`,
`PCMK_XE_NVPAIR`, `PCMK_XE_OBJ_REF`, `PCMK_XE_OP`, `PCMK_XE_OPERATION`,
`PCMK_XE_OPERATIONS`, `PCMK_XE_OPERATION_HISTORY`, `PCMK_XE_OPTION`,
`PCMK_XE_OP_DEFAULTS`, `PCMK_XE_OUTPUT`, `PCMK_XE_OVERRIDE`, `PCMK_XE_OVERRIDES`,
`PCMK_XE_PACEMAKERD`, `PCMK_XE_PACEMAKER_RESULT`, `PCMK_XE_PARAMETER`,
`PCMK_XE_PARAMETERS`, `PCMK_XE_PODMAN`, `PCMK_XE_PORT_MAPPING`,
`PCMK_XE_POSITION`, `PCMK_XE_PRIMITIVE`, `PCMK_XE_PROMOTION_SCORE`,
`PCMK_XE_PROVIDER`, `PCMK_XE_PROVIDERS`, `PCMK_XE_PSEUDO_ACTION`,
`PCMK_XE_REASON`, `PCMK_XE_RECIPIENT`, `PCMK_XE_REPLICA`, `PCMK_XE_RESOURCE`,
`PCMK_XE_RESOURCES`, `PCMK_XE_RESOURCES_CONFIGURED`, `PCMK_XE_RESOURCE_AGENT`,
`PCMK_XE_RESOURCE_AGENT_ACTION`, `PCMK_XE_RESOURCE_CONFIG`,
`PCMK_XE_RESOURCE_HISTORY`, `PCMK_XE_RESOURCE_REF`, `PCMK_XE_RESOURCE_SET`,
`PCMK_XE_RESULT_CODE`, `PCMK_XE_REVISED_CLUSTER_STATUS`,
`PCMK_XE_ROLE`, `PCMK_XE_RSC_ACTION`, `PCMK_XE_RSC_COLOCATION`,
`PCMK_XE_RSC_DEFAULTS`, `PCMK_XE_RSC_LOCATION`, `PCMK_XE_RSC_ORDER`,
`PCMK_XE_RSC_TICKET`, `PCMK_XE_RULE`, `PCMK_XE_RULE_CHECK`, `PCMK_XE_SELECT`,
`PCMK_XE_SELECT_ATTRIBUTES`, `PCMK_XE_SELECT_FENCING`, `PCMK_XE_SELECT_NODES`,
`PCMK_XE_SELECT_RESOURCES`, `PCMK_XE_SHADOW`, `PCMK_XE_SHORTDESC`,
`PCMK_XE_SOURCE`, `PCMK_XE_SPECIAL`, `PCMK_XE_STACK`, `PCMK_XE_STATUS`,
`PCMK_XE_STORAGE`, `PCMK_XE_STORAGE_MAPPING`, `PCMK_XE_SUMMARY`, `PCMK_XE_TAG`,
`PCMK_XE_TAGS`, `PCMK_XE_TARGET`, `PCMK_XE_TEMPLATE`, `PCMK_XE_TICKET`,
`PCMK_XE_TICKETS`, `PCMK_XE_TIMING`, `PCMK_XE_TIMINGS`, `PCMK_XE_TRANSITION`,
`PCMK_XE_UTILIZATION`, `PCMK_XE_UTILIZATIONS`, `PCMK_XE_VALIDATE`,
`PCMK_XE_VERSION`, `PCMK_XE_XML`, and `PCMK_XE_XML_PATCHSET`
+ **libcrmcommon:** add functions `pcmk_action_text()`, `pcmk_find_node()`,
`pcmk_foreach_active_resource()`, `pcmk_get_dc()`, `pcmk_parse_interval_spec()`,
`pcmk_get_no_quorum_policy()`, `pcmk_has_quorum()`,
`pcmk_node_is_clean()`, `pcmk_update_configured_schema()`,
`pcmk_node_is_in_maintenance()`, `pcmk_node_is_online()`,
`pcmk_node_is_pending()`, `pcmk_node_is_shutting_down()`, `pcmk_on_fail_text()`,
`pcmk_parse_action()`, `pcmk_parse_role()`, `pcmk_resource_id()`,
`pcmk_resource_is_managed()`, `pcmk_role_text()`, and `pcmk_set_scheduler_cib()`
+ **libcrmcommon:** add type `pcmk_rule_input_t`
+ **libcrmcommon:** deprecate globals `crm_log_level`, `crm_trace_nonlog`,
`was_processing_error`, and `was_processing_warning`
+ **libcrmcommon:** deprecate functions `add_message_xml()`, `add_node_copy()`,
`can_prune_leaf()`, `cli_config_update()`, `copy_in_properties()`, `copy_xml()`,
`create_hello_message()`, `pcmk_parse_action()`, `create_reply()`,
`create_reply_adv()`, `create_request()`, `create_request_adv()`,
`create_xml_node()`, `crm_map_element_name()`, `crm_next_same_xml()`,
`crm_parse_interval_spec()`, `crm_xml_escape()`, `diff_xml_object()`,
`dump_xml_formatted()`, `dump_xml_formatted_with_text()`,
`dump_xml_unformatted()`, `expand_plus_plus()`, `filename2xml()`,
`find_xml_children()`, `find_xml_node()`, `first_named_child()`,
`fix_plus_plus_recursive()`, `get_message_xml()`, `get_schema_name()`,
`get_schema_version()`, `get_xpath_object_relative()`, `ID()`,
`pcmk_action_text()`, `pcmk_create_html_node()`, `pcmk_create_xml_text_node()`,
`pcmk_hostname()`, `pcmk_on_fail_text()`, `purge_diff_markers()`,
`replace_xml_child()`, `stdin2xml()`, `string2xml()`, `subtract_xml_object()`,
`update_validation()`, `update_xml_child()`, `validate_xml()`,
`validate_xml_verbose()`, `write_xml_fd()`, `write_xml_file()`,
`xml_latest_schema()`, and `xml_remove_prop()`
+ **libcrmcommon:** deprecate constants `CIB_OPTIONS_FIRST`, `CRM_INFINITY_S`,
`CRM_MINUS_INFINITY_S`, `CRM_OP_LOCAL_SHUTDOWN`, `CRM_PLUS_INFINITY_S`,
`CRM_SCORE_INFINITY`, `F_CLIENTNAME`, `F_CRM_DATA`, `F_CRM_DC_LEAVING`,
`F_CRM_ELECTION_AGE_S`, `F_CRM_ELECTION_AGE_US`, `F_CRM_ELECTION_ID`,
`F_CRM_ELECTION_OWNER`, `F_CRM_HOST_FROM`, `F_CRM_HOST_TO`, `F_CRM_JOIN_ID`,
`F_CRM_MSG_TYPE`, `F_CRM_ORIGIN`, `F_CRM_REFERENCE`, `F_CRM_SYS_FROM`,
`F_CRM_SYS_TO`, `F_CRM_TASK`, `F_CRM_TGRAPH`, `F_CRM_TGRAPH_INPUT`,
`F_CRM_THROTTLE_MAX`, `F_CRM_THROTTLE_MODE`, `F_CRM_USER`, `F_CRM_VERSION`,
`F_ORIG`, `F_SEQ`, `F_SUBTYPE`, `F_TYPE`, `F_XML_TAGNAME`, `INFINITY`, `INFINITY_S`,
`MINUS_INFINITY_S`, `OFFLINESTATUS`, `ONLINESTATUS`,
`PCMK_XA_PROMOTED_MAX_LEGACY`, `PCMK_XA_PROMOTED_NODE_MAX_LEGACY`,
`PCMK_XA_TARGET_PATTERN`, `PCMK_XA_UNAME`, `PCMK_XE_PROMOTABLE_LEGACY`,
`SUPPORT_UPSTART`, `T_ATTRD`, `T_CRM`, `XML_ACL_ATTR_ATTRIBUTE`,
`XML_ACL_ATTR_KIND`, `XML_ACL_ATTR_REF`, `XML_ACL_ATTR_REFv1`, `XML_ACL_ATTR_TAG`,
`XML_ACL_ATTR_TAGv1`, `XML_ACL_ATTR_XPATH`, `XML_ACL_TAG_DENY`,
`XML_ACL_TAG_GROUP`, `XML_ACL_TAG_PERMISSION`, `XML_ACL_TAG_READ`,
`XML_ACL_TAG_ROLE`, `XML_ACL_TAG_ROLE_REF`, `XML_ACL_TAG_ROLE_REFv1`,
`XML_ACL_TAG_USER`, `XML_ACL_TAG_USERv1`, `XML_ACL_TAG_WRITE`,
`XML_AGENT_ATTR_CLASS`, `XML_AGENT_ATTR_PROVIDER`, `XML_ALERT_ATTR_PATH`,
`XML_ALERT_ATTR_REC_VALUE`, `XML_ALERT_ATTR_TIMEOUT`,
`XML_ALERT_ATTR_TSTAMP_FORMAT`, `XML_ATTR_CRM_VERSION`, `XML_ATTR_DC_UUID`,
`XML_ATTR_DESC`, `XML_ATTR_DIGEST`, `XML_ATTR_GENERATION`,
`XML_ATTR_GENERATION_ADMIN`, `XML_ATTR_HAVE_QUORUM`, `XML_ATTR_HAVE_WATCHDOG`,
`XML_ATTR_ID`, `XML_ATTR_IDREF`, `XML_ATTR_ID_LONG`, `XML_ATTR_NAME`,
`XML_ATTR_NUMUPDATES`, `XML_ATTR_OP`, `XML_ATTR_ORIGIN`, `XML_ATTR_QUORUM_PANIC`,
`XML_ATTR_REFERENCE`, `XML_ATTR_REQUEST`, `XML_ATTR_RESPONSE`,
`XML_ATTR_STONITH_DEVICES`, `XML_ATTR_STONITH_INDEX`, `XML_ATTR_STONITH_TARGET`,
`XML_ATTR_STONITH_TARGET_ATTRIBUTE`, `XML_ATTR_STONITH_TARGET_VALUE`,
`XML_ATTR_TE_NOWAIT`, `XML_ATTR_TE_TARGET_RC`, `XML_ATTR_TIMEOUT`,
`XML_ATTR_TRANSITION_KEY`, `XML_ATTR_TRANSITION_MAGIC`, `XML_ATTR_TSTAMP`,
`XML_ATTR_TYPE`, `XML_ATTR_UPDATE_CLIENT`, `XML_ATTR_UPDATE_ORIGIN`,
`XML_ATTR_UPDATE_USER`, `XML_ATTR_VALIDATION`, `XML_ATTR_VERSION`,
`XML_BOOLEAN_FALSE`, `XML_BOOLEAN_NO`, `XML_BOOLEAN_TRUE`, `XML_BOOLEAN_YES`,
`XML_CIB_ATTR_PRIORITY`, `XML_CIB_ATTR_SHUTDOWN`, `XML_CIB_ATTR_WRITTEN`,
`XML_CIB_TAG_ACLS`, `XML_CIB_TAG_ALERT`, `XML_CIB_TAG_ALERTS`,
`XML_CIB_TAG_ALERT_ATTR`, `XML_CIB_TAG_ALERT_ATTRIBUTES`,
`XML_CIB_TAG_ALERT_FENCING`, `XML_CIB_TAG_ALERT_NODES`,
`XML_CIB_TAG_ALERT_RECIPIENT`, `XML_CIB_TAG_ALERT_RESOURCES`,
`XML_CIB_TAG_ALERT_SELECT`, `XML_CIB_TAG_CONFIGURATION`,
`XML_CIB_TAG_CONSTRAINTS`, `XML_CIB_TAG_CONTAINER`, `XML_CIB_TAG_CRMCONFIG`,
`XML_CIB_TAG_GENERATION_TUPPLE`, `XML_CIB_TAG_GROUP`, `XML_CIB_TAG_INCARNATION`,
`XML_CIB_TAG_LRM`, `XML_CIB_TAG_NODE`, `XML_CIB_TAG_NODES`, `XML_CIB_TAG_NVPAIR`,
`XML_CIB_TAG_OBJ_REF`, `XML_CIB_TAG_OPCONFIG`, `XML_CIB_TAG_PROPSET`,
`XML_CIB_TAG_RESOURCE`, `XML_CIB_TAG_RESOURCES`, `XML_CIB_TAG_RSCCONFIG`,
`XML_CIB_TAG_RSC_TEMPLATE`, `XML_CIB_TAG_SECTION_ALL`, `XML_CIB_TAG_STATE`,
`XML_CIB_TAG_STATUS`, `XML_CIB_TAG_TAG`, `XML_CIB_TAG_TAGS`,
`XML_CIB_TAG_TICKETS`, `XML_CIB_TAG_TICKET_STATE`, `XML_COLOC_ATTR_INFLUENCE`,
`XML_COLOC_ATTR_NODE_ATTR`, `XML_COLOC_ATTR_SOURCE`,
`XML_COLOC_ATTR_SOURCE_INSTANCE`, `XML_COLOC_ATTR_SOURCE_ROLE`,
`XML_COLOC_ATTR_TARGET`, `XML_COLOC_ATTR_TARGET_INSTANCE`,
`XML_COLOC_ATTR_TARGET_ROLE`, `XML_CONFIG_ATTR_DC_DEADTIME`,
`XML_CONFIG_ATTR_ELECTION_FAIL`, `XML_CONFIG_ATTR_FENCE_REACTION`,
`XML_CONFIG_ATTR_FORCE_QUIT`, `XML_CONFIG_ATTR_NODE_PENDING_TIMEOUT`,
`XML_CONFIG_ATTR_PRIORITY_FENCING_DELAY`, `XML_CONFIG_ATTR_RECHECK`,
`XML_CONFIG_ATTR_SHUTDOWN_LOCK`, `XML_CONFIG_ATTR_SHUTDOWN_LOCK_LIMIT`,
`XML_CONS_ATTR_SYMMETRICAL`, `XML_CONS_TAG_RSC_DEPEND`,
`XML_CONS_TAG_RSC_LOCATION`, `XML_CONS_TAG_RSC_ORDER`, `XML_CONS_TAG_RSC_SET`,
`XML_CONS_TAG_RSC_TICKET`, `XML_CRM_TAG_PING`, `XML_DIFF_ATTR`, `XML_DIFF_CHANGE`,
`XML_DIFF_LIST`, `XML_DIFF_MARKER`, `XML_DIFF_OP`, `XML_DIFF_PATH`,
`XML_DIFF_POSITION`, `XML_DIFF_RESULT`, `XML_DIFF_VERSION`, `XML_DIFF_VSOURCE`,
`XML_DIFF_VTARGET`, `XML_EXPR_ATTR_ATTRIBUTE`, `XML_EXPR_ATTR_OPERATION`,
`XML_EXPR_ATTR_TYPE`, `XML_EXPR_ATTR_VALUE`, `XML_EXPR_ATTR_VALUE_SOURCE`,
`XML_FAILCIB_ATTR_ID`, `XML_FAILCIB_ATTR_OBJTYPE`, `XML_FAILCIB_ATTR_OP`,
`XML_FAILCIB_ATTR_REASON`, `XML_FAIL_TAG_CIB`, `XML_GRAPH_TAG_CRM_EVENT`,
`XML_GRAPH_TAG_DOWNED`, `XML_GRAPH_TAG_MAINTENANCE`,
`XML_GRAPH_TAG_PSEUDO_EVENT`, `XML_GRAPH_TAG_RSC_OP`,
`XML_LOCATION_ATTR_DISCOVERY`, `XML_LOC_ATTR_SOURCE`,
`XML_LOC_ATTR_SOURCE_PATTERN`, `XML_LRM_ATTR_CALLID`,
`XML_LRM_ATTR_EXIT_REASON`, `XML_LRM_ATTR_INTERVAL`, `XML_LRM_ATTR_INTERVAL_MS`,
`XML_LRM_ATTR_MIGRATE_SOURCE`, `XML_LRM_ATTR_MIGRATE_TARGET`,
`XML_LRM_ATTR_OPSTATUS`, `XML_LRM_ATTR_OP_DIGEST`, `XML_LRM_ATTR_OP_RESTART`,
`XML_LRM_ATTR_OP_SECURE`, `XML_LRM_ATTR_RC`, `XML_LRM_ATTR_RESTART_DIGEST`,
`XML_LRM_ATTR_ROUTER_NODE`, `XML_LRM_ATTR_RSCID`, `XML_LRM_ATTR_SECURE_DIGEST`,
`XML_LRM_ATTR_TARGET`, `XML_LRM_ATTR_TARGET_UUID`, `XML_LRM_ATTR_TASK`,
`XML_LRM_ATTR_TASK_KEY`, `XML_LRM_TAG_RESOURCE`, `XML_LRM_TAG_RESOURCES`,
`XML_LRM_TAG_RSC_OP`, `XML_NODE_ATTR_RSC_DISCOVERY`, `XML_NODE_IS_FENCED`,
`XML_NODE_IS_MAINTENANCE`, `XML_NODE_IS_REMOTE`, `XML_NVPAIR_ATTR_NAME`,
`XML_NVPAIR_ATTR_VALUE`, `XML_OP_ATTR_ALLOW_MIGRATE`, `XML_OP_ATTR_DIGESTS_ALL`,
`XML_OP_ATTR_DIGESTS_SECURE`, `XML_OP_ATTR_INTERVAL_ORIGIN`,
`XML_OP_ATTR_ON_FAIL`, `XML_OP_ATTR_PENDING`, `XML_OP_ATTR_START_DELAY`,
`XML_ORDER_ATTR_FIRST`, `XML_ORDER_ATTR_FIRST_ACTION`, `XML_ORDER_ATTR_KIND`,
`XML_ORDER_ATTR_THEN`, `XML_ORDER_ATTR_THEN_ACTION`, `XML_PING_ATTR_CRMDSTATE`,
`XML_PING_ATTR_PACEMAKERDSTATE`, `XML_PING_ATTR_PACEMAKERDSTATE_INIT`,
`XML_PING_ATTR_PACEMAKERDSTATE_REMOTE`,
`XML_PING_ATTR_PACEMAKERDSTATE_RUNNING`,
`XML_PING_ATTR_PACEMAKERDSTATE_SHUTDOWNCOMPLETE`,
`XML_PING_ATTR_PACEMAKERDSTATE_SHUTTINGDOWN`,
`XML_PING_ATTR_PACEMAKERDSTATE_STARTINGDAEMONS`,
`XML_PING_ATTR_PACEMAKERDSTATE_WAITPING`, `XML_PING_ATTR_STATUS`,
`XML_PING_ATTR_SYSFROM`, `XML_REMOTE_ATTR_RECONNECT_INTERVAL`,
`XML_RSC_ATTR_CLEAR_INTERVAL`, `XML_RSC_ATTR_CLEAR_OP`,
`XML_RSC_ATTR_CONTAINER`, `XML_RSC_ATTR_CRITICAL`, `XML_RSC_ATTR_INCARNATION`,
`XML_RSC_ATTR_INTERLEAVE`, `XML_RSC_ATTR_INTERNAL_RSC`,
`XML_RSC_ATTR_MAINTENANCE`, `XML_RSC_ATTR_MANAGED`, `XML_RSC_ATTR_MULTIPLE`,
`XML_RSC_ATTR_NOTIFY`, `XML_RSC_ATTR_ORDERED`, `XML_RSC_ATTR_PROMOTABLE`,
`XML_RSC_ATTR_REMOTE_NODE`, `XML_RSC_ATTR_REMOTE_RA_ADDR`,
`XML_RSC_ATTR_REMOTE_RA_PORT`, `XML_RSC_ATTR_REMOTE_RA_SERVER`,
`XML_RSC_ATTR_REQUIRES`, `XML_RSC_ATTR_RESTART`, `XML_RSC_ATTR_STICKINESS`,
`XML_RSC_ATTR_TARGET`, `XML_RSC_ATTR_TARGET_ROLE`, `XML_RSC_ATTR_UNIQUE`,
`XML_RSC_OP_LAST_CHANGE`, `XML_RSC_OP_T_EXEC`, `XML_RSC_OP_T_QUEUE`,
`XML_RULE_ATTR_BOOLEAN_OP`, `XML_RULE_ATTR_ROLE`, `XML_RULE_ATTR_SCORE`,
`XML_RULE_ATTR_SCORE_ATTRIBUTE`, `XML_TAG_ATTRS`, `XML_TAG_ATTR_SETS`,
`XML_TAG_CIB`, `XML_TAG_DIFF`, `XML_TAG_EXPRESSION`, `XML_TAG_FAILED`,
`XML_TAG_FENCING_LEVEL`, `XML_TAG_FENCING_TOPOLOGY`, `XML_TAG_GRAPH`,
`XML_TAG_META_SETS`, `XML_TAG_OPTIONS`, `XML_TAG_PARAM`, `XML_TAG_PARAMS`,
`XML_TAG_RESOURCE_REF`, `XML_TAG_RULE`, `XML_TAG_TRANSIENT_NODEATTRS`,
`XML_TAG_UTILIZATION`, `XML_TICKET_ATTR_LOSS_POLICY`, and
`XML_TICKET_ATTR_TICKET`
+ **libcrmcommon:** deprecate direct access to all members of `pcmk_scheduler_t`,
`pcmk_tag_t`, and `pcmk_ticket_t`
+ **libcrmcommon:** deprecate `pcmk_rsc_methods_t`, `pcmk_assignment_methods_t`,
struct `pe_action_s`, struct `pe_resource_s`, struct `resource_alloc_functions_s`,
struct `resource_object_functions_s`, struct `pe_node_s`, and
struct `pe_node_shared_s`, including all their members
+ **libcrmcommon:** deprecate enums `action_fail_response`, `action_tasks`,
`expression_type`, `node_type`, `pcmk_rsc_flags`, `pcmk_scheduler_flags`,
`pe_action_flags`, `pe_discover_e`, `pe_obj_types`, `rsc_recovery_type`, and
`rsc_start_requirement`, including all their values
+ **liblrmd:** deprecate constants `F_LRMD_ALERT`, `F_LRMD_ALERT_ID`,
`F_LRMD_ALERT_PATH`, `F_LRMD_CALLBACK_TOKEN`, `F_LRMD_CALLDATA`, `F_LRMD_CALLID`,
`F_LRMD_CALLOPTS`, `F_LRMD_CLASS`, `F_LRMD_CLIENTID`, `F_LRMD_CLIENTNAME`,
`F_LRMD_EXEC_RC`, `F_LRMD_IPC_CLIENT`, `F_LRMD_IPC_IPC_SERVER`, `F_LRMD_IPC_MSG`,
`F_LRMD_IPC_MSG_FLAGS`, `F_LRMD_IPC_MSG_ID`, `F_LRMD_IPC_OP`, `F_LRMD_IPC_USER`,
`F_LRMD_IS_IPC_PROVIDER`, `F_LRMD_OPERATION`, `F_LRMD_OP_STATUS`, `F_LRMD_ORIGIN`,
`F_LRMD_PROTOCOL_VERSION`, `F_LRMD_PROVIDER`, `F_LRMD_RC`, `F_LRMD_REMOTE_MSG_ID`,
`F_LRMD_REMOTE_MSG_TYPE`, `F_LRMD_RSC`, `F_LRMD_RSC_ACTION`, `F_LRMD_RSC_DELETED`,
`F_LRMD_RSC_EXEC_TIME`, `F_LRMD_RSC_EXIT_REASON`, `F_LRMD_RSC_ID`,
`F_LRMD_RSC_INTERVAL`, `F_LRMD_RSC_OUTPUT`, `F_LRMD_RSC_QUEUE_TIME`,
`F_LRMD_RSC_RCCHANGE_TIME`, `F_LRMD_RSC_RUN_TIME`, `F_LRMD_RSC_START_DELAY`,
`F_LRMD_RSC_USERDATA_STR`, `F_LRMD_TIMEOUT`, `F_LRMD_TYPE`, `F_LRMD_WATCHDOG`,
`T_LRMD`, `T_LRMD_IPC_PROXY`, `T_LRMD_NOTIFY`, `T_LRMD_REPLY`, and `T_LRMD_RSC_OP`
+ **libpacemaker:** distribute pacemaker.h header to allow high-level API usage
+ **libpe_rules:** deprecate functions `find_expression_type()`,
`pe_evaluate_rules()`, `pe_eval_expr()`, `pe_eval_rules()`, `pe_eval_subexpr()`,
`pe_expand_re_matches()`, `pe_test_expression()`, and `pe_test_rule()`
+ **libpe_rules,libpe_status:** move enum `expression_type` and globals
`was_processing_error` and `was_processing_warning` to libcrmcommon
+ **libpe_rules,libpe_status:** deprecate role member of `pe_op_eval_data`
+ **libpe_rules,libpe_status:** deprecate functions `text2task()`, `fail2text()`,
`recovery2text()`, `role2text()`, `task2text()`, and `text2role()`
+ **libpe_status:** deprecate functions `pe_find_node()`, `pe_pref()`,
`pe_rsc_is_anon_clone()`, `pe_rsc_is_bundled()`, `pe_rsc_is_clone()`,
and `pe_rsc_is_unique_clone()`
+ **libpe_status:** deprecate global `resource_class_functions`
+ **libstonithd:** deprecate constants `T_STONITH_NOTIFY_DISCONNECT`,
`T_STONITH_NOTIFY_FENCE`, `T_STONITH_NOTIFY_HISTORY`, and
`T_STONITH_NOTIFY_HISTORY_SYNCED`
# Pacemaker-2.1.7 (19 Dec 2023)
- 1388 commits with 358 files changed, 23771 insertions(+), 17219 deletions(-)
## Features added since Pacemaker-2.1.6
+ **build:** allow building with libxml2 2.12.0 and greater
+ **CIB:** deprecate "ordering" attribute of "resource_set"
+ **CIB:** new cluster option "node-pending-timeout" (defaulting to 0, meaning
no timeout, to preserve existing behavior) allows fencing of nodes that do
not join Pacemaker's controller group within this much time after joining
the cluster
+ **controller:** `PCMK_node_start_state` now works with Pacemaker Remote nodes
+ **tools:** `crm_verify` now supports `--quiet` option (currently same as default
behavior, but in the future, verbose behavior might become the default,
so script writers are recommended to explicitly add `--quiet` if they do not
want output)
+ **tools:** `crm_node` supports standard `--output-as/--output-to` arguments
+ **tests:** CTSlab.py was renamed to cts-lab
## Fixes since Pacemaker-2.1.6
+ **logging:** restore ability to enable XML trace logs by file and function
*(regression introduced in 2.1.6)*
+ **scheduler:** avoid double free with disabled recurring actions
*(regression introduced in 2.1.5)*
+ **tools:** consider dampening argument when setting values with `attrd_updater`
*(regression introduced in 2.1.5)*
+ **tools:** wait for reply from `crm_node -R` *(regression introduced in 2.0.5)*
+ **agents:** handle dampening parameter consistently and correctly
+ **CIB:** be more strict about ignoring colocation elements without an ID
+ **controller:** do not check whether watchdog fencing is enabled
if "stonith-watchdog-timeout" is not configured
+ **controller:** don't try to execute agent action at shutdown
+ **controller:** avoid race condition when updating node state during join
+ **controller:** correctly determine state of a fenced node without a name
+ **controller:** wait a second between fencer connection attempts
+ **libpacemaker:** avoid shuffling clone instances unnecessarily
+ **libpacemaker:** get bundle container's promotion score from correct node
+ **libpacemaker:** role-based colocations now work with bundles
+ **libpacemaker:** clone-node-max now works with cloned groups
+ **scheduler:** compare anti-colocation dependent negative preferences against
stickiness
+ **scheduler:** consider explicit colocations with group members
+ **scheduler:** avoid fencing a pending node without a name
+ **scheduler:** properly evaluate rules in action meta-attributes
+ **scheduler:** properly sort rule-based blocks when overwriting values
+ **tools:** `crm_resource` `--wait` will now wait if any actions are pending
(previously it would wait only if new actions were planned)
+ **tools:** `crm_verify` `--output-as=xml` now includes detailed messages
+ **tools:** avoid showing pending nodes as having "<3.15.1" feature set in
`crm_mon`
+ **tools:** fix display of clone descriptions
+ **tools:** `crm_resource` now reports an error rather than timing out when
trying to restart an unmanaged resource
+ **tools:** `crm_resource` now properly detects which promoted role name to use
in ban and move constraints
## Public API changes since Pacemaker-2.1.6 (all API/ABI backward-compatible)
+ **libcib:** `cib_t` now supports transactions via new `cib_api_operations_t`
methods, new `cib_transaction` value in enum `cib_call_options`, and new
`cib_t` transaction and user members
+ **libcib:** `cib_t` now supports setting the ACL user for methods via new
`cib_api_operations_t` `set_user()` method
+ **libcib:** deprecate `cib_api_operations_t` methods `inputfd()`, `noop()`,
`quit()`, `set_op_callback()`, and `signon_raw()`
+ **libcib:** deprecate `cib_call_options` values `cib_mixed_update`,
`cib_scope_local`, and `cib_zero_copy`
+ **libcib:** deprecate `cib_t` `op_callback` member
+ **libcrmcluster:** deprecate `set_uuid()`
+ **libcrmcluster:** `send_cluster_message()'s` data argument is const
+ **libcrmcommon:** add enum `pcmk_rc_e` values `pcmk_rc_compression`,
`pcmk_rc_ns_resolution`, and `pcmk_rc_no_transaction`
+ **libcrmcommon,libpe_rules,libpe_status:** many APIs have been moved from
`libpe_rules` and `libpe_status` to libcrmcommon, sometimes with new names
(deprecating the old ones), as described below
+ **libcrmcommon:** add (and deprecate) `PCMK_DEFAULT_METADATA_TIMEOUT_MS` defined
constant
+ **libcrmcommon:** add enum `pcmk_rsc_flags`
+ **libcrmcommon:** add enum `pcmk_scheduler_flags`
+ **libcrmcommon:** add `pcmk_action_added_to_graph`
+ **libcrmcommon:** add `pcmk_action_always_in_graph`
+ **libcrmcommon:** add `pcmk_action_attrs_evaluated`
+ **libcrmcommon:** add `PCMK_ACTION_CANCEL` string constant
+ **libcrmcommon:** add `PCMK_ACTION_CLEAR_FAILCOUNT` string constant
+ **libcrmcommon:** add `PCMK_ACTION_CLONE_ONE_OR_MORE` string constant
+ **libcrmcommon:** add `PCMK_ACTION_DELETE` string constant
+ **libcrmcommon:** add `PCMK_ACTION_DEMOTE` string constant
+ **libcrmcommon:** add `pcmk_action_demote` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_DEMOTED` string constant
+ **libcrmcommon:** add `pcmk_action_demoted` to enum `action_tasks`
+ **libcrmcommon:** add `pcmk_action_detect_loop`
+ **libcrmcommon:** add `PCMK_ACTION_DO_SHUTDOWN` string constant
+ **libcrmcommon:** add `pcmk_action_fence` to enum `action_tasks`
+ **libcrmcommon:** add `pcmk_action_inputs_deduplicated`
+ **libcrmcommon:** add `PCMK_ACTION_LIST` string constant
+ **libcrmcommon:** add `PCMK_ACTION_LOAD_STOPPED` string constant
+ **libcrmcommon:** add `PCMK_ACTION_LRM_DELETE` string constant
+ **libcrmcommon:** add `PCMK_ACTION_MAINTENANCE_NODES` string constant
+ **libcrmcommon:** add `PCMK_ACTION_META_DATA` string constant
+ **libcrmcommon:** add `pcmk_action_migratable`
+ **libcrmcommon:** add `PCMK_ACTION_MIGRATE_FROM` string constant
+ **libcrmcommon:** add `PCMK_ACTION_MIGRATE_TO` string constant
+ **libcrmcommon:** add `pcmk_action_migration_abort`
+ **libcrmcommon:** add `pcmk_action_min_runnable`
+ **libcrmcommon:** add `PCMK_ACTION_MONITOR` string constant
+ **libcrmcommon:** add `pcmk_action_monitor` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_NOTIFIED` string constant
+ **libcrmcommon:** add `pcmk_action_notified` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_NOTIFY` string constant
+ **libcrmcommon:** add `pcmk_action_notify` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_OFF` string constant
+ **libcrmcommon:** add `PCMK_ACTION_ON` string constant
+ **libcrmcommon:** add `PCMK_ACTION_ONE_OR_MORE` string constant
+ **libcrmcommon:** add `pcmk_action_on_dc`
+ **libcrmcommon:** add `pcmk_action_optional`
+ **libcrmcommon:** add `PCMK_ACTION_PROMOTE` string constant
+ **libcrmcommon:** add `pcmk_action_promote` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_PROMOTED` string constant
+ **libcrmcommon:** add `pcmk_action_promoted` to enum `action_tasks`
+ **libcrmcommon:** add `pcmk_action_pseudo`
+ **libcrmcommon:** add `PCMK_ACTION_REBOOT` string constant
+ **libcrmcommon:** add `PCMK_ACTION_RELOAD` string constant
+ **libcrmcommon:** add `PCMK_ACTION_RELOAD_AGENT` string constant
+ **libcrmcommon:** add `pcmk_action_reschedule`
+ **libcrmcommon:** add `pcmk_action_runnable`
+ **libcrmcommon:** add `PCMK_ACTION_RUNNING` string constant
+ **libcrmcommon:** add `pcmk_action_shutdown` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_START` string constant
+ **libcrmcommon:** add `pcmk_action_start` to enum `action_tasks`
+ **libcrmcommon:** add `pcmk_action_started` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_STATUS` string constant
+ **libcrmcommon:** add `PCMK_ACTION_STONITH` string constant
+ **libcrmcommon:** add `PCMK_ACTION_STOP` string constant
+ **libcrmcommon:** add `pcmk_action_stop` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_STOPPED` string constant
+ **libcrmcommon:** add `pcmk_action_stopped` to enum `action_tasks`
+ **libcrmcommon:** add `pcmk_action_t` type
+ **libcrmcommon:** add `pcmk_action_unspecified` to enum `action_tasks`
+ **libcrmcommon:** add `PCMK_ACTION_VALIDATE_ALL` string constant
+ **libcrmcommon:** add `pcmk_assignment_methods_t` type
+ **libcrmcommon:** add `PCMK_DEFAULT_ACTION_TIMEOUT_MS` defined constant
+ **libcrmcommon:** add `pcmk_log_xml_as()`
+ **libcrmcommon:** add `PCMK_META_CLONE_MAX` string constant
+ **libcrmcommon:** add `PCMK_META_CLONE_MIN` string constant
+ **libcrmcommon:** add `PCMK_META_CLONE_NODE_MAX` string constant
+ **libcrmcommon:** add `PCMK_META_FAILURE_TIMEOUT` string constant
+ **libcrmcommon:** add `PCMK_META_MIGRATION_THRESHOLD` string constant
+ **libcrmcommon:** add `PCMK_META_PROMOTED_MAX` string constant
+ **libcrmcommon:** add `PCMK_META_PROMOTED_NODE_MAX` string constant
+ **libcrmcommon:** add `pcmk_multiply_active_block` to enum `rsc_recovery_type`
+ **libcrmcommon:** add `pcmk_multiply_active_restart` to enum `rsc_recovery_type`
+ **libcrmcommon:** add `pcmk_multiply_active_stop` to enum `rsc_recovery_type`
+ **libcrmcommon:** add `pcmk_multiply_active_unexpected` to enum `rsc_recovery_type`
+ **libcrmcommon:** add `PCMK_NODE_ATTR_TERMINATE` string constant
+ **libcrmcommon:** add `pcmk_node_t` type
+ **libcrmcommon:** add `pcmk_node_variant_cluster`
+ **libcrmcommon:** add `pcmk_node_variant_remote`
+ **libcrmcommon:** add `pcmk_no_action_flags`
+ **libcrmcommon:** add `pcmk_no_quorum_demote`
+ **libcrmcommon:** add `pcmk_no_quorum_fence`
+ **libcrmcommon:** add `pcmk_no_quorum_freeze`
+ **libcrmcommon:** add `pcmk_no_quorum_ignore`
+ **libcrmcommon:** add `pcmk_no_quorum_stop`
+ **libcrmcommon:** add `pcmk_on_fail_ban` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_block` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_demote` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_fence_node` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_ignore` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_reset_remote` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_restart` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_restart_container` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_standby_node` to `action_fail_response`
+ **libcrmcommon:** add `pcmk_on_fail_stop` to enum `action_fail_response`
+ **libcrmcommon:** add `pcmk_probe_always` to enum `pe_discover_e`
+ **libcrmcommon:** add `pcmk_probe_exclusive` to enum `pe_discover_e`
+ **libcrmcommon:** add `pcmk_probe_never` to enum `pe_discover_e`
+ **libcrmcommon:** add `pcmk_requires_fencing` to enum `rsc_start_requirement`
+ **libcrmcommon:** add `pcmk_requires_nothing` to enum `rsc_start_requirement`
+ **libcrmcommon:** add `pcmk_requires_quorum` to enum `rsc_start_requirement`
+ **libcrmcommon:** add `pcmk_resource_t` type
+ **libcrmcommon:** add `pcmk_role_promoted` to enum `rsc_role_e`
+ **libcrmcommon:** add `pcmk_role_started` to enum `rsc_role_e`
+ **libcrmcommon:** add `pcmk_role_stopped` to enum `rsc_role_e`
+ **libcrmcommon:** add `pcmk_role_unknown` to enum `rsc_role_e`
+ **libcrmcommon:** add `pcmk_role_unpromoted` to enum `rsc_role_e`
+ **libcrmcommon:** add `pcmk_rsc_match_anon_basename`
+ **libcrmcommon:** add `pcmk_rsc_match_basename`
+ **libcrmcommon:** add `pcmk_rsc_match_clone_only`
+ **libcrmcommon:** add `pcmk_rsc_match_current_node`
+ **libcrmcommon:** add `pcmk_rsc_match_history`
+ **libcrmcommon:** add `pcmk_rsc_methods_t` type
+ **libcrmcommon:** add `pcmk_rsc_variant_bundle`
+ **libcrmcommon:** add `pcmk_rsc_variant_clone`
+ **libcrmcommon:** add `pcmk_rsc_variant_group`
+ **libcrmcommon:** add `pcmk_rsc_variant_primitive`
+ **libcrmcommon:** add `pcmk_rsc_variant_unknown`
+ **libcrmcommon:** add `pcmk_scheduler_t` type
+ **libcrmcommon:** add `pcmk_tag_t` type
+ **libcrmcommon:** add `pcmk_ticket_t` type
+ **libcrmcommon:** add `PCMK_XA_FORMAT` string constant
+ **libcrmcommon:** `crm_ipc_send()'s` message argument is now const
+ **libcrmcommon:** deprecate `action_demote` in enum `action_tasks`
+ **libcrmcommon:** deprecate `action_demoted` in enum `action_tasks`
+ **libcrmcommon:** deprecate `action_fail_block` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_demote` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_fence` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_ignore` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_migrate` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_recover` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_reset_remote` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_standby` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `action_fail_stop` in `action_fail_response`
+ **libcrmcommon:** deprecate `action_notified` in enum `action_tasks`
+ **libcrmcommon:** deprecate `action_notify` in enum `action_tasks`
+ **libcrmcommon:** deprecate `action_promote` in enum `action_tasks`
+ **libcrmcommon:** deprecate `action_promoted` in enum `action_tasks`
+ **libcrmcommon:** deprecate `action_restart_container` in enum `action_fail_response`
+ **libcrmcommon:** deprecate `CRMD_ACTION_CANCEL` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_DELETE` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_DEMOTE` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_DEMOTED` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_METADATA` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_MIGRATE` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_MIGRATED` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_NOTIFIED` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_NOTIFY` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_PROMOTE` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_PROMOTED` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_RELOAD` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_RELOAD_AGENT` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_START` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_STARTED` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_STATUS` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_STOP` string constant
+ **libcrmcommon:** deprecate `CRMD_ACTION_STOPPED` string constant
+ **libcrmcommon:** deprecate `CRMD_METADATA_CALL_TIMEOUT` defined constant
+ **libcrmcommon:** deprecate `crm_action_str()`
+ **libcrmcommon:** deprecate `CRM_DEFAULT_OP_TIMEOUT_S` string constant
+ **libcrmcommon:** deprecate `crm_element_name()`
+ **libcrmcommon:** deprecate `CRM_OP_FENCE` string constant
+ **libcrmcommon:** deprecate `CRM_OP_RELAXED_CLONE` string constant
+ **libcrmcommon:** deprecate `CRM_OP_RELAXED_SET` string constant
+ **libcrmcommon:** deprecate `crm_xml_replace()`
+ **libcrmcommon:** deprecate enum `pe_link_state`
+ **libcrmcommon:** deprecate `getDocPtr()`
+ **libcrmcommon:** deprecate `monitor_rsc` in enum `action_tasks`
+ **libcrmcommon:** deprecate `node_member`
+ **libcrmcommon:** deprecate `node_remote`
+ **libcrmcommon:** deprecate `no_action` in enum `action_tasks`
+ **libcrmcommon:** deprecate `no_quorum_demote`
+ **libcrmcommon:** deprecate `no_quorum_freeze`
+ **libcrmcommon:** deprecate `no_quorum_ignore`
+ **libcrmcommon:** deprecate `no_quorum_stop`
+ **libcrmcommon:** deprecate `no_quorum_suicide`
+ **libcrmcommon:** deprecate `pcmk_log_xml_impl()`
+ **libcrmcommon:** deprecate `pcmk_scheduler_t` localhost member
+ **libcrmcommon:** deprecate `pe_action_dangle`
+ **libcrmcommon:** deprecate `pe_action_dc`
+ **libcrmcommon:** deprecate `pe_action_dedup`
+ **libcrmcommon:** deprecate `pe_action_dumped`
+ **libcrmcommon:** deprecate `pe_action_have_node_attrs`
+ **libcrmcommon:** deprecate `pe_action_implied_by_stonith`
+ **libcrmcommon:** deprecate `pe_action_migrate_runnable`
+ **libcrmcommon:** deprecate `pe_action_optional`
+ **libcrmcommon:** deprecate `pe_action_print_always`
+ **libcrmcommon:** deprecate `pe_action_processed`
+ **libcrmcommon:** deprecate `pe_action_pseudo`
+ **libcrmcommon:** deprecate `pe_action_requires_any`
+ **libcrmcommon:** deprecate `pe_action_reschedule`
+ **libcrmcommon:** deprecate `pe_action_runnable`
+ **libcrmcommon:** deprecate `pe_action_tracking`
+ **libcrmcommon:** deprecate `pe_clone`
+ **libcrmcommon:** deprecate `pe_container`
+ **libcrmcommon:** deprecate `pe_discover_always` in enum `pe_discover_e`
+ **libcrmcommon:** deprecate `pe_discover_exclusive` in enum `pe_discover_e`
+ **libcrmcommon:** deprecate `pe_discover_never` in enum `pe_discover_e`
+ **libcrmcommon:** deprecate `pe_find_anon`
+ **libcrmcommon:** deprecate `pe_find_any`
+ **libcrmcommon:** deprecate `pe_find_clone`
+ **libcrmcommon:** deprecate `pe_find_current`
+ **libcrmcommon:** deprecate `pe_find_inactive`
+ **libcrmcommon:** deprecate `pe_find_renamed`
+ **libcrmcommon:** deprecate `pe_group`
+ **libcrmcommon:** deprecate `pe_native`
+ **libcrmcommon:** deprecate `pe_unknown`
+ **libcrmcommon:** deprecate `recovery_block` in enum `rsc_recovery_type`
+ **libcrmcommon:** deprecate `recovery_stop_only` in enum `rsc_recovery_type`
+ **libcrmcommon:** deprecate `recovery_stop_start` in enum `rsc_recovery_type`
+ **libcrmcommon:** deprecate `recovery_stop_unexpected` in enum `rsc_recovery_type`
+ **libcrmcommon:** deprecate `RSC_CANCEL` string constant
+ **libcrmcommon:** deprecate `RSC_DELETE` string constant
+ **libcrmcommon:** deprecate `RSC_DEMOTE` string constant
+ **libcrmcommon:** deprecate `RSC_DEMOTED` string constant
+ **libcrmcommon:** deprecate `RSC_METADATA` string constant
+ **libcrmcommon:** deprecate `RSC_MIGRATE` string constant
+ **libcrmcommon:** deprecate `RSC_MIGRATED` string constant
+ **libcrmcommon:** deprecate `RSC_NOTIFIED` string constant
+ **libcrmcommon:** deprecate `RSC_NOTIFY` string constant
+ **libcrmcommon:** deprecate `RSC_PROMOTE` string constant
+ **libcrmcommon:** deprecate `RSC_PROMOTED` string constant
+ **libcrmcommon:** deprecate `rsc_req_nothing` in enum `rsc_start_requirement`
+ **libcrmcommon:** deprecate `rsc_req_quorum` in enum `rsc_start_requirement`
+ **libcrmcommon:** deprecate `rsc_req_stonith` in enum `rsc_start_requirement`
+ **libcrmcommon:** deprecate `RSC_ROLE_PROMOTED` in enum `rsc_role_e`
+ **libcrmcommon:** deprecate `RSC_ROLE_STARTED` in enum `rsc_role_e`
+ **libcrmcommon:** deprecate `RSC_ROLE_STOPPED` in enum `rsc_role_e`
+ **libcrmcommon:** deprecate `RSC_ROLE_UNKNOWN` in enum `rsc_role_e`
+ **libcrmcommon:** deprecate `RSC_ROLE_UNPROMOTED`
+ **libcrmcommon:** deprecate `RSC_START` string constant
+ **libcrmcommon:** deprecate `RSC_STARTED` string constant
+ **libcrmcommon:** deprecate `RSC_STATUS` string constant
+ **libcrmcommon:** deprecate `RSC_STOP` string constant
+ **libcrmcommon:** deprecate `RSC_STOPPED` string constant
+ **libcrmcommon:** deprecate `shutdown_crm` in enum `action_tasks`
+ **libcrmcommon:** deprecate `started_rsc` in enum `action_tasks`
+ **libcrmcommon:** deprecate `start_rsc` in enum `action_tasks`
+ **libcrmcommon:** deprecate `stonith_node` in enum `action_tasks`
+ **libcrmcommon:** deprecate `stopped_rsc` in enum `action_tasks`
+ **libcrmcommon:** deprecate `stop_rsc` in enum `action_tasks`
+ **libcrmcommon:** deprecate `TYPE()` macro
+ **libcrmcommon:** deprecate `XML_ATTR_VERBOSE` string constant
+ **libcrmcommon:** deprecate `XML_CIB_ATTR_SOURCE` string constant
+ **libcrmcommon:** deprecate `XML_CIB_TAG_DOMAINS` string constant
+ **libcrmcommon:** deprecate `xml_has_children()`
+ **libcrmcommon:** deprecate `XML_NODE_EXPECTED` string constant
+ **libcrmcommon:** deprecate `XML_NODE_IN_CLUSTER` string constant
+ **libcrmcommon:** deprecate `XML_NODE_IS_PEER` string constant
+ **libcrmcommon:** deprecate `XML_NODE_JOIN_STATE` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_FAIL_STICKINESS` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_FAIL_TIMEOUT` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_INCARNATION_MAX` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_INCARNATION_MIN` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_INCARNATION_NODEMAX` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_PROMOTED_MAX` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_PROMOTED_NODEMAX` string constant
+ **libcrmcommon:** deprecate `XML_TAG_DIFF_ADDED` string constant
+ **libcrmcommon:** deprecate `XML_TAG_DIFF_REMOVED` string constant
+ **libcrmcommon:** deprecate `XML_TAG_FRAGMENT`
+ **libcrmcommon:** `dump_xml_formatted()'s` argument is now const
+ **libcrmcommon:** `dump_xml_formatted_with_text()'s` argument is const
+ **libcrmcommon:** `dump_xml_unformatted()'s` argument is now const
+ **libcrmcommon:** `save_xml_to_file()'s` xml argument is now const
+ **libcrmcommon:** `validate_xml_verbose()'s` `xml_blob` argument is const
+ **libcrmcommon:** `write_xml_fd()'s` xml argument is now const
+ **libcrmcommon:** `write_xml_file()'s` xml argument is now const
+ **libcrmcommon:** `xml_top` argument of `xpath_search()` is now const
+ **libcrmcommon,libpe_rules,libpe_status:** move enum `pe_ordering`, struct
`pe_action_wrapper_s`, struct `pe_tag_s`, struct `pe_ticket_s`, struct
`resource_object_functions_s`, enum `node_type`, enum `pe_action_flags`, enum
`pe_discover_e`, enum `pe_find`, enum `pe_link_state`, enum `pe_obj_types`, enum
`pe_quorum_policy`, enum `pe_restart`, struct `pe_action_s`, struct `pe_node_s`,
struct `pe_node_shared_s`, struct `pe_resource_s`, struct `pe_working_set_s`,
enum `action_fail_response`, enum `action_tasks`, enum `pe_print_options`, enum
`rsc_recovery_type`, enum `rsc_role_e`, and enum `rsc_start_requirement` to
libcrmcommon
+ **libpacemaker,libpe_rules,libpe_status:** use `pcmk_action_t` instead of
`pe_action_t`, `pcmk_node_t` instead of `pe_node_t`, `pcmk_resource_t` instead of
`pe_resource_t`, and `pcmk_scheduler_t` instead of `pe_working_set_t` in all API
structs and functions
+ **libpacemaker:** add `pcmk_list_alternatives()`, `pcmk_list_providers()`,
`pcmk_list_standards()`, and `pcmk_list_agents()` for functionality equivalent
to `crm_resource` `--list-ocf-alternatives,` `--list-ocf-providers`,
`--list-standards,` and `--list-agents`
+ **libpe_rules,libpe_status:** deprecate `pe_action_t` type
+ **libpe_rules,libpe_status:** deprecate `pe_action_wrapper_t`
+ **libpe_rules,libpe_status:** deprecate `pe_node_t` type
+ **libpe_rules,libpe_status:** deprecate `pe_resource_t` type
+ **libpe_rules,libpe_status:** deprecate `pe_tag_t`
+ **libpe_rules,libpe_status:** deprecate `pe_ticket_t`
+ **libpe_rules,libpe_status:** deprecate `pe_working_set_t` type
+ **libpe_rules,libpe_status:** deprecate `resource_alloc_functions_t` type
+ **libpe_rules,libpe_status:** deprecate `resource_object_functions_t`
+ **libpe_status,libpe_rules:** deprecate enum `pe_ordering` and all its values
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_MAX`
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_PROMOTED_LEGACY_S` string constant
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_PROMOTED_S` string constant
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_STARTED_S` string constant
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_STOPPED_S` string constant
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_UNKNOWN_S`
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_UNPROMOTED_LEGACY_S` string constant
+ **libpe_status,libpe_rules:** deprecate `RSC_ROLE_UNPROMOTED_S` string constant
+ **libpe_status:** deprecate enum `pe_check_parameters`
+ **libpe_status:** deprecate `pe_flag_check_config`
+ **libpe_status:** deprecate `pe_flag_concurrent_fencing`
+ **libpe_status:** deprecate `pe_flag_enable_unfencing`
+ **libpe_status:** deprecate `pe_flag_have_quorum`
+ **libpe_status:** deprecate `pe_flag_have_remote_nodes`
+ **libpe_status:** deprecate `pe_flag_have_status`
+ **libpe_status:** deprecate `pe_flag_have_stonith_resource`
+ **libpe_status:** deprecate `pe_flag_maintenance_mode`
+ **libpe_status:** deprecate `pe_flag_no_compat`
+ **libpe_status:** deprecate `pe_flag_no_counts`
+ **libpe_status:** deprecate `pe_flag_quick_location`
+ **libpe_status:** deprecate `pe_flag_sanitized`
+ **libpe_status:** deprecate `pe_flag_show_scores`
+ **libpe_status:** deprecate `pe_flag_show_utilization`
+ **libpe_status:** deprecate `pe_flag_shutdown_lock`
+ **libpe_status:** deprecate `pe_flag_startup_fencing`
+ **libpe_status:** deprecate `pe_flag_startup_probes`
+ **libpe_status:** deprecate `pe_flag_start_failure_fatal`
+ **libpe_status:** deprecate `pe_flag_stonith_enabled`
+ **libpe_status:** deprecate `pe_flag_stop_action_orphans`
+ **libpe_status:** deprecate `pe_flag_stop_everything`
+ **libpe_status:** deprecate `pe_flag_stop_rsc_orphans`
+ **libpe_status:** deprecate `pe_flag_symmetric_cluster`
+ **libpe_status:** deprecate `pe_rsc_allow_migrate`
+ **libpe_status:** deprecate `pe_rsc_allow_remote_remotes`
+ **libpe_status:** deprecate `pe_rsc_assigning`
+ **libpe_status:** deprecate `pe_rsc_block`
+ **libpe_status:** deprecate `pe_rsc_critical`
+ **libpe_status:** deprecate `pe_rsc_detect_loop`
+ **libpe_status:** deprecate `pe_rsc_failed`
+ **libpe_status:** deprecate `pe_rsc_failure_ignored`
+ **libpe_status:** deprecate `pe_rsc_fence_device`
+ **libpe_status:** deprecate `pe_rsc_is_container`
+ **libpe_status:** deprecate `pe_rsc_maintenance`
+ **libpe_status:** deprecate `pe_rsc_managed`
+ **libpe_status:** deprecate `pe_rsc_merging`
+ **libpe_status:** deprecate `pe_rsc_needs_fencing`
+ **libpe_status:** deprecate `pe_rsc_needs_quorum`
+ **libpe_status:** deprecate `pe_rsc_needs_unfencing`
+ **libpe_status:** deprecate `pe_rsc_notify`
+ **libpe_status:** deprecate `pe_rsc_orphan`
+ **libpe_status:** deprecate `pe_rsc_orphan_container_filler`
+ **libpe_status:** deprecate `pe_rsc_promotable`
+ **libpe_status:** deprecate `pe_rsc_provisional`
+ **libpe_status:** deprecate `pe_rsc_reload`
+ **libpe_status:** deprecate `pe_rsc_replica_container`
+ **libpe_status:** deprecate `pe_rsc_restarting`
+ **libpe_status:** deprecate `pe_rsc_runnable`
+ **libpe_status:** deprecate `pe_rsc_start_pending`
+ **libpe_status:** deprecate `pe_rsc_stop`
+ **libpe_status:** deprecate `pe_rsc_stop_unexpected`
+ **libpe_status:** deprecate `pe_rsc_unique`
# Pacemaker-2.1.6 (24 May 2023)
- 1124 commits with 402 files changed, 25220 insertions(+), 14751 deletions(-)
## Features added since Pacemaker-2.1.5
+ **CIB:** deprecate "moon" in `date_spec` elements in rules
+ **CIB:** deprecate support for Nagios resources
+ **CIB:** utilization attributes may be set as transient
+ **CIB:** alerts and alert recipients support an "enabled" meta-attribute
+ All daemons support `--output-as/--output-to` options including
XML output with interactive options
+ **tools:** `attrd_updater` supports `--wait` parameter that can be set to
"no" (return immediately after submitting request, which is
the default and previous behavior), "local" (return after the
new value has taken effect on the local node), or "cluster" (return
after new value has taken effect on all nodes)
+ **tools:** `attrd_updater` supports `-z/--utilization` to modify
utilization attributes
+ **tools:** `attrd_updater` supports `--pattern` to affect all attributes
matching a given pattern
+ **tools:** `crm_attribute` supports `--pattern` with permanent node attributes
(in addition to previous support for transient attributes)
+ **tools:** `crm_mon` displays resource descriptions if `--show-description` or
`--show-detail` is given
+ **tools:** `crm_mon` shows maintenance mode when enabled per-resource
+ **tools:** `crm_mon` `--interval` can be used to update XML and text output
(in addition to previous support for HTML)
+ **tools:** `crm_mon` fencing history includes microseconds in timestamps
+ **tools:** `crm_mon` shows which node it was run on
+ **tools:** `crm_mon` shows whether Pacemaker or Pacemaker Remote is running or
shutting down
+ **tools:** deprecate `crm_mon --simple-status`
+ **tools:** `crm_resource` `--element` option can be used with
`--get-parameter`, `--set-parameter,` and `--delete-parameter` to modify
resource properties such as class, provider, type, and description
+ **tools:** `crm_resource` `--list` shows resource descriptions better
(including when `--output-as=xml` is used)
+ **tools:** `crm_shadow` supports standard `--output-as/--output-to` arguments
## Fixes since Pacemaker-2.1.5
+ **pacemakerd:** -S should wait for cluster to shut down before returning
*(regression introduced in 2.1.1)*
+ **Pacemaker Remote:** remote nodes wait for all-clear from cluster before
shutting down *(regression introduced in 2.1.5)*
+ **tools:** `attrd_updater` `--query` without `--node` shows attributes from all
nodes instead of local node *(regression introduced in 2.1.5)*
+ **pacemaker-attrd:** Preserve a Pacemaker Remote node's transient attributes
if its connection to the cluster is lost but reconnects
+ **CIB manager:** successful CIB schema upgrade always forces a write
+ **controller:** avoid election storm when joining node has CIB newer than DC
can accept
+ **controller:** avoid election storm due to incompatible CIB
+ **controller:** avoid use-after-free when disconnecting proxy IPCs during shutdown
+ **controller:** avoid double-incrementing failcount for simulated failures
+ **controller:** avoid reprobing remote nodes when target is a cluster node
+ **controller:** delay join finalization if a transition is in progress
+ **controller:** initial fencing timeout includes any priority-fencing-delay
+ **controller:** shutdown gracefully if scheduler connection is interrupted
during shutdown
+ **fencer:** avoid crash during shutdown when action is pending
+ **fencer:** calculate fencing timeout correctly when watchdog is used with
topology
+ **fencer:** apply priority-fencing-delay only to first device tried
+ **fencer:** total and per-device fencing timeouts include any
priority-fencing-delay and `pcmk_delay_base`
+ **scheduler:** fix a number of corner cases with colocations, including
preventing a resource from starting if it has a mandatory
colocation with a group whose start is blocked, preventing
optional anti-colocation from overriding infinite stickiness,
and correctly considering a group's colocation dependents
when the group is itself colocated with a clone
+ **scheduler:** honor as many mandatory colocations as possible before
considering any optional ones
+ **scheduler:** ensure earlier group member starts occur after later member stops
+ **scheduler:** handle orderings with bundles more correctly
+ **scheduler:** ensure expired results never affect resource state
+ **scheduler:** handle cleaned `migrate_from` history correctly
+ **scheduler:** prevent pending monitor of one clone instance from causing
unexpected stop of other instances
+ **scheduler:** prevent inactive clone instances from starting if probe is
unrunnable on any node
+ **agents:** SysInfo calculates `cpu_load` correctly
+ **tools:** `cibadmin --scope` accepts status
+ **tools:** `crm_attribute -p ""` works same as `-p` when called from
resource agent
+ **tools:** `crm_attribute` recognizes "-INFINITY" as value instead of options
+ **tools:** `crm_mon` avoids displaying recurring monitors as pending if first
attempt fails
+ **tools:** `crm_mon` `--daemonize` shows disconnected message when CIB connection
is lost instead of continuing to display last known state
+ **tools:** `crm_mon` avoids crash when built without curses library support
+ **tools:** `crm_mon` supports `--output-as=none` correctly
+ **tools:** `crm_resource` `--ban` or `--move` works with single-replica bundles
+ **tools:** `crm_shadow` `--commit` now works with `CIB_file`
+ **tools:** `crm_simulate` failure injection avoids crash if node name is unknown
## Public API changes since Pacemaker-2.1.5
+ **Python:** New "pacemaker" Python module (packaged as python3-pacemaker in
RPMs built with "make rpm") contains supported public API with
BuildOptions and ExitStatus classes
+ **libcib:** add `client_id()` method member to `cib_api_operations_t`
+ **libcib:** deprecate `cib_database`
+ **libcib:** deprecate `cib_quorum_override`
+ **libcib:** deprecate the `cib_api_operations_t:update()` method
+ **libcrmcluster:** add `pcmk_cluster_new()`
+ **libcrmcluster:** add `crm_join_nack_quiet`
+ **libcrmcluster:** add `pcmk_cluster_free()`
+ **libcrmcluster:** node argument to `send_cluster_message()` is now const
+ **libcrmcluster:** node argument to `send_cluster_text()` is now const
+ **libcrmcommon:** add `crm_time_usecs`
+ **libcrmcommon:** add `PCMK_META_ENABLED`
+ **libcrmcommon:** add `pcmk_pacemakerd_state_remote`
+ **libcrmcommon:** add `pcmk_rc_bad_xml_patch`
+ **libcrmcommon:** add `pcmk_rc_bad_input`
+ **libcrmcommon:** add `pcmk_rc_disabled`
+ **libcrmcommon:** deprecate `add_xml_nocopy()`
+ **libcrmcommon:** deprecate `log_data_element()`
+ **libcrmcommon:** deprecate `PCMK_RESOURCE_CLASS_NAGIOS`
+ **libcrmcommon:** deprecate `PCMK_RESOURCE_CLASS_UPSTART`
+ **libcrmcommon:** deprecate `XML_ATTR_UUID`
+ **libcrmcommon:** deprecate `XML_CIB_ATTR_REPLACE`
+ **libcrmcommon:** deprecate `xml_log_changes()`
+ **libcrmcommon:** deprecate `xml_log_options` enum
+ **libcrmcommon:** deprecate `xml_log_patchset()`
+ **libcrmcommon:** argument to `pcmk_xe_is_probe()` is now const
+ **libcrmcommon:** argument to `pcmk_xe_mask_probe_failure()` is now const
+ **libcrmcommon:** patchset argument of `xml_log_patchset()` is now const
+ **libcrmcommon:** `rsc_op_expected_rc()` argument is now const
+ **libcrmcommon:** second argument to `copy_in_properties()` is now const
+ **libcrmcommon:** xml argument of `xml_log_changes()` is now const
+ **libcrmservice:** deprecate enum `nagios_exitcode`
+ **libpacemaker:** add `pcmk_query_node_info()`
+ **libpacemaker:** add `pcmk_query_node_name()`
+ **libpacemaker:** multiple arguments to `pcmk_simulate()` are now const
+ **libpacemaker:** node argument to `pcmk_resource_digests()` is now const
+ **libpacemaker:** `node_types` argument to `pcmk_list_nodes()` is now const
+ **libpacemaker:** `pcmk_controller_status()` node name arg is now const
+ **libpe_rules:** last argument to `pe_expand_re_matches()` is now const
+ **libpe_rules:** `rule_data` argument to `pe_eval_nvpairs()` is now const
+ **libpe_rules:** second argument to `pe_eval_expr()` and `pe_eval_subexpr()` is now const
+ **libpe_rules:** second argument to `pe_eval_rules()` is now const
+ **libpe_rules:** second argument to `pe_unpack_nvpairs()` is now const
+ **libpe_status:** add `pe_rsc_detect_loop`
+ **libpe_status:** add `pe_rsc_replica_container`
+ **libpe_status:** deprecate fixed member of `pe_node_t`
+ **libpe_status:** argument to `pe_rsc_is_bundled()` is now const
+ **libpe_status:** argument to `rsc_printable_id()` is now const
+ **libpe_status:** first argument to `calculate_active_ops()` is now const
+ **libpe_status:** first argument to `pe_find_node()` is now const
+ **libpe_status:** first argument to `pe_find_node_any()` is now const
+ **libpe_status:** first argument to `pe_find_node_id()` is now const
+ **libpe_status:** first argument to `resource_object_functions_t:is_filtered()` is now const
# Pacemaker-2.1.5 (7 Dec 2022)
- 1287 commits with 447 files changed, 33546 insertions(+), 21518 deletions(-)
## Features added since Pacemaker-2.1.4
+ **CIB:** access control lists (ACLs) for groups are supported
+ **CIB:** ACL target and group XML supports "name" attribute to specify a name
that is not a unique XML ID
+ **CIB:** deprecate pacemaker-next schema
+ **CIB:** deprecate first-instance and then-instance in ordering constraints
and rsc-instance and with-rsc-instance in colocation constraints
(only usable with pacemaker-next schema)
+ **CIB:** deprecate "collocated" and "ordered" meta-attributes for groups
(true is default, and resource sets should be used instead of false)
+ **build:** support building with -D_TIME_BITS=64
+ **build:** support building with compilers that are strict about void
+ **build:** allow building RPMs from source distribution
+ **fencer:** deprecate stand-alone mode
+ **agents:** ClusterMon, controld, HealthCPU, ifspeed, and SysInfo agents support
OCF 1.1 standard
+ **agents:** non-functional SystemHealth agent has been removed
+ **tools:** non-functional ipmiservicelogd and notifyServicelogEvent tools have
been removed
+ **tools:** `crm_attribute` supports querying all attributes on a given node
+ **tools:** `crm_attribute` `--query,` --delete, and `--update` support
regular expressions
+ **tools:** `crm_error` supports standard `--output-as/--output-to` arguments
+ **tools:** `crm_error` lists all return codes if none are specified
+ **tools:** `crm_mon` `--show-detail` displays the CRM feature set of each node
and makes display of fencing actions more technical
+ **tools:** `crm_resource` `--why` checks node health
+ **tools:** `crm_resource` `--constraints` now accepts `--recursive` (equivalent to
`--stack)` and `--force` (to show constraints for a group member
instead of the group)
## Fixes since Pacemaker-2.1.4
+ **tools:** `crm_error` does not print spurious output when given a negative
return code argument *(regression introduced in 2.0.4)*
+ **tools:** avoid crash if `crm_resource` is given extraneous arguments
*(regression introduced in 2.0.5)*
+ **tools:** `stonith_admin` `--validate` XML output shows correct validation status
and errors *(regressions introduced in 2.0.5 and 2.1.2)*
+ **tools:** `crm_resource` `--list-operations` shows pending operations as pending
instead of complete *(regression introduced in 2.1.0)*
+ **controller:** move resources if appropriate after they are reordered in CIB
*(regression introduced in 2.1.3)*
+ **fencing:** allow fence devices to be registered if local node is not in CIB
*(regression introduced in 2.1.3)*
+ **tools:** `crm_mon` `--one-shot` should succeed even if pacemaker is shutting down
*(regression introduced in 2.1.3)*
+ **tools:** avoid memory leak in `crm_mon` *(regression introduced in 2.1.3)*
+ **tools:** `crm_attribute` `--quiet` outputs nothing instead of "(null)" if
attribute has no value *(regression introduced in 2.1.3)*
+ **tools:** accept deprecated and unused `attrd_updater` `--quiet` option
*(regression introduced in 2.1.3)*
+ **CIB:** avoid crashes when XML IDs grow very long
+ **controller:** pre-load agent metadata asynchronously to avoid timeout when
agent's metadata action runs `crm_node`
+ **controller:** avoid timing issue that increments resource fail count twice
+ **fencing:** unfence all nodes after device configuration changes
+ **fencing:** ignore node that executed action when checking whether actions
are equivalent
+ **scheduler,controller:** calculate secure digest consistently
+ **scheduler:** consider roles when blocking colocation dependents
+ **scheduler:** prioritize group colocations properly
+ **scheduler:** properly consider effect of "with group" colocations
+ **scheduler:** handle corner cases in live migrations
+ **scheduler:** avoid perpetual moving of bundle containers in certain situations
+ **scheduler:** properly calculate resource parameter digests without history
+ **scheduler:** do not enforce stop if newer monitor indicates resource was not
running on target of failed `migrate_to`
+ **scheduler:** do not enforce stop on rejoined node after failed `migrate_to`
+ **scheduler:** don't demote on expected node when multiple-active is set to
`stop_unexpected`
+ **scheduler:** prevent resources running on multiple nodes after partial live
migration
+ **scheduler:** restart resource instead of reload if extra parameters in
operation change
+ **schemas:** Consider days, minutes, seconds, and yeardays in date expressions
valid
+ **schemas:** Consider `in_range` with an end and duration valid
+ **schemas:** Consider score and score-attribute optional in rules
+ **tools:** `crm_resource` `--digests` uses most recent operation history entry
+ **tools:** if multiple return code options are given to `crm_error`, use the
last one
+ **tools:** `crm_resource` correctly detects if a resource is unmanaged
or disabled even if there are multiple settings using rules
## Public API changes since Pacemaker-2.1.4
+ **libcib:** add `cib_api_operations_t:set_primary`
+ **libcib:** add `cib_api_operations_t:set_secondary`
+ **libcib:** deprecate `cib_api_operations_t:delete_absolute()`
+ **libcib:** deprecate `cib_api_operations_t:is_master`
+ **libcib:** deprecate `cib_api_operations_t:set_master`
+ **libcib:** deprecate `cib_api_operations_t:set_slave`
+ **libcib:** deprecate `cib_api_operations_t:set_slave_all`
+ **libcrmcommon:** deprecated `XML_CIB_TAG_MASTER` constant is usable again
*(regression introduced in 2.1.0)*
+ **libcrmcommon:** `pcmk_ipc_api_t` supports attribute manager IPC
+ **libcrmcommon:** add `pcmk_rc_unpack_error`
+ **libcrmcommon:** add `CRM_EX_FAILED_PROMOTED`
+ **libcrmcommon:** add `CRM_EX_NONE`
+ **libcrmcommon:** add `CRM_EX_PROMOTED`
+ **libcrmcommon:** add `pcmk_readable_score()`
+ **libcrmcommon:** add `PCMK_XA_PROMOTED_MAX_LEGACY` string constant
+ **libcrmcommon:** add `PCMK_XA_PROMOTED_NODE_MAX_LEGACY` string constant
+ **libcrmcommon:** argument to `crm_time_check()` is now const
+ **libcrmcommon:** argument to `pcmk_controld_api_replies_expected()` is now const
+ **libcrmcommon:** argument to `pcmk_xml_attrs2nvpairs()` is now const
+ **libcrmcommon:** argument to `xml2list()` is now const
+ **libcrmcommon:** argument to `xml_acl_denied()` is now const
+ **libcrmcommon:** argument to `xml_acl_enabled()` is now const
+ **libcrmcommon:** argument to `xml_get_path()` is now const
+ **libcrmcommon:** arguments to `crm_time_add()` are now const
+ **libcrmcommon:** arguments to `crm_time_compare()` are now const
+ **libcrmcommon:** arguments to `crm_time_subtract()` are now const
+ **libcrmcommon:** argv argument to `crm_log_preinit()` is now char *const *
+ **libcrmcommon:** `crm_time_calculate_duration()` arguments are now const
+ **libcrmcommon:** deprecate `CRM_ATTR_RA_VERSION`
+ **libcrmcommon:** deprecate `crm_destroy_xml()`
+ **libcrmcommon:** deprecate `crm_ipc_server_error`
+ **libcrmcommon:** deprecate `crm_ipc_server_info`
+ **libcrmcommon:** deprecate `CRM_OP_LRM_QUERY`
+ **libcrmcommon:** deprecate `crm_str()`
+ **libcrmcommon:** deprecate `PCMK_XE_PROMOTED_MAX_LEGACY` string constant
+ **libcrmcommon:** deprecate `PCMK_XE_PROMOTED_NODE_MAX_LEGACY` constant
+ **libcrmcommon:** deprecate `score2char()`
+ **libcrmcommon:** deprecate `score2char_stack()`
+ **libcrmcommon:** deprecate `XML_ATTR_RA_VERSION`
+ **libcrmcommon:** deprecate `xml_get_path()`
+ **libcrmcommon:** deprecate `XML_PARANOIA_CHECKS`
+ **libcrmcommon:** deprecate `XML_TAG_OP_VER_ATTRS`
+ **libcrmcommon:** deprecate `XML_TAG_OP_VER_META`
+ **libcrmcommon:** deprecate `XML_TAG_RSC_VER_ATTRS`
+ **libcrmcommon:** dt argument of `crm_time_get_gregorian()` is now const
+ **libcrmcommon:** dt argument of `crm_time_get_isoweek()` is now const
+ **libcrmcommon:** dt argument of `crm_time_get_ordinal()` is now const
+ **libcrmcommon:** dt argument of `crm_time_get_timeofday()` is now const
+ **libcrmcommon:** dt argument of `crm_time_get_timezone()` is now const
+ **libcrmcommon:** first argument to `create_reply()` is now const
+ **libcrmcommon:** first argument to `crm_copy_xml_element()` is now const
+ **libcrmcommon:** first argument to `find_xml_node()` is now const
+ **libcrmcommon:** first argument to `get_message_xml()` is now const
+ **libcrmcommon:** first argument to `pcmk_ipc_name()` is now const
+ **libcrmcommon:** first argument to `xml_patch_versions()` is now const
+ **libcrmcommon:** last argument to `crm_write_blackbox()` is now const
+ **libcrmcommon:** add `pcmk_rc_duplicate_id`
+ **libcrmcommon:** add `pcmk_result_get_strings()`
+ **libcrmcommon:** add `pcmk_result_type` enum
+ **libcrmcommon:** add `PCMK_XE_DATE_EXPRESSION` constant
+ **libcrmcommon:** add `PCMK_XE_OP_EXPRESSION` constant
+ **libcrmcommon:** add `PCMK_XE_RSC_EXPRESSION` constant
+ **libcrmcommon:** first argument to `crm_time_as_string()` is now const
+ **libcrmcommon:** `crm_time_t` argument to `crm_time_log_alias()` is now const
+ **libcrmcommon:** argument to `crm_time_get_seconds()` is now const
+ **libcrmcommon:** argument to `crm_time_get_seconds_since_epoch()` is now const
+ **libcrmcommon:** sixth argument to `log_data_element()` is now const
+ **libcrmcommon:** source argument to `crm_time_set()` is now const
+ **libcrmcommon:** source argument to `crm_time_set_timet()` is now const
+ **libcrmcommon:** source argument to `pcmk_copy_time()` is now const
+ **libpacemaker:** add `pcmk_check_rule()`
+ **libpacemaker:** add `pcmk_check_rules()`
+ **libpacemaker:** add `pcmk_show_result_code()`
+ **libpacemaker:** add `pcmk_list_result_codes()`
+ **libpacemaker:** add `pcmk_rc_disp_flags` enum
+ **libpacemaker:** `ipc_name` argument to `pcmk_pacemakerd_status()` is now const
+ **libpe_rules:** deprecate `version_expr` enum value
+ **libpe_rules:** second argument to `pe_eval_nvpairs()` is now const
+ **libpe_status:** argument to `pe_rsc_is_anon_clone()` is now const
+ **libpe_status:** argument to `pe_rsc_is_unique_clone()` is now const
+ **libpe_status:** deprecate enum `pe_graph_flags`
+ **libpe_status:** first argument to `pe_rsc_is_clone()` is now const
+ **libpe_status:** two arguments to `get_rsc_attributes()` are now const
+ **libpe_status:** second argument to `pe_eval_versioned_attributes()` is now const
+ **libpe_status:** second argument to `pe_rsc_params()` is now const
+ **libstonithd:** deprecate `stonith_event_t:message`
+ **libstonithd:** deprecate `stonith_event_t:type`
+ **libstonithd:** last argument to `stonith_api_operations_t:register_device()`
is now const
+ **libstonithd:** last argument to `stonith_api_operations_t:register_level()` is
now const
+ **libstonithd:** last argument to
`stonith_api_operations_t:register_level_full()` is now const
+ **libstonithd:** params argument to `stonith_api_operations_t:validate()` is now
const
# Pacemaker-2.1.4 (13 Jun 2022)
- 17 commits with 9 files changed, 52 insertions(+), 18 deletions(-)
## Fixes since Pacemaker-2.1.3
+ **fencing:** get target-by-attribute working again *(regression in 2.1.3)*
+ **fencing:** avoid use-after-free when processing self-fencing requests
with topology *(regression in 2.1.3)*
+ **resources:** typo in HealthSMART meta-data *(regression in 2.1.3)*
+ **fencing:** avoid memory leaks when processing topology requests
+ **fencing:** delegate shouldn't matter when checking equivalent fencing
+ **tools:** fix CSS syntax error in `crm_mon --output-as=html`
# Pacemaker-2.1.3 (1 Jun 2022)
- 814 commits with 332 files changed, 23435 insertions(+), 12137 deletions(-)
## Features added since Pacemaker-2.1.2
+ Internal failures of resource actions (such as an OCF agent not being
found) are shown with a detailed exit reason in logs, `crm_mon` output, etc.
+ Support for CIB <node> entries with type="ping" is deprecated (this was an
undocumented means of defining a quorum-only node)
+ **build:** configure script supports experimental `--enable-nls` option to
enable native language translations (currently only Chinese
translations of certain help text are available)
+ **rpm:** `crm_attribute` is now part of the pacemaker-cli package instead of
the pacemaker package
+ **CIB:** resources support allow-unhealthy-nodes meta-attribute to exempt
the resource from bans due to node health checks (particularly useful
for health monitoring resources themselves)
+ **CIB:** multiple-active cluster property can be set to "stop_unexpected" to
leave the expected instance running and stop only any unexpected ones
+ **CIB:** bundles support resource utilization
+ **pacemakerd:** regularly check that subdaemons are active and accepting IPC
connections so sbd can self-fence a node with a malfunctioning
subdaemon
+ **pacemaker-schedulerd:** support `--output-as/--output-to` options including
XML output with interactive options
+ **tools:** cibadmin `--show-access` option to show CIB colorized for ACLs
+ **tools:** `crm_attribute` supports standard `--output-as/--output-to` options
+ **tools:** `crm_mon` output indicates if a node's health is yellow or red
+ **tools:** for probes that failed because the service is not installed or
locally configured, `crm_mon` displays the resource as stopped rather
than failed
+ **tools:** `crm_rule` supports standard `--output-as/--output-to` options,
allows passing multiple -r options, and is no longer experimental
+ **tools:** `stonith_admin` fencing commands display reasons for failures
+ **resource agents:** HealthSMART supports OCF 1.1 standard and new
`OCF_RESKEY_dampen` and `OCF_RESKEY_smartctl` parameters
## Fixes since Pacemaker-2.1.2
+ **build:** avoid circular library dependency *(regression introduced in 2.1.0)*
+ **systemd:** if pacemakerd exits immediately after starting, wait 1 second
before trying to respawn, and allow 5 attempts
+ **fencer:** get fencing completion time correctly
*(regression introduced in 2.1.2)*
+ **fencer:** avoid memory leak when broadcasting history differences
*(regression introduced in 2.1.0)*
+ **controller:** correctly match "node down" events so remote nodes don't get
fenced when connection is stopped
*(regression introduced in 2.1.2)*
+ **executor:** avoid possible double free during notify operation
*(regression introduced in 2.1.1)*
+ **tools:** get `stonith_admin` -T/--tag option working again
*(regression introduced in 2.0.3)*
+ **resources:** use correct syntax in Stateful meta-data
*(regression introduced in 2.1.0)*
+ **corosync:** repeat `corosync_cfg_trackstart` if first attempt fails
+ **libcrmcommon:** retry IPC requests after EAGAIN errors
+ **executor,scheduler:** treat "no secrets" fence results as a hard error
+ **fencing:** handle dynamic target query failures better
+ **fencing:** don't set stonith action to pending if fork fails
+ **pacemakerd:** avoid race condition when subdaemaon is checked while exiting
+ **scheduler:** avoid memory leak when displaying clones in certain conditions
+ **scheduler:** properly set data set flags when scheduling actions
+ **tools:** support command-line `crm_attribute` calls on Pacemaker remote nodes
whose node name in the cluster differs from their local hostname
+ **tools:** prevent possible `crm_resource` crashes if multiple commands specified
## Public API changes since Pacemaker-2.1.2
+ **libcrmcommon:** `pcmk_ipc_api_t` supports scheduler IPC
+ **libpacemaker:** add `pcmk_status()` (equivalent to `crm_mon)`
+ **libcib:** deprecate `get_object_parent()`
+ **libcib:** deprecate `get_object_path()`
+ **libcib:** deprecate `get_object_root()`
+ **libcrmcommon:** add `pcmk_cib_parent_name_for()`
+ **libcrmcommon:** add `pcmk_cib_xpath_for()`
+ **libcrmcommon:** add `pcmk_find_cib_element()`
+ **libcrmcommon:** deprecate `crm_xml_add_boolean()`
+ **libpe_status:** add `pe_flag_check_config`
+ **libpe_status:** add `pe_node_shared_s:data_set`
+ **libpe_status:** add `pe_rsc_restarting` flag
+ **libpe_status:** add `pe_rsc_stop_unexpected` flag
+ **libpe_status:** add `recovery_stop_unexpected` to enum `rsc_recovery_type`
+ **libpe_status:** deprecate `node_ping`
+ **libpe_status:** deprecate `pe_order_stonith_stop`
+ **libpe_status:** deprecate `pe_rsc_starting` and `pe_rsc_stopping`
+ **libstonithd:** add `exit_reason` member to `stonith_history_t`
+ **libstonithd:** deprecate `stonith_t:call_timeout`
+ **libstonithd:** `stonith_api_del_notification()` with NULL second argument
removes all notifications
# Pacemaker-2.1.2 (23 Nov 2021)
- 462 commits with 223 files changed, 16518 insertions(+), 11743 deletions(-)
## Features added since Pacemaker-2.1.1
+ **build:** when built with `--with-initdir,` Pacemaker uses the value to find
LSB resources (in addition to being where Pacemaker's own
init scripts are installed)
+ **build:** cmocka is new dependency for unit tests ("make check")
+ **rpm:** `fence_watchdog` now comes with pacemaker package (not pacemaker-cli)
+ **daemons:** metadata for cluster options supports OCF 1.1 standard
+ **executor:** nagios warning results now map to OCF "degraded" result code
+ **fencing:** `pcmk_delay_base` can optionally specify different delays per node
+ **fencing:** `pcmk_host_map` supports escaped characters such as spaces in values
+ **resources:** HealthIOWait agent supports OCF 1.1 standard, and validate works
+ **tools:** `crm_mon` shows exit reasons for actions failed due to internal errors
+ **tools:** `crm_mon` failed action display is more human-friendly by default
+ **tools:** `crm_resource` `--force-*` now outputs exit reasons if available
## Fixes since Pacemaker-2.1.1
+ **pkg-config:** return correct value for ocfdir *(regression introduced in 2.1.0)*
+ **tools:** fix `crm_mon` `--hide-headers` and related options
*(regression introduced in 2.0.4)*
+ **attrd:** check election status upon node loss to avoid election timeout
+ **controller:** improved handling of executor connection failures
+ **executor:** properly detect systemd unit existence
+ **pacemakerd:** recover properly from Corosync crash
+ **fencing:** fencing results are now sorted with sub-second resolution
+ **fencing:** fix `fence_watchdog` version output, metadata output, and man page
+ **fencing:** mark state as done if remapped "on" times out
+ **tools:** map LSB status to OCF correctly with `crm_resource` --force-check
## Public API changes since Pacemaker-2.1.1
+ **libcrmcommon:** deprecate `PCMK_OCF_EXEC_ERROR`
+ **libcrmcommon:** deprecate `PCMK_OCF_PENDING`
+ **libcrmcommon:** deprecate `PCMK_OCF_SIGNAL`
+ **libcrmcommon:** add `CRM_EX_DEGRADED` and `CRM_EX_DEGRADED_PROMOTED`
+ **libcrmcommon:** add enum `pcmk_exec_status`
+ **libcrmcommon:** add `PCMK_EXEC_MAX`
+ **libcrmcommon:** add `PCMK_EXEC_NO_FENCE_DEVICE`
+ **libcrmcommon:** add `PCMK_EXEC_NO_SECRETS`
+ **libcrmcommon:** add `pcmk_exec_status_str()`
+ **libcrmcommon:** add `pcmk_rc2ocf()`
+ **libcrmcommon:** deprecate `PCMK_OCF_TIMEOUT`
+ **libcrmservice:** add `services_result2ocf()`
+ **libcrmservice:** deprecate enum `op_status`
+ **libcrmservice:** deprecate `LSB_ROOT_DIR`
+ **libcrmservice:** deprecate `NAGIOS_NOT_INSTALLED`
+ **libcrmservice:** deprecate `NAGIOS_STATE_DEPENDENT`
+ **libcrmservice:** deprecate `services_get_ocf_exitcode()`
+ **libcrmservice:** deprecate `services_list()` and `services_action_create()`
+ **libcrmservice:** deprecate `services_lrm_status_str()`
+ **libpacemaker:** add enum `pcmk_sim_flags`
+ **libpacemaker:** add `pcmk_injections_t`
+ **libpacemaker:** add `pcmk_free_injections()`
+ **libpacemaker:** add `pcmk_simulate()`
+ **libstonithd:** add opaque member to `stonith_event_t`
+ **libstonithd:** add opaque member to `stonith_callback_data_t`
# Pacemaker-2.1.1 (09 Sep 2021)
- 231 commits with 102 files changed, 4912 insertions(+), 3428 deletions(-)
## Features added since Pacemaker-2.1.0
+ enhanced support for OCF Resource Agent API 1.1 standard
+ **agents:** ocf:pacemaker:attribute and ocf:pacemaker:ping agents now
support 1.1
+ **tools:** `crm_resource` passes output format to agents so they will use it if
supported
+ **tools:** `crm_resource` `--validate` and `--force-check` options accept optional
check level to pass to agent
+ **tools:** `crm_mon` XML output includes stonith-timeout-ms and
priority-fencing-delay-ms cluster properties
+ **pacemakerd:** support `--output-as/--output-to` options including XML output
with interactive options such as --features
## Fixes since Pacemaker-2.1.0
+ **pacemaker-attrd:** avoid repeated unfencing of remote nodes when DC
joined cluster after remote was up
+ **controller:** ensure newly joining node learns the node names of non-DCs
+ **controller:** ensure lost node's transient attributes are cleared without
DC
+ **scheduler:** avoid invalid transition when group member is unmanaged
(CLBZ#5423)
+ **scheduler:** don't schedule probes of unmanaged resources on pending
nodes
+ **executor:** avoid crash after TLS connection errors
*(regression introduced in 2.0.4)*
+ **fencing:** avoid repeated attempts to get (nonexistent) meta-data for
watchdog device *(regression introduced in 2.1.0)*
+ **fencing:** select correct device when `pcmk_host_check="dynamic-list"`
and `pcmk_host_map` are both specified (CLBZ#5474)
+ **tools:** `crm_attribute` supports node attribute values starting with a
'-' again *(regression introduced in 2.1.0)*
+ **tools:** `crm_attribute` deprecated `--get-value` option does not require
an argument *(regression introduced in 2.1.0)*
+ **tools:** avoid `crm_mon` memory leaks when filtering by resource or node
*(regressions introduced in 2.0.4 and 2.0.5)*
+ **tools:** exit with status 0 (not 64) for `--version` argument to
`crm_rule`, `crm_error`, `crm_diff`, and `crm_node` *(regression introduced
in 2.0.4)* and `crm_attribute` *(regression introduced in 2.1.0)*
+ **tools:** `crm_mon` should show active unmanaged resources on offline
nodes without requiring `-r/--inactive`
+ **tools:** better `crm_resource` error messages for unsupported resource
types
+ **tools:** `crm_simulate` `--show-failcounts` includes all relevant
information
+ **tools:** `crm_mon` should not show inactive group members without
`--inactive`
+ **tools:** `crm_mon` XML output should show members of cloned groups
+ **libcrmcommon:** correctly handle case-sensitive XML IDs
## Public API changes since Pacemaker-2.1.0
+ **libcrmcommon:** add `pcmk_section_e` type
+ **libcrmcommon:** add `pcmk_show_opt_e` type
+ **libcrmcommon:** add `pcmk_pacemakerd_api_shutdown()`
+ **libpe_status:** deprecate enum `pe_print_options`
# Pacemaker-2.1.0 (08 Jun 2021)
- 849 commits with 327 files changed, 22089 insertions(+), 12633 deletions(-)
## Features added since Pacemaker-2.0.5
+ support for OCF Resource Agent API 1.1 standard
- allow Promoted and Unpromoted role names in CIB (in addition to Master
and Slave, which are deprecated), and use new role names in output,
logs, and constraints created by `crm_resource --ban`
- advertise 1.1 support to agents, and provide `notify_promoted_*` and
`notify_unpromoted_*` environment variables to agents with notify actions
- support `reloadable` parameter attribute and `reload-agent` action in
agents that advertise 1.1 support
- support 1.1 standard in ocf:pacemaker:Dummy, ocf:pacemaker:remote, and
ocf:pacemaker:Stateful resource agents
- add "promoted-only" (in addition to "master-only", which is deprecated)
in `crm_mon` XML output for bans
+ support for noncritical resources
- colocation constraints accept an "influence" attribute that determines
whether dependent influences main resource's location (the default of
"true" preserves the previous behavior, while "false" makes the
dependent stop if it reaches its migration-threshold in failures rather
than cause both resources to move to another node)
- resources accept a "critical" meta-attribute that serves as default for
all colocation constraints involving the resource as the dependent, as
well as groups involving the resource
+ detail log uses millisecond-resolution timestamps when Pacemaker is built
with libqb 2.0 or later
+ **CIB:** deprecate the remove-after-stop cluster property, `can_fail` action
meta-attribute, and support for Upstart-based resources
+ **controller:** the `PCMK_panic_action` environment variable may be set to
sync-crash or sync-reboot to attempt to synchronize local
disks before crashing or rebooting, which can be helpful to
record cached log messages but runs the risk of the sync
hanging and leaving the host running after a critical error
+ **tools:** `CIB_file="-"` can be used to get the CIB from standard input
+ **tools:** crmadmin, `crm_resource`, `crm_simulate`, and `crm_verify` support
standard `--output-as/--output-to` options (including XML output,
intended for parsing by scripts and higher-level tools)
+ **tools:** `crm_attribute` accepts `-p/--promotion` option to operate on
promotion score (replacing `crm_master`, which is deprecated)
+ **tools:** `crm_resource` accepts `--promoted` option (replacing `--master,` which
is deprecated)
+ **tools:** `crm_resource` accepts `--digests` advanced option
+ **tools:** `crm_simulate` accepts `--show-attrs` and `--show-failcounts` options
- Build process changes since Pacemaker-2.0.5
+ Pacemaker requires newer versions of certain dependencies, including
Python 3.2 or later (support for Python 2 has been dropped), glib 2.32.0
or later, libqb 0.17.0 or later, GnuTLS 2.12.0 or later (to enable
Pacemaker Remote support), rpm 4.11.0 (if building RPMs), and a C library
that provides setenv() and unsetenv()
+ **configure:** `--enable-legacy-links` (which is deprecated) defaults to
"no", meaning that symbolic links will not be created for the Pacemaker 1
daemon names
+ **configure:** `--enable-compat-2.0` prevents certain output changes (most
significantly role names) to maintain compatibility with older tools,
scripts, and resource agents that rely on previous output
+ **configure:** `--with-resource-stickiness-default` sets a
`resource-stickiness` default in newly created CIBs
+ **configure:** `--with-concurrent-fencing-default` specifies default for
`concurrent-fencing` cluster property
+ **configure:** `--with-sbd-sync-default` specifies default for syncing
start-up with sbd
+ **configure:** `--with-ocfrapath` specifies resource agent directories to
search
+ **configure:** `--with-ocfrainstalldir` specifies where to install
ocf:pacemaker agents
+ **configure:** `--with-gnutls="no"` explicitly disables support for
Pacemaker Remote and the `remote-tls-port` cluster property
+ **configure:** `--with-acl` has been removed (support for ACLs is always
built)
+ **configure:** deprecated `--with-pkgname,` `--with-pkg-name`,
`--with-brand`, `--enable-ansi,` and `--enable-no-stack` options have been
removed
+ environment variables file (typically /etc/sysconfig/pacemaker or
/etc/default/pacemaker) will be installed when `make install` is run
+ documentation has dependency on python3-sphinx instead of publican, and is
generated beneath doc/sphinx
## Fixes since Pacemaker-2.0.5
+ **controller:** always refresh agent meta-data after start, in case agent was
updated *(regression introduced in 1.1.18)*
+ **tools:** avoid crash when running `crm_mon` in daemonized mode with CGI output
*(regression introduced in 2.0.3)*
+ **tools:** correctly treat unspecified node as all nodes instead of local node
when `attrd_updater` `--query` or `crm_resource` `--cleanup` is run on a
Pacemaker Remote node *(regressions introduced in 1.1.14 and 1.1.17)*
+ **tools:** exit with status 0 (not 64) for `--version` argument to `crm_simulate`
*(regression introduced in 2.0.4)* and `crm_resource` and crmadmin
*(regression introduced in 2.0.5)*
+ **pacemaker-attrd:** avoid race condition where transient attributes for a
leaving node could be reinstated when the node rejoins,
potentially causing a node that was just rebooted to exit
the cluster immediately after rejoining
+ **controller,scheduler:** fix year 2038 issues affecting shutdowns,
remote node fencing, last-rc-change, and
ticket last-granted dates
+ **controller:** retry scheduler connection after failure, to avoid cluster
stopping on a node without stopping resources (clbz#5445)
+ **fencing:** avoid pending fencing entry getting "stuck" in history if
originating node was unreachable when result was received
+ **fencing:** retry getting agent meta-data if initial attempt fails
+ **fencing:** detect when devices have been removed from configuration
+ **scheduler:** constrain clone-min, clone-max, clone-node-max, promoted-max,
and promoted-node-max options to non-negative values
+ **scheduler:** constrain resource priorities and node-health-base to score range
+ **scheduler:** treat invalid duration fields in time-based rules as 0, not -1
+ **scheduler:** node attribute rule expressions with a value-source of "param"
or "meta" work when rsc or rsc-pattern with an inverted match
is given, as well as when rsc-pattern with a regular
match is given
+ **scheduler:** node attribute rule expressions with a value-source of "param"
work with a resource parameter that itself is determined by a
node attribute rule expression
+ **scheduler:** avoid remote connection shutdown hanging when connection
history for node hosting connection is not last in CIB status
+ **scheduler:** route monitor cancellations behind moving remote connections correctly
+ **libcrmcommon:** avoid potential integer overflow when adding seconds to times
+ **tools:** cibsecret syncs to remote nodes and guest nodes as well as
cluster nodes
+ **tools:** show other available cluster information in `crm_mon` even if
fence history is not available
+ **tools:** retry failed fencer and CIB connections in `crm_mon`
+ **tools:** `crm_mon` reports if Pacemaker is waiting for sbd at start-up
+ **tools:** respect rules when showing node attributes in `crm_mon`
+ **tools:** improve messages when `crm_mon` is run on disconnected remote node
+ **tools:** constrain node IDs to non-negative values for `crm_node -N`
+ **tools:** `crm_node` -l on restarted node works even when Corosync 2 is used
without node names specified in Corosync configuration
+ **tools:** fix issues in calculation of non-sensitive resource parameter
digests that made `crm_simulate` wrongly think configuration changed
- C API changes since Pacemaker-2.0.5
+ **all:** new `PCMK_ALLOW_DEPRECATED` constant controls API availability
+ **libcrmcluster:** deprecate `crm_terminate_member()`
+ **libcrmcluster:** deprecate `crm_terminate_member_no_mainloop()`
+ **libcrmcommon:** add `CRMD_ACTION_RELOAD_AGENT` string constant
+ **libcrmcommon:** add `PCMK_OCF_MAJOR_VERSION` string constant
+ **libcrmcommon:** add `PCMK_OCF_MINOR_VERSION` string constant
+ **libcrmcommon:** add `PCMK_OCF_RUNNING_PROMOTED` enum value
+ **libcrmcommon:** add `PCMK_OCF_VERSION` string constant
+ **libcrmcommon:** add `PCMK_XE_PROMOTABLE_LEGACY` string constant
+ **libcrmcommon:** add `PCMK_XE_PROMOTED_MAX_LEGACY` string constant
+ **libcrmcommon:** add `PCMK_XE_PROMOTED_NODE_MAX_LEGACY` string constant
+ **libcrmcommon:** add enum `ocf_exitcode` (moved from libcrmservice)
+ **libcrmcommon:** deprecate `__builtin_expect()`
+ **libcrmcommon:** deprecate `__likely()`
+ **libcrmcommon:** deprecate `__unlikely()`
+ **libcrmcommon:** deprecate `crm_atoi()`
+ **libcrmcommon:** deprecate `crm_build_path()`
+ **libcrmcommon:** deprecate `crm_config_error` global variable
+ **libcrmcommon:** deprecate `crm_config_warning` global variable
+ **libcrmcommon:** deprecate `crm_ftoa()`
+ **libcrmcommon:** deprecate `crm_hash_table_size()`
+ **libcrmcommon:** deprecate `crm_itoa()`
+ **libcrmcommon:** deprecate `crm_itoa_stack()`
+ **libcrmcommon:** deprecate `crm_log_cli_init()`
+ **libcrmcommon:** deprecate `crm_parse_int()`
+ **libcrmcommon:** deprecate `crm_parse_ll()`
+ **libcrmcommon:** deprecate `crm_str_hash()`
+ **libcrmcommon:** deprecate `crm_str_table_dup()`
+ **libcrmcommon:** deprecate `crm_str_table_new()`
+ **libcrmcommon:** deprecate `crm_strcase_equal()`
+ **libcrmcommon:** deprecate `crm_strcase_hash()`
+ **libcrmcommon:** deprecate `crm_strcase_table_new()`
+ **libcrmcommon:** deprecate `crm_strip_trailing_newline()`
+ **libcrmcommon:** deprecate `crm_ttoa()`
+ **libcrmcommon:** deprecate `EOS` constant
+ **libcrmcommon:** deprecate `GListPtr` type
+ **libcrmcommon:** deprecate `g_str_hash_traditional()`
+ **libcrmcommon:** deprecate `MAX_IPC_DELAY` constant
+ **libcrmcommon:** deprecate `pcmk_format_named_time()`
+ **libcrmcommon:** deprecate `pcmk_format_nvpair()`
+ **libcrmcommon:** deprecate `pcmk_numeric_strcasecmp()`
+ **libcrmcommon:** deprecate `PCMK_OCF_DEGRADED_MASTER` enum value
+ **libcrmcommon:** deprecate `PCMK_OCF_FAILED_MASTER` enum value
+ **libcrmcommon:** deprecate `PCMK_OCF_RUNNING_MASTER` enum value
+ **libcrmcommon:** deprecate `pcmk_scan_nvpair()`
+ **libcrmcommon:** deprecate `XML_CIB_TAG_MASTER` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_MASTER_MAX` string constant
+ **libcrmcommon:** deprecate `XML_RSC_ATTR_MASTER_NODEMAX` string constant
+ **libcrmservice:** enum `ocf_exitcode` is obtained from libcrmcommon
+ **libpacemaker:** add `pcmk_controller_status()` function
+ **libpacemaker:** add `pcmk_designated_controller()` function
+ **libpacemaker:** add `pcmk_list_nodes()` function
+ **libpacemaker:** add `pcmk_pacemakerd_status()` function
+ **libpacemaker:** add `pcmk_resource_digests()` function
+ **libpe_status:** add `parameter_cache` member to `pe_resource_t`
+ **libpe_status:** add `pe_order_promoted_implies_first` enum value
+ **libpe_status:** add `pe_rsc_params()`
+ **libpe_status:** add `RSC_ROLE_PROMOTED` enum value
+ **libpe_status:** add `RSC_ROLE_PROMOTED_LEGACY_S` string constant
+ **libpe_status:** add `RSC_ROLE_PROMOTED_S` string constant
+ **libpe_status:** add `RSC_ROLE_UNPROMOTED` enum value
+ **libpe_status:** add `RSC_ROLE_UNPROMOTED_LEGACY_S` string constant
+ **libpe_status:** add `RSC_ROLE_UNPROMOTED_S` string constant
+ **libpe_status:** add priv member to `pcmk_working_set_t`, for Pacemaker use only
+ **libpe_status:** deprecate `pe_order_implies_first_master` enum value
+ **libpe_status:** deprecate `pe_print_details` enum value
+ **libpe_status:** deprecate `pe_print_dev` enum value
+ **libpe_status:** deprecate `pe_print_html` enum value
+ **libpe_status:** deprecate `pe_print_log` enum value
+ **libpe_status:** deprecate `pe_print_max_details` enum value
+ **libpe_status:** deprecate `pe_print_ncurses` enum value
+ **libpe_status:** deprecate `pe_print_xml` enum value
+ **libpe_status:** deprecate `pe_resource_t` parameters member
+ **libpe_status:** deprecate `RSC_ROLE_MASTER` enum value
+ **libpe_status:** deprecate `RSC_ROLE_MASTER_S` string constant
+ **libpe_status:** deprecate `RSC_ROLE_SLAVE` enum value
+ **libpe_status:** deprecate `RSC_ROLE_SLAVE_S` string constant
+ **libpe_status:** ignore `->parameter()` resource object function's create
argument
# Pacemaker-2.0.5 (02 Dec 2020)
- 534 commits with 286 files changed, 23133 insertions(+), 14626 deletions(-)
## Features added since Pacemaker-2.0.4
+ **configuration:** Add type="integer" to rule elements, allowing for
specifying 64-bit integers and specifying
double-precision floating point numbers when
type="number".
+ **daemons:** Recognize new OCF agent status codes 190 (degraded) and 191
(degraded master) to be treated as success but displayed as
errors.
+ **sbd-integration:** support `SBD_SYNC_RESOURCE_STARTUP` environment
variable to better synchronize Pacemaker start-up
and shutdown with SBD
+ **scheduler:** Add rule-based tests to `op_defaults` and `rsc_defaults.`
+ **scheduler:** Add on-fail=demote and no-quorum-policy=demote recovery
policies for promoted resources.
+ **tools:** Add `--resource=` to filter `crm_mon` output for a resource.
+ **tools:** Add -P to crmadmin to show pacemakerd status.
+ **tools:** In cibsecret, read value to set from input (or stdin) if not specified.
## Fixes for regressions introduced in Pacemaker-2.0.4
+ **tools:** Add the node name back to bundle instances in `crm_mon.`
+ **tools:** get `stonith_admin` `--list-targets` working again
## Fixes for regressions introduced in Pacemaker-2.0.3
+ **tools:** Fix adding HTTP headers to `crm_mon` in daemon mode.
+ **tools:** Show expected score of ping attributes in `crm_mon` XML output
## Fixes for regressions introduced in Pacemaker-2.0.1
+ **scheduler:** require pre-/post-start notifications correctly.
## Changes since Pacemaker-2.0.4
+ Prevent the bypassing of ACLs by direct IPC (CVE-2020-25654)
+ **build:** Fix a build issue on Fedora s390x.
+ **build:** Fix python2 vs. python3 packaging problems on openSUSE Tumbleweed
+ **build:** Update pkgconfig files for CI testing
+ **controller:** avoid recovery delay when shutdown locks expire
+ **controller:** Log correct timeout for timed-out stonith monitor
+ **fencer:** avoid infinite loop if device is removed during operation
+ **fencer:** avoid possible use-of-NULL when parsing metadata
+ **libfencing:** add `port` or `plug` parameter according to metadata on
`validate` if no `pcmk_host_argument` specified
+ **libfencing:** respect `pcmk_host_argument=none` on `validate`
+ **scheduler:** disallow on-fail=stop for stop operations
+ **scheduler:** don't schedule a dangling migration stop if one already occurred
+ **scheduler:** don't select instance to be promoted on a guest that can't run
+ **scheduler:** fix build when `DEFAULT_CONCURRENT_FENCING_TRUE` is set
+ **scheduler:** Remove `pe_print_expanded_xml` print option.
+ **scheduler:** Use `pcmk_monitor_timeout` as stonith start timeout
+ **tools:** Add management status to `crm_mon`'s group output.
+ **tools:** Add "No active resources" to one case in `crm_mon` output.
+ **tools:** Allow tags and alerts in `cibadmin --scope`
+ **tools:** Avoid `crm_node` on Pacemaker Remote nodes failing when other
executor actions are occurring.
+ **tools:** Avoid pending fence actions getting stuck in `crm_mon` display
+ **tools:** "Connectivity is lost" may not be displayed even if the ping
communication is lost.
+ **tools:** Display stop-all-resources in `crm_mon's` cluster options.
+ **tools:** don't use pssh -q option in cibsecret unless supported
+ **tools:** Fix adding the http-equiv header to `crm_mon` in daemon mode.
+ **tools:** If a clone or group is disabled, display that in `crm_mon` as part
of the resource's header.
+ **tools:** `crm_node` -l and -p now work from Pacemaker Remote nodes
+ **tools:** Don't display `crm_resource` error messages twice.
+ **tools:** Print inactive resources by default with `crm_mon` xml output.
+ **tools:** properly detect local node name in cibsecret
+ **tools:** Revert some `crm_resource` string-related checks on variables
that aren't strings
+ **tools:** Use bash in cibsecret
+ **xml:** Mark new `crm_mon` attributes as optional in schema
# Pacemaker-2.0.4 (15 Jun 2020)
- 515 commits with 269 files changed, 22560 insertions(+), 13726 deletions(-)
## Features added since Pacemaker-2.0.3
+ **build:** Add support for glib-based unit tests.
+ **controller:** add new feature 'shutdown-locks'
Optionally allow a gracefully shutdown node to have the resources
locked to it for a configurable time.
So that it can be restarted with exactly the same resources
running as before.
+ **controller/fencing/scheduler:** add new feature 'priority-fencing-delay'
Optionally derive the priority of a node from the resource-priorities
of the resources it is running.
In a fencing-race the node with the highest priority has a certain
advantage over the others as fencing requests for that node are
executed with an additional delay.
controlled via cluster option priority-fencing-delay (default = 0)
+ **tools:** `stonith_admin`: add `--delay` option (default = 0) to support enforced
fencing delay
+ **tools:** Add `--include=` and `--exclude=` to `crm_mon.`
+ **tools:** Add `--node=` to filter `crm_mon` output for a node (or tag).
+ **tools:** Allow more fine grained filtering of fence-history in `crm_mon.`
+ **tools:** Allow `crm_rule` to check some `date_specs.`
## Fixes for regressions introduced in Pacemaker-2.0.0
+ **tools:** ensure that getting the local node name using `crm_node`
always works from Pacemaker Remote node command line
## Changes since Pacemaker-2.0.3
+ **build:** improve checking headers for C++ fitness
+ **build:** restore build on latest OpenSUSE
+ **fencing:** Report an error getting history in output instead of empty history
+ **fencing:** Improve parameter handling for RHCS-style fence-agents
- make parameter `action` shown as not required
- add `port` or `plug` parameter according to metadata
- `plug` parameter shown as non-required in the metadata
+ **controller:** clear leaving node's transient attributes even if there is no DC
+ **controller:** don't ack same request twice
+ **iso8601:** Fix `crm_time_parse_offset()` to parse offset with plus sign.
+ **libcrmcommon, libpe:** validate interval specs correctly
+ **libcrmcommon:** Add `pcmk_str_is_infinity`, `pcmk_str_is_minus_infinity` and
`pcmk__unregister_formats.`
+ **libcrmcommon:** Extend what `pcmk__split_range` can do.
+ **libfencing:** Export formatted output functions.
+ **libpe_status:** Add node list arg to output messages preventing weird
behavior + potential segfaults
+ **libpe_status:** Update the maint mode message for HTML output.
+ **fencing, tools:** Fix arguments to stonith-event.
+ **scheduler:** don't incorporate dependencies' colocations multiple times
+ **scheduler:** ensure attenuated scores still have some effect
+ **scheduler:** ignore colocation constraints with 0 score
+ **scheduler:** make sure cluster-wide maintenance-mode=true overrides
per-resource settings
+ **scheduler:** properly detect whether guest node is fenceable
+ **scheduler:** do not differentiate case where all the nodes have equal priority
+ **tests:** Add tests for `pe_cron_range_satisfied.`
+ **tests:** Add tests for the current behavior of `pcmk__split_range.`
+ **tools, lib:** Use standard pacemaker return codes in `crm_rule.`
+ **tools:** Correct stonith-event arguments in `crm_mon.`
+ **tools:** Fix man pages for `crm_mon` & `crm_diff.`
+ **tools:** Make `crm_mon` `--interval` understand more formats.
+ **tools:** Fix `--html-title=` in `crm_mon.`
+ **tools:** Print errors to stderr for `crm_mon's` legacy xml.
+ **tools:** use return codes consistently in `stonith_admin`
+ **tools:** Use glib for cmdline handling in `crm_diff`, `crm_simulate` & `crm_error`
+ **xml:** Add a new version of the tags schema.
+ **based:** populate `cib_nodes` when `cibadmin -R --scope=configuration`
+ **cibsecret:** don't use pssh -q option unless supported
+ **fencing:** avoid infinite loop if device is removed during operation
+ **fencing:** avoid possible use-of-NULL when parsing metadata
+ **fencing:** Remove dangling 'pending' for completed DC fencing. (CLBZ#5401)
# Pacemaker-2.0.3 (25 Nov 2019)
- 601 commits with 227 files changed, 17862 insertions(+), 10471 deletions(-)
## Features added since Pacemaker-2.0.2
+ **controller:** new 'fence-reaction' cluster option specifies whether local node
should 'stop' or 'panic' if notified of own fencing
+ **controller:** more cluster properties support ISO 8601 time specifications
+ **controller:** calculate cluster recheck interval dynamically when possible
+ **Pacemaker Remote:** allow file for environment variables when used in bundle
+ **Pacemaker Remote:** allow configurable listen address and TLS priorities
+ **tools:** `crm_mon` now supports standard `--output-as/--output-to` options
+ **tools:** `crm_mon` HTML output supports user-defined CSS stylesheet
+ **tools:** `stonith_admin` supports HTML output in addition to text and XML
+ **tools:** `crm_simulate` supports `--repeat` option to repeat profiling tests
+ **tools:** new `pcmk_simtimes` tool compares `crm_simulate` profiling output
+ **agents:** SysInfo supports K, T, and P units in addition to Kb and G
## Changes since Pacemaker-2.0.2
+ **fencer:** do not block concurrent fencing actions on a device
*(regression since 2.0.2)*
+ **all:** avoid Year 2038 issues
+ **all:** allow ISO 8601 strings of form "<date>T<time> <offset>"
+ **rpm:** pacemaker-cts package now explicitly requires pacemaker-cli
+ **controller:** set timeout on scheduler responses to avoid infinite wait
+ **controller:** confirm cancel of failed monitors, to avoid transition timeout
+ **executor:** let controller cancel monitors, to avoid transition timeout
+ **executor:** return error for stonith probes if stonith connection was lost
+ **fencer:** ensure concurrent fencing commands always get triggered to execute
+ **fencer:** fail pending actions and re-sync history after crash and restart
+ **fencer:** don't let command with long delay block other pending commands
+ **fencer:** allow functioning even if CIB updates arrive unceasingly
+ **scheduler:** wait for probe actions to complete to prevent unnecessary
restart/re-promote of dependent resources
+ **scheduler:** avoid invalid transition when guest node host is not fenceable
+ **scheduler:** properly detect dangling migrations, to avoid restart loop
+ **scheduler:** avoid scheduling actions on remote node that is shutting down
+ **scheduler:** avoid delay in recovery of failed remote connections
+ **scheduler:** clarify action failure log messages by including failure time
+ **scheduler:** calculate secure digests for unfencing, for replaying saved CIBs
+ **libcrmcommon:** avoid possible use-of-NULL when applying XML diffs
+ **libcrmcommon:** correctly apply XML diffs with multiple move/create changes
+ **libcrmcommon:** return error when applying XML diffs with unknown operations
+ **tools:** avoid duplicate lines between nodes in `crm_simulate` dot graph
+ **tools:** count disabled/blocked resources correctly in `crm_mon/crm_simulate`
+ **tools:** `crm_mon` `--interval` now accepts ISO 8601 and has correct help
+ **tools:** organize `crm_mon` text output with list headings, indents, bullets
+ **tools:** `crm_report:` fail if tar is not available
+ **tools:** `crm_report:` correct argument parsing
+ **tools:** `crm_report:` don't ignore log if unrelated file is too large
+ **tools:** `stonith_admin` `--list-targets` should show what fencer would use
+ **agents:** calculate #health_disk correctly in SysInfo
+ **agents:** handle run-as-user properly in ClusterMon
# Pacemaker-2.0.2 (04 Jun 2019)
- 288 commits with 225 files changed, 28494 insertions(+), 24465 deletions(-)
## Features added since Pacemaker-2.0.1
+ **tools:** `crm_resource` `--validate` can get resource parameters from command line
+ **tools:** `crm_resource` `--clear` prints out any cleared constraints
+ **tools:** new `crm_rule` tool for checking rule expiration (experimental)
+ **tools:** `stonith_admin` supports XML output for machine parsing (experimental)
+ **resources:** new HealthIOWait resource agent for node health tracking
## Changes since Pacemaker-2.0.1
+ Important security fixes for CVE-2018-16878, CVE-2018-16877, CVE-2019-3885
+ **build:** `crm_report` bug report URL is now configurable at build time
+ **build:** private libpengine/libtransitioner libraries combined as libpacemaker
+ **controller:** avoid memory leak when duplicate monitor is scheduled
+ **scheduler:** respect order constraints when resources are being probed
+ **scheduler:** one group stop shouldn't make another required
+ **libcrmcommon:** handle out-of-range integers in configuration better
+ **libcrmcommon:** export logfile environment variable if using default
+ **libcrmcommon:** avoid segmentation fault when beginning formatted text list
+ **libcrmservice:** fix use-after-free memory error in alert handling
+ **libstonithd:** handle more than 64KB output from fence agents
# Pacemaker-2.0.1 (4 Mar 2019)
- 592 commits with 173 files changed, 9268 insertions(+), 5344 deletions(-)
## Features added since Pacemaker-2.0.0
+ Pacemaker bundles support podman for container management
+ **fencing:** SBD may be used in a cluster that has guest nodes or bundles
+ **fencing:** fencing history is synchronized among all nodes
+ **fencing:** `stonith_admin` has option to clear fence history
+ **tools:** `crm_mon` can show fencing action failures and history
+ **tools:** `crm_resource --clear` supports new `--expired` option
+ **Pacemaker Remote:** new options to restrict TLS Diffie-Hellman prime length
## Changes since Pacemaker-2.0.0
+ **scheduler:** clone notifications could be scheduled for a stopped
Pacemaker Remote node and block all further cluster actions
*(regression since 2.0.0)*
+ **libcrmcommon:** correct behavior for completing interrupted live migrations
*(regression since 2.0.0)*
+ **tools:** `crm_resource -C` could fail to clean up all failures in one run
*(regression since 2.0.0)*
+ **Pacemaker Remote:** avoid unnecessary downtime when moving resource to
Pacemaker Remote node that fails to come up *(regression since 1.1.18)*
+ **tools:** restore `stonith_admin` ability to confirm unseen nodes are down
*(regression since 1.1.12)*
+ **build:** minor logging fixes to allow compatibility with GCC 9 -Werror
+ **build:** spec file now puts XML schemas in new pacemaker-schemas package
+ **build:** spec file now provides virtual pcmk-cluster-manager package
+ **pacemaker-attrd:** wait a short time before re-attempting failed writes
+ **pacemaker-attrd:** ignore attribute delays when writing after node (re-)join
+ **pacemaker-attrd:** start new election immediately if writer is lost
+ **pacemaker-attrd:** clear election dampening when the writer leaves
+ **pacemaker-attrd:** detect alert configuration changes when CIB is replaced
+ **CIB:** inform originator of CIB upgrade failure
+ **controller:** support resource agents that require node name even for meta-data
+ **controller:** don't record pending clone notifications in CIB
+ **controller:** DC detects completion of another node's shutdown more accurately
+ **controller:** shut down DC if unable to update node attributes
+ **controller:** handle corosync peer/join notifications for new node in any order
+ **controller:** clear election dampening when DC is lost
+ **executor:** cancel recurring monitors if fence device registration is lost
+ **fencing:** check for fence device update when resource defaults change
+ **fencing:** avoid pacemaker-fenced crash possible with `stonith_admin` misuse
+ **fencing:** limit fencing history to 500 entries
+ **fencing:** `stonith_admin` now complains if no action option is specified
+ **pacemakerd:** do not modify kernel.sysrq on Linux
+ **scheduler:** regression test compatibility with glib 2.59.0
+ **scheduler:** avoid unnecessary recovery of cleaned guest nodes and bundles
+ **scheduler:** ensure failures causing fencing not expired until fencing done
+ **scheduler:** start unique clone instances in numerical order
+ **scheduler:** convert unique clones to anonymous clones when not supported
+ **scheduler:** associate pending tasks with correct clone instance
+ **scheduler:** ensure bundle clone notifications are directed to correct host
+ **scheduler:** avoid improper bundle monitor rescheduling or fail count clearing
+ **scheduler:** honor asymmetric orderings even when restarting
+ **scheduler:** don't order non-DC shutdowns before DC fencing
+ **ACLs:** assume unprivileged ACL user if can't get user info
+ **Pacemaker Remote:** get Diffie-Hellman prime bit length from GnuTLS API
+ **libcrmservice:** cancel DBus call when cancelling systemd/upstart actions
+ **libcrmservice:** order systemd resources relative to `pacemaker_remote`
+ **libpe_status:** add public API constructor/destructor for `pe_working_set_t`
+ **tools:** fix `crm_resource` `--clear` when lifetime was used with ban/move
+ **tools:** fix `crm_resource` `--move` when lifetime was used with previous move
+ **tools:** make `crm_mon` CIB connection errors non-fatal if previously successful
+ **tools:** improve `crm_mon` messages when generating HTML output
+ **tools:** `crm_mon` cluster connection failure is now "critical" in nagios mode
+ **tools:** `crm_mon` listing of standby nodes shows if they have active resources
+ **tools:** `crm_diff` now ignores attribute ordering when comparing in CIB mode
+ **tools:** improve `crm_report` detection of logs, CIB directory, and processes
+ **tools:** `crm_verify` returns reliable exit codes
+ **tools:** `crm_simulate` resource history uses same name as live cluster would
# Pacemaker-2.0.0 (6 Jul 2018)
- 885 commits with 549 files changed, 89865 insertions(+), 95100 deletions(-)
- Deprecated features removed since Pacemaker-1.1.18
+ All of these have newer forms, and the cluster will automatically convert
most older syntax usage in saved configurations to newer syntax as needed
+ Drop support for heartbeat and corosync 1 (whether using CMAN or plugin)
+ Drop support for rolling upgrades from Pacemaker versions older than 1.1.11
+ Drop support for built-in SMTP and SNMP in `crm_mon`
+ Drop support for legacy option aliases including default-action-timeout,
default-resource-stickiness, resource-failure-stickiness,
default-resource-failure-stickiness, is-managed-default,
and all names using underbar instead of dash
+ Drop support for "requires" operation meta-attribute
+ Drop support for the `pcmk_*_cmd`, `pcmk_arg_map`, and `pcmk_poweroff_action`
fence resource parameters
+ Drop support for deprecated command-line options to crmadmin,
`crm_attribute`, `crm_resource`, `crm_verify`, `crm_mon`, and `stonith_admin`
+ Drop support for operation meta-attributes in `instance_attributes`
+ Drop support for `PCMK_legacy` and `LRMD_MAX_CHILDREN` environment variables
+ Drop support for undocumented resource isolation feature
+ Drop support for processing very old saved CIB files (including
pre-0.6.0 start failure entries, pre-0.6.5 operation history entries,
pre-0.7 transition keys, pre-1.1.4 migration history entries,
pre-1.0 XML configuration schemas, pre-1.1.6 ticket state entries, and
pre-1.1.7 failed recurring operation history entries)
## Features added since Pacemaker-1.1.18
+ The pacemaker daemons have been renamed to make logs more intuitive
and easier to search
+ The default location of the Pacemaker detail log is now
/var/log/pacemaker/pacemaker.log (instead of being directly in /var/log),
and Pacemaker will no longer use Corosync's logging preferences;
configure script options are available to change default log locations
+ The detail log's message format has been improved
+ The master XML tag is deprecated in favor of using a standard clone tag
with a new "promotable" meta-attribute set to true, and the "master-max"
and "master-node-max" master meta-attributes are deprecated in favor of
new "promoted-max" and "promoted-node-max" clone meta-attributes;
documentation now refers to these as promotable clones rather than
master/slave, stateful, or multistate clones, and refers to
promotion scores instead of master scores
+ Administration-related documentation has been moved from the
"Pacemaker Explained" document to a new "Pacemaker Administration" document
+ record-pending now defaults to TRUE (pending actions are shown in status)
+ All Python code in Pacemaker now supports both Python 2.7 and Python 3
+ The command-line tools now return consistent, well-defined exit codes;
`crm_error` has an `--exit` option to list these
+ Pacemaker's systemd unit files now remove systemd's spawned process limit
+ mount, path, and timer systemd unit types are now supported as resources
+ A negative stonith-watchdog-timeout now tells the cluster to automatically
calculate the value based on `SBD_WATCHDOG_TIMEOUT` (which was the behavior
of 0 before 1.1.15; 0 retains its post-1.1.15 behavior of disabling use of
the watchdog as a fencing device)
- + The undocumented restart-type and `role_after_failure`
- resource meta-attributes are now deprecated
+ + The undocumented `restart-type` resource option and `role_after_failure`
+ operation option are now deprecated
+ Regression testing code has been consolidated and overhauled
(the most obvious change is new command names)
+ **build:** create /etc/pacemaker directory when installing
+ **build:** improved portability to BSD-based platforms
+ **tools:** `crm_resource` `--cleanup` now cleans only failed operation history;
`crm_resource` `--reprobe` retains the previous behavior of cleaning all
operation history
+ **tools:** add `stonith_admin` `--validate` option to check device configuration
+ **tools:** `crm_node` is now in the pacemaker-cli package (instead of pacemaker)
+ **alerts:** add epoch and usec alert variables for improved SNMP alerts
+ **controller:** deprecate "crmd-*" cluster options in favor of new names
+ **scheduler:** deprecate stonith-action value "poweroff" (use "off" instead)
+ **scheduler:** deprecate require-all in `rsc_order`
+ **libcrmcluster:** prefer corosync name over `ring0_addr`
+ **xml:** allow local "kind" in `resource_set` within `rsc_order`
## Changes since Pacemaker-1.1.18
+ Restore systemd unit dependency on DBus *(regression in 1.1.17)*
+ **CIB:** handle mixed-case node names when modifying attributes *(regression in 1.1.17)*
+ **scheduler:** avoid crash when logging ignored failure timeout *(regression in 1.1.17)*
+ **attrd:** ensure node name is broadcast at start-up *(regression in 1.1.18)*
+ **scheduler:** unfence before probing or starting fence devices *(regression in 1.1.18)*
+ **tools:** treat INFINITY correctly in `crm_failcount` *(regression in 1.1.17)*
+ **tools:** show master scores with `crm_simulate -sL` *(regression in 1.1.18)*
+ **tools:** `crm_master` did not work without explicit `--lifetime` *(regression in 1.1.18)*
+ Numerous changes to public C API of libraries
+ Choose current node correctly when a resource is multiply active
+ **controller,executor,tools:** avoid minor memory leaks
+ **CIB:** don't use empty CIB if real CIB has bad permissions
+ **controller:** avoid double free after ACL rejection of resource deletion
+ **controller:** don't record pending clone notifications in CIB
+ **controller:** always write faked failures to CIB whenever possible
+ **controller:** quorum gain without a node join should cause new transition
+ **executor:** handle systemd actions correctly when used with "service:"
+ **executor:** find absolute LSB paths when used with "service:"
+ **scheduler:** handle "requires" of "quorum" or "nothing" properly
+ **scheduler:** ensure orphaned recurring monitors have interval set
+ **scheduler:** handle pending migrations correctly when record-pending is true
+ **scheduler:** don't time out failures that cause fencing until fencing completes
+ **scheduler:** handle globally-unique bundle children correctly
+ **scheduler:** use correct default timeout for monitors
+ **scheduler:** "symmetrical" defaults to "false" for serialize orders
+ **scheduler:** avoid potential use-of-NULL when unpacking ordering constraint
+ **scheduler:** properly cancel recurring monitors
+ **scheduler:** do not schedule notifications for unrunnable actions
+ **scheduler:** ensure stops occur after stopped remote connections come back up
+ **scheduler:** consider only allowed nodes when ordering start after all recovery
+ **scheduler:** avoid graph loop from ordering bundle child stops/demotes after container fencing
+ **scheduler:** remote connection resources are safe to require only quorum
+ **scheduler:** correctly observe colocation with bundles in Master role
+ **scheduler:** restart resource after failed demote when appropriate
+ **Pacemaker Remote:** always use most recent remote proxy
+ **tools:** `crm_node` now gets correct node name and ID on Pacemaker Remote nodes
+ **tools:** correctly check `crm_resource` `--move` for master role
+ **tools:** cibsecret `--help/--version` doesn't require cluster to be running
+ **tools:** ignore attribute placement when `crm_diff` compares in cib mode
+ **tools:** prevent notify actions from causing `crm_resource` `--wait` to hang
+ **resources:** drop broken configdir parameter from ocf:pacemaker:controld
- **For further details, see:**
https://projects.clusterlabs.org/w/projects/pacemaker/pacemaker_2.0_changes/
# Pacemaker-1.1.18 (14 Nov 2017)
- 644 commits with 167 files changed, 9753 insertions(+), 5596 deletions(-)
## Features added since Pacemaker-1.1.17
+ warnings are now logged when using legacy syntax to be removed in 2.0
+ **agents:** ifspeed agent is now installed when building
+ **agents:** ifspeed agent can optionally detect interface name from IP address
+ **alerts:** support alert filters
+ **alerts:** experimental support for alerts for node attribute changes
+ **crmd,pengine:** support unfencing of remote nodes
+ **pengine:** bundles now support all constraint types
+ **pengine:** bundles now support rkt containers
+ **pengine:** bundles support new container-attribute-target parameter
+ **pengine,tools:** logs and `crm_mon` show why resources changed state
+ **stonith-ng:** support new fencing resource parameter `pcmk_delay_base`
+ **tools:** new `crm_resource` option `--why` explains why resources are stopped
## Changes since Pacemaker-1.1.17
+ many documentation improvements
+ **agents:** ifspeed properly calculates speed of hfi1 interfaces
+ **agents:** ClusterMon now interprets "update" less than 1000 as seconds
+ **attrd:** don't lose attributes set between attrd start-up and cluster join
+ **attrd:** fix multiple minor memory leaks
+ **crmd:** correctly record that unfencing is complete
+ **crmd:** error more quickly if remote start fails due to missing key
+ **lrmd:** remote resource operations return immediate error if key setup fails
+ **lrmd:** allow pre-1.1.15 cluster nodes to connect to current Pacemaker Remote
+ **pengine:** guest nodes are now probed like other nodes
+ **pengine:** probe remote nodes for guest node resources
+ **pengine:** do not probe guest/bundle connections until guest/bundle is active
+ **pengine:** allow resources to stop prior to probes completing
+ **pengine:** bundles wait only for other containers on same node to be probed
+ **pengine:** have bundles log to stderr so 'docker logs'/'journalctl -M' works
+ **pengine:** only pass requests for promote/demote flags onto the bundle's child
+ **pengine:** do not map ports into Docker container when net=host is specified
+ **pengine:** allow resources inside bundles to receive clone notifications
+ **pengine:** default to non-interleaved bundle ordering for safety
+ **pengine:** ensure bundle nodes and child resources are correctly cleaned up
+ **pengine:** prevent graph loops when fencing the host underneath a bundle
+ **pengine:** fix multiple memory issues (use-after-free, use-of-NULL) with bundles
+ **pengine:** resources in bundles respect failcounts
+ **pengine:** ensure nested container connections run on the same host
+ **pengine:** ensure unrecoverable remote nodes are fenced even with no resources
+ **pengine:** handle resource migrating behind a migrating remote connection
+ **pengine:** don't prefer to keep unique instances on same node
+ **pengine:** exclude exclusive resources and nodes from symmetric default score
+ **pengine:** if ignoring failure, also ignore migration-threshold
+ **pengine:** restore the ability to send the transition graph via the disk if it gets too big
+ **pengine:** validate no-quorum-policy=suicide correctly
+ **pengine:** avoid crash when alerts section has comments
+ **pengine:** detect permanent master scores at start-up
+ **pengine:** do not re-add a node's default score for each location constraint
+ **pengine:** make sure calculated resource scores are consistent on different architectures
+ **pengine:** retrigger unfencing for changed device parameters only when necessary
+ **pengine:** don't schedule reload and restart in same transition (CLBZ#5309, regression introduced in 1.1.15)
+ **stonith-ng:** make fencing-device reappear properly after reenabling
+ **stonith-ng:** include `pcmk_on_action` in meta-data so 'on' can be overridden
+ **tools:** allow `crm_report` to work with no log files specified
+ **tools:** fix use-after-free in `crm_diff` introduced in 1.1.17
+ **tools:** allow `crm_resource` to operate on anonymous clones in unknown states
+ **tools:** `crm_resource --cleanup` on appropriate nodes if we don't know state of resource
+ **tools:** prevent disconnection from crmd during `crm_resource --cleanup`
+ **tools:** improve messages for `crm_resource` `--force-*` options
+ **tools:** `crm_mon:` avoid infinite process spawning if -E script can't be run
+ **tools:** `crm_mon:` don't show previous exit-reason for failed action with none
+ **libcrmservice:** list systemd unit files, not only active units (CLBZ#5299)
+ **libcrmservice:** parse long description correctly for LSB meta-data
# Pacemaker-1.1.17 (06 Jul 2017)
- 539 commits with 177 files changed, 11525 insertions(+), 5036 deletions(-)
## Features added since Pacemaker-1.1.16
+ New `bundle` resource type for Docker container use cases (experimental)
+ New `PCMK_node_start_state` environment variable to start node in standby
+ New `value-source` rule expression attribute in location constraints to
compare a node attribute against a resource parameter
+ New `stonith-max-attempts` cluster option to specify how many times
fencing can fail for a target before the cluster will no longer
immediately re-attempt it (previously hard-coded at 10)
+ New `cluster-ipc-limit` cluster option to avoid IPC client eviction in
large clusters
+ Failures are now tracked per operation type, as well as per node and
resource (the `fail-count` and `last-failure` node attribute names now end
in `#OPERATION_INTERVAL`)
+ **attrd:** Pacemaker Remote node attributes and regular expressions are now
supported on legacy cluster stacks (heartbeat, CMAN, and corosync plugin)
+ **tools:** New `crm_resource --validate` option
+ **tools:** New `stonith_admin --list-targets` option
+ **tools:** New `crm_attribute --pattern` option to match a regular
expression
+ **tools:** `crm_resource --cleanup` and `crm_failcount` can now take
`--operation` and `--interval` options to operate on a single operation
type
## Changes since Pacemaker-1.1.16
+ Fix multiple memory issues (leaks, use-after-free) in libraries
+ **pengine:** unmanaging a guest node resource puts guest in maintenance mode
+ **cib:** broadcasts of cib changes should always pass ACL checks
+ **crmd,libcrmcommon:** update throttling when CPUs are hot-plugged
+ **crmd:** abort transition whenever we lose quorum
+ **crmd:** avoid attribute write-out on join when atomic attrd is used
+ **crmd:** check for too many stonith failures only when aborting for that reason
+ **crmd:** correctly clear failure counts only for a specified node
+ **crmd:** don't fence old DC if it's shutting down as soon-to-be DC joins
+ **crmd:** forget stonith failures when forgetting node
+ **crmd:** all nodes should track stonith failure counts in case they become DC
+ **crmd:** update cache status for guest node whose host is fenced
+ **dbus:** prevent lrmd from hanging on dbus calls
+ **fencing:** detect newly added constraints for stonith devices
+ **pengine:** order remote actions after connection recovery
*(regression introduced in 1.1.15)*
+ **pengine:** quicker recovery from failed demote
+ **libcib:** determine remote nodes correctly from node status entries
+ **libcrmcommon:** avoid evicting IPC client if messages spike briefly
+ **libcrmcommon:** better XML comment handling prevents infinite election loop
+ **libcrmcommon:** set month correctly in date/time string sent to alert agents
+ **libfencing,fencing:** intelligently remap "action" wrongly specified in config
+ **libservices:** ensure completed ops aren't on blocked ops list
+ **libservices:** properly detect and cancel in-flight systemd/upstart ops
+ **libservices:** properly watch writable DBus handles
+ **libservices:** systemd service that is reloading doesn't cause monitor failure
+ **pacemaker_remoted:** allow graceful shutdown while unmanaged
+ **pengine,libpe_status:** don't clear same fail-count twice
+ **pengine:** consider guest node unclean if its host is unclean
+ **pengine:** do not re-add a node's default score for each location constraint
+ **pengine:** avoid restarting services when recovering remote connection
+ **pengine:** better guest node recovery when host fails
+ **pengine:** guest node fencing doesn't require stonith enabled
+ **pengine:** allow probes of guest node connection resources
+ **pengine:** properly handle allow-migrate explicitly set for remote connection
+ **pengine:** fence failed remote nodes even if no resources can run on them
+ **tools:** resource agents will now get the correct node name on
Pacemaker Remote nodes when using `crm_node` and `crm_attribute`
+ **tools:** avoid grep crashes in `crm_report` when looking for system logs
+ **tools:** `crm_resource -C` now clears `last-failure` as well as
`fail-count`
+ **tools:** implement `crm_failcount` command-line options correctly
+ **tools:** properly ignore version with `crm_diff --no-version`
# Pacemaker-1.1.16 (30 Nov 2016)
- 382 commits with 145 files changed, 7200 insertions(+), 5621 deletions(-)
## Features added since Pacemaker-1.1.15
+ Location constraints may use rsc-pattern, with submatches expanded
+ node-health-base available with node-health-strategy=progressive
+ new Pacemaker Development document for working on pacemaker code base
+ new `PCMK_panic_action` variable allows crash instead of reboot on panic
+ **resources:** add resource agent for managing a node attribute
+ **systemd:** include socket units when listing all systemd agents
## Changes since Pacemaker-1.1.15
+ Important security fix for CVE-2016-7035
+ Logging is now synchronous when blackboxes are enabled
+ All python code except CTS is now compatible with python 2.6+ and 3.2+
+ **build:** take advantage of compiler features for security and performance
+ **build:** update SuSE spec modifications for recent spec changes
+ **build:** avoid watchdog reboot when upgrading `pacemaker_remote` with sbd
+ **build:** numerous other improvements in environment detection, etc.
+ **cib:** fix infinite loop when no schema validates
+ **crmd:** cl#5185 - record pending operations in CIB before they are performed
+ **crmd:** don't abort transitions for CIB comment changes
+ **crmd:** resend shutdown request if DC loses original request
+ **documentation:** install improved README in doc instead of now-removed AUTHORS
+ **documentation:** clarify licensing and provide copy of all licenses
+ **documentation:** document various features and upgrades better
+ **fence_legacy:** use "list" action when searching cluster-glue agents
+ **libcib:** don't stop sending alerts after releasing DC role
+ **libcrmcommon:** properly handle XML comments when comparing v2 patchset diffs
+ **libcrmcommon:** report errors consistently when waiting for data on connection
+ **libpengine:** avoid potential use-of-NULL
+ **libservices:** use DBusError API properly
+ **pacemaker_remote:** init script stop should always return 0
+ **pacemaker_remote:** allow remote clients to timeout/reconnect
+ **pacemaker_remote:** correctly calculate remaining timeout when receiving messages
+ **pengine:** avoid transition loop for start-then-stop + unfencing
+ **pengine:** correctly update dependent actions of un-runnable clones
+ **pengine:** do not fence a node in maintenance mode if it shuts down cleanly
+ **pengine:** set `OCF_RESKEY_CRM_meta_notify_active_*` for multistate resources
+ **resources:** ping - avoid temporary files for fping check, support FreeBSD
+ **resources:** SysInfo - better support for FreeBSD
+ **resources:** variable name typo in docker-wrapper
+ **systemd:** order pacemaker after time-sync target
+ **tools:** correct `attrd_updater` help and error messages when using CMAN
+ **tools:** `crm_standby` `--version/--help` should work without cluster running
+ **tools:** make `crm_report` sanitize CIB before generating readable version
+ **tools:** display pending resource state by default when available
+ **tools:** avoid matching other process with same PID in ClusterMon
# Pacemaker-1.1.15 (21 Jun 2016)
- 533 commits with 219 files changed, 6659 insertions(+), 3989 deletions(-)
## Features added since Pacemaker-1.1.14
+ Event-driven alerts allow scripts to be called after significant events
+ **build:** Some files moved from pacemaker package to pacemaker-cli for cleaner pacemaker-remote dependencies
+ **build:** ./configure `--with-configdir` argument for /etc/sysconfig, /etc/default, etc.
+ **fencing:** Simplify watchdog integration
+ **fencing:** Support concurrent fencing actions via new `pcmk_action_limit` option
+ **remote:** `pacemaker_remote` may be stopped without disabling resource first
+ **remote:** Report integration status of Pacemaker Remote nodes in CIB `node_state`
+ **tools:** `crm_mon` now reports why resources are not starting
+ **tools:** `crm_report` now obscures passwords in logfiles
+ **tools:** `attrd_updater --update-both/--update-delay` options allow
changing dampening value
+ **tools:** allow `stonith_admin -H '*'` to show history for all nodes
## Changes since Pacemaker-1.1.14
+ Fix multiple memory issues (leaks, use-after-free) in daemons, libraries and tools
+ Make various log messages more user-friendly
+ Improve FreeBSD and Hurd support
+ **attrd:** Prevent possible segfault on exit
+ **cib:** Fix regression to restore support for compressed CIB larger than 1MB
+ **common:** fix regression in 1.1.14 that made have-watchdog always true
+ **controld:** handle DLM "wait fencing" state better
+ **crmd:** Fix regression so that fenced unseen nodes do not remain unclean
+ **crmd:** Take start-delay into account when calculation action timeouts
+ **crmd:** Avoid timeout on older peers when cancelling a resource operation
+ **fencing:** Allow fencing by node ID (e.g. by DLM) even if node left cluster
+ **lrmd:** Fix potential issues when cluster is stopped via systemd shutdown
+ **pacemakerd:** Properly respawn stonithd if it fails
+ **pengine:** Fix regression with multiple monitor levels that could ignore failure
+ **pengine:** Correctly set `OCF_RESKEY_CRM_meta_timeout` when start-delay is configured
+ **pengine:** Properly order actions for master/slave resources in anti-colocations
+ **pengine:** Respect asymmetrical ordering when trying to move resources
+ **pengine:** Properly order stop actions on guest node relative to host stonith
+ **pengine:** Correctly block actions dependent on unrunnable clones
+ **remote:** Allow remote nodes to have node attributes even with legacy attrd
+ **remote:** Recover from remote node fencing more quickly
+ **remote:** Place resources on newly rejoined remote nodes more quickly
+ **resources:** ping agent can now use fping6 for IPv6 hosts
+ **resources:** SysInfo now resets #health_disk to green when there's sufficient free disk
+ **tools:** `crm_report` is now more efficient and handles Pacemaker Remote nodes better
+ **tools:** Prevent `crm_resource` segfault when `--resource` is not supplied with --restart
+ **tools:** `crm_shadow` `--display` option now works
+ **tools:** `crm_resource` `--restart` handles groups, target-roles and moving resources better
# Pacemaker-1.1.14 (14 Jan 2016)
- 724 commits with 179 files changed, 13142 insertions(+), 7695 deletions(-)
## Features added since Pacemaker-1.1.13
+ **crm_resource:** Indicate common reasons why a resource may not start after a cleanup
+ **crm_resource:** New `--force-promote` and `--force-demote` options for debugging
+ **fencing:** Support targeting fencing topologies by node name pattern or node attribute
+ **fencing:** Remap sequential topology reboots to all-off-then-all-on
+ **pengine:** Allow resources to start and stop as soon as their state is known on all nodes
+ **pengine:** Include a list of all and available nodes with clone notifications
+ **pengine:** Addition of the clone resource clone-min metadata option
+ **pengine:** Support of multiple-active=block for resource groups
+ **remote:** Resources that create guest nodes can be included in a group resource
+ **remote:** `reconnect_interval` option for remote nodes to delay reconnect after fence
## Changes since Pacemaker-1.1.13
+ improve support for building on FreeBSD and Debian
+ fix multiple memory issues (leaks, use-after-free, double free, use-of-NULL) in components and tools
+ **cib:** Do not terminate due to badly behaving clients
+ **cman:** handle corosync-invented node names of the form Node{id} for peers not in its node list
+ **controld:** replace bashism
+ **crm_node:** Display node state with -l and quorum status with -q, if available
+ **crmd:** resources would sometimes be restarted when only non-unique parameters changed
+ **crmd:** fence remote node after connection failure only once
+ **crmd:** handle resources named the same as cluster nodes
+ **crmd:** Pre-emptively fail in-flight actions when lrmd connections fail
+ **crmd:** Record actions in the CIB as failed if we cannot execute them
+ **crm_report:** Enable password sanitizing by default
+ **crm_report:** Allow log file discovery to be disabled
+ **crm_resource:** Allow the resource configuration to be modified for
`--force-{check,start,..}` calls
+ **crm_resource:** Compensate for -C and -p being called with the child resource for clones
+ **crm_resource:** Correctly clean up all children for anonymous cloned groups
+ **crm_resource:** Correctly clean up failcounts for inactive anonymous clones
+ **crm_resource:** Correctly observe `--force` when deleting and updating attributes
+ **crm_shadow:** Fix `crm_shadow --diff`
+ **crm_simulate:** Prevent segfault on arches with 64bit `time_t`
+ **fencing:** ensure `required`/`automatic` apply only to `on` actions
+ **fencing:** Return a provider for the internal fencing agent `#watchdog`
instead of logging an error
+ **fencing:** ignore stderr output of fence agents (often used for debug
messages)
+ **fencing:** fix issue where deleting a fence device attribute can delete
the device
+ **libcib:** potential user input overflow
+ **libcluster:** overhaul peer cache management
+ **log:** make syslog less noisy
+ **log:** fix various misspellings in log messages
+ **lrmd:** cancel currently pending STONITH op if stonithd connection is
lost
+ **lrmd:** Finalize all pending and recurring operations when cleaning up a
resource
+ **pengine:** Bug cl#5247 - Imply resources running on a container are
stopped when the container is stopped
+ **pengine:** cl#5235 - Prevent graph loops that can be introduced by
`load_stopped -> migrate_to` ordering
+ **pengine:** Correctly bypass fencing for resources that do not require it
+ **pengine:** do not timeout remote node recurring monitor op failure until
after fencing
+ **pengine:** Ensure recurring monitor operations are cancelled when clone
instances are de-allocated
+ **pengine:** fixes segfault in pengine when fencing remote node
+ **pengine:** properly handle blocked clone actions
+ **pengine:** ensure failed actions that occurred in node shutdown are
displayed
+ **remote:** correctly display the usage of the ocf:pacemaker:remote
resource agent
+ **remote:** do not fail operations because of a migration
+ **remote:** enable reloads for select remote connection options
+ **resources:** allow for top output with or without percent sign in
HealthCPU
+ **resources:** Prevent an error message on stopping `Dummy` resource
+ **systemd:** Prevent segfault when logging failed operations
+ **systemd:** Reconnect to System DBus if the connection is closed
+ **systemd:** set systemd resources' timeout values higher than systemd's
own default
+ **tools:** Do not send command lines to syslog
+ **tools:** update SNMP MIB
+ **upstart:** Ensure pending structs are correctly unreferenced
# Pacemaker-1.1.13 (24 Jun 2015)
- 750 commits with 156 files changed, 11323 insertions(+), 3725 deletions(-)
## Features added since Pacemaker-1.1.12
+ Allow fail-counts to be removed en-mass when the new attrd is in operation
+ attrd supports private attributes (not written to CIB)
+ **crmd:** Ensure a watchdog device is in use if stonith-watchdog-timeout is configured
+ **crmd:** If configured, trigger the watchdog immediately if we lose quorum and no-quorum-policy=suicide
+ **crm_diff:** Support generating a difference without versions details if `--no-version/-u` is supplied
+ **crm_resource:** Implement an intelligent restart capability
+ **Fencing:** Advertise the watchdog device for fencing operations
+ **Fencing:** Allow the cluster to recover resources if the watchdog is in use
+ **fencing:** cl#5134 - Support random fencing delay to avoid double fencing
+ **mcp:** Allow orphan children to initiate node panic via SIGQUIT
+ **mcp:** Turn on sbd integration if pacemakerd finds it running
+ **mcp:** Two new error codes that result in machine reset or power off
+ Officially support the resource-discovery attribute for location constraints
+ **PE:** Allow natural ordering of colocation sets
+ **PE:** Support non-actionable degraded mode for OCF
+ **pengine:** cl#5207 - Display "UNCLEAN" for resources running on unclean offline nodes
+ **remote:** pcmk remote client tool for use with container wrapper script
+ Support machine panics for some kinds of errors (via sbd if available)
+ **tools:** add `crm_resource` `--wait` option
+ **tools:** `attrd_updater` supports `--query` and `--all` options
+ **tools:** `attrd_updater:` Allow attributes to be set for other nodes
## Changes since Pacemaker-1.1.12
+ **pengine:** exclusive discovery implies rsc is only allowed on exclusive subset of nodes
+ **acl:** Correctly implement the 'reference' acl directive
+ **acl:** Do not delay evaluation of added nodes in some situations
+ **attrd:** b22b1fe did uuid test too early
+ **attrd:** Clean out the node cache when requested by the admin
+ **attrd:** fixes double free in attrd legacy
+ **attrd:** properly write attributes for peers once uuid is discovered
+ **attrd:** refresh should force an immediate write-out of all attributes
+ **attrd:** Simplify how node deletions happen
+ **Bug rhbz#1067544 - Tools:** Correctly handle `--ban,` --move and `--locate` for master/slave groups
+ Bug rhbz#1181824 - Ensure the DC can be reliably fenced
+ **cib:** Ability to upgrade cib validation schema in legacy mode
+ **cib:** Always generate digests for cib diffs in legacy mode
+ **cib:** assignment where comparison intended
+ **cib:** Avoid nodeid conflicts we don't care about
+ **cib:** Correctly add "update-origin", "update-client" and "update-user" attributes for cib
+ **cib:** Correctly set up signal handlers
+ **cib:** Correctly track node state
+ **cib:** Do not update on disk backups if we're just querying them
+ **cib:** Enable cib legacy mode for plugin-based clusters
+ **cib:** Ensure file-based backends treat '-o section' consistently with the native backend
+ **cib:** Ensure upgrade operations from a non-DC get an acknowledgement
+ **cib:** No need to enforce cib digests for v2 diffs in legacy mode
+ **cib:** Revert d153b86 to instantly get cib synchronized in legacy mode
+ **cib:** tls sock cleanup for remote cib connections
+ **cli:** Ensure subsequent unknown long options are correctly detected
+ **cluster:** Invoke `crm_remove_conflicting_peer()` only when the new node's uname is being assigned in the node cache
+ **common:** Increment current and age for lib common as a result of APIs being added
+ **corosync:** Bug cl#5232 - Somewhat gracefully handle nodes with invalid UUIDs
+ **corosync:** Avoid unnecessary repeated CMAP API calls
+ **crmd/pengine:** handle on-fail=ignore properly
+ **crmd:** Add "on_node" attribute for *_last_failure_0 lrm resource operations
+ **crmd:** All peers need to track node shutdown requests
+ **crmd:** Cached copies of transient attributes cease to be valid once a node leaves the membership
+ **crmd:** Correctly add the local option that validates against schema for pengine to calculate
+ **crmd:** Disable debug logging that results in significant overhead
+ **crmd:** do not remove connection resources during re-probe
+ **crmd:** don't update fail count twice for same failure
+ **crmd:** Ensure remote connection resources timeout properly during 'migrate_from' action
+ **crmd:** Ensure `throttle_mode()` does something on Linux
+ **crmd:** Fixes crash when remote connection migration fails
+ **crmd:** gracefully handle remote node disconnects during op execution
+ **crmd:** Handle remote connection failures while executing ops on remote connection
+ **crmd:** include remote nodes when forcing cluster wide resource reprobe
+ **crmd:** never stop recurring monitor ops for pcmk remote during incomplete migration
+ **crmd:** Prevent the old version of DC from being fenced when it shuts down for rolling-upgrade
+ **crmd:** Prevent use-of-NULL during reprobe
+ **crmd:** properly update job limit for baremetal remote-nodes
+ **crmd:** Remote-node throttle jobs count towards cluster-node hosting conneciton rsc
+ **crmd:** Reset stonith failcount to recover transitioner when the node rejoins
+ **crmd:** resolves memory leak in crmd.
+ **crmd:** respect `start-failure-is-fatal` even for artifically injected
events
+ **crmd:** Wait for all pending operations to complete before poking the
policy engine
+ **crmd:** When container's host is fenced, cancel in-flight operations
+ **crm_attribute:** Correctly update config options when `-o crm_config` is
specified
+ **crm_failcount:** Better error reporting when no resource is specified
+ **crm_mon:** add exit reason to resource failure output
+ **crm_mon:** Fill `CRM_notify_node` in traps with node's uname rather than node's id if possible
+ **crm_mon:** Repair notification delivery when the v2 patch format is in use
+ **crm_node:** Correctly remove nodes from the CIB by nodeid
+ **crm_report:** More patterns for finding logs on non-DC nodes
+ **crm_resource:** Allow resource restart operations to be node specific
+ **crm_resource:** avoid deletion of lrm cache on node with resource discovery disabled.
+ **crm_resource:** Calculate how long to wait for a restart based on the resource timeouts
+ **crm_resource:** Clean up memory in `--restart` error paths
+ **crm_resource:** Display the locations of all anonymous clone children when supplying the children's common ID
+ **crm_resource:** Ensure `--restart` sets/clears meta attributes
+ **crm_resource:** Ensure fail-counts are purged when we redetect the state of all resources
+ **crm_resource:** Implement `--timeout` for resource restart operations
+ **crm_resource:** Include group members when calculating the next timeout
+ **crm_resource:** Memory leak in error paths
+ **crm_resource:** Prevent use-after-free
+ **crm_resource:** Repair regression test outputs
+ **crm_resource:** Use-after-free when restarting a resource
+ **dbus:** ref count leaks
+ **dbus:** Ensure both the read and write queues get dispatched
+ **dbus:** Fail gracefully if malloc fails
+ **dbus:** handle dispatch queue when multiple replies need to be processed
+ **dbus:** Notice when dbus connections get disabled
+ **dbus:** Remove double-free introduced while trying to make coverity shut up
+ ensure if B is colocated with A, B can never run without A
+ **fence_legacy:** Avoid passing 'port' to cluster-glue agents
+ **fencing:** Allow nodes to be purged from the member cache
+ **fencing:** Correctly make args for fencing agents
+ **fencing:** Correctly wait for self-fencing to occur when the watchdog is in use
+ **fencing:** Ensure the hostlist parameter is set for watchdog agents
+ **fencing:** Force 'stonith-ng' as the system name
+ **fencing:** Gracefully handle invalid metadata from agents
+ **fencing:** If configured, wait stonith-watchdog-timer seconds for self-fencing to complete
+ **fencing:** Reject actions for devices that haven't been explicitly registered yet
+ **ipc:** properly allocate server enforced buffer size on client
+ **ipc:** use server enforced buffer during ipc client send
+ **lrmd, services:** interpret LSB status codes properly
+ **lrmd:** add back support for class heartbeat agents
+ **lrmd:** cancel pending async connection during disconnect
+ **lrmd:** enable ipc proxy for docker-wrapper privileged mode
+ **lrmd:** fix rescheduling of systemd monitor op during start
+ **lrmd:** Handle systemd reporting 'done' before a resource is actually stopped
+ **lrmd:** Hint to child processes that using `sd_notify` is not required
+ **lrmd:** Log with the correct personality
+ **lrmd:** Prevent glib assert triggered by timers being removed from mainloop more than once
+ **lrmd:** report original timeout when systemd operation completes
+ **lrmd:** store failed operation exit reason in cib
+ **mainloop:** resolves race condition mainloop poll involving modification of ipc connections
+ make targetted reprobe for remote node work, `crm_resource` -C -N <remote node>
+ **mcp:** Allow a configurable delay when debugging shutdown issues
+ **mcp:** Avoid requiring 'export' for SYS-V sysconfig options
+ **Membership:** Detect and resolve nodes that change their ID
+ **pacemakerd:** resolves memory leak of xml structure in pacemakerd
+ **pengine:** ability to launch resources in isolated containers
+ **pengine:** add #kind=remote for baremetal remote-nodes
+ **pengine:** allow baremetal remote-nodes to recover without requiring fencing when cluster-node fails
+ **pengine:** allow remote-nodes to be placed in maintenance mode
+ **pengine:** Avoid trailing whitespaces when printing resource state
+ **pengine:** cl#5130 - Choose nodes capable of running all the colocated utilization resources
+ **pengine:** cl#5130 - Only check the capacities of the nodes that are allowed to run the resource
+ **pengine:** Correctly compare feature set to determine how to unpack meta attributes
+ **pengine:** disable migrations for resources with isolation containers
+ **pengine:** disable reloading of resources within isolated container wrappers
+ **pengine:** Do not aggregate children in a pending state into the started/stopped/etc lists
+ **pengine:** Do not record duplicate copies of the failed actions
+ **pengine:** Do not reschedule monitors that are no longer needed while resource definitions have changed
+ **pengine:** Fence baremetal remote when recurring monitor op fails
+ **pengine:** Fix colocation with unmanaged resources
+ **pengine:** Fix the behaviors of multi-state resources with asymmetrical ordering
+ **pengine:** fixes pengine crash with orphaned remote node connection resource
+ **pengine:** fixes segfault caused by malformed log warning
+ **pengine:** handle cloned isolated resources in a sane way
+ **pengine:** handle isolated resource scenario, cloned group of isolated resources
+ **pengine:** Handle ordering between stateful and migratable resources
+ **pengine:** imply stop in container node resources when host node is fenced
+ **pengine:** only fence baremetal remote when connection can fails or can not be recovered
+ **pengine:** only kill process group on timeout when on-fail does not equal block.
+ **pengine:** per-node control over resource discovery
+ **pengine:** prefer migration target for remote node connections
+ **pengine:** prevent disabling rsc discovery per node in certain situations
+ **pengine:** Prevent use-after-free in `sort_rsc_process_order()`
+ **pengine:** properly handle ordering during remote connection partial migration
+ **pengine:** properly recover remote-nodes when cluster-node proxy goes offline
+ **pengine:** remove unnecessary whitespace from notify environment variables
+ **pengine:** require-all feature for ordered clones
+ **pengine:** Resolve memory leaks
+ **pengine:** resource discovery mode for location constraints
+ **pengine:** restart master instances on instance attribute changes
+ **pengine:** Turn off legacy unpacking of resource options into the meta hashtable
+ **pengine:** Watchdog integration is sufficient for fencing
+ Perform systemd reloads asynchronously
+ **ping:** Correctly advertise multiplier default
+ Prefer to inherit the watchdog timeout from SBD
+ properly record stop args after reload
+ provide fake meta data for ra class heartbeat
+ **remote:** report timestamps for remote connection resource operations
+ **remote:** Treat recv msg timeout as a disconnect
+ **service:** Prevent potential use-of-NULL in metadata lookups
+ **solaris:** Allow compilation when dirent.d_type is not available
+ **solaris:** Correctly replace the linux swab functions
+ **solaris:** Disable throttling since /proc doesn't exist
+ **stonith-ng:** Correctly observe the watchdog completion timeout
+ **stonith-ng:** Correctly track node state
+ **stonith-ng:** Reset mainloop source IDs after removing them
+ **systemd:** Correctly handle long running stop actions
+ **systemd:** Ensure failed monitor operations always return
+ **systemd:** Ensure we don't call `dbus_message_unref()` with NULL
+ **systemd:** fix crash caused when canceling in-flight operation
+ **systemd:** Kindly ask dbus NOT to kill the process if the dbus connection fails
+ **systemd:** Perform actions asynchronously
+ **systemd:** Perform monitor operations without blocking
+ **systemd:** Tell systemd not to take DBus down from underneath us
+ **systemd:** Trick systemd into not stopping our services before us during shutdown
+ **tools:** Improve `crm_mon` output with certain option combinations
+ **upstart:** Monitor actions always return 'ok' or 'not running'
+ **upstart:** Perform more parts of monitor operations without blocking
+ **xml:** add 'require-all' to xml schema for constraints
+ **xml:** cl#5231 - Unset the deleted attributes in the resulting diffs
+ **xml:** Clone the latest constraint schema in preparation for changes"
+ **xml:** Correctly create v1 patchsets when deleting attributes
+ **xml:** Do not change the ordering of properties when applying v1 cib diffs
+ **xml:** Do not dump deleted attributes
+ **xml:** Do not prune leaves from v1 cib diffs that are being created with digests
+ **xml:** Ensure ACLs are reapplied before calculating what a replace operation changed
+ **xml:** Fix upgrade-1.3.xsl to correctly transform ACL rules with "attribute"
+ **xml:** Prevent assert errors in `crm_element_value()` on applying a patch without version information
+ **xml:** Prevent potential use-of-NULL
# Pacemaker-1.1.12 (22 Jul 2014)
- 795 commits with 195 files changed, 13772 insertions(+), 6176 deletions(-)
## Features added since Pacemaker-1.1.11
+ Changes to the ACL schema to support nodes and unix groups
+ **cib:** Check ACLs prior to making the update instead of parsing the diff afterwards
+ **cib:** Default ACL support to on
+ **cib:** Enable the more efficient xml patchset format
+ **cib:** Implement zero-copy status update
+ **cib:** Send all r/w operations via the cluster connection and have all
nodes process them
+ **crmd:** Set `cluster-name` property to Corosync's `cluster_name` by
default for Corosync 2
+ **crm_mon:** Display brief output if `-b/--brief` is supplied or `b` is
toggled
+ **crm_report:** Allow ssh alternatives to be used
+ **crm_ticket:** Support multiple modifications for a ticket in an atomic operation
+ **extra:** Add logrotate configuration file for /var/log/pacemaker.log
+ **Fencing:** Add the ability to call `stonith_api_time()` from `stonith_admin`
+ **logging:** daemons always get a log file, unless explicitly set to configured 'none'
+ **logging:** allows the user to specify a log level that is output to syslog
+ **PE:** Automatically re-unfence a node if the fencing device definition changes
+ **pengine:** cl#5174 - Allow resource sets and templates for location constraints
+ **pengine:** Support cib object tags
+ **pengine:** Support cluster-specific instance attributes based on rules
+ **pengine:** Support id-ref in nvpair with optional "name"
+ **pengine:** Support per-resource maintenance mode
+ **pengine:** Support site-specific instance attributes based on rules
+ **tools:** Allow `crm_shadow` to create older configuration versions
+ **tools:** Display pending state in `crm_mon/crm_resource/crm_simulate` if `--pending/-j` is supplied (cl#5178)
+ **xml:** Add the ability to have lightweight schema revisions
+ **xml:** Enable resource sets in location constraints for 1.2 schema
+ **xml:** Support resources that require unfencing
## Changes since Pacemaker-1.1.11
+ **acl:** Authenticate pacemaker-remote requests with the node name as the client
+ **acl:** Read access must be explicitly granted
+ **attrd:** Ensure attribute dampening is always observed
+ **attrd:** Remove offline nodes from node cache for "peer-remove" requests
+ Bug cl#5055 - Improved migration support.
+ Bug cl#5184 - Ensure pending probes that ultimately fail are correctly updated
+ **Bug cl#5196 - pengine:** Check values after expanding templates
+ Bug cl#5212 - Do not promote instances when quorum is lots and no-quorum-policy=freeze
+ Bug cl#5213 - Ensure role colocation with -INFINITY is enforced
+ Bug cl#5213 - Limit the scope of the previous commit to the masters role
+ **Bug cl#5219 - pengine:** Allow unrelated resources with a common colocation target to remain promoted
+ **Bug cl#5222 - cib:** Repair rolling update capability
+ Bug cl#5222 - Enable legacy mode whenever a broadcast update is detected
+ Bug rhbz#1036631 - Stop members of cloned groups when dependencies are stopped
+ Bug rhbz#1054307 - cname pattern match should be more restrictive in init script
+ Bug rhbz#1057697 - Use native DBus library for systemd/upstart support to avoid problematic use of threads
+ Bug rhbz#1097457 - Limit the scope of the previous fix and include a helpful comment
+ Bug rhbz#1097457 - Prevent invalid transition when resource are ordered to start after the container they're started in
+ **cib:** allow setting permanent remote-node attributes
+ **cib:** Auto-detect which patchset format to use
+ **cib:** Determine the best value of `validate-with` if one is not supplied
+ **cib:** Do not disable cib disk writes if on-disk cib is corrupt
+ **cib:** Ensure `cibadmin -R/--replace` commands get replies
+ **cib:** Erasing the cib is an admin action, bump the `admin_epoch` instead
+ **cib:** Fix remote cib based on TLS
+ **cib:** Ignore patch failures if we already have their contents
+ **cib:** Validate that everyone still sees the same configuration once all updates have completed
+ **cibadmin:** Allow priviliged clients to perform tasks as unpriviliged users
+ **cibadmin:** Remove dangerous commands that exposed unnecessary implementation internal details
+ **cluster:** Fix segfault on removing a node
+ **cluster:** Prevent search of unames from attempting to create node entries for unknown nodes
+ **cluster:** Remove unknown offline nodes with conflicting unames from node cache
+ **controld:** Do not consider the dlm up until the address list is present
+ **controld:** handling startup fencing within the controld agent, not the dlm
+ **controld:** Return `OCF_ERR_INSTALLED` instead of `OCF_NOT_INSTALLED`
+ **crmd:** Ack pending operations that were cancelled due to rsc deletion
+ **crmd:** Actions can only be executed if their pre-requisits completed successfully
+ **crmd:** avoid double free caused by nested hash table removal
+ **crmd:** Avoid spamming the cib by triggering a transition only once per non-status change
+ **crmd:** Correctly react to successful unfencing operations
+ **crmd:** Correctly recognise operation cancellations we initiated
+ **crmd:** Do not erase the status section for unfenced nodes
+ **crmd:** Do not overwrite existing node state when fencing completes
+ **crmd:** Do not start timers for already completed operations
+ **crmd:** Ensure `crm_config` options are re-read on updates
+ **crmd:** Fenced nodes that return prior to an election do not need to have their status section reset
+ **crmd:** make `lrm_state` hash table not case sensitive
+ **crmd:** make `node_state` erase correctly
+ **crmd:** Only write `fence_averride` if open() returns a positive file descriptor
+ **crmd:** Prevent manual fencing confirmations from attempting to create node entries for unknown nodes
+ **crmd:** Prevent SIGPIPE when notifying CMAN about fencing operations
+ **crmd:** Remove state of unknown nodes with conflicting unames from CIB
+ **crmd:** Remove unknown nodes with conflicting unames from CIB
+ **crmd:** Report unsuccessful unfencing operations
+ **crm_diff:** Allow the generation of xml patchsets without digests
+ **crm_mon:** Allow the file created by `--as-html` to be world readable
+ **crm_mon:** Ensure resource attributes have been unpacked before displaying connectivity data
+ **crm_node:** Only remove the named resource from the cib
+ **crm_report:** Gracefully handle rediculously large logfiles
+ **crm_report:** Only gather dlm data if `dlm_controld` is running
+ **crm_resource:** Gracefully handle -EACCESS when querying the cib
+ **crm_verify:** Perform a full set of calculations whenever the status section is present
+ **fencing:** Advertise support for reboot/on/off in the metadata for legacy agents
+ **fencing:** Automatically switch from 'list' to 'status' to 'static-list' if those actions are not advertised in the metadata
+ **fencing:** Cache metadata lookups to avoid repeated blocking during device registration
+ **fencing:** Correctly record which peer performed the fencing operation
+ **fencing:** default to 'off' when agent does not advertise 'reboot' in metadata
+ **fencing:** Do not unregister/register all stonith devices on every resource agent change
+ **fencing:** Execute all required fencing devices regardless of what topology level they are at
+ **fencing:** Fence using all required devices
+ **fencing:** Pass the correct options when looking up the history by node name
+ **fencing:** Update stonith device list only if stonith is enabled
+ **get_cluster_type:** failing concurrent tool invocations on heartbeat
+ ignore SIGPIPE when gnutls is in use
+ **iso8601:** Different logic is needed when logging and calculating durations
+ **iso8601:** Fix memory leak in duration calculation
+ **Logging:** Bootstrap daemon logging before processing arguments but configure it afterwards
+ **lrmd:** Cancel recurring operations before stop action is executed
+ **lrmd:** Expose logging variables expected by OCF agents
+ **lrmd:** Handle systemd reporting 'done' before a resource is actually stopped/started
+ **lrmd:** Merge duplicate recurring monitor operations
+ **lrmd:** Prevent OCF agents from logging to random files due to "value" of setenv() being NULL
+ **lrmd:** Provide stderr output from agents if available, otherwise fall back to stdout
+ **mainloop:** Better handle the killing of processes in the act of exiting
+ **mainloop:** Canceling in-flight operations should not fail if child process has already exited.
+ **mainloop:** Fixes use after free in process monitor code
+ **mcp:** Tell systemd not to respawn us if we exit with rc=100
+ **membership:** Avoid duplicate peer entries in the peer cache
+ **pengine:** Allow container nodes to migrate with connection resource
+ **pengine:** avoid assert by searching for stop action on correct node during LogActions
+ **pengine:** Block restart of resources if any dependent resource in a group is unmanaged
+ **pengine:** cl#5186 - Avoid running rsc on two nodes when node is fenced during migration
+ **pengine:** cl#5187 - Prevent resources in an anti-colocation from even temporarily running on a same node
+ **pengine:** cl#5200 - Before migrating utilization-using resources to a node, take off the load that will no longer run there if it's not introducing transition loop
+ **pengine:** Correctly handle origin offsets in the future
+ **pengine:** Correctly observe requires=nothing
+ **pengine:** Default sequential to TRUE for resource sets for consistency with colocation sets
+ **pengine:** Delay unfencing until after we know the state of all resources that require unfencing
+ **pengine:** Do not initiate fencing for unclean nodes when fencing is disabled
+ **pengine:** Ensure instance numbers are preserved for cloned templates
+ **pengine:** Ensure unfencing only happens once, even if the transition is interrupted
+ **pengine:** Fencing devices default to only requiring quorum in order to start
+ **pengine:** fixes invalid transition caused by clones with more than 10 instances
+ **pengine:** Force record pending for `migrate_to` actions
+ **pengine:** handles edge case where container order constraints are not honored during migration
+ **pengine:** Ignore failure-timeout only if the failed operation has on-fail="block"
+ **pengine:** Mark unrunnable stop actions as "blocked" and show the correct current locations
+ **pengine:** Memory leaks
+ **pengine:** properly handle fencing of container remote-nodes when the container is orphaned
+ **pengine:** properly place resource within a container when container is a remote-node.
+ **pengine:** Unfencing is based on device probes, there is no need to unfence when normal resources are found active
+ **pengine:** Use "#cluster-name" in rules for setting cluster-specific instance attributes
+ **pengine:** Use "#site-name" in rules for setting site-specific instance attributes
+ **remote:** Allow baremetal remote-node connection resources to migrate
+ **remote:** clear remote-node status correctly
+ **remote:** Enable migration support for baremetal connection resources by default
+ **remote:** Handle request/response ipc proxy correctly
+ **services:** Correctly reset the nice value for lrmd's children
+ **services:** Do not allow duplicate recurring op entries
+ **services:** Do not block synced service executions
+ **services:** Fixes segfault associated with cancelling in-flight recurring operations.
+ **services:** Remove cancelled recurring ops from internal lists as early as possible
+ **services:** Remove file descriptors from mainloop as soon as we have drained them
+ **services:** Reset the scheduling policy and priority for lrmd's children without replying on `SCHED_RESET_ON_FORK`
+ **services_action_cancel:** Interpret return code from `mainloop_child_kill()` correctly
+ **stonith_admin:** Ensure pointers passed to sscanf() are properly initialized
+ `stonith_api_time_helper` now returns when the most recent fencing operation completed
+ **systemd:** Prevent use-of-NULL when determining if an agent exists
+ **systemd:** Try to handle dbus actions that complete prior to configuring a callback
+ **Tools:** Non-daemons shouldn't abort just because xml parsing failed
+ **Upstart:** Allow comilation with glib versions older than 2.28
+ **Upstart:** Do not attempt upstart jobs if we cannot connect to dbus
+ When data was old, it fixed so that the newest cib might not be acquired.
+ **xml:** Check all available schemas when doing upgrades
+ **xml:** Correctly determine the lowest allowed schema version
+ **xml:** Correctly enforce ACLs after a replace operation
+ **xml:** Correctly infer attribute changes after a replace operation
+ **xml:** Create the correct diff when only part of a document is changed
+ **xml:** Detect attribute ordering changes
+ **xml:** Detect content that is added and removed in the same update
+ **xml:** Do not prune meaningful leaves from v1 patchsets
+ **xml:** Empty patchsets are considered to have applied cleanly
+ **xml:** Ensure patches always have version details set
+ **xml:** Find the minimal set of changes when part of a document is replaced
+ **xml:** If validate-with is missing, we find the most recent schema that accepts it and go from there
+ **xml:** Introduce a 'move' primitive for v2 patch sets
+ **xml:** Preserve the attribute order in the patch for subsequent digest validation
+ **xml:** Resolve memory leak when logging xml blobs
+ **xml:** Update xml validation to allow '<node type=remote />'
# Pacemaker-1.1.11 (13 Feb 2014)
- 462 commits with 147 files changed, 6810 insertions(+), 4057 deletions(-)
## Features added since Pacemaker-1.1.10
+ **attrd:** A truly atomic version of attrd for use where CPG is used for cluster communication
+ **cib:** Allow values to be added/updated and removed in a single update
+ **cib:** Support XML comments in diffs
+ **Core:** Allow blackbox logging to be disabled with SIGUSR2
+ **crmd:** Do not block on proxied calls from `pacemaker_remoted`
+ **crmd:** Enable cluster-wide throttling when the cib heavily exceeds its target load
+ **crmd:** Make the per-node action limit directly configurable in the CIB
+ **crmd:** Slow down recovery on nodes with IO load
+ **crmd:** Track CPU usage on cluster nodes and slow down recovery on nodes with high CPU/IO load
+ **crm_mon:** add `--hide-headers` option to hide all headers
+ **crm_node:** Display partition output in sorted order
+ **crm_report:** Collect logs directly from journald if available
+ **Fencing:** On timeout, clean up the agent's entire process group
+ **Fencing:** Support agents that need the host to be unfenced at startup
+ **ipc:** Raise the default buffer size to 128k
+ **PE:** Add a special attribute for distinguishing between real nodes and containers in constraint rules
+ **PE:** Allow location constraints to take a regex pattern to match against resource IDs
+ **pengine:** Distinguish between the agent being missing and something the agent needs being missing
+ **remote:** Properly version the remote connection protocol
## Changes since Pacemaker-1.1.10
+ Bug rhbz#1011618 - Consistently use 'Slave' as the role for unpromoted master/slave resources
+ Bug rhbz#1057697 - Use native DBus library for systemd and upstart support to avoid problematic use of threads
+ **attrd:** Any variable called 'cluster' makes the daemon crash before reaching main()
+ **attrd:** Avoid infinite write loop for unknown peers
+ **attrd:** Drop all attributes for peers that left the cluster
+ **attrd:** Give remote-nodes ability to set attributes with attrd
+ **attrd:** Prevent inflation of attribute dampen intervals
+ **attrd:** Support SI units for attribute dampening
+ **pengine:** Don't prevent clones from running due to dependent resources
(bug cl#5171)
+ **corosync:** Attempt to retrieve a peer's node name if it is not already
known (bug cl#5179)
+ **corosync:** Ensure node IDs are written to the CIB as unsigned integers
(bug cl#5181)
+ **tools:** Handle `crm_resource --ban` for master/slave resources as
advertised (bug rhbz#902407)
+ **cib:** Correctly check for archived configuration files
+ **cib:** Correctly log short-form xml diffs
+ **cib:** Fix remote cib based on TLS
+ **cibadmin:** Report errors during sign-off
+ **cli:** Do not enabled blackbox for cli tools
+ **cluster:** Fix segfault on removing a node
+ **cman:** Do not start pacemaker if cman startup fails
+ **cman:** Start clvmd and friends from the init script if enabled
+ Command-line tools should stop after an assertion failure
+ **controld:** Use the correct variant of `dlm_controld` for corosync-2 clusters
+ **cpg:** Correctly set the group name length
+ **cpg:** Ensure the CPG group is always null-terminated
+ **cpg:** Only process one message at a time to allow other priority jobs to be performed
+ **crmd:** Correctly observe the configured batch-limit
+ **crmd:** Correctly update expected state when the previous DC shuts down
+ **crmd:** Correcty update the history cache when recurring ops change their return code
+ **crmd:** Don't add `node_state` to cib, if we have not seen or fenced this node yet
+ **crmd:** don't segfault on shutdown when using heartbeat
+ **crmd:** Prevent recurring monitors being cancelled due to notify operations
+ **crmd:** Reliably detect and act on reprobe operations from the policy engine
+ **crmd:** When a peer expectedly shuts down, record the new join and expected states into the cib
+ **crmd:** When the DC gracefully shuts down, record the new expected state into the cib
+ **crm_attribute:** Do not swallow hostname lookup failures
+ **crm_mon:** Do not display duplicates of failed actions
+ **crm_mon:** Reduce flickering in interactive mode
+ **crm_resource:** Observe `--master` modifier for `--move`
+ **crm_resource:** Provide a meaningful error if `--master` is used for primitives and groups
+ **fencing:** Allow fencing for node after topology entries are deleted
+ **fencing:** Apply correct score to the resource of group
+ **fencing:** Ignore changes to non-fencing resources
+ **fencing:** Observe `pcmk_host_list` during automatic unfencing
+ **fencing:** Put all fencing agent processes into their own process group
+ **fencing:** Wait until all possible replies are recieved before continuing with unverified devices
+ **ipc:** Compress msgs based on client's actual max send size
+ **ipc:** Have the ipc server enforce a minimum buffer size all clients must use.
+ **iso8601:** Prevent dates from jumping backwards a day in some timezones
+ **lrmd:** Correctly calculate metadata for the 'service' class
+ **lrmd:** Correctly cancel monitor actions for lsb/systemd/service resources on cleaning up
+ **mcp:** Remove LSB hints that instruct chkconfig to start pacemaker at boot time
+ **mcp:** Some distros complain when LSB scripts do not include Default-Start/Stop directives
+ **pengine:** Allow fencing of baremetal remote nodes
+ **pengine:** cl#5186 - Avoid running rsc on two nodes when node is fenced during migration
+ **pengine:** Correctly account for the location preferences of things colocated with a group
+ **pengine:** Correctly handle demotion of grouped masters that are partially demoted
+ **pengine:** Disable container node probes due to constraint conflicts
+ **pengine:** Do not allow colocation with blocked clone instances
+ **pengine:** Do not re-allocate clone instances that are blocked in the Stopped state
+ **pengine:** Do not restart resources that depend on unmanaged resources
+ **pengine:** Force record pending for `migrate_to` actions
+ **pengine:** Location constraints with role=Started should prevent masters from running at all
+ **pengine:** Order demote/promote of resources on remote nodes to happen only once the connection is up
+ **pengine:** Properly handle orphaned multistate resources living on remote-nodes
+ **pengine:** Properly shutdown orphaned remote connection resources
+ **pengine:** Recover unexpectedly running container nodes.
+ **remote:** Add support for ipv6 into `pacemaker_remote` daemon
+ **remote:** Handle endian changes between client and server and improve forward compatibility
+ **services:** Fixes segfault associated with cancelling in-flight recurring operations.
+ **services:** Reset the scheduling policy and priority for lrmd's children without replying on `SCHED_RESET_ON_FORK`
# Pacemaker-1.1.10 (26 Jul 2013)
- 602 commits with 143 files changed, 8162 insertions(+), 5159 deletions(-)
## Features added since Pacemaker-1.1.9
+ **Core:** Convert all exit codes to positive errno values
+ **crm_error:** Add the ability to list and print error symbols
+ **crm_resource:** Allow individual resources to be reprobed
+ **crm_resource:** Allow options to be set recursively
+ **crm_resource:** Implement `--ban` for moving resources away from nodes
and `--clear` (replaces `--unmove`)
+ **crm_resource:** Support OCF tracing when using
`--force-(check|start|stop)`
+ **PE:** Allow active nodes in our current membership to be fenced without quorum
+ **PE:** Suppress meaningless IDs when displaying anonymous clone status
+ Turn off auto-respawning of systemd services when the cluster starts them
+ **PE:** Support maintenance mode for a single node (bug cl#5128)
## Changes since Pacemaker-1.1.9
+ **crmd:** cib: stonithd: Memory leaks resolved and improved use of glib reference counting
+ **attrd:** Fixes deleted attributes during dc election
+ Bug cf#5153 - Correctly display clone failcounts in `crm_mon`
+ **Bug cl#5133 - pengine:** Correctly observe on-fail=block for failed demote operation
+ **Bug cl#5148 - legacy:** Correctly remove a node that used to have a different nodeid
+ Bug cl#5151 - Ensure node names are consistently compared without case
+ **Bug cl#5152 - crmd:** Correctly clean up fenced nodes during membership changes
+ Bug cl#5154 - Do not expire failures when on-fail=block is present
+ **Bug cl#5155 - pengine:** Block the stop of resources if any depending resource is unmanaged
+ Bug cl#5157 - Allow migration in the absence of some colocation constraints
+ **Bug cl#5161 - crmd:** Prevent memory leak in operation cache
+ **Bug cl#5164 - crmd:** Fixes crash when using pacemaker-remote
+ **Bug cl#5164 - pengine:** Fixes segfault when calculating transition with remote-nodes.
+ **Bug cl#5167 - `crm_mon:**` Only print "stopped" node list for incomplete clone sets
+ Bug cl#5168 - Prevent clones from being bounced around the cluster due to location constraints
+ Bug cl#5170 - Correctly support on-fail=block for clones
+ **cib:** Correctly read back archived configurations if the primary is corrupted
+ **cib:** The result is not valid when diffs fail to apply cleanly for CLI tools
+ **cib:** Restore the ability to embed comments in the configuration
+ **cluster:** Detect and warn about node names with capitals
+ **cman:** Do not pretend we know the state of nodes we've never seen
+ **cman:** Do not unconditionally start cman if it is already running
+ **cman:** Support non-blocking CPG calls
+ **Core:** Ensure the blackbox is saved on abnormal program termination
+ **corosync:** Detect the loss of members for which we only know the nodeid
+ **corosync:** Do not pretend we know the state of nodes we've never seen
+ **corosync:** Ensure removed peers are erased from all caches
+ **corosync:** Nodes that can persist in sending CPG messages must be alive afterall
+ **crmd:** Do not get stuck in `S_POLICY_ENGINE` if a node we couldn't fence returns
+ **crmd:** Do not update fail-count and last-failure for old failures
+ **crmd:** Ensure all membership operations can complete while trying to cancel a transition
+ **crmd:** Ensure operations for cleaned up resources don't block recovery
+ **crmd:** Ensure we return to a stable state if there have been too many fencing failures
+ **crmd:** Initiate node shutdown if another node claims to have successfully fenced us
+ **crmd:** Prevent messages for remote crmd clients from being relayed to wrong daemons
+ **crmd:** Properly handle recurring monitor operations for remote-node agent
+ **crmd:** Store last-run and last-rc-change for all operations
+ **crm_mon:** Ensure stale pid files are updated when a new process is started
+ **crm_report:** Correctly collect logs when 'uname -n' reports fully qualified names
+ **fencing:** Fail the operation once all peers have been exhausted
+ **fencing:** Restore the ability to manually confirm that fencing completed
+ **ipc:** Allow unpriviliged clients to clean up after server failures
+ **ipc:** Restore the ability for members of the haclient group to connect to the cluster
+ **legacy:** Support "crm_node `--remove"` with a node name for corosync plugin (bnc#805278)
+ **lrmd:** Default to the upstream location for resource agent scratch directory
+ **lrmd:** Pass errors from lsb metadata generation back to the caller
+ **pengine:** Correctly handle resources that recover before we operate on them
+ **pengine:** Delete the old resource state on every node whenever the resource type is changed
+ **pengine:** Detect constraints with inappropriate actions (ie. promote for a clone)
+ **pengine:** Ensure per-node resource parameters are used during probes
+ **pengine:** If fencing is unavailable or disabled, block further recovery for resources that fail to stop
+ **pengine:** Implement the rest of `get_timet_now()` and rename to `get_effective_time`
+ **pengine:** Re-initiate `_active_` recurring monitors that previously failed but have timed out
+ **remote:** Workaround for inconsistent tls handshake behavior between gnutls versions
+ **systemd:** Ensure we get shut down correctly by systemd
+ **systemd:** Reload systemd after adding/removing override files for cluster services
+ **xml:** Check for and replace non-printing characters with their octal equivalent while exporting xml text
+ **xml:** Prevent lockups by setting a more reliable buffer allocation strategy
# Pacemaker-1.1.9 (08 Mar 2013)
- 731 commits with 1301 files changed, 92909 insertions(+), 57455 deletions(-)
## Features added in Pacemaker-1.1.9
+ **corosync:** Allow cman and corosync 2.0 nodes to use a name other than uname()
+ **corosync:** Use queues to avoid blocking when sending CPG messages
+ **ipc:** Compress messages that exceed the configured IPC message limit
+ **ipc:** Use queues to prevent slow clients from blocking the server
+ **ipc:** Use shared memory by default
+ **lrmd:** Support nagios remote monitoring
+ **lrmd:** Pacemaker Remote Daemon for extending pacemaker functionality outside corosync cluster.
+ **pengine:** Check for master/slave resources that are not OCF agents
+ **pengine:** Support a 'requires' resource meta-attribute for controlling whether it needs quorum, fencing or nothing
+ **pengine:** Support for resource container
+ **pengine:** Support resources that require unfencing before start
## Changes since Pacemaker-1.1.8
+ **attrd:** Correctly handle deletion of non-existant attributes
+ Bug cl#5135 - Improved detection of the active cluster type
+ Bug rhbz#913093 - Use `crm_node` instead of uname
+ **cib:** Avoid use-after-free by correctly support `cib_no_children` for non-xpath queries
+ **cib:** Correctly process XML diff's involving element removal
+ **cib:** Performance improvements for non-DC nodes
+ **cib:** Prevent error message by correctly handling peer replies
+ **cib:** Prevent ordering changes when applying xml diffs
+ **cib:** Remove text nodes from cib replace operations
+ **cluster:** Detect node name collisions in corosync
+ **cluster:** Preserve corosync membership state when matching node name/id entries
+ **cman:** Force fenced to terminate on shutdown
+ **cman:** Ignore qdisk 'nodes'
+ **core:** Drop per-user core directories
+ **corosync:** Avoid errors when closing failed connections
+ **corosync:** Ensure peer state is preserved when matching names to nodeids
+ **corosync:** Clean up CMAP connections after querying node name
+ **corosync:** Correctly detect corosync 2.0 clusters even if we don't have permission to access it
+ **crmd:** Bug cl#5144 - Do not updated the expected status of failed nodes
+ **crmd:** Correctly determin if cluster disconnection was abnormal
+ **crmd:** Correctly relay messages for remote clients (bnc#805626, bnc#804704)
+ **crmd:** Correctly stall the FSA when waiting for additional inputs
+ **crmd:** Detect and recover when we are evicted from CPG
+ **crmd:** Differentiate between a node that is up and coming up in `peer_update_callback()`
+ **crmd:** Have cib operation timeouts scale with node count
+ **crmd:** Improved continue/wait logic in `do_dc_join_finalize()`
+ **crmd:** Prevent election storms caused by getrusage() values being too close
+ **crmd:** Prevent timeouts when performing pacemaker level membership negotiation
+ **crmd:** Prevent use-after-free of `fsa_message_queue` during exit
+ **crmd:** Store all current actions when stalling the FSA
+ **crm_mon:** Do not try to render a blank cib and indicate the previous output is now stale
+ **crm_mon:** Fixes `crm_mon` crash when using snmp traps.
+ **crm_mon:** Look for the correct error codes when applying configuration updates
+ **crm_report:** Ensure policy engine logs are found
+ **crm_report:** Fix node list detection
+ **crm_resource:** Have `crm_resource` generate a valid transition key when sending resource commands to the crmd
+ **date/time:** Bug cl#5118 - Correctly convert seconds-since-epoch to the current time
+ **fencing:** Attempt to provide more information that just 'generic error' for failed actions
+ **fencing:** Correctly record completed but previously unknown fencing operations
+ **fencing:** Correctly terminate when all device options have been exhausted
+ **fencing:** cov#739453 - String not null terminated
+ **fencing:** Do not merge new fencing requests with stale ones from dead nodes
+ **fencing:** Do not start fencing until entire device topology is found or query results timeout.
+ **fencing:** Do not wait for the query timeout if all replies have arrived
+ **fencing:** Fix passing of parameters from CMAN containing '='
+ **fencing:** Fix non-comparison when sorting devices by priority
+ **fencing:** On failure, only try a topology device once from the remote level.
+ **fencing:** Only try peers for non-topology based operations once
+ **fencing:** Retry stonith device for duration of action's timeout period.
+ **heartbeat:** Remove incorrect assert during cluster connect
+ **ipc:** Bug cl#5110 - Prevent 100% CPU usage when looking for synchronous replies
+ **ipc:** Use 50k as the default compression threshold
+ **legacy:** Prevent assertion failure on routing ais messages (bnc#805626)
+ **legacy:** Re-enable logging from the pacemaker plugin
+ **legacy:** Relax the 'active' check for plugin based clusters to avoid false negatives
+ **legacy:** Skip peer process check if the process list is empty in `crm_is_corosync_peer_active()`
+ **mcp:** Only define `HA_DEBUGLOG` to avoid agent calls to `ocf_log` printing everything twice
+ **mcp:** Re-attach to existing pacemaker components when mcp fails
+ **pengine:** Any location constraint for the slave role applies to all roles
+ **pengine:** Avoid leaking memory when cleaning up failcounts and using containers
+ **pengine:** Bug cl#5101 - Ensure stop order is preserved for partially active groups
+ **pengine:** Bug cl#5140 - Allow set members to be stopped when the subseqent set has require-all=false
+ **pengine:** Bug cl#5143 - Prevent shuffling of anonymous master/slave instances
+ **pengine:** Bug rhbz#880249 - Ensure orphan masters are demoted before being stopped
+ **pengine:** Bug rhbz#880249 - Teach the PE how to recover masters into primitives
+ **pengine:** cl#5025 - Automatically clear failcount for start/monitor failures after resource parameters change
+ **pengine:** cl#5099 - Probe operation uses the timeout value from the minimum interval monitor by default (#bnc776386)
+ **pengine:** cl#5111 - When clone/master child rsc has on-fail=stop, insure all children stop on failure.
+ **pengine:** cl#5142 - Do not delete orphaned children of an anonymous clone
+ **pengine:** Correctly unpack active anonymous clones
+ **pengine:** Ensure previous migrations are closed out before attempting another one
+ **pengine:** Introducing the whitebox container resources feature
+ **pengine:** Prevent double-free for cloned primitive from template
+ **pengine:** Process `rsc_ticket` dependencies earlier for correctly allocating resources (bnc#802307)
+ **pengine:** Remove special cases for fencing resources
+ **pengine:** rhbz#902459 - Remove rsc node status for orphan resources
+ **systemd:** Gracefully handle unexpected DBus return types
+ Replace the use of the insecure mktemp(3) with mkstemp(3)
# Pacemaker-1.1.8 (20 Sep 2012)
- 1019 commits with 2107 files changed, 117258 insertions(+), 73606 deletions(-)
## Changes since Pacemaker-1.1.7
+ All APIs have been cleaned up and reduced to essentials
+ Pacemaker now includes a replacement lrmd that supports systemd and upstart agents
+ Config and state files (cib.xml, PE inputs and core files) have moved to new locations
+ The crm shell has become a separate project and no longer included with Pacemaker
+ All daemons/tools now have a unified set of error codes based on errno.h (see `crm_error)`
+ **Core:** Bug cl#5032 - Rewrite the iso8601 date handling code
+ **Core:** Correctly extract the version details from a diff
+ **Core:** Log blackbox contents, if enabled, when an error occurs
+ **Core:** Only `LOG_NOTICE` and higher are sent to syslog
+ **Core:** Replace use of IPC from clplumbing with IPC from libqb
+ **Core:** SIGUSR1 now enables blackbox logging, SIGTRAP to write out
+ **Core:** Support a blackbox for additional logging detail after crashes/errors
+ Promote support for advanced fencing logic to the stable schema
+ Promote support for node starting scores to the stable schema
+ Promote support for service and systemd to the stable schema
+ **attrd:** Differentiate between updating all our attributes and everybody updating all theirs too
+ **attrd:** Have single-shot clients wait for an ack before disconnecting
+ **cib:** cl#5026 - Synced cib updates should not return until the cpg broadcast is complete.
+ **corosync:** Detect when the first corosync has not yet formed and handle it gracefully
+ **corosync:** Obtain a full list of configured nodes, including their names, when we connect to the quorum API
+ **corosync:** Obtain a node name from DNS if one was not already known
+ **corosync:** Populate the cib nodelist from corosync if available
+ **corosync:** Use the CFG API and DNS to determine node names if not configured in corosync.conf
+ **crmd:** Block after 10 failed fencing attempts for a node
+ **crmd:** cl#5051 - Fixes file leak in PE ipc connection initialization.
+ **crmd:** cl#5053 - Fixes fail-count not being updated properly.
+ **crmd:** cl#5057 - Restart sub-systems correctly (bnc#755671)
+ **crmd:** cl#5068 - Fixes `crm_node` -R option so it works with corosync 2.0
+ **crmd:** Correctly re-establish failed attrd connections
+ **crmd:** Detect when the quorum API isn't configured for corosync 2.0
+ **crmd:** Do not overwrite any configured node type (eg. quorum node)
+ **crmd:** Enable use of new lrmd daemon and client library in crmd.
+ **crmd:** Overhaul the way node state is recorded and updated in the CIB
+ **fencing:** Bug rhbz#853537 - Prevent use-of-NULL when the cib libraries are not available
+ **fencing:** cl#5073 - Add 'off' as an valid value for stonith-action option.
+ **fencing:** cl#5092 - Always timeout stonith operations if timeout period expires.
+ **fencing:** cl#5093 - Stonith per device timeout option
+ **fencing:** Clean up if we detect a failed connection
+ **fencing:** Delegate complex self fencing requests - we wont be around to see it to completion
+ **fencing:** Ensure all peers are notified of complex fencing op completion
+ **fencing:** Fix passing of `fence_legacy` parameters containing '='
+ **fencing:** Gracefully handle metadata requests for unknown agents
+ **fencing:** Return cached dynamic target list for busy devices.
+ **fencing:** rhbz#801355 - Abort transition on DC when external fencing operation is detected
+ **fencing:** rhbz#801355 - Merge fence requests for identical operations already in progress.
+ **fencing:** rhbz#801355 - Report fencing operations external of pacemaker to cib
+ **fencing:** Specify the action to perform using action= instead of the older option=
+ **fencing:** Stop building fake metadata for broken agents
+ **fencing:** Tolerate agents that report empty metadata in the admin tool
+ **mcp:** Correctly retry the connection to corosync on failure
+ **mcp:** Do not shut down IPC until the last client exits
+ **mcp:** Prevent use-after-free when running against corosync 1.x
+ **pengine:** Bug cl#5059 - Use the correct action's status when calculating required actions for interleaved clones
+ **pengine:** Bypass online/offline checking resource detection for ping/quorum nodes
+ **pengine:** cl#5044 - `migrate_to` no longer requires `load_stopped` for avoiding possible transition loop
+ **pengine:** cl#5069 - Honor 'on-fail=ignore' even when operation is disabled.
+ **pengine:** cl#5070 - Allow influence of promotion score when multistate rsc is left hand of colocation
+ **pengine:** cl#5072 - Fixes monitor op stopping after rsc promotion.
+ **pengine:** cl#5072 - Fixes pengine regression test failures
+ **pengine:** Correctly set the status for nodes not intended to run Pacemaker
+ **pengine:** Do not append instance numbers to anonymous clones
+ **pengine:** Fix failcount expiration
+ **pengine:** Fix memory leaks found by valgrind
+ **pengine:** Fix use-after-free and use-of-NULL errors detected by coverity
+ **pengine:** Fixes use of colocation scores other than +/- INFINITY
+ **pengine:** Improve detection of rejoining nodes
+ **pengine:** Prevent use-of-NULL when tracing is enabled
+ **pengine:** Stonith resources are allowed to start even if their probes haven't completed on partially active nodes
+ **services:** New class called 'service' which expands to the correct (LSB/systemd/upstart) standard
+ **services:** Support Asynchronous systemd/upstart actions
+ **Tools:** `crm_shadow` - Bug cl#5062 - Correctly set argv[0] when forking a shell process
+ **Tools:** `crm_report:` Always include system logs (if we can find them)
# Pacemaker-1.1.7 (28 Mar 2012)
- 513 commits with 1171 files changed, 90472 insertions(+), 19368 deletions(-)
## Changes since Pacemaker-1.1.6.1
+ **ais:** Prepare for corosync versions using IPC from libqb
+ **cib:** Correctly shutdown in the presence of peers without relying on timers
+ **cib:** Don't halt disk writes if the previous digest is missing
+ **cib:** Determine when there are no peers to respond to our shutdown request and exit
+ **cib:** Ensure no additional messages are processed after we begin terminating
+ **Cluster:** Hook up the callbacks to the corosync quorum notifications
+ **Core:** basename() may modify its input, do not pass in a constant
+ **Core:** Bug cl#5016 - Prevent failures in recurring ops from being lost
+ **Core:** Bug rhbz#800054 - Correctly retrieve heartbeat uuids
+ **Core:** Correctly determine when an XML file should be decompressed
+ **Core:** Correctly track the length of a string without reading from uninitialzied memory (valgrind)
+ **Core:** Ensure signals are handled eventually in the absense of timer sources or IPC messages
+ **Core:** Prevent use-of-NULL in `crm_update_peer()`
+ **Core:** Strip text nodes from on disk xml files
+ **Core:** Support libqb for logging
+ **corosync:** Consistently set the correct uuid with `get_node_uuid()`
+ **Corosync:** Correctly disconnect from corosync variants
+ **Corosync:** Correctly extract the node id from membership udpates
+ **corosync:** Correctly infer lost members from the quorum API
+ **Corosync:** Default to using the nodeid as the node's uuid (instead of uname)
+ **corosync:** Ensure we catch nodes that leave the membership, even if the ringid doesn't change
+ **corosync:** Hook up CPG membership
+ **corosync:** Relax a development assert and gracefully handle the error condition
+ **corosync:** Remove deprecated member of the CFG API
+ **corosync:** Treat `CS_ERR_QUEUE_FULL` the same as `CS_ERR_TRY_AGAIN`
+ **corosync:** Unset the process list when nodes dissappear on us
+ **crmd:** Also purge fencing results when we enter `S_NOT_DC`
+ **crmd:** Bug cl#5015 - Remove the failed operation as well as the resulting fail-count and last-failure attributes
+ **crmd:** Correctly determine when a node can suicide with fencing
+ **crmd:** Election - perform the age comparison only once
+ **crmd:** Fast-track shutdown if we couldn't request it via attrd
+ **crmd:** Leave it up to the PE to decide which ops can/cannot be reload
+ **crmd:** Prevent use-after-free when calling `delete_resource` due to `CRM_OP_REPROBE`
+ **crmd:** Supply format arguments in the correct order
+ **fencing:** Add missing format parameter
+ **fencing:** Add the fencing topology section to the 1.1 configuration schema
+ **fencing:** `fence_legacy` - Drop spurilous host argument from status query
+ **fencing:** `fence_legacy` - Ensure port is available as an environment variable when calling monitor
+ **fencing:** `fence_pcmk` - don't block if nothing is specified on stdin
+ **fencing:** Fix log format error
+ **fencing:** Fix segfault caused by passing garbage to dlsym()
+ **fencing:** Fix use-of-NULL in `process_remote_stonith_query()`
+ **fencing:** Fix use-of-NULL when listing installed devices
+ **fencing:** Implement support for advanced fencing topologies: eg. kdump || (network && disk) || power
+ **fencing:** More gracefully handle failed 'list' operations for devices that only support a single connection
+ **fencing:** Prevent duplicate free when listing devices
+ **fencing:** Prevent uninitialized pointers being passed to free
+ **fencing:** Prevent use-after-free, we may need the query result for subsequent operations
+ **fencing:** Provide enough data to construct an entry in the node's fencing history
+ **fencing:** Standardize on /one/ method for clients to request members be fenced
+ **fencing:** Supress errors when listing all registered devices
+ **mcp:** `corosync_cfg_state_track` was removed from the corosync API, luckily we didnt use it for anything
+ **mcp:** Do not specify a WorkingDirectory in the systemd unit file - startup fails if its not available
+ **mcp:** Set the `HA_quorum_type` env variable consistently with our corosync plugin
+ **mcp:** Shut down if one of our child processes can/should not be respawned
+ **pengine:** Bug cl#5000 - Ensure ordering is preserved when depending on partial sets
+ **pengine:** Bug cl#5028 - Unmanaged services should block shutdown unless in maintenance mode
+ **pengine:** Bug cl#5038 - Prevent restart of anonymous clones when clone-max decreases
+ **pengine:** Bug cl#5007 - Fixes use of colocation constraints with multi-state resources
+ **pengine:** Bug cl#5014 - Prevent asymmetrical order constraints from causing resource stops
+ **pengine:** Bug cl#5000 - Implements ability to create `rsc_order` constraint sets such that A can start after B or C has started.
+ **pengine:** Correctly migrate a resource that has just migrated
+ **pengine:** Correct return from error path
+ **pengine:** Detect reloads of previously migrated resources
+ **pengine:** Ensure post-migration stop actions occur before node shutdown
+ **pengine:** Log as loudly as possible when we cannot shut down a cluster node
+ **pengine:** Reload of a resource no longer causes a restart of dependent resources
+ **pengine:** Support limiting the number of concurrent live migrations
+ **pengine:** Support referencing templates in constraints
+ **pengine:** Support of referencing resource templates in resource sets
+ **pengine:** Support to make tickets standby for relinquishing tickets gracefully
+ **stonith:** A "start" operation of a stonith resource does a "monitor" on the device beyond registering it
+ **stonith:** Bug rhbz#745526 - Ensure `stonith_admin` actually gets called by `fence_pcmk`
+ **Stonith:** Ensure all nodes receive and deliver notifications of the manual override
+ **stonith:** Fix the stonith timeout issue (cl#5009, bnc#727498)
+ **Stonith:** Implement a manual override for when nodes are known to be safely off
+ **Tools:** Bug cl#5003 - Prevent use-after-free in `crm_simlate`
+ **Tools:** `crm_mon` - Support to display tickets (based on Yuusuke Iida's work)
+ **Tools:** `crm_simulate` - Support to grant/revoke/standby/activate tickets from the new ticket state section
+ **Tools:** Implement `crm_node` functionality for native corosync
+ Fix a number of potential problems reported by coverity
# Pacemaker-1.1.6 (31 Aug 2011)
- 376 commits with 1761 files changed, 36259 insertions(+), 140578 deletions(-)
## Changes since Pacemaker-1.1.5
+ **ais:** check for retryable errors when dispatching AIS messages
+ **ais:** Correctly disconnect from Corosync and Cman based clusters
+ **ais:** Followup to previous patch - Ensure we drain the corosync queue of messages when Glib tells us there is input
+ **ais:** Handle IPC error before checking for NULL data (bnc#702907)
+ **cib:** Check the validation version before adding the originator details of a CIB change
+ **cib:** Remove disconnected remote connections from mainloop
+ **cman:** Correctly override existing fenced operations
+ **cman:** Dequeue all the cman emitted events and not only the first one leaving the others in the event's queue.
+ **cman:** Don't call `fenced_join` and `fenced_leave` when notifying cman of a fencing event.
+ **cman:** We need to run the crmd as root for CMAN so that we can ACK fencing operations
+ **Core:** Cancelled and pending operations do not count as failed
+ **Core:** Ensure there is sufficient space for EOS when building short-form option strings
+ **Core:** Fix variable expansion in pkg-config files
+ **Core:** Partial revert of accidental commit in previous patch
+ **Core:** Use dlopen to load heartbeat libraries on-demand
+ **crmd:** Bug lf#2509 - Watch for config option changes from the CIB even if we're not the DC
+ **crmd:** Bug lf#2528 - Introduce a slight delay when creating a transition to allow attrd time to perform its updates
+ **crmd:** Bug lf#2559 - Fail actions that were scheduled for a failed/fenced node
+ **crmd:** Bug lf#2584 - Allow nodes to fence themselves if they're the last one standing
+ **crmd:** Bug lf#2632 - Correctly handle nodes that return faster than stonith
+ **crmd:** Cancel timers for actions that were pending on dead nodes
+ **crmd:** Catch fence operations that claim to succeed but did not really
+ **crmd:** Do not wait for actions that were pending on dead nodes
+ **crmd:** Ensure we do not attempt to perform action on failed nodes
+ **crmd:** Prevent use-of-NULL by `g_hash_table_iter_next()`
+ **crmd:** Recurring actions shouldn't cause the last non-recurring action to be forgotten
+ **crmd:** Store only the last and last failed operation in the CIB
+ **mcp:** dirname() modifies the input path - pass in a copy of the logfile path
+ **mcp:** Enable stack detection logic instead of forcing 'corosync'
+ **mcp:** Fix spelling mistake in systemd service script that prevents shutdown
+ **mcp:** Shut down if corosync becomes unavailable
+ **mcp:** systemd control file is now functional
+ **pengine:** Before migrating an utilization-using resource to a node, take off the load which will no longer run there (lf#2599, bnc#695440)
+ **pengine:** Before migrating an utilization-using resource to a node, take off the load which will no longer run there (regression tests) (lf#2599, bnc#695440)
+ **pengine:** Bug lf#2574 - Prevent shuffling by choosing the correct clone instance to stop
+ **pengine:** Bug lf#2575 - Use uname for migration variables, id is a UUID on heartbeat
+ **pengine:** Bug lf#2581 - Avoid group restart when clone (re)starts on an unrelated node
+ **pengine:** Bug lf#2613, lf#2619 - Group migration after failures and non-default utilization policies
+ **pengine:** Bug suse#707150 - Prevent services being active if dependencies on clones are not satisfied
+ **pengine:** Correctly recognise which recurring operations are currently active
+ **pengine:** Demote from Master does not clear previous errors
+ **pengine:** Ensure restarts due to definition changes cause the start action to be re-issued not probes
+ **pengine:** Ensure role is preserved for unmanaged resources
+ **pengine:** Ensure unmanaged resources have the correct role set so the correct monitor operation is chosen
+ **pengine:** Fix memory leak for re-allocated resources reported by valgrind
+ **pengine:** Implement cluster ticket and deadman
+ **pengine:** Implement resource template
+ **pengine:** Correctly determine the state of multi-state resources with a partial operation history
+ **pengine:** Only allocate master/slave resources once
+ **pengine:** Partial revert of 'Minor code cleanup CS: cf6bca32376c On: 2011-08-15'
+ **pengine:** Resolve memory leak reported by valgrind
+ **pengine:** Restore the ability to save inputs to disk
+ **Shell:** implement -w,--wait option to wait for the transition to finish
+ **Shell:** repair template list command
+ **Shell:** set of commands to examine logs, reports, etc
+ **Stonith:** Consolidate `pcmk_host_map` into `run_stonith_agent` so that it is applied consistently
+ **Stonith:** Deprecate `pcmk_arg_map` for the saner `pcmk_host_argument`
+ **Stonith:** Fix use-of-NULL by `g_hash_table_lookup`
+ **Stonith:** Improved `pcmk_host_map` parsing
+ **Stonith:** Prevent use-of-NULL by `g_hash_table_lookup`
+ **Stonith:** Prevent use-of-NULL when no Linux-HA stonith agents are present
+ **stonith:** Add missing entries to `stonith_error2string()`
+ **Stonith:** Correctly finish sending agent options if the initial write is interrupted
+ **stonith:** Correctly handle synchronous calls
+ **stonith:** Coverity - Correctly construct result list for the query API call
+ **stonith:** Coverity - Remove badly constructed memory allocation from the query API call
+ **stonith:** Ensure completed operations are recorded as such in the history
+ **Stonith:** Ensure device parameters are passed to the daemon during registration
+ **stonith:** Fix use-of-NULL in `stonith_api_device_list()`
+ **stonith:** `stonith_admin` - Prevent use of uninitialized pointer by `--history` command
+ **Tools:** Bug lf#2528 - Make progress when `attrd_updater` is called repeatedly within the dampen interval but with the same value
+ **Tools:** `crm_report` - Correctly extract data from the local node
+ **Tools:** `crm_report` - Remove newlines when detecting the node list
+ **Tools:** `crm_report` - Repair the ability to extract data from the local machine
+ **Tools:** `crm_report` - Report on all detected backtraces
# Pacemaker-1.1.5 (11 Feb 2011)
- 184 commits with 605 files changed, 46103 insertions(+), 26417 deletions(-)
## Changes since Pacemaker-1.1.4
+ Add the ability to delegate sub-sections of the cluster to non-root users via ACLs
Needs to be enabled at compile time, not enabled by default.
+ **ais:** Bug lf#2550 - Report failed processes immediately
+ **Core:** Prevent recently introduced use-after-free in `replace_xml_child()`
+ **Core:** Reinstate the logic that skips past non-XML_ELEMENT_NODE children
+ **Core:** Remove extra calls to xmlCleanupParser resulting in use-after-free
+ **Core:** Repair reference to child-of-child after removal of `xml_child_iter_filter` from `get_message_xml()`
+ **crmd:** Bug lf#2545 - Ensure notify variables are accurate for stop operations
+ **crmd:** Cancel recurring operations while we're still connected to the lrmd
+ **crmd:** Reschedule the `PE_START` action if its not already running when we try to use it
+ **crmd:** Update failcount for failed promote and demote operations
+ **pengine:** Bug lf#2445 - Avoid relying on stickness for stable clone placement
+ **pengine:** Bug lf#2445 - Do not override configured clone stickiness values
+ **pengine:** Bug lf#2493 - Don't imply colocation requirements when applying ordering constraints with clones
+ **pengine:** Bug lf#2495 - Prevent segfault by validating the contents of ordering sets
+ **pengine:** Bug lf#2508 - Correctly reconstruct the status of anonymous cloned groups
+ **pengine:** Bug lf#2518 - Avoid spamming the logs with errors for orphan resources
+ **pengine:** Bug lf#2544 - Prevent unstable clone placement by factoring in the current node's score before all others
+ **pengine:** Bug lf#2554 - target-role alone is not sufficient to promote resources
+ **pengine:** Correct `target_rc` for probes of inactive resources (fix regression introduced by cs:ac3f03006e95)
+ **pengine:** Ensure that fencing has completed for stop actions on stonith-dependent resources (lf#2551)
+ **pengine:** Only update the node's promotion score if the resource is active there
+ **pengine:** Only use the promotion score from the current clone instance
+ **pengine:** Prevent use-of-NULL resulting from variable shadowing spotted by Coverity
+ **pengine:** Prevent use-of-NULL when there is status for an undefined node
+ **pengine:** Prevet use-after-free resulting from unintended recursion when chosing a node to promote master/slave resources
+ **Shell:** don't create empty optional sections (bnc#665131)
+ **Stonith:** Teach `stonith_admin` to automagically obtain the current node attributes for the target from the CIB
+ **tools:** Bug lf#2527 - Prevent use-of-NULL in `crm_simulate`
+ **Tools:** Prevent `crm_resource` commands from being lost due to the use of `cib_scope_local`
# Pacemaker-1.1.4 (20 Oct 2010)
- 169 commits with 772 files changed, 56172 insertions(+), 39309 deletions(-)
## Changes since Pacemaker-1.1.3
+ Italian translation of Clusters from Scratch
+ Significant performance enhancements to the Policy Engine and CIB
+ **cib:** Bug lf#2506 - Don't remove client's when notifications fail, they might just be too big
+ **cib:** Drop invalid/failed connections from the client hashtable
+ **cib:** Ensure all diffs sent to peers have sufficient ordering information
+ **cib:** Ensure non-change diffs can preserve the ordering on the other side
+ **cib:** Fix the feature set check
+ **cib:** Include version information on our synthesised diffs when nothing changed
+ **cib:** Optimize the way we detect group/set ordering changes - 15% speedup
+ **cib:** Prevent false detection of config updates with the new diff format
+ **cib:** Reduce unnecessary copying when comparing xml objects
+ **cib:** Repair the processing of updates sent from peer nodes
+ **cib:** Revert part of a recent commit that purged still valid connections
+ **cib:** The feature set version check is only valid if the current value is non-NULL
+ **Core:** Actually removing diff markers is necessary
+ **Core:** Bug lf#2506 - Drop the compression limit because Heartbeat's IPC code sucks
+ **Core:** Cache Relax-NG schemas - profiling indicates many cycles are wasted needlessly re-parsing them
+ **Core:** Correctly compare against `crm_log_level` in the logging macros
+ **Core:** Correctly extract the version details from a diff
+ **Core:** Correctly hook up the RNG schema cache
+ **Core:** Correctly use `lazy_xml_sort()` for v2 digests
+ **Core:** Don't compress large payload elements unless we're approaching message limits
+ **Core:** Don't insert empty ID tags when applying diffs
+ **Core:** Enable the improve v2 digests
+ **Core:** Ensure ordering is preserved when applying diffs
+ **Core:** Fix the `CRM_CHECK` macro
+ **Core:** Modify the v2 digest algorithm so that some fields are sorted
+ **Core:** Prevent use-after-free when creating a CIB update for a timed out action
+ **Core:** Prevent use-of-NULL when cleaning up RelaxNG data structures
+ **Core:** Provide significant performance improvements by implementing versioned diffs and digests
+ **crmd:** All pending operations should be recorded, even recurring ones with high start delays
+ **crmd:** Don't abort transitions when probes are completed on a node
+ **crmd:** Don't hide stop events that time out - allowing faster recovery in the presence of overloaded hosts
+ **crmd:** Ensure the CIB is always writable on the DC by removing a timing hole
+ **crmd:** Include the correct transition details for timed out operations
+ **crmd:** Prevent use of NULL by making copies of the operation's hash table
+ **crmd:** There's no need to check the cib version from the 'added' part of diff updates
+ **crmd:** Use the supplied timeout for stop actions
+ **mcp:** Ensure valgrind is able to log its output somewhere
+ **mcp:** Use 99/01 for the start/stop sequence to avoid problems with services (such as libvirtd) started by init - Patch from Vladislav Bogdanov
+ **pengine:** Ensure fencing of the DC preceeds the `STONITH_DONE` operation
+ **pengine:** Fix memory leak introduced as part of the conversion to GHashTables
+ **pengine:** Fix memory leak when processing completed migration actions
+ **pengine:** Fix typo leading to use-of-NULL in the new ordering code
+ **pengine:** Free memory in recently introduced helper function
+ **pengine:** lf#2478 - Implement improved handling and recovery of atomic resource migrations
+ **pengine:** Obtain massive speedup by prepending to the list of ordering constraints (which can grow quite large)
+ **pengine:** Optimize the logic for deciding which non-grouped anonymous clone instances to probe for
+ **pengine:** Prevent clones from being stopped because resources colocated with them cannot be active
+ **pengine:** Try to ensure atomic migration ops occur within a single transition
+ **pengine:** Use hashtables instead of linked lists for performance sensitive datastructures
+ **pengine:** Use the original digest algorithm for parameter lists
+ **stonith:** cleanup children on timeout in `fence_legacy`
+ **Stonith:** Fix two memory leaks
+ **Tools:** `crm_shadow` - Avoid replacing the entire configuration (including status)
# Pacemaker-1.1.3 (21 Sep 2010)
- 352 commits with 481 files changed, 14130 insertions(+), 11156 deletions(-)
## Changes since Pacemaker-1.1.2.1
+ **ais:** Bug lf#2401 - Improved processing when the peer crmd processes join/leave
+ **ais:** Correct the logic for conecting to plugin based clusters
+ **ais:** Do not supply a process list in mcp-mode
+ **ais:** Drop support for whitetank in the 1.1 release series
+ **ais:** Get an initial dump of the node membership when connecting to quorum-based clusters
+ **ais:** Guard against saturated cpg connections
+ **ais:** Handle `CS_ERR_TRY_AGAIN` in more cases
+ **ais:** Move the code for finding uid before the fork so that the child does no logging
+ **ais:** Never allow quorum plugins to affect connection to the pacemaker plugin
+ **ais:** Sign everyone up for peer process updates, not just the crmd
+ **ais:** The cluster type needs to be set before initializing classic openais connections
+ **cib:** Also free query result for xpath operations that return more than one hit
+ **cib:** Attempt to resolve memory corruption when forking a child to write the cib to disk
+ **cib:** Correctly free memory when writing out the cib to disk
+ **cib:** Fix the application of unversioned diffs
+ **cib:** Remove old developmental error logging
+ **cib:** Restructure the 'valid peer' check for deciding which instructions to ignore
+ **cman:** Correctly process membership/quorum changes from the pcmk plugin. Allow other message types through untouched
+ **cman:** Filter directed messages not intended for us
+ **cman:** Grab the initial membership when we connect
+ **cman:** Keep the list of peer processes up-to-date
+ **cman:** Make sure our common hooks are called after a cman membership update
+ **cman:** Make sure we can compile without cman present
+ **cman:** Populate sender details for cpg messages
+ **cman:** Update the ringid for cman based clusters
+ **Core:** Correctly unpack `HA_Messages` containing multiple entries with the same name
+ **Core:** `crm_count_member()` should only track nodes that have the full stack up
+ **Core:** New developmental logging system inspired by the kernel and a PoC from Lars Ellenberg
+ **crmd:** All nodes should see status updates, not just he DC
+ **crmd:** Allow non-DC nodes to clear failcounts too
+ **crmd:** Base DC election on process relative uptime
+ **crmd:** Bug lf#2439 - `cancel_op()` can also return `HA_RSCBUSY`
+ **crmd:** Bug lf#2439 - Handle asynchronous notification of resource deletion events
+ **crmd:** Bug lf#2458 - Ensure stop actions always have the relevant resource attributes
+ **crmd:** Disable age as a criteria for cman based clusters, its not reliable enough
+ **crmd:** Ensure we activate the DC timer if we detect an alternate DC
+ **crmd:** Factor the nanosecond component of process uptime in elections
+ **crmd:** Fix assertion failure when performing async resource failures
+ **crmd:** Fix handling of async resource deletion results
+ **crmd:** Include the action for crm graph operations
+ **crmd:** Make sure the membership cache is accurate after a sucessful fencing operation
+ **crmd:** Make sure we always poke the FSA after a transition to clear any `TE_HALT` actions
+ **crmd:** Offer crm-level membership once the peer starts the crmd process
+ **crmd:** Only need to request quorum update for plugin based clusters
+ **crmd:** Prevent assertion failure for stop actions resulting from cs: 3c0bc17c6daf
+ **crmd:** Prevent everyone from loosing DC elections by correctly initializing all relevant variables
+ **crmd:** Prevent segmentation fault
+ **crmd:** several fixes for async resource delete (thanks to beekhof)
+ **crmd:** Use the correct define/size for lrm resource IDs
+ Introduce two new cluster types 'cman' and 'corosync', replaces 'quorum_provider' concept
+ **mcp:** Add missing headers when built without heartbeat support
+ **mcp:** Correctly initialize the string containing the list of active daemons
+ **mcp:** Fix macro expansion in init script
+ **mcp:** Fix the expansion of the pid file in the init script
+ **mcp:** Handle `CS_ERR_TRY_AGAIN` when connecting to libcfg
+ **mcp:** Make sure we can compile the mcp without cman present
+ **mcp:** New master control process for (re)spawning pacemaker daemons
+ **mcp:** Read config early so we can re-initialize logging asap if daemonizing
+ **mcp:** Rename the mcp binary to pacemakerd and create a 'pacemaker' init script
+ **mcp:** Resend our process list after every CPG change
+ **mcp:** Tell chkconfig we need to shut down early on
+ **pengine:** Avoid creating invalid ordering constraints for probes that are not needed
+ **pengine:** Bug lf#1959 - Fail unmanaged resources should not prevent other services from shutting down
+ **pengine:** Bug lf#2422 - Ordering dependencies on partially active groups not observed properly
+ **pengine:** Bug lf#2424 - Use notify oepration definition if it exists in the configuration
+ **pengine:** Bug lf#2433 - No services should be stopped until probes finish
+ **pengine:** Bug lf#2453 - Enforce clone ordering in the absense of colocation constraints
+ **pengine:** Bug lf#2476 - Repair on-fail=block for groups and primitive resources
+ **pengine:** Correctly detect when there is a real failcount that expired and needs to be cleared
+ **pengine:** Correctly handle pseudo action creation
+ **pengine:** Correctly order clone startup after group/clone start
+ **pengine:** Correct use-after-free introduced in the prior patch
+ **pengine:** Do not demote resources because something that requires it can not run
+ **pengine:** Fix colocation for interleaved clones
+ **pengine:** Fix colocation with partially active groups
+ **pengine:** Fix potential use-after-free defect from coverity
+ **pengine:** Fix previous merge
+ **pengine:** Fix use-after-free in `order_actions()` reported by valgrind
+ **pengine:** Make the current data set a global variable so it does not need to be passed around everywhere
+ **pengine:** Prevent endless loop when looking for operation definitions in the configuration
+ **pengine:** Prevent segfault by ensuring the arguments to `do_calculations()` are initialized
+ **pengine:** Rewrite the ordering constraint logic to be simplicity, clarity and maintainability
+ **pengine:** Wait until stonith is available, do not fall back to shutdown for nodes requesting termination
+ Resolve coverity `RESOURCE_LEAK` defects
+ **Shell:** Complete the transition to using `crm_attribute` instead of `crm_failcount` and `crm_standby`
+ **stonith:** Advertise stonith-ng options in the metadata
+ **stonith:** Bug lf#2461 - Prevent segfault by not looking up operations if the hashtable has not been initialized yet
+ **stonith:** Bug lf#2473 - Add the timeout at the top level where the daemon is looking for it
+ **Stonith:** Bug lf#2473 - Ensure stonith operations complete within the timeout and are terminated if they run too long
+ **stonith:** Bug lf#2473 - Ensure timeouts are included for fencing operations
+ **stonith:** Bug lf#2473 - Gracefully handle remote operations that arrive late (after we have done notifications)
+ **stonith:** Correctly parse `pcmk_host_list` parameters that appear on a single line
+ **stonith:** Map poweron/poweroff back to on/off expected by the stonith tool from cluster-glue
+ **stonith:** pass the configuration to the stonith program via environment variables (bnc#620781)
+ **Stonith:** Use the timeout specified by the user
+ Support starting plugin-based Pacemaker clusters with the MCP as well
+ **Tools:** Bug lf#2456 - Fix assertion failure in `crm_resource`
+ **tools:** `crm_node` - Repair the ability to connect to openais based clusters
+ **tools:** `crm_node` - Use the correct short option for --cman
+ **tools:** `crm_report` - corosync.conf wont necessarily contain the text 'pacemaker' anymore
+ **Tools:** `crm_simulate` - Fix use-after-free in when terminating
+ **tools:** `crm_simulate` - Resolve coverity `USE_AFTER_FREE` defect
+ **Tools:** Drop the 'pingd' daemon and resource agent in favor of ocf:pacemaker:ping
+ **Tools:** Fix recently introduced use-of-NULL
+ **Tools:** Fix use-after-free defects from coverity
# Pacemaker-1.1.2 (12 May 2010)
- 339 commits with 708 files changed, 37918 insertions(+), 10584 deletions(-)
## Changes since Pacemaker-1.1.1
+ **ais:** Do not count votes from offline nodes and calculate current votes before sending quorum data
+ **ais:** Ensure the list of active processes sent to clients is always up-to-date
+ **ais:** Look for the correct conf variable for turning on file logging
+ **ais:** Need to find a better and thread-safe way to set `core_uses_pid.` Disable for now.
+ **ais:** Use the threadsafe version of getpwnam
+ **Core:** Bump the feature set due to the new failcount expiry feature
+ **Core:** fix memory leaks exposed by valgrind
+ **Core:** Bug lf#2414 - Prevent use-after-free reported by valgrind when doing xpath based deletions
+ **crmd:** Bug lf#2414 - Prevent use-after-free of the PE connection after it dies
+ **crmd:** Bug lf#2414 - Prevent use-after-free of the stonith-ng connection
+ **crmd:** Bug lf#2401 - Improved detection of partially active peers
+ **crmd:** Bug lf#2379 - Ensure the cluster terminates when the PE is not available
+ **crmd:** Do not allow the `target_rc` to be misused by resource agents
+ **crmd:** Do not ignore action timeouts based on FSA state
+ **crmd:** Ensure we don't get stuck in `S_PENDING` if we lose an election to someone that never talks to us again
+ **crmd:** Fix memory leaks exposed by valgrind
+ **crmd:** Remove race condition that could lead to multiple instances of a clone being active on a machine
+ **crmd:** Send `erase_status_tag()` calls to the local CIB when the DC is fenced, since there is no DC to accept them
+ **crmd:** Use global fencing notifications to prevent secondary fencing operations of the DC
+ **pengine:** Bug lf#2317 - Avoid needless restart of primitive depending on a clone
+ **pengine:** Bug lf#2361 - Ensure clones observe mandatory ordering constraints if the LHS is unrunnable
+ **pengine:** Bug lf#2383 - Combine failcounts for all instances of an anonymous clone on a host
+ **pengine:** Bug lf#2384 - Fix intra-set colocation and ordering
+ **pengine:** Bug lf#2403 - Enforce mandatory promotion (colocation) constraints
+ **pengine:** Bug lf#2412 - Correctly find clone instances by their prefix
+ **pengine:** Do not be so quick to pull the trigger on nodes that are coming up
+ **pengine:** Fix memory leaks exposed by valgrind
+ **pengine:** Rewrite `native_merge_weights()` to avoid Fix use-after-free
+ **Shell:** Bug bnc#590035 - always reload status if working with the cluster
+ **Shell:** Bug bnc#592762 - Default to using the status section from the live CIB
+ **Shell:** Bug lf#2315 - edit multiple `meta_attributes` sets in resource management
+ **Shell:** Bug lf#2221 - enable comments
+ **Shell:** Bug bnc#580492 - implement new cibstatus interface and commands
+ **Shell:** Bug bnc#585471 - new cibstatus import command
+ **Shell:** check timeouts also against the default-action-timeout property
+ **Shell:** new configure filter command
+ **Tools:** `crm_mon` - fix memory leaks exposed by valgrind
# Pacemaker-1.1.1 (16 Feb 2010)
- First public release of Pacemaker 1.1
- Package reference documentation in a doc subpackage
- Move cts into a subpackage so that it can be easily consumed by others
- New stonith daemon that supports global notifications
- Service placement influenced by the physical resources
- A new tool for simulating failures and the cluster’s reaction to them
- Ability to serialize an otherwise unrelated a set of resource actions (eg. Xen migrations)
# Pacemaker-1.0.7 (18 Jan 2010)
- 193 commits with 220 files changed, 15933 insertions(+), 8782 deletions(-)
## Changes since 1.0.5-4
+ **pengine:** Bug 2213 - Ensure groups process location constraints so that clone-node-max works for cloned groups
+ **pengine:** Bug lf#2153 - non-clones should not restart when clones stop/start on other nodes
+ **pengine:** Bug lf#2209 - Clone ordering should be able to prevent startup of dependent clones
+ **pengine:** Bug lf#2216 - Correctly identify the state of anonymous clones when deciding when to probe
+ **pengine:** Bug lf#2225 - Operations that require fencing should wait for 'stonith_complete' not 'all_stopped'.
+ **pengine:** Bug lf#2225 - Prevent clone peers from stopping while another is instance is (potentially) being fenced
+ **pengine:** Correctly anti-colocate with a group
+ **pengine:** Correctly unpack ordering constraints for resource sets to avoid graph loops
+ **Tools:** crm: load help from `crm_cli.txt`
+ **Tools:** crm: resource sets (bnc#550923)
+ **Tools:** crm: support for comments (LF 2221)
+ **Tools:** crm: support for description attribute in resources/operations (bnc#548690)
+ **Tools:** hb2openais: add EVMS2 CSM processing (and other changes) (bnc#548093)
+ **Tools:** hb2openais: do not allow empty rules, clones, or groups (LF 2215)
+ **Tools:** hb2openais: refuse to convert pure EVMS volumes
+ **cib:** Ensure the loop for login message terminates
+ **cib:** Finally fix reliability of receiving large messages over remote plaintext connections
+ **cib:** Fix remote notifications
+ **cib:** For remote connections, default to `CRM_DAEMON_USER` since thats the only one that the cib can validate the password for using PAM
+ **cib:** Remote plaintext - Retry sending parts of the message that did not fit the first time
+ **crmd:** Ensure batch-limit is correctly enforced
+ **crmd:** Ensure we have the latest status after a transition abort
+ **(bnc#547579,547582):** Tools: crm: status section editing support
+ **shell:** Add allow-migrate as allowed meta-attribute (bnc#539968)
+ **Build:** Do not automatically add -L/lib, it could cause 64-bit arches to break
+ **pengine:** Bug lf#2206 - `rsc_order` constraints always use score at the top level
+ **pengine:** Only complain about target-role=master for non m/s resources
+ **pengine:** Prevent non-multistate resources from being promoted through target-role
+ **pengine:** Provide a default action for resource-set ordering
+ **pengine:** Silently fix requires=fencing for stonith resources so that it can be set in `op_defaults`
+ **Tools:** Bug lf#2286 - Allow the shell to accept template parameters on the command line
+ **Tools:** Bug lf#2307 - Provide a way to determin the nodeid of past cluster members
+ **Tools:** crm: add update method to template apply (LF 2289)
+ **Tools:** crm: direct RA interface for ocf class resource agents (LF 2270)
+ **Tools:** crm: direct RA interface for stonith class resource agents (LF 2270)
+ **Tools:** crm: do not add score which does not exist
+ **Tools:** crm: do not consider warnings as errors (LF 2274)
+ **Tools:** crm: do not remove sets which contain id-ref attribute (LF 2304)
+ **Tools:** crm: drop empty attributes elements
+ **Tools:** crm: exclude locations when testing for pathological constraints (LF 2300)
+ **Tools:** crm: fix exit code on single shot commands
+ **Tools:** crm: fix node delete (LF 2305)
+ **Tools:** crm: implement -F (--force) option
+ **Tools:** crm: rename status to cibstatus (LF 2236)
+ **Tools:** crm: revisit configure commit
+ **Tools:** crm: stay in crm if user specified level only (LF 2286)
+ **Tools:** crm: verify changes on exit from the configure level
+ **ais:** Some clients such as `gfs_controld` want a cluster name, allow one to be specified in corosync.conf
+ **cib:** Clean up logic for receiving remote messages
+ **cib:** Create valid notification control messages
+ **cib:** Indicate where the remote connection came from
+ **cib:** Send password prompt to stderr so that stdout can be redirected
+ **cts:** Fix rsh handling when stdout is not required
+ **doc:** Fill in the section on removing a node from an AIS-based cluster
+ **doc:** Update the docs to reflect the 0.6/1.0 rolling upgrade problem
+ **doc:** Use Publican for docbook based documentation
+ **fencing:** stonithd: add metadata for stonithd instance attributes (and support in the shell)
+ **fencing:** stonithd: ignore case when comparing host names (LF 2292)
+ **tools:** Make `crm_mon` functional with remote connections
+ **xml:** Add stopped as a supported role for operations
+ **xml:** Bug bnc#552713 - Treat node unames as text fields not IDs
+ **xml:** Bug lf#2215 - Create an always-true expression for empty rules when upgrading from 0.6
# Pacemaker-1.0.5-4 (29 Oct 2009)
- Include the fixes from CoroSync integration testing
- Move the resource templates - they are not documentation
- Ensure documentation is placed in a standard location
- Exclude documentation that is included elsewhere in the package
- Update the tarball from upstream to version ee19d8e83c2a
+ **cib:** Correctly clean up when both plaintext and tls remote ports are requested
+ **pengine:** Bug bnc#515172 - Provide better defaults for lt(e) and gt(e) comparisions
+ **pengine:** Bug lf#2197 - Allow master instances placemaker to be influenced by colocation constraints
+ **pengine:** Make sure promote/demote pseudo actions are created correctly
+ **pengine:** Prevent target-role from promoting more than master-max instances
+ **ais:** Bug lf#2199 - Prevent expected-quorum-votes from being populated with garbage
+ **ais:** Prevent deadlock - don't try to release IPC message if the connection failed
+ **cib:** For validation errors, send back the full CIB so the client can display the errors
+ **cib:** Prevent use-after-free for remote plaintext connections
+ **crmd:** Bug lf#2201 - Prevent use-of-NULL when running heartbeat
# Pacemaker-1.0.5-3 (13 Oct 2009)
- Update the tarball from upstream to version 38cd629e5c3c
+ **Core:** Bug lf#2169 - Allow dtd/schema validation to be disabled
+ **pengine:** Bug lf#2106 - Not all anonymous clone children are restarted after configuration change
+ **pengine:** Bug lf#2170 - stop-all-resources option had no effect
+ **pengine:** Bug lf#2171 - Prevent groups from starting if they depend on a complex resource which can not
+ **pengine:** Disable resource management if stonith-enabled=true and no stonith resources are defined
+ **pengine:** do not include master score if it would prevent allocation
+ **ais:** Avoid excessive load by checking for dead children every 1s (instead of 100ms)
+ **ais:** Bug rh#525589 - Prevent shutdown deadlocks when running on CoroSync
+ **ais:** Gracefully handle changes to the AIS nodeid
+ **crmd:** Bug bnc#527530 - Wait for the transition to complete before leaving `S_TRANSITION_ENGINE`
+ **crmd:** Prevent use-after-free with `LOG_DEBUG_3`
+ **xml:** Mask the "symmetrical" attribute on `rsc_colocation` constraints (bnc#540672)
+ bnc#520707: Tools: crm: new templates ocfs2 and clvm
+ **Build:** Invert the disable ais/heartbeat logic so that `--without` (ais|heartbeat) is available to rpmbuild
+ **pengine:** Bug lf#2178 - Indicate unmanaged clones
+ **pengine:** Bug lf#2180 - Include node information for all failed ops
+ **pengine:** Bug lf#2189 - Incorrect error message when unpacking simple ordering constraint
+ **pengine:** Correctly log resources that would like to start but can not
+ **pengine:** Stop ptest from logging to syslog
+ **ais:** Include version details in plugin name
+ **crmd:** Requery the resource metadata after every start operation
# Pacemaker-1.0.5-2.1 (21 Aug 2009)
- rebuilt with new openssl
# Pacemaker-1.0.5-2 (19 Aug 2009)
- Add versioned perl dependency as specified by
https://fedoraproject.org/wiki/Packaging/Perl#Packages_that_link_to_libperl
- No longer remove RPATH data, it prevents us finding libperl.so and no other
libraries were being hardcoded
- Compile in support for heartbeat
- Conditionally add heartbeat-devel and corosynclib-devel to the -devel requirements
depending on which stacks are supported
# Pacemaker-1.0.5 (17 Aug 2009)
- Add dependency on resource-agents
- Use the version of the configure macro that supplies `--prefix,` --libdir, etc
- Update the tarball from upstream to version 462f1569a437 (Pacemaker 1.0.5 final)
+ **Tools:** `crm_resource` - Advertise `--move` instead of --migrate
+ **Extra:** New node connectivity RA that uses system ping and `attrd_updater`
+ **crmd:** Note that dc-deadtime can be used to mask the brokeness of some switches
# Tue Aug 11 2009 Ville Skyttä <ville.skytta@iki.fi> - 1.0.5-0.7.c9120a53a6ae.hg
- Use bzipped upstream tarball.
# Wed Jul 29 2009 Andrew Beekhof <andrew@beekhof.net> - 1.0.5-0.6.c9120a53a6ae.hg
- Add back missing build auto* dependencies
- Minor cleanups to the install directive
# Tue Jul 28 2009 Andrew Beekhof <andrew@beekhof.net> - 1.0.5-0.5.c9120a53a6ae.hg
- Add a leading zero to the revision when alphatag is used
# Tue Jul 28 2009 Andrew Beekhof <andrew@beekhof.net> - 1.0.5-0.4.c9120a53a6ae.hg
- Incorporate the feedback from the cluster-glue review
- Realistically, the version is a 1.0.5 pre-release
- Use the global directive instead of define for variables
- Use the haclient/hacluster group/user instead of daemon
- Use the `_configure` macro
- Fix install dependencies
# Pacemaker-1.0.4-3 Fri Jul 24 2009 Andrew Beekhof <andrew@beekhof.net> -
- Initial Fedora checkin
- Include an AUTHORS and license file in each package
- Change the library package name to pacemaker-libs to be more
Fedora compliant
- Remove execute permissions from xml related files
- Reference the new cluster-glue devel package name
- Update the tarball from upstream to version c9120a53a6ae
+ **pengine:** Only prevent migration if the clone dependency is stopping/starting on the target node
+ **pengine:** Bug 2160 - Don't shuffle clones due to colocation
+ **pengine:** New implementation of the resource migration (not stop/start) logic
+ **Tools:** `crm_resource` - Prevent use-of-NULL by requiring a resource name for the -A and -a options
+ **pengine:** Prevent use-of-NULL in `find_first_action()`
# Pacemaker-1.0.4-2 (14 Jul 2009)
- Reference authors from the project AUTHORS file instead of listing in description
- Change Source0 to reference the Mercurial repo
- Cleaned up the summaries and descriptions
- Incorporate the results of Fedora package self-review
# Pacemaker-1.0.4 (04 Jun 2009)
- 209 commits with 266 files changed, 12010 insertions(+), 8276 deletions(-)
## Changes since Pacemaker-1.0.3
+ **(bnc#488291):** ais: do not rely on byte endianness on ptr cast
+ **(bnc#507255):** Tools: crm: delete rsc/op_defaults (these `meta_attributes` are killing me)
+ **(bnc#507255):** Tools: crm: import properly rsc/op_defaults
+ **(LF 2114):** Tools: crm: add support for operation instance attributes
+ **ais:** Bug lf#2126 - Messages replies cannot be routed to transient clients
+ **ais:** Fix compilation for the latest Corosync API (v1719)
+ **attrd:** Do not perform all updates as complete refreshes
+ **cib:** Fix huge memory leak affecting heartbeat-based clusters
+ **Core:** Allow xpath queries to match attributes
+ **Core:** Generate the help text directly from a tool options struct
+ **Core:** Handle differences in 0.6 messaging format
+ **crmd:** Bug lf#2120 - All transient node attribute updates need to go via attrd
+ **crmd:** Correctly calculate how long an FSA action took to avoid spamming the logs with errors
+ **crmd:** Fix another large memory leak affecting Heartbeat based clusters
+ **lha:** Restore compatibility with older versions
+ **pengine:** Bug bnc#495687 - Filesystem is not notified of successful STONITH under some conditions
+ **pengine:** Make running a cluster with STONITH enabled but no STONITH resources an error and provide details on resolutions
+ **pengine:** Prevent use-ofNULL when using resource ordering sets
+ **pengine:** Provide inter-notification ordering guarantees
+ **pengine:** Rewrite the notification code to be understanable and extendable
+ **Tools:** attrd - Prevent race condition resulting in the cluster forgetting the node wishes to shut down
+ **Tools:** crm: regression tests
+ **Tools:** `crm_mon` - Fix smtp notifications
+ **Tools:** `crm_resource` - Repair the ability to query meta attributes
+ **Build:** Bug lf#2105 - Debian package should contain pacemaker doc and crm templates
+ bnc#507255: Tools: crm: handle empty rsc/op_defaults properly
+ bnc#507255: Tools: crm: use the right `obj_type` when creating objects from xml nodes
+ LF#2107: Tools: crm: revisit exit codes in configure
+ **cib:** Do not bother validating updates that only affect the status section
+ **Core:** Include supported stacks in version information
+ **crmd:** Record in the CIB, the cluster infrastructure being used
+ **cts:** Do not combine `crm_standby` arguments - the wrapper can not process them
+ **cts:** Fix the CIBAusdit class
+ **Extra:** Refresh showscores script from Dominik
+ **pengine:** Build a statically linked version of ptest
+ **pengine:** Correctly log the actions for resources that are being recovered
+ **pengine:** Correctly log the occurrence of promotion events
+ **pengine:** Implememt node health based on a patch from Mark Hamzy
+ **Tools:** Add examples to help text outputs
+ **Tools:** crm: catch syntax errors for configure load
+ **Tools:** crm: implement erasing nodes in configure erase
+ **Tools:** crm: work with parents only when managing xml objects
+ **Tools:** `crm_mon` - Add option to run custom notification program on resource operations (Patch by Dominik Klein)
+ **Tools:** `crm_resource` - Allow `--cleanup` to function on complex resources and cluster-wide
+ **Tools:** haresource2cib.py - Patch from horms to fix conversion error
+ **Tools:** Include stack information in `crm_mon` output
+ **Tools:** Two new options (`--stack`, `--constraints`) to `crm_resource`
for querying how a resource is configured
# Pacemaker-1.0.3 (08 Apr 2009)
- 383 commits with 329 files changed, 15471 insertions(+), 15119 deletions(-)
## Changes since Pacemaker-1.0.2
+ Added tag SLE11-HAE-GMC for changeset 9196be9830c2
+ **ais plugin:** Fix quorum calculation (bnc#487003)
+ **ais:** Another memory fix leak in error path
+ **ais:** Bug bnc#482847, bnc#482905 - Force a clean exit of OpenAIS once Pacemaker has finished unloading
+ **ais:** Bug bnc#486858 - Fix `update_member()` to prevent spamming clients with membership events containing no changes
+ **ais:** Centralize all quorum calculations in the ais plugin and allow expected votes to be configured int he cib
+ **ais:** Correctly handle a return value of zero from `openais_dispatch_recv()`
+ **ais:** Disable logging to a file
+ **ais:** Fix memory leak in error path
+ **ais:** IPC messages are only in scope until a response is sent
+ All signal handlers used with `CL_SIGNAL()` need to be as minimal as possible
+ **cib:** Bug bnc#482885 - Simplify CIB disk-writes to prevent data loss. Required a change to the backup filename format
+ **cib:** crmd: Revert part of 9782ab035003. Complex shutdown routines need `G_main_add_SignalHandler` to avoid race coditions
+ **crm:** Avoid infinite loop during crm configure edit (bnc#480327)
+ **crmd:** Avoid a race condition by waiting for the attrd update to trigger a transition automatically
+ **crmd:** Bug bnc#480977 - Prevent extra, partial, shutdown when a node restarts too quickly
+ **crmd:** Bug bnc#480977 - Prevent extra, partial, shutdown when a node restarts too quickly (verified)
+ **crmd:** Bug bnc#489063 - Ensure the DC is always unset after we 'lose' an election
+ **crmd:** Bug BSC#479543 - Correctly find the migration source for timed out `migrate_from` actions
+ **crmd:** Call `crm_peer_init()` before we start the FSA - prevents a race condition when used with Heartbeat
+ **crmd:** Erasing the status section should not be forced to the local node
+ **crmd:** Fix memory leak in cib notication processing code
+ **crmd:** Fix memory leak in transition graph processing
+ **crmd:** Fix memory leaks found by valgrind
+ **crmd:** More memory leaks fixes found by valgrind
+ **fencing:** stonithd: `is_heartbeat_cluster` is a no-no if there is no heartbeat support
+ **pengine:** Bug bnc#466788 - Exclude nodes that can not run resources
+ **pengine:** Bug bnc#466788 - Make colocation based on node attributes work
+ **pengine:** Bug BNC#478687 - Do not crash when clone-max is 0
+ **pengine:** Bug bnc#488721 - Fix id-ref expansion for clones, the doc-root for clone children is not the cib root
+ **pengine:** Bug bnc#490418 - Correctly determine node state for nodes wishing to be terminated
+ **pengine:** Bug LF#2087 - Correctly parse the state of anonymous clones that have multiple instances on a given node
+ **pengine:** Bug lf#2089 - Meta attributes are not inherited by clone children
+ **pengine:** Bug lf#2091 - Correctly restart modified resources that were found active by a probe
+ **pengine:** Bug lf#2094 - Fix probe ordering for cloned groups
+ **pengine:** Bug LF:2075 - Fix large pingd memory leaks
+ **pengine:** Correctly attach orphaned clone children to their parent
+ **pengine:** Correctly handle terminate node attributes that are set to the output from time()
+ **pengine:** Ensure orphaned clone members are hooked up to the parent when clone-max=0
+ **pengine:** Fix memory leak in LogActions
+ **pengine:** Fix the determination of whether a group is active
+ **pengine:** Look up the correct promotion preference for anonymous masters
+ **pengine:** Simplify handling of start failures by changing the default migration-threshold to INFINITY
+ **pengine:** The ordered option for clones no longer causes extra start/stop operations
+ **RA:** Bug bnc#490641 - Shut down `dlm_controld` with -TERM instead of -KILL
+ **RA:** pingd: Set default ping interval to 1 instead of 0 seconds
+ **Resources:** pingd - Correctly tell the ping daemon to shut down
+ **Tools:** Bug bnc#483365 - Ensure the command from `cluster_test` includes
a value for `--log-facility`
+ **Tools:** cli: fix and improve delete command
+ **Tools:** crm: add and implement templates
+ **Tools:** crm: add support for command aliases and some common commands (i.e. cd,exit)
+ **Tools:** crm: create top configuration nodes if they are missing
+ **Tools:** crm: fix parsing attributes for rules (broken by the previous changeset)
+ **Tools:** crm: new ra set of commands
+ **Tools:** crm: resource agents information management
+ **Tools:** crm: `rsc/op_defaults`
+ **Tools:** crm: support for no value attribute in nvpairs
+ **Tools:** crm: the new configure monitor command
+ **Tools:** crm: the new configure node command
+ **Tools:** `crm_mon` - Prevent use-of-NULL when summarizing an orphan
+ **Tools:** hb2openais: create clvmd clone for respawn evmsd in ha.cf
+ **Tools:** hb2openais: fix a serious recursion bug in xml node processing
+ **Tools:** hb2openais: fix ocfs2 processing
+ **Tools:** pingd - prevent double free of getaddrinfo() output in error path
+ **Tools:** The default re-ping interval for pingd should be 1s not 1ms
+ bnc#479049: Tools: crm: add validation of resource type for the configure primitive command
+ bnc#479050: Tools: crm: add help for RA parameters in tab completion
+ bnc#479050: Tools: crm: add tab completion for primitive params/meta/op
+ bnc#479050: Tools: crm: reimplement cluster properties completion
+ bnc#486968: Tools: crm: listnodes function requires no parameters (do not mix completion with other stuff)
+ **ais:** Remove the ugly hack for dampening AIS membership changes
+ **cib:** Fix memory leaks by using `mainloop_add_signal`
+ **cib:** Move more logging to the debug level (was info)
+ **cib:** Overhaul the processing of synchronous replies
+ **Core:** Add library functions for instructing the cluster to terminate nodes
+ **crmd:** Add new expected-quorum-votes option
+ **crmd:** Allow up to 5 retires when an attrd update fails
+ **crmd:** Automatically detect and use new values for `crm_config` options
+ **crmd:** Bug bnc#490426 - Escalated shutdowns stall when there are pending resource operations
+ **crmd:** Clean up and optimize the DC election algorithm
+ **crmd:** Fix memory leak in shutdown
+ **crmd:** Fix memory leaks spotted by Valgrind
+ **crmd:** Ignore join messages from hosts other than our DC
+ **crmd:** Limit the scope of resource updates to the status section
+ **crmd:** Prevent the crmd from being respawned if its told to shut down when it did not ask to be
+ **crmd:** Re-check the election status after membership events
+ **crmd:** Send resource updates via the local CIB during elections
+ **pengine:** Bug bnc#491441 - `crm_mon` does not display operations returning 'uninstalled' correctly
+ **pengine:** Bug lf#2101 - For location constraints, role=Slave is equivalent to role=Started
+ **pengine:** Clean up the API - removed ->children() and renamed ->find_child() to `fine_rsc()`
+ **pengine:** Compress the display of healthy anonymous clones
+ **pengine:** Correctly log the actions for resources that are being recovered
+ **pengine:** Determin a promotion score for complex resources
+ **pengine:** Ensure clones always have a value for globally-unique
+ **pengine:** Prevent orphan clones from being allocated
+ **RA:** controld: Return proper exit code for stop op.
+ **Tools:** Bug bnc#482558 - Fix logging test in `cluster_test`
+ **Tools:** Bug bnc#482828 - Fix quoting in `cluster_test` logging setup
+ **Tools:** Bug bnc#482840 - Include directory path to CTSlab.py
+ **Tools:** crm: add more user input checks
+ **Tools:** crm: do not check resource status of we are working with a shadow
+ **Tools:** crm: fix id-refs and allow reference to top objects (i.e. primitive)
+ **Tools:** crm: ignore comments in the CIB
+ **Tools:** crm: multiple column output would not work with small lists
+ **Tools:** crm: refuse to delete running resources
+ **Tools:** crm: rudimentary if-else for templates
+ **Tools:** crm: Start/stop clones via target-role.
+ **Tools:** `crm_mon` - Compress the node status for healthy and offline nodes
+ **Tools:** `crm_shadow` - Return 0/`cib_ok` when `--create-empty` succeeds
+ **Tools:** `crm_shadow` - Support `-e`, the short form of `--create-empty`
+ **Tools:** Make attrd quieter
+ **Tools:** pingd - Avoid using various clplumbing functions as they seem to leak
+ **Tools:** Reduce pingd logging
# Pacemaker-1.0.2 (16 Feb 2009)
- 441 commits with 639 files changed, 20871 insertions(+), 21594 deletions(-)
## Changes since Pacemaker-1.0.1
+ **Tools:** Bug BNC#450815 - crm cli: do not generate id for the operations tag
+ **ais:** Add support for the new AIS IPC layer
+ **ais:** Always set header.error to the correct default: `SA_AIS_OK`
+ **ais:** Bug BNC#456243 - Ensure the membership cache always contains an entry for the local node
+ **ais:** Bug BNC#456208 - Prevent deadlocks by not logging in the child process before exec()
+ **ais:** By default, disable supprt for the WIP openais IPC patch
+ **ais:** Detect and handle situations where ais and the crm disagree on the node name
+ **ais:** Ensure `crm_peer_seq` is updated after a membership update
+ **ais:** Make sure all IPC header fields are set to sane defaults
+ **ais:** Repair and streamline service load now that whitetank startup functions correctly
+ **build:** create and install doc files
+ **cib:** Allow clients without mainloop to connect to the cib
+ **cib:** CID:18 - Fix use-of-NULL in `cib_perform_op`
+ **cib:** CID:18 - Repair errors introduced in b5a18704477b - Fix use-of-NULL in `cib_perform_op`
+ **cib:** Ensure diffs contain the correct values of `admin_epoch`
+ **cib:** Fix four moderately sized memory leaks detected by Valgrind
+ **Core:** CID:10 - Prevent indexing into an array of schemas with a negative value
+ **Core:** CID:13 - Fix memory leak in `log_data_element`
+ **Core:** CID:15 - Fix memory leak in `crm_get_peer`
+ **Core:** CID:6 - Fix use-of-NULL in `copy_ha_msg_input`
+ **Core:** Fix crash in the membership code preventing node shutdown
+ **Core:** Fix more memory leaks foudn by valgrind
+ **Core:** Prevent unterminated strings after decompression
+ **crmd:** Bug BNC:467995 - Delay marking STONITH operations complete until STONITH tells us so
+ **crmd:** Bug LF:1962 - Do not NACK peers because they are not (yet) in our membership. Just ignore them.
+ **crmd:** Bug LF:2010 - Ensure fencing cib updates create the `node_state` entry if needed to preent re-fencing during cluster startup
+ **crmd:** Correctly handle reconnections to attrd
+ **crmd:** Ensure updates for lost migrate operations indicate which node it tried to migrating to
+ **crmd:** If there are no nodes to finalize, start an election.
+ **crmd:** If there are no nodes to welcome, start an election.
+ **crmd:** Prevent node attribute loss by detecting attrd disconnections immediately
+ **crmd:** Prevent node re-probe loops by ensuring mandatory actions always complete
+ **pengine:** Bug 2005 - Fix startup ordering of cloned stonith groups
+ **pengine:** Bug 2006 - Correctly reprobe cloned groups
+ **pengine:** Bug BNC:465484 - Fix the no-quorum-policy=suicide option
+ **pengine:** Bug LF:1996 - Correctly process disabled monitor operations
+ **pengine:** CID:19 - Fix use-of-NULL in `determine_online_status`
+ **pengine:** Clones now default to globally-unique=false
+ **pengine:** Correctly calculate the number of available nodes for the clone to use
+ **pengine:** Only shoot online nodes with no-quorum-policy=suicide
+ **pengine:** Prevent on-fail settings being ignored after a resource is successfully stopped
+ **pengine:** Prevent use-of-NULL for failed migrate actions in `process_rsc_state()`
+ **pengine:** Remove an optimization for the terminate node attribute that caused the cluster to block indefinitly
+ **pengine:** Repar the ability to colocate based on node attributes other than uname
+ **pengine:** Start the correct monitor operation for unmanaged masters
+ **stonith:** CID:3 - Fix another case of exceptionally poor error handling by the original stonith developers
+ **stonith:** CID:5 - Checking for NULL and then dereferencing it anyway is an interesting approach to error handling
+ **stonithd:** Sending IPC to the cluster is a privileged operation
+ **stonithd:** wrong checks for shmid (0 is a valid id)
+ **Tools:** attrd - Correctly determine when an attribute has stopped changing and should be committed to the CIB
+ **Tools:** Bug 2003 - pingd does not correctly detect failures when the interface is down
+ **Tools:** Bug 2003 - pingd does not correctly handle node-down events on multi-NIC systems
+ **Tools:** Bug 2021 - pingd does not detect sequence wrapping correctly, incorrectly reports nodes offline
+ **Tools:** Bug BNC:468066 - Do not use the result of uname() when its no longer in scope
+ **Tools:** Bug BNC:473265 - `crm_resource -L` dumps core
+ **Tools:** Bug LF:2001 - Transient node attributes should be set via attrd
+ **Tools:** Bug LF:2036 - `crm_resource` cannot set/get parameters for cloned resources
+ **Tools:** Bug LF:2046 - Node attribute updates are lost because attrd can take too long to start
+ **Tools:** Cause the correct clone instance to be failed with
`crm_resource -F`
+ **Tools:** `cluster_test` - Allow the user to select a stack and fix CTS invocation
+ **Tools:** crm cli: allow rename only if the resource is stopped
+ **Tools:** crm cli: catch system errors on file operations
+ **Tools:** crm cli: completion for ids in configure
+ **Tools:** crm cli: drop '-rsc' from attributes for order constraint
+ **Tools:** crm cli: exit with an appropriate exit code
+ **Tools:** crm cli: fix wrong order of action and resource in order constraint
+ **Tools:** crm cli: fox wrong exit code
+ **Tools:** crm cli: improve handling of cib attributes
+ **Tools:** crm cli: new command: configure rename
+ **Tools:** crm cli: new command: configure upgrade
+ **Tools:** crm cli: new command: node delete
+ **Tools:** crm cli: prevent key errors on missing cib attributes
+ **Tools:** crm cli: print long help for help topics
+ **Tools:** crm cli: return on syntax error when parsing score
+ **Tools:** crm cli: `rsc_location` can be without nvpairs
+ **Tools:** crm cli: short node preference location constraint
+ **Tools:** crm cli: sometimes, on errors, level would change on single shot use
+ **Tools:** crm cli: syntax: drop a bunch of commas (remains of help tables conversion)
+ **Tools:** crm cli: verify user input for sanity
+ **Tools:** crm: find expressions within rules (do not always skip xml nodes due to used id)
+ **Tools:** `crm_master` should not define a set id now that attrd is used. Defining one can break lookups
+ **Tools:** `crm_mon` Use the OID assigned to the project by IANA for SNMP traps
+ bnc#445622: Tools: crm cli: improve the node show command and drop node status
+ LF#2009: stonithd: improve timeouts for remote fencing
+ **ais:** Allow dead peers to be removed from membership calculations
+ **ais:** Pass node deletion events on to clients
+ **ais:** Sanitize ipc usage
+ **ais:** Supply the node uname in addtion to the id
+ **Build:** Clean up configure to ensure `NON_FATAL_CFLAGS` is consistent with CFLAGS (ie. includes -g)
+ **Build:** Install `cluster_test`
+ **Build:** Use more restrictive CFLAGS and fix the resulting errors
+ **cib:** CID:20 - Fix potential use-after-free in `cib_native_signon`
+ **Core:** Bug BNC:474727 - Set a maximum time to wait for IPC messages
+ **Core:** CID:12 - Fix memory leak in `decode_transition_magic` error path
+ **Core:** CID:14 - Fix memory leak in `calculate_xml_digest` error path
+ **Core:** CID:16 - Fix memory leak in `date_to_string` error path
+ **Core:** Try to track down the cause of XML parsing errors
+ **crmd:** Bug BNC:472473 - Do not wait excessive amounts of time for lost actions
+ **crmd:** Bug BNC:472473 - Reduce the transition timeout to `action_timeout+network_delay`
+ **crmd:** Do not fast-track the processing of LRM refreshes when there are pending actions.
+ **crmd:** `do_dc_join_filter_offer` - Check the 'join' message is for the current instance before deciding to NACK peers
+ **crmd:** Find option values without having to do a config upgrade
+ **crmd:** Implement shutdown using a transient node attribute
+ **crmd:** Update the crmd options to use dashes instead of underscores
+ **cts:** Add 'cluster reattach' to the suite of automated regression tests
+ **cts:** `cluster_test` - Make some usability enhancements
+ **CTS:** `cluster_test` - suggest a valid port number
+ **CTS:** Fix python import order
+ **cts:** Implement an automated SplitBrain test
+ **CTS:** Remove references to deleted classes
+ **Extra:** Resources - Use `HA_VARRUN` instead of `HA_RSCTMP` for state files as Heartbeat removes `HA_RSCTMP` at startup
+ **HB:** Bug 1933 - Fake `crmd_client_status_callback()` calls because HB does not provide them for already running processes
+ **pengine:** CID:17 - Fix memory leak in `find_actions_by_task` error path
+ **pengine:** CID:7,8 - Prevent hypothetical use-of-NULL in LogActions
+ **pengine:** Defer logging the actions performed on a resource until we have processed ordering constraints
+ **pengine:** Remove the symmetrical attribute of colocation constraints
+ **Resources:** pingd - fix the meta defaults
+ **Resources:** Stateful - Add missing meta defaults
+ **stonithd:** exit if we the pid file cannot be locked
+ **Tools:** Allow attrd clients to specify the ID the attribute should be created with
+ **Tools:** attrd - Allow attribute updates to be performed from a hosts peer
+ **Tools:** Bug LF:1994 - Clean up `crm_verify` return codes
+ **Tools:** Change the pingd defaults to ping hosts once every second (instead of 5 times every 10 seconds)
+ **Tools:** cibmin - Detect resource operations with a view to providing email/snmp/cim notification
+ **Tools:** crm cli: add back symmetrical for order constraints
+ **Tools:** crm cli: generate role in location when converting from xml
+ **Tools:** crm cli: handle shlex exceptions
+ **Tools:** crm cli: keep order of help topics
+ **Tools:** crm cli: refine completion for ids in configure
+ **Tools:** crm cli: replace inf with INFINITY
+ **Tools:** crm cli: streamline cib load and parsing
+ **Tools:** crm cli: supply provider only for ocf class primitives
+ **Tools:** `crm_mon` - Add support for sending mail notifications of resource events
+ **Tools:** `crm_mon` - Include the DC version in status summary
+ **Tools:** `crm_mon` - Sanitize startup and option processing
+ **Tools:** `crm_mon` - switch to event-driven updates and add support for sending snmp traps
+ **Tools:** `crm_shadow` - Replace the `--locate` option with the saner
`--edit`
+ **Tools:** hb2openais: do not remove Evmsd resources, but replace them with clvmd
+ **Tools:** hb2openais: replace crmadmin with `crm_mon`
+ **Tools:** hb2openais: replace the lsb class with ocf for o2cb
+ **Tools:** hb2openais: reuse code
+ **Tools:** LF:2029 - Display an error if `crm_resource` is used to reset the operation history of non-primitive resources
+ **Tools:** Make pingd resilient to attrd failures
+ **Tools:** pingd - fix the command line switches
+ **Tools:** Rename `ccm_tool` to `crm_node`
# Pacemaker-1.0.1 (18 Nov 2008)
- 170 commits with 816 files changed, 7633 insertions(+), 6286 deletions(-)
## Changes since Pacemaker-1.0.1
+ **ais:** Allow the crmd to get callbacks whenever a node state changes
+ **ais:** Create an option for starting the mgmtd daemon automatically
+ **ais:** Ensure `HA_RSCTMP` exists for use by resource agents
+ **ais:** Hook up the openais.conf config logging options
+ **ais:** Zero out the PID of disconnecting clients
+ **cib:** Ensure global updates cause a disk write when appropriate
+ **Core:** Add an extra snaity check to getXpathResults() to prevent segfaults
+ **Core:** Do not redefine `__FUNCTION__` unnecessarily
+ **Core:** Repair the ability to have comments in the configuration
+ **crmd:** Bug:1975 - crmd should wait indefinitely for stonith operations to complete
+ **crmd:** Ensure PE processing does not occur for all error cases in `do_pe_invoke_callback`
+ **crmd:** Requests to the CIB should cause any prior PE calculations to be ignored
+ **heartbeat:** Wait for membership 'up' events before removing stale node status data
+ **pengine:** Bug LF:1988 - Ensure recurring operations always have the correct target-rc set
+ **pengine:** Bug LF:1988 - For unmanaged resources we need to skip the usual `can_run_resources()` checks
+ **pengine:** Ensure the terminate node attribute is handled correctly
+ **pengine:** Fix optional colocation
+ **pengine:** Improve up the detection of 'new' nodes joining the cluster
+ **pengine:** Prevent assert failures in `master_color()` by ensuring unmanaged masters are always reallocated to their current location
+ **Tools:** crm cli: parser: return False on syntax error and None for comments
+ **Tools:** crm cli: unify template and edit commands
+ **Tools:** `crm_shadow` - Show more line number information after validation failures
+ **Tools:** hb2openais: add option to upgrade the CIB to v3.0
+ **Tools:** hb2openais: add U option to getopts and update usage
+ **Tools:** hb2openais: backup improved and multiple fixes
+ **Tools:** hb2openais: fix class/provider reversal
+ **Tools:** hb2openais: fix testing
+ **Tools:** hb2openais: move the CIB update to the end
+ **Tools:** hb2openais: update logging and set logfile appropriately
+ **Tools:** LF:1969 - Attrd never sets any properties in the cib
+ **Tools:** Make attrd functional on OpenAIS
+ **ais:** Hook up the options for specifying the expected number of nodes and total quorum votes
+ **ais:** Look for pacemaker options inside the service block with 'name: pacemaker' instead of creating an addtional configuration block
+ **ais:** Provide better feedback when nodes change nodeids (in openais.conf)
+ **cib:** Always store cib contents on disk with `num_updates=0`
+ **cib:** Ensure remote access ports are cleaned up on shutdown
+ **crmd:** Detect deleted resource operations automatically
+ **crmd:** Erase a nodes resource operations and transient attributes after a successful STONITH
+ **crmd:** Find a more appropriate place to update quorum and refresh attrd attributes
+ **crmd:** Fix the handling of unexpected PE exits to ensure the current CIB is stored
+ **crmd:** Fix the recording of pending operations in the CIB
+ **crmd:** Initiate an attrd refresh `_after_` the status section has been fully repopulated
+ **crmd:** Only the DC should update quorum in an openais cluster
+ Ensure meta attributes are used consistantly
+ **pengine:** Allow group and clone level resource attributes
+ **pengine:** Bug N:437719 - Ensure scores from colocated resources count when allocating groups
+ **pengine:** Prevent lsb scripts from being used in globally unique clones
+ **pengine:** Make a best-effort guess at a migration threshold for people with 0.6 configs
+ **Resources:** controld - ensure we are part of a clone with `globally_unique=false`
+ **Tools:** attrd - Automatically refresh all attributes after a CIB replace operation
+ **Tools:** Bug LF:1985 - `crm_mon` - Correctly process failed cib queries to allow reconnection after cluster restarts
+ **Tools:** Bug LF:1987 - `crm_verify` incorrectly warns of configuration upgrades for the most recent version
+ **Tools:** crm (bnc#441028): check for key error in attributes management
+ **Tools:** `crm_mon` - display the meaning of the operation rc code instead of the status
+ **Tools:** `crm_mon` - Fix the display of timing data
+ **Tools:** `crm_verify` - check that we are being asked to validate a complete config
+ **xml:** Relax the restriction on the contents of `rsc_locaiton.node`
# Pacemaker-1.0.0 (16 Oct 2008)
- 261 commits with 3021 files changed, 244985 insertions(+), 111596 deletions(-)
## Changes since f805e1b30103
+ add the crm cli program
+ **ais:** Move the service id definition to a common location and make sure it is always used
+ **build:** rename hb2openais.sh to .in and replace paths with vars
+ **cib:** Implement `--create` for `crm_shadow`
+ **cib:** Remove dead files
+ **Core:** Allow the expected number of quorum votes to be configrable
+ **Core:** `cl_malloc` and friends were removed from Heartbeat
+ **Core:** Only call xmlCleanupParser() if we parsed anything. Doing so unconditionally seems to cause a segfault
+ **hb2openais.sh:** improve pingd handling; several bugs fixed
+ **hb2openais:** fix clone creation; replace EVMS strings
+ new hb2openais.sh conversion script
+ **pengine:** Bug LF:1950 - Ensure the current values for all notification variables are always set (even if empty)
+ **pengine:** Bug LF:1955 - Ensure unmanaged masters are unconditionally repromoted to ensure they are monitored correctly.
+ **pengine:** Bug LF:1955 - Fix another case of filtering causing unmanaged master failures
+ **pengine:** Bug LF:1955 - Umanaged mode prevents master resources from being allocated correctly
+ **pengine:** Bug N:420538 - Anit-colocation caused a positive node preference
+ **pengine:** Correctly handle unmanaged resources to prevent them from being started elsewhere
+ **pengine:** `crm_resource` - Fix the `--migrate` command
+ **pengine:** MAke stonith-enabled default to true and warn if no STONITH resources are found
+ **pengine:** Make sure orphaned clone children are created correctly
+ **pengine:** Monitors for unmanaged resources do not need to wait for start/promote/demote actions to complete
+ **stonithd (LF 1951):** fix remote stonith operations
+ **stonithd:** fix handling of timeouts
+ **stonithd:** fix logic for stonith resource priorities
+ **stonithd:** implement the fence-timeout instance attribute
+ **stonithd:** initialize value before reading fence-timeout
+ **stonithd:** set timeouts for fencing ops to the timeout of the start op
+ **stonithd:** stonith rsc priorities (new feature)
+ **Tools:** Add hb2openais - a tool for upgrading a Heartbeat cluster to use OpenAIS instead
+ **Tools:** `crm_verify` - clean up the upgrade logic to prevent crash on invalid configurations
+ **Tools:** Make pingd functional on Linux
+ Update version numbers for 1.0 candidates
+ **ais:** Add support for a synchronous call to retrieve the nodes nodeid
+ **ais:** Use the agreed service number
+ **Build:** Reliably detect heartbeat libraries during configure
+ **Build:** Supply prototypes for libreplace functions when needed
+ **Build:** Teach configure how to find corosync
+ **Core:** Provide better feedback if Pacemaker is started by a stack it does not support
+ **crmd:** Avoid calling GHashTable functions with NULL
+ **crmd:** Delay raising `I_ERROR` when the PE exits until we have had a chance to save the current CIB
+ **crmd:** Hook up the stonith-timeout option to stonithd
+ **crmd:** Prevent potential use-of-NULL in `global_timer_callback`
+ **crmd:** Rationalize the logging of graph aborts
+ **pengine:** Add a `stonith_timeout` option and remove new options that are better set in `rsc_defaults`
+ **pengine:** Allow external entities to ask for a node to be shot by creating a terminate=true transient node attribute
+ **pengine:** Bug LF:1950 - Notifications do not contain all documented resource state fields
+ **pengine:** Bug N:417585 - Do not restart group children whos individual score drops below zero
+ **pengine:** Detect clients that disconnect before receiving their reply
+ **pengine:** Implement a true maintenance mode
+ **pengine:** Implement on-fail=standby for NTT. Derived from a patch by Satomi TANIGUCHI
+ **pengine:** Print the correct message when stonith is disabled
+ **pengine:** ptest - check the input is valid before proceeding
+ **pengine:** Revert group stickiness to the 'old way'
+ **pengine:** Use the correct attribute for action 'requires' (was prereq)
+ **stonithd:** Fix compilation without full heartbeat install
+ **stonithd:** exit with better code on empty host list
+ **tools:** Add a new regression test for CLI tools
+ **tools:** `crm_resource` - return with non-zero when a resource migration command is invalid
+ **tools:** `crm_shadow` - Allow the admin to start with an empty CIB (and no cluster connection)
+ **xml:** pacemaker-0.7 is now an alias for the 1.0 schema
# Pacemaker-0.7.3 (22 Sep 2008)
- 133 commits with 89 files changed, 7492 insertions(+), 1125 deletions(-)
## Changes since f805e1b30103
+ **Tools:** add the crm cli program
+ **Core:** `cl_malloc` and friends were removed from Heartbeat
+ **Core:** Only call xmlCleanupParser() if we parsed anything. Doing so unconditionally seems to cause a segfault
+ new hb2openais.sh conversion script
+ **pengine:** Bug LF:1950 - Ensure the current values for all notification variables are always set (even if empty)
+ **pengine:** Bug LF:1955 - Ensure unmanaged masters are unconditionally repromoted to ensure they are monitored correctly.
+ **pengine:** Bug LF:1955 - Fix another case of filtering causing unmanaged master failures
+ **pengine:** Bug LF:1955 - Umanaged mode prevents master resources from being allocated correctly
+ **pengine:** Bug N:420538 - Anit-colocation caused a positive node preference
+ **pengine:** Correctly handle unmanaged resources to prevent them from being started elsewhere
+ **pengine:** `crm_resource` - Fix the `--migrate` command
+ **pengine:** MAke stonith-enabled default to true and warn if no STONITH resources are found
+ **pengine:** Make sure orphaned clone children are created correctly
+ **pengine:** Monitors for unmanaged resources do not need to wait for start/promote/demote actions to complete
+ **stonithd (LF 1951):** fix remote stonith operations
+ **Tools:** `crm_verify` - clean up the upgrade logic to prevent crash on invalid configurations
+ **ais:** Add support for a synchronous call to retrieve the nodes nodeid
+ **ais:** Use the agreed service number
+ **pengine:** Allow external entities to ask for a node to be shot by creating a terminate=true transient node attribute
+ **pengine:** Bug LF:1950 - Notifications do not contain all documented resource state fields
+ **pengine:** Bug N:417585 - Do not restart group children whos individual score drops below zero
+ **pengine:** Implement a true maintenance mode
+ **pengine:** Print the correct message when stonith is disabled
+ **stonithd:** exit with better code on empty host list
+ **xml:** pacemaker-0.7 is now an alias for the 1.0 schema
# Pacemaker-0.7.1 (20 Aug 2008)
- 184 commits with 513 files changed, 43408 insertions(+), 43783 deletions(-)
## Changes since 0.7.0-19
+ Fix compilation when GNUTLS isn't found
+ **admin:** Fix use-after-free in `crm_mon`
+ **Build:** Remove testing code that prevented heartbeat-only builds
+ **cib:** Use single quotes so that the xpath queries for nvpairs will succeed
+ **crmd:** Always connect to stonithd when the TE starts and ensure we notice if it dies
+ **crmd:** Correctly handle a dead PE process
+ **crmd:** Make sure async-failures cause the failcount to be incremented
+ **pengine:** Bug LF:1941 - Handle failed clone instance probes when clone-max < #nodes
+ **pengine:** Parse resource ordering sets correctly
+ **pengine:** Prevent use-of-NULL - order->rsc_rh will not always be non-NULL
+ **pengine:** Unpack colocation sets correctly
+ **Tools:** `crm_mon` - Prevent use-of-NULL for orphaned resources
+ **ais:** Add support for a synchronous call to retrieve the nodes nodeid
+ **ais:** Allow transient clients to receive membership updates
+ **ais:** Avoid double-free in error path
+ **ais:** Include in the mebership nodes for which we have not determined their hostname
+ **ais:** Spawn the PE from the ais plugin instead of the crmd
+ **cib:** By default, new configurations use the latest schema
+ **cib:** Clean up the CIB if it was already disconnected
+ **cib:** Only increment `num_updates` if something actually changed
+ **cib:** Prevent use-after-free in client after abnormal termination of the CIB
+ **Core:** Fix memory leak in xpath searches
+ **Core:** Get more details regarding parser errors
+ **Core:** Repair `expand_plus_plus` - do not call char2score on unexpanded values
+ **Core:** Switch to the libxml2 parser - its significantly faster
+ **Core:** Use a libxml2 library function for xml -> text conversion
+ **crmd:** Asynchronous failure actions have no parameters
+ **crmd:** Avoid calling glib functions with NULL
+ **crmd:** Do not allow an election to promote a node from `S_STARTING`
+ **crmd:** Do not vote if we have not completed the local startup
+ **crmd:** Fix `te_update_diff()` now that `get_object_root()` functions differently
+ **crmd:** Fix the lrmd xpath expressions to not contain quotes
+ **crmd:** If we get a join offer during an election, better restart the election
+ **crmd:** No further processing is needed when using the LRMs API call for failing resources
+ **crmd:** Only update have-quorum if the value changed
+ **crmd:** Repair the input validation logic in `do_te_invoke`
+ **cts:** CIBs can no longer contain comments
+ **cts:** Enable a bunch of tests that were incorrectly disabled
+ **cts:** The libxml2 parser wont allow v1 resources to use integers as parameter names
+ Do not use the cluster UID and GID directly. Look them up based on the configured value of `HA_CCMUSER`
+ Fix compilation when heartbeat is not supported
+ **pengine:** Allow groups to be involved in optional ordering constraints
+ **pengine:** Allow sets of operations to be reused by multiple resources
+ **pengine:** Bug LF:1941 - Mark extra clone instances as orphans and do not show inactive ones
+ **pengine:** Determin the correct migration-threshold during resource expansion
+ **pengine:** Implement no-quorum-policy=suicide (FATE #303619)
+ **pengine:** Clean up resources after stopping old copies of the PE
+ **pengine:** Teach the PE how to stop old copies of itself
+ **Tools:** Backport `hb_report` updates
+ **Tools:** `cib_shadow` - On create, spawn a new shell with `CIB_shadow` and PS1 set accordingly
+ **Tools:** Rename `cib_shadow` to `crm_shadow`
# Pacemaker-0.7.0-19 (18 Jul 2008)
- 108 commits with 216 files changed, 4632 insertions(+), 4173 deletions(-)
## Changes added since unstable-0.7
+ **admin:** Fix use-after-free in `crm_mon`
+ **ais:** Change the tag for the ais plugin to "pacemaker" (used in openais.conf)
+ **ais:** Log terminated processes as an error
+ **cib:** Performance - Reorganize things to avoid calculating the XML diff twice
+ **pengine:** Bug LF:1941 - Handle failed clone instance probes when clone-max < #nodes
+ **pengine:** Fix memory leak in action2xml
+ **pengine:** Make `OCF_ERR_ARGS` a node-level error rather than a cluster-level one
+ **pengine:** Properly handle clones that are not installed on all nodes
+ **admin:** cibadmin - Show any validation errors if the upgrade failed
+ **admin:** `cib_shadow` - Implement `--locate` to display the underlying filename
+ **admin:** `cib_shadow` - Implement a `--diff` option
+ **admin:** `cib_shadow` - Implement a `--switch` option
+ **admin:** `crm_resource` - create more compact constraints that do not use lifetime (which is deprecated)
+ **ais:** Approximate `born_on` for OpenAIS based clusters
+ **cib:** Remove `do_id_check`, it is a poor substitute for ID validation by a schema
+ **cib:** Skip construction of pre-notify messages if no-one wants one
+ **Core:** Attempt to streamline some key functions to increase performance
+ **Core:** Clean up XML parser after validation
+ **crmd:** Detect and optimize the CRMs behavior when processing diffs of an LRM refresh
+ Fix memory leaks when resetting the name of an XML object
+ **pengine:** Prefer the current location if it is one of a group of nodes with the same (highest) score
# Pacemaker-0.7.0 (25 Jun 2008)
- 439 commits with 676 files changed, 41310 insertions(+), 52071 deletions(-)
## Changes added since stable-0.6
+ A new tool for setting up and invoking CTS
+ **Admin:** All tools now use `--node` (-N) for specifying node unames
+ **Admin:** All tools now use `--xml-file` (-x) and `--xml-text` (-X) for specifying where to find XML blobs
+ **cib:** Cleanup the API - remove redundant input fields
+ **cib:** Implement `CIB_shadow` - a facility for making and testing changes before uploading them to the cluster
+ **cib:** Make registering per-op callbacks an API call and renamed (for clarity) the API call for requesting notifications
+ **Core:** Add a facility for automatically upgrading old configurations
+ **Core:** Adopt libxml2 as the XML processing library - all external clients need to be recompiled
+ **Core:** Allow sending TLS messages larger than the MTU
+ **Core:** Fix parsing of time-only ISO dates
+ **Core:** Smarter handling of XML values containing quotes
+ **Core:** XML memory corruption - catch, and handle, cases where we are overwriting an attribute value with itself
+ **Core:** The xml ID type does not allow UUIDs that start with a number
+ **Core:** Implement XPath based versions of query/delete/replace/modify
+ **Core:** Remove some HA2.0.(3,4) compatibility code
+ **crmd:** Overhaul the detection of nodes that are starting vs. failed
+ **pengine:** Bug LF:1459 - Allow failures to expire
+ **pengine:** Have the PE do non-persistent configuration upgrades before performing calculations
+ **pengine:** Replace failure-stickiness with a simple 'migration-threshold'
+ **tengine:** Simplify the design by folding the tengine process into the crmd
+ **Admin:** Bug LF:1438 - Allow the list of all/active resource operations to be queried by `crm_resource`
+ **Admin:** Bug LF:1708 - `crm_resource` should print a warning if an attribute is already set as a meta attribute
+ **Admin:** Bug LF:1883 - `crm_mon` should display fail-count and operation history
+ **Admin:** Bug LF:1883 - `crm_mon` should display operation timing data
+ **Admin:** Bug N:371785 - `crm_resource` -C does not also clean up fail-count attributes
+ **Admin:** `crm_mon` - include timing data for failed actions
+ **ais:** Read options from the environment since objdb is not completely usable yet
+ **cib:** Add sections for `op_defaults` and `rsc_defaults`
+ **cib:** Better matching notification callbacks (for detecting duplicates and removal)
+ **cib:** Bug LF:1348 - Allow rules and attribute sets to be referenced for use in other objects
+ **cib:** BUG LF:1918 - By default, all cib calls now timeout after 30s
+ **cib:** Detect updates that decrease the version tuple
+ **cib:** Implement a client-side operation timeout - Requires LHA update
+ **cib:** Implement callbacks and async notifications for remote connections
+ **cib:** Make cib->cmds->update() an alias for modify at the API level (also implemented in cibadmin)
+ **cib:** Mark the CIB as disconnected if the IPC connection is terminated
+ **cib:** New call option 'cib_can_create' which can be passed to modify actions - allows the object to be created if it does not exist yet
+ **cib:** Reimplement get|set|delete attributes using XPath
+ **cib:** Remove some useless parts of the API
+ **cib:** Remove the 'attributes' scaffolding from the new format
+ **cib:** Implement the ability for clients to connect to remote servers
+ **Core:** Add support for validating xml against RelaxNG schemas
+ **Core:** Allow more than one item to be modified/deleted in XPath based operations
+ **Core:** Fix the `sort_pairs` function for creating sorted xml objects
+ **Core:** iso8601 - Implement `subtract_duration` and fix `subtract_time`
+ **Core:** Reduce the amount of xml copying
+ **Core:** Support value='value+=N' XML updates (in addtion to value='value++')
+ **crmd:** Add support for `lrm_ops->fail_rsc` if its available
+ **crmd:** HB - watch link status for node leaving events
+ **crmd:** Bug LF:1924 - Improved handling of lrmd disconnects and shutdowns
+ **crmd:** Do not wait for actions with a `start_delay` over 5 minutes. Confirm them immediately
+ **pengine:** Bug LF:1328 - Do not fencing nodes in clusters without managed resources
+ **pengine:** Bug LF:1461 - Give transient node attributes (in <status/>) preference over persistent ones (in <nodes/>)
+ **pengine:** Bug LF:1884, Bug LF:1885 - Implement N:M ordering and colocation constraints
+ **pengine:** Bug LF:1886 - Create a resource and operation 'defaults' config section
+ **pengine:** Bug LF:1892 - Allow recurring actions to be triggered at known times
+ **pengine:** Bug LF:1926 - Probes should complete before stop actions are invoked
+ **pengine:** Fix the standby when its set as a transient attribute
+ **pengine:** Implement a global 'stop-all-resources' option
+ **pengine:** Implement cibpipe, a tool for performing/simulating config changes "offline"
+ **pengine:** We do not allow colocation with specific clone instances
+ **Tools:** pingd - Implement a stack-independent version of pingd
+ **xml:** Ship an xslt for upgrading from 0.6 to 0.7
# Pacemaker-0.6.5 (19 Jun 2008)
- 48 commits with 37 files changed, 1204 insertions(+), 234 deletions(-)
## Changes since Pacemaker-0.6.4
+ **Admin:** Repair the ability to delete failcounts
+ **ais:** Audit IPC handling between the AIS plugin and CRM processes
+ **ais:** Have the plugin create needed /var/lib directories
+ **ais:** Make sure the sync and async connections are assigned correctly (not swapped)
+ **cib:** Correctly detect configuration changes - `num_updates` does not count
+ **pengine:** Apply stickiness values to the whole group, not the individual resources
+ **pengine:** Bug N:385265 - Ensure groups are migrated instead of remaining partially active on the current node
+ **pengine:** Bug N:396293 - Enforce mandatory group restarts due to ordering constraints
+ **pengine:** Correctly recover master instances found active on more than one node
+ **pengine:** Fix memory leaks reported by Valgrind
+ **Admin:** `crm_mon` - Misc improvements from Satomi Taniguchi
+ Bug LF#1900 - Resource stickiness should not allow placement in asynchronous clusters
+ **crmd:** Ensure joins are completed promptly when a node taking part dies
+ **pengine:** Avoid clone instance shuffling in more cases
+ **pengine:** Bug LF:1906 - Remove an optimization in `native_merge_weights()` causing group scores to behave eratically
+ **pengine:** Make use of `target_rc` data to correctly process resource operations
+ **pengine:** Prevent a possible use of NULL in `sort_clone_instance()`
+ **tengine:** Include target rc in the transition key - used to correctly determin operation failure
# Pacemaker-0.6.4 (22 May 2008)
- 55 commits with 199 files changed, 7103 insertions(+), 12378 deletions(-)
## Changes since Pacemaker-0.6.3
+ **crmd:** Bug LF:1881 LF:1882 - Overhaul the logic for operation cancelation and deletion
+ **crmd:** Bug LF:1894 - Make sure cancelled recurring operations are cleaned out from the CIB
+ **pengine:** Bug N:387749 - Colocation with clones causes unnecessary clone instance shuffling
+ **pengine:** Ensure 'master' monitor actions are cancelled `_before_` we demote the resource
+ **pengine:** Fix assert failure leading to core dump - make sure variable is properly initialized
+ **pengine:** Make sure 'slave' monitoring happens after the resource has been demoted
+ **pengine:** Prevent failure stickiness underflows (where too many failures become a `_positive_` preference)
+ **Admin:** `crm_mon` - Only complain if the output file could not be opened
+ **Common:** `filter_action_parameters` - enable legacy handling only for older versions
+ **pengine:** Bug N:385265 - The failure stickiness of group children is ignored until it reaches -INFINITY
+ **pengine:** Implement master and clone colocation by exlcuding nodes rather than setting ones score to INFINITY (similar to cs: 756afc42dc51)
+ **tengine:** Bug LF:1875 - Correctly find actions to cancel when their node leaves the cluster
# Pacemaker-0.6.3 (23 Apr 2008)
- 117 commits with 354 files changed, 19094 insertions(+), 11338 deletions(-)
## Changes since Pacemaker-0.6.2
+ **Admin:** Bug LF:1848 - `crm_resource` - Pass set name and id to `delete_resource_attr()` in the correct order
+ **Build:** SNMP has been moved to the management/pygui project
+ **crmd:** Bug LF1837 - Unmanaged resources prevent crmd from shutting down
+ **crmd:** Prevent use-after-free in lrm interface code (Patch based on work by Keisuke MORI)
+ **pengine:** Allow the cluster to make progress by not retrying failed demote actions
+ **pengine:** Anti-colocation with slave should not prevent master colocation
+ **pengine:** Bug LF 1768 - Wait more often for STONITH ops to complete before starting resources
+ **pengine:** Bug LF1836 - Allow is-managed-default=false to be overridden by individual resources
+ **pengine:** Bug LF185 - Prevent pointless master/slave instance shuffling by ignoring the master-pref of stopped instances
+ **pengine:** Bug N-191176 - Implement interleaved ordering for clone-to-clone scenarios
+ **pengine:** Bug N-347004 - Ensure clone notifications are always sent when an instance is stopped/started
+ **pengine:** Bug N-347004 - Include notification ordering is correct for interleaved clones
+ **pengine:** Bug PM-11 - Directly link `probe_complete` to starting clone instances
+ **pengine:** Bug PM1 - Fix setting failcounts when applied to complex resources
+ **pengine:** Bug PM12, LF1648 - Extensive revision of group ordering
+ **pengine:** Bug PM7 - Ensure masters are always demoted before they are stopped
+ **pengine:** Create probes after allocation to allow smarter handling of anonymous clones
+ **pengine:** Do not prioritize clone instances that must be moved
+ **pengine:** Fix error in previous commit that allowed more than the required number of masters to be promoted
+ **pengine:** Group start ordering fixes
+ **pengine:** Implement promote/demote ordering for cloned groups
+ **tengine:** Repair failcount updates
+ **tengine:** Use the correct offset when updating failcount
+ **Admin:** Add a summary output that can be easily parsed by CTS for audit purposes
+ **Build:** Make configure fail if bz2 or libxml2 are not present
+ **Build:** Re-instate a better default for LCRSODIR
+ **CIB:** Bug LF-1861 - Filter irrelvant error status from synchronous CIB clients
+ **Core:** Bug 1849 - Invalid conversion of ordinal leap year to gregorian date
+ **Core:** Drop compatibility code for 2.0.4 and 2.0.5 clusters
+ **crmd:** Bug LF-1860 - Automatically cancel recurring ops before demote and promote operations (not only stops)
+ **crmd:** Save the current CIB contents if we detect the PE crashed
+ **pengine:** Bug LF:1866 - Fix version check when applying compatibility handling for failed start operations
+ **pengine:** Bug LF:1866 - Restore the ability to have start failures not be fatal
+ **pengine:** Bug PM1 - Failcount applies to all instances of non-unique clone
+ **pengine:** Correctly set the state of partially active master/slave groups
+ **pengine:** Do not claim to be stopping an already stopped orphan
+ **pengine:** Ensure `implies_left` ordering constraints are always effective
+ **pengine:** Indicate each resources 'promotion' score
+ **pengine:** Prevent a possible use-of-NULL
+ **pengine:** Reprocess the current action if it changed (so that any prior dependencies are updated)
+ **tengine:** Bug LF-1859 - Wait for fail-count updates to complete before terminating the transition
+ **tengine:** Bug LF:1859 - Do not abort graphs due to our own failcount updates
+ **tengine:** Bug LF:1859 - Prevent the TE from interupting itself
# Pacemaker-0.6.2 (14 Feb 2008)
- 11 commits with 7 files changed, 58 insertions(+), 18 deletions(-)
## Changes since Pacemaker-0.6.1
+ **haresources2cib.py:** set default-action-timeout to the default (20s)
+ **haresources2cib.py:** update ra parameters lists
+ **SNMP:** Allow the snmp subagent to be built (patch from MATSUDA, Daiki)
+ **Tools:** Make sure the autoconf variables in haresources2cib are expanded
# Pacemaker-0.6.1 (12 Feb 2008)
- 25 commits with 37 files changed, 1323 insertions(+), 227 deletions(-)
## Changes since Pacemaker-0.6.0
+ **CIB:** Ensure changes to top-level attributes (like `admin_epoch)` cause a disk write
+ **CIB:** Ensure the archived file hits the disk before returning
+ **CIB:** Repair the ability to do 'atomic increment' updates (value="value++")
+ **crmd:** Bug #7 - Connecting to the crmd immediately after startup causes use-of-NULL
+ **CIB:** Mask `cib_diff_resync` results from the caller - they do not need to know
+ **crmd:** Delay starting the IPC server until we are fully functional
+ **CTS:** Fix the startup patterns
+ **pengine:** Bug 1820 - Allow the first resource in a group to be migrated
+ **pengine:** Bug 1820 - Check the colocation dependencies of resources to be migrated
# Pacemaker-0.6.0 (14 Jan 2008)
- 347 commits with 2272 files changed, 132508 insertions(+), 305991 deletions(-)
- This is the first release of the Pacemaker Cluster Resource Manager formerly part of Heartbeat.
- For those looking for the GUI, mgmtd, CIM or TSA components, they are now found in
the new pacemaker-pygui project. Build dependencies prevent them from being
included in Heartbeat (since the built-in CRM is no longer supported) and,
being non-core components, are not included with Pacemaker.
- Test hardware:
+ 6-node vmware cluster (sles10-sp1/256MB/vmware stonith) on a single host (opensuse10.3/2GB/2.66GHz Quad Core2)
+ 7-node EMC Centera cluster (sles10/512MB/2GHz Xeon/ssh stonith)
- Notes: Heartbeat Stack
+ All testing was performed with STONITH enabled
+ The CRM was enabled using the "crm respawn" directive
- Notes: OpenAIS Stack
+ This release contains a preview of support for the OpenAIS cluster stack
+ The current release of the OpenAIS project is missing two important
patches that we require. OpenAIS packages containing these patches are
available for most major distributions at:
http://download.opensuse.org/repositories/server:/ha-clustering
+ The OpenAIS stack is not currently recommended for use in clusters that
have shared data as STONITH support is not yet implimented
+ pingd is not yet available for use with the OpenAIS stack
+ 3 significant OpenAIS issues were found during testing of 4 and 6 node
clusters. We are activly working together with the OpenAIS project to
get these resolved.
- Pending bugs encountered during testing:
+ OpenAIS #1736 - Openais membership took 20s to stabilize
+ Heartbeat #1750 - `ipc_bufpool_update:` magic number in head does not match
+ OpenAIS #1793 - Assertion failure in `memb_state_gather_enter()`
+ OpenAIS #1796 - Cluster message corruption
## Changes since Heartbeat-2.1.2-24
+ Add OpenAIS support
+ **Admin:** `crm_uuid` - Look in the right place for Heartbeat UUID files
+ **admin:** Exit and indicate a problem if the crmd exits while crmadmin is performing a query
+ **cib:** Fix `CIB_OP_UPDATE` calls that modify the whole CIB
+ **cib:** Fix compilation when supporting the heartbeat stack
+ **cib:** Fix memory leaks caused by the switch to `get_message_xml()`
+ **cib:** `HA_VALGRIND_ENABLED` needs to be set `_and_` set to 1|yes|true
+ **cib:** Use `get_message_xml()` in preference to `cl_get_struct()`
+ **cib:** Use the return value from call to write() in `cib_send_plaintext()`
+ **Core:** ccm nodes can legitimately have a node id of 0
+ **Core:** Fix peer-process tracking for the Heartbeat stack
+ **Core:** Heartbeat does not send status notifications for nodes that were already part of the cluster. Fake them instead
+ **CRM:** Add children to `HA_Messages` such that the field name matches `F_XML_TAGNAME`
+ **crm:** Adopt a more flexible appraoch to enabling Valgrind
+ **crm:** Fix compilation when bzip2 is not installed
+ **CRM:** Future-proof `get_message_xml()`
+ **crmd:** Filter election responses based on time not FSA state
+ **crmd:** Handle all possible peer states in `crmd_ha_status_callback()`
+ **crmd:** Make sure the current date/time is set - prevents use-of-NULL when evaluating rules
+ **crmd:** Relax an assertion regrading ccm membership instances
+ **crmd:** Use (node->processes&crm_proc_ais) to accurately update the CIB after replace operations
+ **crmd:** Heartbeat: Accurately record peer client status
+ **pengine:** Bug 1777 - Allow colocation with a resource in the Stopped state
+ **pengine:** Bug 1822 - Prevent use-of-NULL in PromoteRsc()
+ **pengine:** Implement three recovery policies based on `op_status` and `op_rc`
+ **pengine:** Parse fail-count correctly (it may be set to ININFITY)
+ **pengine:** Prevent graph-loop when stonith agents need to be moved around before a STONITH op
+ **pengine:** Prevent graph-loops when two operations have the same name+interval
+ **tengine:** Cancel active timers when destroying graphs
+ **tengine:** Ensure failcount is set correctly for failed stops/starts
+ **tengine:** Update failcount for oeprations that time out
+ **admin:** Prevent hang in `crm_mon -1` when there is no cib connection
(patch from Junko IKEDA)
+ **cib:** Require `--force|-f` when performing potentially dangerous commands with cibadmin
+ **cib:** Tweak the shutdown code
+ **Common:** Only count peer processes of active nodes
+ **Core:** Create generic cluster sign-in method
+ **core:** Fix compilation when Heartbeat support is disabled
+ **Core:** General cleanup for supporting two stacks
+ **Core:** iso6601 - Support parsing of time-only strings
+ **core:** Isolate more code that is only needed when `SUPPORT_HEARTBEAT` is enabled
+ **crm:** Improved logging of errors in the XML parser
+ **crmd:** Fix potential use-of-NULL in string comparison
+ **crmd:** Reimpliment syncronizing of CIB queries and updates when invoking the PE
+ **tools:** `crm_mon`: Indicate when a node is both in standby mode and offline
+ **pengine:** Bug 1822 - Do not try an promote groups if not all of it is active
+ **pengine:** `on_fail=nothing` is an alias for 'ignore' not 'restart'
+ **pengine:** Prevent a potential use-of-NULL in `cron_range_satisfied()`
+ **snmp subagent:** fix a problem on displaying an unmanaged group
+ **snmp subagent:** use the syslog setting
+ **snmp:** v2 support (thanks to Keisuke MORI)
+ **snmp subagent:** made it not complain about some things if shutting down
diff --git a/daemons/based/based_io.c b/daemons/based/based_io.c
index f55722b284..7ea3cf83ec 100644
--- a/daemons/based/based_io.c
+++ b/daemons/based/based_io.c
@@ -1,462 +1,463 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU General Public License version 2
* or later (GPLv2+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
#include <errno.h>
#include <fcntl.h>
#include <dirent.h>
#include <sys/param.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/stat.h>
#include <glib.h>
#include <libxml/tree.h>
#include <crm/crm.h>
#include <crm/cib.h>
#include <crm/common/util.h>
#include <crm/common/xml.h>
#include <crm/cib/internal.h>
#include <crm/cluster.h>
#include <pacemaker-based.h>
crm_trigger_t *cib_writer = NULL;
int write_cib_contents(gpointer p);
static void
cib_rename(const char *old)
{
int new_fd;
char *new = crm_strdup_printf("%s/cib.auto.XXXXXX", cib_root);
umask(S_IWGRP | S_IWOTH | S_IROTH);
new_fd = mkstemp(new);
if ((new_fd < 0) || (rename(old, new) < 0)) {
crm_err("Couldn't archive unusable file %s (disabling disk writes and continuing)",
old);
cib_writes_enabled = FALSE;
} else {
crm_err("Archived unusable file %s as %s", old, new);
}
if (new_fd > 0) {
close(new_fd);
}
free(new);
}
/*
* It is the callers responsibility to free the output of this function
*/
static xmlNode *
retrieveCib(const char *filename, const char *sigfile)
{
xmlNode *root = NULL;
crm_info("Reading cluster configuration file %s (digest: %s)",
filename, sigfile);
switch (cib_file_read_and_verify(filename, sigfile, &root)) {
case -pcmk_err_cib_corrupt:
crm_warn("Continuing but %s will NOT be used.", filename);
break;
case -pcmk_err_cib_modified:
/* Archive the original files so the contents are not lost */
crm_warn("Continuing but %s will NOT be used.", filename);
cib_rename(filename);
cib_rename(sigfile);
break;
}
return root;
}
/*
* for OSs without support for direntry->d_type, like Solaris
*/
#ifndef DT_UNKNOWN
# define DT_UNKNOWN 0
# define DT_FIFO 1
# define DT_CHR 2
# define DT_DIR 4
# define DT_BLK 6
# define DT_REG 8
# define DT_LNK 10
# define DT_SOCK 12
# define DT_WHT 14
#endif /*DT_UNKNOWN*/
static int cib_archive_filter(const struct dirent * a)
{
int rc = 0;
/* Looking for regular files (d_type = 8) starting with 'cib-' and not ending in .sig */
struct stat s;
char *a_path = crm_strdup_printf("%s/%s", cib_root, a->d_name);
if(stat(a_path, &s) != 0) {
rc = errno;
crm_trace("%s - stat failed: %s (%d)", a->d_name, pcmk_rc_str(rc), rc);
rc = 0;
} else if ((s.st_mode & S_IFREG) != S_IFREG) {
unsigned char dtype;
#ifdef HAVE_STRUCT_DIRENT_D_TYPE
dtype = a->d_type;
#else
switch (s.st_mode & S_IFMT) {
case S_IFREG: dtype = DT_REG; break;
case S_IFDIR: dtype = DT_DIR; break;
case S_IFCHR: dtype = DT_CHR; break;
case S_IFBLK: dtype = DT_BLK; break;
case S_IFLNK: dtype = DT_LNK; break;
case S_IFIFO: dtype = DT_FIFO; break;
case S_IFSOCK: dtype = DT_SOCK; break;
default: dtype = DT_UNKNOWN; break;
}
#endif
crm_trace("%s - wrong type (%d)", a->d_name, dtype);
} else if(strstr(a->d_name, "cib-") != a->d_name) {
crm_trace("%s - wrong prefix", a->d_name);
} else if (pcmk__ends_with_ext(a->d_name, ".sig")) {
crm_trace("%s - wrong suffix", a->d_name);
} else {
crm_debug("%s - candidate", a->d_name);
rc = 1;
}
free(a_path);
return rc;
}
static int cib_archive_sort(const struct dirent ** a, const struct dirent **b)
{
/* Order by creation date - most recently created file first */
int rc = 0;
struct stat buf;
time_t a_age = 0;
time_t b_age = 0;
char *a_path = crm_strdup_printf("%s/%s", cib_root, a[0]->d_name);
char *b_path = crm_strdup_printf("%s/%s", cib_root, b[0]->d_name);
if(stat(a_path, &buf) == 0) {
a_age = buf.st_ctime;
}
if(stat(b_path, &buf) == 0) {
b_age = buf.st_ctime;
}
free(a_path);
free(b_path);
if(a_age > b_age) {
rc = 1;
} else if(a_age < b_age) {
rc = -1;
}
crm_trace("%s (%lu) vs. %s (%lu) : %d",
a[0]->d_name, (unsigned long)a_age,
b[0]->d_name, (unsigned long)b_age, rc);
return rc;
}
xmlNode *
readCibXmlFile(const char *dir, const char *file, gboolean discard_status)
{
struct dirent **namelist = NULL;
int lpc = 0;
char *sigfile = NULL;
char *sigfilepath = NULL;
char *filename = NULL;
const char *name = NULL;
const char *value = NULL;
const char *use_valgrind = pcmk__env_option(PCMK__ENV_VALGRIND_ENABLED);
xmlNode *root = NULL;
xmlNode *status = NULL;
sigfile = crm_strdup_printf("%s.sig", file);
if (pcmk__daemon_can_write(dir, file) == FALSE
|| pcmk__daemon_can_write(dir, sigfile) == FALSE) {
cib_status = -EACCES;
return NULL;
}
filename = crm_strdup_printf("%s/%s", dir, file);
sigfilepath = crm_strdup_printf("%s/%s", dir, sigfile);
free(sigfile);
cib_status = pcmk_ok;
root = retrieveCib(filename, sigfilepath);
free(filename);
free(sigfilepath);
if (root == NULL) {
crm_warn("Primary configuration corrupt or unusable, trying backups in %s", cib_root);
lpc = scandir(cib_root, &namelist, cib_archive_filter, cib_archive_sort);
if (lpc < 0) {
crm_err("scandir(%s) failed: %s", cib_root, pcmk_rc_str(errno));
}
}
while (root == NULL && lpc > 1) {
crm_debug("Testing %d candidates", lpc);
lpc--;
filename = crm_strdup_printf("%s/%s", cib_root, namelist[lpc]->d_name);
sigfile = crm_strdup_printf("%s.sig", filename);
crm_info("Reading cluster configuration file %s (digest: %s)",
filename, sigfile);
if (cib_file_read_and_verify(filename, sigfile, &root) < 0) {
crm_warn("Continuing but %s will NOT be used.", filename);
} else {
crm_notice("Continuing with last valid configuration archive: %s", filename);
}
free(namelist[lpc]);
free(filename);
free(sigfile);
}
free(namelist);
if (root == NULL) {
root = createEmptyCib(0);
crm_warn("Continuing with an empty configuration.");
}
if (cib_writes_enabled && (use_valgrind != NULL)
&& (crm_is_true(use_valgrind)
|| (strstr(use_valgrind, PCMK__SERVER_BASED) != NULL))) {
cib_writes_enabled = FALSE;
crm_err("*** Disabling disk writes to avoid confusing Valgrind ***");
}
status = pcmk__xe_first_child(root, PCMK_XE_STATUS, NULL, NULL);
if (discard_status && status != NULL) {
// Strip out the PCMK_XE_STATUS section if there is one
pcmk__xml_free(status);
status = NULL;
}
if (status == NULL) {
pcmk__xe_create(root, PCMK_XE_STATUS);
}
/* Do this before schema validation happens */
/* fill in some defaults */
name = PCMK_XA_ADMIN_EPOCH;
value = crm_element_value(root, name);
if (value == NULL) {
crm_warn("No value for %s was specified in the configuration.", name);
crm_warn("The recommended course of action is to shutdown,"
" run crm_verify and fix any errors it reports.");
crm_warn("We will default to zero and continue but may get"
" confused about which configuration to use if"
" multiple nodes are powered up at the same time.");
crm_xml_add_int(root, name, 0);
}
name = PCMK_XA_EPOCH;
value = crm_element_value(root, name);
if (value == NULL) {
crm_xml_add_int(root, name, 0);
}
name = PCMK_XA_NUM_UPDATES;
value = crm_element_value(root, name);
if (value == NULL) {
crm_xml_add_int(root, name, 0);
}
// Unset (DC should set appropriate value)
pcmk__xe_remove_attr(root, PCMK_XA_DC_UUID);
if (discard_status) {
crm_log_xml_trace(root, "[on-disk]");
}
if (!pcmk__configured_schema_validates(root)) {
cib_status = -pcmk_err_schema_validation;
}
return root;
}
gboolean
uninitializeCib(void)
{
xmlNode *tmp_cib = the_cib;
if (tmp_cib == NULL) {
crm_debug("The CIB has already been deallocated.");
return FALSE;
}
the_cib = NULL;
crm_debug("Deallocating the CIB.");
pcmk__xml_free(tmp_cib);
crm_debug("The CIB has been deallocated.");
return TRUE;
}
/*
* This method will free the old CIB pointer on success and the new one
* on failure.
*/
int
activateCibXml(xmlNode * new_cib, gboolean to_disk, const char *op)
{
if (new_cib) {
xmlNode *saved_cib = the_cib;
pcmk__assert(new_cib != saved_cib);
the_cib = new_cib;
pcmk__xml_free(saved_cib);
if (cib_writes_enabled && cib_status == pcmk_ok && to_disk) {
crm_debug("Triggering CIB write for %s op", op);
mainloop_set_trigger(cib_writer);
}
return pcmk_ok;
}
crm_err("Ignoring invalid CIB");
if (the_cib) {
crm_warn("Reverting to last known CIB");
} else {
crm_crit("Could not write out new CIB and no saved version to revert to");
}
return -ENODATA;
}
static void
cib_diskwrite_complete(mainloop_child_t * p, pid_t pid, int core, int signo, int exitcode)
{
const char *errmsg = "Could not write CIB to disk";
if ((exitcode != 0) && cib_writes_enabled) {
cib_writes_enabled = FALSE;
errmsg = "Disabling CIB disk writes after failure";
}
if ((signo == 0) && (exitcode == 0)) {
crm_trace("Disk write [%d] succeeded", (int) pid);
} else if (signo == 0) {
crm_err("%s: process %d exited %d", errmsg, (int) pid, exitcode);
} else {
crm_err("%s: process %d terminated with signal %d (%s)%s",
errmsg, (int) pid, signo, strsignal(signo),
(core? " and dumped core" : ""));
}
mainloop_trigger_complete(cib_writer);
}
int
write_cib_contents(gpointer p)
{
int exit_rc = pcmk_ok;
xmlNode *cib_local = NULL;
/* Make a copy of the CIB to write (possibly in a forked child) */
if (p) {
/* Synchronous write out */
cib_local = pcmk__xml_copy(NULL, p);
} else {
int pid = 0;
int bb_state = qb_log_ctl(QB_LOG_BLACKBOX, QB_LOG_CONF_STATE_GET, 0);
/* Turn it off before the fork() to avoid:
* - 2 processes writing to the same shared mem
* - the child needing to disable it
* (which would close it from underneath the parent)
* This way, the shared mem files are already closed
*/
qb_log_ctl(QB_LOG_BLACKBOX, QB_LOG_CONF_ENABLED, QB_FALSE);
pid = fork();
if (pid < 0) {
crm_err("Disabling disk writes after fork failure: %s", pcmk_rc_str(errno));
cib_writes_enabled = FALSE;
return FALSE;
}
if (pid) {
/* Parent */
mainloop_child_add(pid, 0, "disk-writer", NULL, cib_diskwrite_complete);
if (bb_state == QB_LOG_STATE_ENABLED) {
/* Re-enable now that it it safe */
qb_log_ctl(QB_LOG_BLACKBOX, QB_LOG_CONF_ENABLED, QB_TRUE);
}
return -1; /* -1 means 'still work to do' */
}
/* Asynchronous write-out after a fork() */
/* In theory, we can scribble on the_cib here and not affect the parent,
* but let's be safe anyway.
*/
cib_local = pcmk__xml_copy(NULL, the_cib);
}
/* Write the CIB */
exit_rc = cib_file_write_with_digest(cib_local, cib_root, "cib.xml");
/* A nonzero exit code will cause further writes to be disabled */
pcmk__xml_free(cib_local);
if (p == NULL) {
crm_exit_t exit_code = CRM_EX_OK;
switch (exit_rc) {
case pcmk_ok:
exit_code = CRM_EX_OK;
break;
case pcmk_err_cib_modified:
exit_code = CRM_EX_DIGEST; // Existing CIB doesn't match digest
break;
case pcmk_err_cib_backup: // Existing CIB couldn't be backed up
case pcmk_err_cib_save: // New CIB couldn't be saved
exit_code = CRM_EX_CANTCREAT;
break;
default:
exit_code = CRM_EX_ERROR;
break;
}
/* Use _exit() because exit() could affect the parent adversely */
+ pcmk_common_cleanup();
_exit(exit_code);
}
return exit_rc;
}
diff --git a/daemons/controld/controld_join_dc.c b/daemons/controld/controld_join_dc.c
index dfb4ae6cc3..7ada26949d 100644
--- a/daemons/controld/controld_join_dc.c
+++ b/daemons/controld/controld_join_dc.c
@@ -1,1092 +1,1095 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU General Public License version 2
* or later (GPLv2+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <inttypes.h> // PRIu32
#include <stdbool.h> // bool, true, false
#include <stdio.h> // NULL
#include <stdlib.h> // free(), etc.
#include <glib.h> // gboolean, etc.
#include <libxml/tree.h> // xmlNode
#include <crm/crm.h>
#include <crm/common/xml.h>
#include <crm/cluster.h>
#include <pacemaker-controld.h>
static char *max_generation_from = NULL;
static xmlNodePtr max_generation_xml = NULL;
/*!
* \internal
* \brief Nodes from which a CIB sync has failed since the peer joined
*
* This table is of the form (<tt>node_name -> join_id</tt>). \p node_name is
* the name of a client node from which a CIB \p sync_from() call has failed in
* \p do_dc_join_finalize() since the client joined the cluster as a peer.
* \p join_id is the ID of the join round in which the \p sync_from() failed,
* and is intended for use in nack log messages.
*/
static GHashTable *failed_sync_nodes = NULL;
void finalize_join_for(gpointer key, gpointer value, gpointer user_data);
void finalize_sync_callback(xmlNode * msg, int call_id, int rc, xmlNode * output, void *user_data);
gboolean check_join_state(enum crmd_fsa_state cur_state, const char *source);
/* Numeric counter used to identify join rounds (an unsigned int would be
* appropriate, except we get and set it in XML as int)
*/
static int current_join_id = 0;
/*!
* \internal
* \brief Get log-friendly string equivalent of a controller group join phase
*
* \param[in] phase Join phase
*
* \return Log-friendly string equivalent of \p phase
*/
static const char *
join_phase_text(enum controld_join_phase phase)
{
switch (phase) {
case controld_join_nack:
return "nack";
case controld_join_none:
return "none";
case controld_join_welcomed:
return "welcomed";
case controld_join_integrated:
return "integrated";
case controld_join_finalized:
return "finalized";
case controld_join_confirmed:
return "confirmed";
default:
return "invalid";
}
}
/*!
* \internal
* \brief Destroy the hash table containing failed sync nodes
*/
void
controld_destroy_failed_sync_table(void)
{
if (failed_sync_nodes != NULL) {
g_hash_table_destroy(failed_sync_nodes);
failed_sync_nodes = NULL;
}
}
/*!
* \internal
* \brief Remove a node from the failed sync nodes table if present
*
* \param[in] node_name Node name to remove
*/
void
controld_remove_failed_sync_node(const char *node_name)
{
if (failed_sync_nodes != NULL) {
g_hash_table_remove(failed_sync_nodes, (gchar *) node_name);
}
}
/*!
* \internal
* \brief Add to a hash table a node whose CIB failed to sync
*
* \param[in] node_name Name of node whose CIB failed to sync
* \param[in] join_id Join round when the failure occurred
*/
static void
record_failed_sync_node(const char *node_name, gint join_id)
{
if (failed_sync_nodes == NULL) {
failed_sync_nodes = pcmk__strikey_table(g_free, NULL);
}
/* If the node is already in the table then we failed to nack it during the
* filter offer step
*/
CRM_LOG_ASSERT(g_hash_table_insert(failed_sync_nodes, g_strdup(node_name),
GINT_TO_POINTER(join_id)));
}
/*!
* \internal
* \brief Look up a node name in the failed sync table
*
* \param[in] node_name Name of node to look up
* \param[out] join_id Where to store the join ID of when the sync failed
*
* \return Standard Pacemaker return code. Specifically, \p pcmk_rc_ok if the
* node name was found, or \p pcmk_rc_node_unknown otherwise.
* \note \p *join_id is set to -1 if the node is not found.
*/
static int
lookup_failed_sync_node(const char *node_name, gint *join_id)
{
*join_id = -1;
if (failed_sync_nodes != NULL) {
gpointer result = g_hash_table_lookup(failed_sync_nodes,
(gchar *) node_name);
if (result != NULL) {
*join_id = GPOINTER_TO_INT(result);
return pcmk_rc_ok;
}
}
return pcmk_rc_node_unknown;
}
void
crm_update_peer_join(const char *source, pcmk__node_status_t *node,
enum controld_join_phase phase)
{
enum controld_join_phase last = controld_get_join_phase(node);
CRM_CHECK(node != NULL, return);
/* Remote nodes do not participate in joins */
if (pcmk_is_set(node->flags, pcmk__node_status_remote)) {
return;
}
if (phase == last) {
crm_trace("Node %s join-%d phase is still %s "
QB_XS " nodeid=%" PRIu32 " source=%s",
node->name, current_join_id, join_phase_text(last),
node->cluster_layer_id, source);
return;
}
if ((phase <= controld_join_none) || (phase == (last + 1))) {
- struct controld_node_status_data *data =
- pcmk__assert_alloc(1, sizeof(struct controld_node_status_data));
+ struct controld_node_status_data *data = NULL;
+ if (node->user_data == NULL) {
+ node->user_data =
+ pcmk__assert_alloc(1, sizeof(struct controld_node_status_data));
+ }
+ data = node->user_data;
data->join_phase = phase;
- node->user_data = data;
crm_trace("Node %s join-%d phase is now %s (was %s) "
QB_XS " nodeid=%" PRIu32 " source=%s",
node->name, current_join_id, join_phase_text(phase),
join_phase_text(last), node->cluster_layer_id,
source);
return;
}
crm_warn("Rejecting join-%d phase update for node %s because can't go from "
"%s to %s " QB_XS " nodeid=%" PRIu32 " source=%s",
current_join_id, node->name, join_phase_text(last),
join_phase_text(phase), node->cluster_layer_id, source);
}
static void
start_join_round(void)
{
GHashTableIter iter;
pcmk__node_status_t *peer = NULL;
crm_debug("Starting new join round join-%d", current_join_id);
g_hash_table_iter_init(&iter, pcmk__peer_cache);
while (g_hash_table_iter_next(&iter, NULL, (gpointer *) &peer)) {
crm_update_peer_join(__func__, peer, controld_join_none);
}
if (max_generation_from != NULL) {
free(max_generation_from);
max_generation_from = NULL;
}
if (max_generation_xml != NULL) {
pcmk__xml_free(max_generation_xml);
max_generation_xml = NULL;
}
controld_clear_fsa_input_flags(R_HAVE_CIB);
}
/*!
* \internal
* \brief Create a join message from the DC
*
* \param[in] join_op Join operation name
* \param[in] host_to Recipient of message
*/
static xmlNode *
create_dc_message(const char *join_op, const char *host_to)
{
xmlNode *msg = pcmk__new_request(pcmk_ipc_controld, CRM_SYSTEM_DC, host_to,
CRM_SYSTEM_CRMD, join_op, NULL);
/* Identify which election this is a part of */
crm_xml_add_int(msg, PCMK__XA_JOIN_ID, current_join_id);
/* Add a field specifying whether the DC is shutting down. This keeps the
* joining node from fencing the old DC if it becomes the new DC.
*/
pcmk__xe_set_bool_attr(msg, PCMK__XA_DC_LEAVING,
pcmk_is_set(controld_globals.fsa_input_register,
R_SHUTDOWN));
return msg;
}
static void
join_make_offer(gpointer key, gpointer value, gpointer user_data)
{
/* @TODO We don't use user_data except to distinguish one particular call
* from others. Make this clearer.
*/
xmlNode *offer = NULL;
pcmk__node_status_t *member = (pcmk__node_status_t *) value;
pcmk__assert(member != NULL);
if (!pcmk__cluster_is_node_active(member)) {
crm_info("Not making join-%d offer to inactive node %s",
current_join_id, pcmk__s(member->name, "with unknown name"));
if ((member->expected == NULL)
&& pcmk__str_eq(member->state, PCMK__VALUE_LOST, pcmk__str_none)) {
/* You would think this unsafe, but in fact this plus an
* active resource is what causes it to be fenced.
*
* Yes, this does mean that any node that dies at the same
* time as the old DC and is not running resource (still)
* won't be fenced.
*
* I'm not happy about this either.
*/
pcmk__update_peer_expected(__func__, member, CRMD_JOINSTATE_DOWN);
}
return;
}
if (member->name == NULL) {
crm_info("Not making join-%d offer to node uuid %s with unknown name",
current_join_id, member->xml_id);
return;
}
if (controld_globals.membership_id != controld_globals.peer_seq) {
controld_globals.membership_id = controld_globals.peer_seq;
crm_info("Making join-%d offers based on membership event %llu",
current_join_id, controld_globals.peer_seq);
}
if (user_data != NULL) {
enum controld_join_phase phase = controld_get_join_phase(member);
if (phase > controld_join_none) {
crm_info("Not making join-%d offer to already known node %s (%s)",
current_join_id, member->name, join_phase_text(phase));
return;
}
}
crm_update_peer_join(__func__, (pcmk__node_status_t*) member,
controld_join_none);
offer = create_dc_message(CRM_OP_JOIN_OFFER, member->name);
// Advertise our feature set so the joining node can bail if not compatible
crm_xml_add(offer, PCMK_XA_CRM_FEATURE_SET, CRM_FEATURE_SET);
crm_info("Sending join-%d offer to %s", current_join_id, member->name);
pcmk__cluster_send_message(member, pcmk_ipc_controld, offer);
pcmk__xml_free(offer);
crm_update_peer_join(__func__, member, controld_join_welcomed);
}
/* A_DC_JOIN_OFFER_ALL */
void
do_dc_join_offer_all(long long action,
enum crmd_fsa_cause cause,
enum crmd_fsa_state cur_state,
enum crmd_fsa_input current_input, fsa_data_t * msg_data)
{
int count;
/* Reset everyone's status back to down or in_ccm in the CIB.
* Any nodes that are active in the CIB but not in the cluster membership
* will be seen as offline by the scheduler anyway.
*/
current_join_id++;
start_join_round();
update_dc(NULL);
if (cause == C_HA_MESSAGE && current_input == I_NODE_JOIN) {
crm_info("A new node joined the cluster");
}
g_hash_table_foreach(pcmk__peer_cache, join_make_offer, NULL);
count = crmd_join_phase_count(controld_join_welcomed);
crm_info("Waiting on join-%d requests from %d outstanding node%s",
current_join_id, count, pcmk__plural_s(count));
// Don't waste time by invoking the scheduler yet
}
/* A_DC_JOIN_OFFER_ONE */
void
do_dc_join_offer_one(long long action,
enum crmd_fsa_cause cause,
enum crmd_fsa_state cur_state,
enum crmd_fsa_input current_input, fsa_data_t * msg_data)
{
pcmk__node_status_t *member = NULL;
ha_msg_input_t *welcome = NULL;
int count;
const char *join_to = NULL;
if (msg_data->data == NULL) {
crm_info("Making join-%d offers to any unconfirmed nodes "
"because an unknown node joined", current_join_id);
g_hash_table_foreach(pcmk__peer_cache, join_make_offer, &member);
check_join_state(cur_state, __func__);
return;
}
welcome = fsa_typed_data(fsa_dt_ha_msg);
if (welcome == NULL) {
// fsa_typed_data() already logged an error
return;
}
join_to = crm_element_value(welcome->msg, PCMK__XA_SRC);
if (join_to == NULL) {
crm_err("Can't make join-%d offer to unknown node", current_join_id);
return;
}
member = pcmk__get_node(0, join_to, NULL, pcmk__node_search_cluster_member);
/* It is possible that a node will have been sick or starting up when the
* original offer was made. However, it will either re-announce itself in
* due course, or we can re-store the original offer on the client.
*/
crm_update_peer_join(__func__, member, controld_join_none);
join_make_offer(NULL, member, NULL);
/* If the offer isn't to the local node, make an offer to the local node as
* well, to ensure the correct value for max_generation_from.
*/
if (!controld_is_local_node(join_to)) {
member = controld_get_local_node_status();
join_make_offer(NULL, member, NULL);
}
/* This was a genuine join request; cancel any existing transition and
* invoke the scheduler.
*/
abort_transition(PCMK_SCORE_INFINITY, pcmk__graph_restart, "Node join",
NULL);
count = crmd_join_phase_count(controld_join_welcomed);
crm_info("Waiting on join-%d requests from %d outstanding node%s",
current_join_id, count, pcmk__plural_s(count));
// Don't waste time by invoking the scheduler yet
}
static int
compare_int_fields(xmlNode * left, xmlNode * right, const char *field)
{
const char *elem_l = crm_element_value(left, field);
const char *elem_r = crm_element_value(right, field);
long long int_elem_l;
long long int_elem_r;
int rc = pcmk_rc_ok;
rc = pcmk__scan_ll(elem_l, &int_elem_l, -1LL);
if (rc != pcmk_rc_ok) { // Shouldn't be possible
crm_warn("Comparing current CIB %s as -1 "
"because '%s' is not an integer", field, elem_l);
}
rc = pcmk__scan_ll(elem_r, &int_elem_r, -1LL);
if (rc != pcmk_rc_ok) { // Shouldn't be possible
crm_warn("Comparing joining node's CIB %s as -1 "
"because '%s' is not an integer", field, elem_r);
}
if (int_elem_l < int_elem_r) {
return -1;
} else if (int_elem_l > int_elem_r) {
return 1;
}
return 0;
}
/* A_DC_JOIN_PROCESS_REQ */
void
do_dc_join_filter_offer(long long action,
enum crmd_fsa_cause cause,
enum crmd_fsa_state cur_state,
enum crmd_fsa_input current_input, fsa_data_t * msg_data)
{
xmlNode *generation = NULL;
int cmp = 0;
int join_id = -1;
int count = 0;
gint value = 0;
gboolean ack_nack_bool = TRUE;
ha_msg_input_t *join_ack = fsa_typed_data(fsa_dt_ha_msg);
const char *join_from = crm_element_value(join_ack->msg, PCMK__XA_SRC);
const char *ref = crm_element_value(join_ack->msg, PCMK_XA_REFERENCE);
const char *join_version = crm_element_value(join_ack->msg,
PCMK_XA_CRM_FEATURE_SET);
pcmk__node_status_t *join_node = NULL;
if (join_from == NULL) {
crm_err("Ignoring invalid join request without node name");
return;
}
join_node = pcmk__get_node(0, join_from, NULL,
pcmk__node_search_cluster_member);
crm_element_value_int(join_ack->msg, PCMK__XA_JOIN_ID, &join_id);
if (join_id != current_join_id) {
crm_debug("Ignoring join-%d request from %s because we are on join-%d",
join_id, join_from, current_join_id);
check_join_state(cur_state, __func__);
return;
}
generation = join_ack->xml;
if (max_generation_xml != NULL && generation != NULL) {
int lpc = 0;
const char *attributes[] = {
PCMK_XA_ADMIN_EPOCH,
PCMK_XA_EPOCH,
PCMK_XA_NUM_UPDATES,
};
/* It's not obvious that join_ack->xml is the PCMK__XE_GENERATION_TUPLE
* element from the join client. The "if" guard is for clarity.
*/
if (pcmk__xe_is(generation, PCMK__XE_GENERATION_TUPLE)) {
for (lpc = 0; cmp == 0 && lpc < PCMK__NELEM(attributes); lpc++) {
cmp = compare_int_fields(max_generation_xml, generation,
attributes[lpc]);
}
} else { // Should always be PCMK__XE_GENERATION_TUPLE
CRM_LOG_ASSERT(false);
}
}
if (ref == NULL) {
ref = "none"; // for logging only
}
if (lookup_failed_sync_node(join_from, &value) == pcmk_rc_ok) {
crm_err("Rejecting join-%d request from node %s because we failed to "
"sync its CIB in join-%d " QB_XS " ref=%s",
join_id, join_from, value, ref);
ack_nack_bool = FALSE;
} else if (!pcmk__cluster_is_node_active(join_node)) {
if (match_down_event(join_from) != NULL) {
/* The join request was received after the node was fenced or
* otherwise shutdown in a way that we're aware of. No need to log
* an error in this rare occurrence; we know the client was recently
* shut down, and receiving a lingering in-flight request is not
* cause for alarm.
*/
crm_debug("Rejecting join-%d request from inactive node %s "
QB_XS " ref=%s", join_id, join_from, ref);
} else {
crm_err("Rejecting join-%d request from inactive node %s "
QB_XS " ref=%s", join_id, join_from, ref);
}
ack_nack_bool = FALSE;
} else if (generation == NULL) {
crm_err("Rejecting invalid join-%d request from node %s "
"missing CIB generation " QB_XS " ref=%s",
join_id, join_from, ref);
ack_nack_bool = FALSE;
} else if ((join_version == NULL)
|| !feature_set_compatible(CRM_FEATURE_SET, join_version)) {
crm_err("Rejecting join-%d request from node %s because feature set %s"
" is incompatible with ours (%s) " QB_XS " ref=%s",
join_id, join_from, (join_version? join_version : "pre-3.1.0"),
CRM_FEATURE_SET, ref);
ack_nack_bool = FALSE;
} else if (max_generation_xml == NULL) {
const char *validation = crm_element_value(generation,
PCMK_XA_VALIDATE_WITH);
if (pcmk__get_schema(validation) == NULL) {
crm_err("Rejecting join-%d request from %s (with first CIB "
"generation) due to %s schema version %s " QB_XS " ref=%s",
join_id, join_from,
((validation == NULL)? "missing" : "unknown"),
pcmk__s(validation, ""), ref);
ack_nack_bool = FALSE;
} else {
crm_debug("Accepting join-%d request from %s (with first CIB "
"generation) " QB_XS " ref=%s",
join_id, join_from, ref);
max_generation_xml = pcmk__xml_copy(NULL, generation);
pcmk__str_update(&max_generation_from, join_from);
}
} else if ((cmp < 0)
|| ((cmp == 0) && controld_is_local_node(join_from))) {
const char *validation = crm_element_value(generation,
PCMK_XA_VALIDATE_WITH);
if (pcmk__get_schema(validation) == NULL) {
crm_err("Rejecting join-%d request from %s (with better CIB "
"generation than current best from %s) due to %s "
"schema version %s " QB_XS " ref=%s",
join_id, join_from, max_generation_from,
((validation == NULL)? "missing" : "unknown"),
pcmk__s(validation, ""), ref);
ack_nack_bool = FALSE;
} else {
crm_debug("Accepting join-%d request from %s (with better CIB "
"generation than current best from %s) " QB_XS " ref=%s",
join_id, join_from, max_generation_from, ref);
crm_log_xml_debug(max_generation_xml, "Old max generation");
crm_log_xml_debug(generation, "New max generation");
pcmk__xml_free(max_generation_xml);
max_generation_xml = pcmk__xml_copy(NULL, join_ack->xml);
pcmk__str_update(&max_generation_from, join_from);
}
} else {
crm_debug("Accepting join-%d request from %s " QB_XS " ref=%s",
join_id, join_from, ref);
}
if (!ack_nack_bool) {
crm_update_peer_join(__func__, join_node, controld_join_nack);
pcmk__update_peer_expected(__func__, join_node, CRMD_JOINSTATE_NACK);
} else {
crm_update_peer_join(__func__, join_node, controld_join_integrated);
pcmk__update_peer_expected(__func__, join_node, CRMD_JOINSTATE_MEMBER);
}
count = crmd_join_phase_count(controld_join_integrated);
crm_debug("%d node%s currently integrated in join-%d",
count, pcmk__plural_s(count), join_id);
if (check_join_state(cur_state, __func__) == FALSE) {
// Don't waste time by invoking the scheduler yet
count = crmd_join_phase_count(controld_join_welcomed);
crm_debug("Waiting on join-%d requests from %d outstanding node%s",
join_id, count, pcmk__plural_s(count));
}
}
/* A_DC_JOIN_FINALIZE */
void
do_dc_join_finalize(long long action,
enum crmd_fsa_cause cause,
enum crmd_fsa_state cur_state,
enum crmd_fsa_input current_input, fsa_data_t * msg_data)
{
char *sync_from = NULL;
int rc = pcmk_ok;
int count_welcomed = crmd_join_phase_count(controld_join_welcomed);
int count_finalizable = crmd_join_phase_count(controld_join_integrated)
+ crmd_join_phase_count(controld_join_nack);
/* This we can do straight away and avoid clients timing us out
* while we compute the latest CIB
*/
if (count_welcomed != 0) {
crm_debug("Waiting on join-%d requests from %d outstanding node%s "
"before finalizing join", current_join_id, count_welcomed,
pcmk__plural_s(count_welcomed));
crmd_join_phase_log(LOG_DEBUG);
/* crmd_fsa_stall(FALSE); Needed? */
return;
} else if (count_finalizable == 0) {
crm_debug("Finalization not needed for join-%d at the current time",
current_join_id);
crmd_join_phase_log(LOG_DEBUG);
check_join_state(controld_globals.fsa_state, __func__);
return;
}
controld_clear_fsa_input_flags(R_HAVE_CIB);
if ((max_generation_from == NULL)
|| controld_is_local_node(max_generation_from)) {
controld_set_fsa_input_flags(R_HAVE_CIB);
}
if (!controld_globals.transition_graph->complete) {
crm_warn("Delaying join-%d finalization while transition in progress",
current_join_id);
crmd_join_phase_log(LOG_DEBUG);
crmd_fsa_stall(FALSE);
return;
}
if (pcmk_is_set(controld_globals.fsa_input_register, R_HAVE_CIB)) {
// Send our CIB out to everyone
sync_from = pcmk__str_copy(controld_globals.cluster->priv->node_name);
crm_debug("Finalizing join-%d for %d node%s (sync'ing from local CIB)",
current_join_id, count_finalizable,
pcmk__plural_s(count_finalizable));
crm_log_xml_debug(max_generation_xml, "Requested CIB version");
} else {
// Ask for the agreed best CIB
sync_from = pcmk__str_copy(max_generation_from);
crm_notice("Finalizing join-%d for %d node%s (sync'ing CIB from %s)",
current_join_id, count_finalizable,
pcmk__plural_s(count_finalizable), sync_from);
crm_log_xml_notice(max_generation_xml, "Requested CIB version");
}
crmd_join_phase_log(LOG_DEBUG);
rc = controld_globals.cib_conn->cmds->sync_from(controld_globals.cib_conn,
sync_from, NULL, cib_none);
fsa_register_cib_callback(rc, sync_from, finalize_sync_callback);
}
void
free_max_generation(void)
{
free(max_generation_from);
max_generation_from = NULL;
pcmk__xml_free(max_generation_xml);
max_generation_xml = NULL;
}
void
finalize_sync_callback(xmlNode * msg, int call_id, int rc, xmlNode * output, void *user_data)
{
CRM_LOG_ASSERT(-EPERM != rc);
if (rc != pcmk_ok) {
const char *sync_from = (const char *) user_data;
do_crm_log(((rc == -pcmk_err_old_data)? LOG_WARNING : LOG_ERR),
"Could not sync CIB from %s in join-%d: %s",
sync_from, current_join_id, pcmk_strerror(rc));
if (rc != -pcmk_err_old_data) {
record_failed_sync_node(sync_from, current_join_id);
}
/* restart the whole join process */
register_fsa_error_adv(C_FSA_INTERNAL, I_ELECTION_DC, NULL, NULL,
__func__);
} else if (!AM_I_DC) {
crm_debug("Sync'ed CIB for join-%d but no longer DC", current_join_id);
} else if (controld_globals.fsa_state != S_FINALIZE_JOIN) {
crm_debug("Sync'ed CIB for join-%d but no longer in S_FINALIZE_JOIN "
"(%s)", current_join_id,
fsa_state2string(controld_globals.fsa_state));
} else {
controld_set_fsa_input_flags(R_HAVE_CIB);
/* make sure dc_uuid is re-set to us */
if (!check_join_state(controld_globals.fsa_state, __func__)) {
int count_finalizable = 0;
count_finalizable = crmd_join_phase_count(controld_join_integrated)
+ crmd_join_phase_count(controld_join_nack);
crm_debug("Notifying %d node%s of join-%d results",
count_finalizable, pcmk__plural_s(count_finalizable),
current_join_id);
g_hash_table_foreach(pcmk__peer_cache, finalize_join_for, NULL);
}
}
}
static void
join_node_state_commit_callback(xmlNode *msg, int call_id, int rc,
xmlNode *output, void *user_data)
{
const char *node = user_data;
if (rc != pcmk_ok) {
fsa_data_t *msg_data = NULL; // for register_fsa_error() macro
crm_crit("join-%d node history update (via CIB call %d) for node %s "
"failed: %s",
current_join_id, call_id, node, pcmk_strerror(rc));
crm_log_xml_debug(msg, "failed");
register_fsa_error(C_FSA_INTERNAL, I_ERROR, NULL);
}
crm_debug("join-%d node history update (via CIB call %d) for node %s "
"complete",
current_join_id, call_id, node);
check_join_state(controld_globals.fsa_state, __func__);
}
/* A_DC_JOIN_PROCESS_ACK */
void
do_dc_join_ack(long long action,
enum crmd_fsa_cause cause,
enum crmd_fsa_state cur_state,
enum crmd_fsa_input current_input, fsa_data_t * msg_data)
{
int join_id = -1;
ha_msg_input_t *join_ack = fsa_typed_data(fsa_dt_ha_msg);
const char *op = crm_element_value(join_ack->msg, PCMK__XA_CRM_TASK);
char *join_from = crm_element_value_copy(join_ack->msg, PCMK__XA_SRC);
pcmk__node_status_t *peer = NULL;
enum controld_join_phase phase = controld_join_none;
enum controld_section_e section = controld_section_lrm;
char *xpath = NULL;
xmlNode *state = join_ack->xml;
xmlNode *execd_state = NULL;
cib_t *cib = controld_globals.cib_conn;
int rc = pcmk_ok;
// Sanity checks
if (join_from == NULL) {
crm_warn("Ignoring message received without node identification");
goto done;
}
if (op == NULL) {
crm_warn("Ignoring message received from %s without task", join_from);
goto done;
}
if (strcmp(op, CRM_OP_JOIN_CONFIRM)) {
crm_debug("Ignoring '%s' message from %s while waiting for '%s'",
op, join_from, CRM_OP_JOIN_CONFIRM);
goto done;
}
if (crm_element_value_int(join_ack->msg, PCMK__XA_JOIN_ID, &join_id) != 0) {
crm_warn("Ignoring join confirmation from %s without valid join ID",
join_from);
goto done;
}
peer = pcmk__get_node(0, join_from, NULL, pcmk__node_search_cluster_member);
phase = controld_get_join_phase(peer);
if (phase != controld_join_finalized) {
crm_info("Ignoring out-of-sequence join-%d confirmation from %s "
"(currently %s not %s)",
join_id, join_from, join_phase_text(phase),
join_phase_text(controld_join_finalized));
goto done;
}
if (join_id != current_join_id) {
crm_err("Rejecting join-%d confirmation from %s "
"because currently on join-%d",
join_id, join_from, current_join_id);
crm_update_peer_join(__func__, peer, controld_join_nack);
goto done;
}
crm_update_peer_join(__func__, peer, controld_join_confirmed);
/* Update CIB with node's current executor state. A new transition will be
* triggered later, when the CIB manager notifies us of the change.
*
* The delete and modify requests are part of an atomic transaction.
*/
rc = cib->cmds->init_transaction(cib);
if (rc != pcmk_ok) {
goto done;
}
// Delete relevant parts of node's current executor state from CIB
if (pcmk_is_set(controld_globals.flags, controld_shutdown_lock_enabled)) {
section = controld_section_lrm_unlocked;
}
controld_node_state_deletion_strings(join_from, section, &xpath, NULL);
rc = cib->cmds->remove(cib, xpath, NULL,
cib_xpath|cib_multiple|cib_transaction);
if (rc != pcmk_ok) {
goto done;
}
// Update CIB with node's latest known executor state
if (controld_is_local_node(join_from)) {
// Use the latest possible state if processing our own join ack
execd_state = controld_query_executor_state();
if (execd_state != NULL) {
crm_debug("Updating local node history for join-%d from query "
"result",
current_join_id);
state = execd_state;
} else {
crm_warn("Updating local node history from join-%d confirmation "
"because query failed",
current_join_id);
}
} else {
crm_debug("Updating node history for %s from join-%d confirmation",
join_from, current_join_id);
}
rc = cib->cmds->modify(cib, PCMK_XE_STATUS, state,
cib_can_create|cib_transaction);
pcmk__xml_free(execd_state);
if (rc != pcmk_ok) {
goto done;
}
// Commit the transaction
rc = cib->cmds->end_transaction(cib, true, cib_none);
fsa_register_cib_callback(rc, join_from, join_node_state_commit_callback);
if (rc > 0) {
// join_from will be freed after callback
join_from = NULL;
rc = pcmk_ok;
}
done:
if (rc != pcmk_ok) {
crm_crit("join-%d node history update for node %s failed: %s",
current_join_id, join_from, pcmk_strerror(rc));
register_fsa_error(C_FSA_INTERNAL, I_ERROR, NULL);
}
free(join_from);
free(xpath);
}
void
finalize_join_for(gpointer key, gpointer value, gpointer user_data)
{
xmlNode *acknak = NULL;
xmlNode *tmp1 = NULL;
pcmk__node_status_t *join_node = value;
const char *join_to = join_node->name;
enum controld_join_phase phase = controld_get_join_phase(join_node);
bool integrated = false;
switch (phase) {
case controld_join_integrated:
integrated = true;
break;
case controld_join_nack:
break;
default:
crm_trace("Not updating non-integrated and non-nacked node %s (%s) "
"for join-%d",
join_to, join_phase_text(phase), current_join_id);
return;
}
/* Update the <node> element with the node's name and UUID, in case they
* weren't known before
*/
crm_trace("Updating node name and UUID in CIB for %s", join_to);
tmp1 = pcmk__xe_create(NULL, PCMK_XE_NODE);
crm_xml_add(tmp1, PCMK_XA_ID, pcmk__cluster_node_uuid(join_node));
crm_xml_add(tmp1, PCMK_XA_UNAME, join_to);
fsa_cib_anon_update(PCMK_XE_NODES, tmp1);
pcmk__xml_free(tmp1);
join_node = pcmk__get_node(0, join_to, NULL,
pcmk__node_search_cluster_member);
if (!pcmk__cluster_is_node_active(join_node)) {
/*
* NACK'ing nodes that the membership layer doesn't know about yet
* simply creates more churn
*
* Better to leave them waiting and let the join restart when
* the new membership event comes in
*
* All other NACKs (due to versions etc) should still be processed
*/
pcmk__update_peer_expected(__func__, join_node, CRMD_JOINSTATE_PENDING);
return;
}
// Acknowledge or nack node's join request
crm_debug("%sing join-%d request from %s",
integrated? "Acknowledg" : "Nack", current_join_id, join_to);
acknak = create_dc_message(CRM_OP_JOIN_ACKNAK, join_to);
pcmk__xe_set_bool_attr(acknak, CRM_OP_JOIN_ACKNAK, integrated);
if (integrated) {
// No change needed for a nacked node
crm_update_peer_join(__func__, join_node, controld_join_finalized);
pcmk__update_peer_expected(__func__, join_node, CRMD_JOINSTATE_MEMBER);
/* Iterate through the remote peer cache and add information on which
* node hosts each to the ACK message. This keeps new controllers in
* sync with what has already happened.
*/
if (pcmk__cluster_num_remote_nodes() > 0) {
GHashTableIter iter;
pcmk__node_status_t *node = NULL;
xmlNode *remotes = pcmk__xe_create(acknak, PCMK_XE_NODES);
g_hash_table_iter_init(&iter, pcmk__remote_peer_cache);
while (g_hash_table_iter_next(&iter, NULL, (gpointer *) &node)) {
xmlNode *remote = NULL;
if (!node->conn_host) {
continue;
}
remote = pcmk__xe_create(remotes, PCMK_XE_NODE);
pcmk__xe_set_props(remote,
PCMK_XA_ID, node->name,
PCMK__XA_NODE_STATE, node->state,
PCMK__XA_CONNECTION_HOST, node->conn_host,
NULL);
}
}
}
pcmk__cluster_send_message(join_node, pcmk_ipc_controld, acknak);
pcmk__xml_free(acknak);
return;
}
gboolean
check_join_state(enum crmd_fsa_state cur_state, const char *source)
{
static unsigned long long highest_seq = 0;
if (controld_globals.membership_id != controld_globals.peer_seq) {
crm_debug("join-%d: Membership changed from %llu to %llu "
QB_XS " highest=%llu state=%s for=%s",
current_join_id, controld_globals.membership_id,
controld_globals.peer_seq, highest_seq,
fsa_state2string(cur_state), source);
if (highest_seq < controld_globals.peer_seq) {
/* Don't spam the FSA with duplicates */
highest_seq = controld_globals.peer_seq;
register_fsa_input_before(C_FSA_INTERNAL, I_NODE_JOIN, NULL);
}
} else if (cur_state == S_INTEGRATION) {
if (crmd_join_phase_count(controld_join_welcomed) == 0) {
int count = crmd_join_phase_count(controld_join_integrated);
crm_debug("join-%d: Integration of %d peer%s complete "
QB_XS " state=%s for=%s",
current_join_id, count, pcmk__plural_s(count),
fsa_state2string(cur_state), source);
register_fsa_input_before(C_FSA_INTERNAL, I_INTEGRATED, NULL);
return TRUE;
}
} else if (cur_state == S_FINALIZE_JOIN) {
if (!pcmk_is_set(controld_globals.fsa_input_register, R_HAVE_CIB)) {
crm_debug("join-%d: Delaying finalization until we have CIB "
QB_XS " state=%s for=%s",
current_join_id, fsa_state2string(cur_state), source);
return TRUE;
} else if (crmd_join_phase_count(controld_join_welcomed) != 0) {
int count = crmd_join_phase_count(controld_join_welcomed);
crm_debug("join-%d: Still waiting on %d welcomed node%s "
QB_XS " state=%s for=%s",
current_join_id, count, pcmk__plural_s(count),
fsa_state2string(cur_state), source);
crmd_join_phase_log(LOG_DEBUG);
} else if (crmd_join_phase_count(controld_join_integrated) != 0) {
int count = crmd_join_phase_count(controld_join_integrated);
crm_debug("join-%d: Still waiting on %d integrated node%s "
QB_XS " state=%s for=%s",
current_join_id, count, pcmk__plural_s(count),
fsa_state2string(cur_state), source);
crmd_join_phase_log(LOG_DEBUG);
} else if (crmd_join_phase_count(controld_join_finalized) != 0) {
int count = crmd_join_phase_count(controld_join_finalized);
crm_debug("join-%d: Still waiting on %d finalized node%s "
QB_XS " state=%s for=%s",
current_join_id, count, pcmk__plural_s(count),
fsa_state2string(cur_state), source);
crmd_join_phase_log(LOG_DEBUG);
} else {
crm_debug("join-%d: Complete " QB_XS " state=%s for=%s",
current_join_id, fsa_state2string(cur_state), source);
register_fsa_input_later(C_FSA_INTERNAL, I_FINALIZED, NULL);
return TRUE;
}
}
return FALSE;
}
void
do_dc_join_final(long long action,
enum crmd_fsa_cause cause,
enum crmd_fsa_state cur_state,
enum crmd_fsa_input current_input, fsa_data_t * msg_data)
{
crm_debug("Ensuring DC, quorum and node attributes are up-to-date");
crm_update_quorum(pcmk__cluster_has_quorum(), TRUE);
}
int crmd_join_phase_count(enum controld_join_phase phase)
{
int count = 0;
pcmk__node_status_t *peer;
GHashTableIter iter;
g_hash_table_iter_init(&iter, pcmk__peer_cache);
while (g_hash_table_iter_next(&iter, NULL, (gpointer *) &peer)) {
if (controld_get_join_phase(peer) == phase) {
count++;
}
}
return count;
}
void crmd_join_phase_log(int level)
{
pcmk__node_status_t *peer;
GHashTableIter iter;
g_hash_table_iter_init(&iter, pcmk__peer_cache);
while (g_hash_table_iter_next(&iter, NULL, (gpointer *) &peer)) {
do_crm_log(level, "join-%d: %s=%s", current_join_id, peer->name,
join_phase_text(controld_get_join_phase(peer)));
}
}
diff --git a/daemons/execd/remoted_schemas.c b/daemons/execd/remoted_schemas.c
index e553551659..15258ff486 100644
--- a/daemons/execd/remoted_schemas.c
+++ b/daemons/execd/remoted_schemas.c
@@ -1,286 +1,290 @@
/*
* Copyright 2023-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <ftw.h>
#include <unistd.h>
#include <sys/stat.h>
#include <crm/cib.h>
#include <crm/cib/cib_types.h>
#include <crm/cib/internal.h>
#include <crm/crm.h>
#include <crm/common/mainloop.h>
#include <crm/common/xml.h>
#include "pacemaker-execd.h"
static pid_t schema_fetch_pid = 0;
static int
rm_files(const char *pathname, const struct stat *sbuf, int type, struct FTW *ftwb)
{
/* Don't delete PCMK__REMOTE_SCHEMA_DIR . */
if (ftwb->level == 0) {
return 0;
}
if (remove(pathname) != 0) {
int rc = errno;
crm_err("Could not remove %s: %s", pathname, pcmk_rc_str(rc));
return -1;
}
return 0;
}
static void
clean_up_extra_schema_files(void)
{
const char *remote_schema_dir = pcmk__remote_schema_dir();
struct stat sb;
int rc;
rc = stat(remote_schema_dir, &sb);
if (rc == -1) {
if (errno == ENOENT) {
/* If the directory doesn't exist, try to make it first. */
if (mkdir(remote_schema_dir, 0755) != 0) {
rc = errno;
crm_err("Could not create directory for schemas: %s",
pcmk_rc_str(rc));
}
} else {
rc = errno;
crm_err("Could not create directory for schemas: %s",
pcmk_rc_str(rc));
}
} else if (!S_ISDIR(sb.st_mode)) {
/* If something exists with the same name that's not a directory, that's
* an error.
*/
crm_err("%s already exists but is not a directory", remote_schema_dir);
} else {
/* It's a directory - clear it out so we can download potentially new
* schema files.
*/
rc = nftw(remote_schema_dir, rm_files, 10, FTW_DEPTH|FTW_MOUNT|FTW_PHYS);
if (rc != 0) {
crm_err("Could not remove %s: %s", remote_schema_dir, pcmk_rc_str(rc));
}
}
}
static void
write_extra_schema_file(xmlNode *xml, void *user_data)
{
const char *remote_schema_dir = pcmk__remote_schema_dir();
const char *file = NULL;
char *path = NULL;
int rc;
file = crm_element_value(xml, PCMK_XA_PATH);
if (file == NULL) {
crm_warn("No destination path given in schema request");
return;
}
path = crm_strdup_printf("%s/%s", remote_schema_dir, file);
/* The schema is a CDATA node, which is a child of the <file> node. Traverse
* all children and look for the first CDATA child. There can't be more than
* one because we only have one file attribute on the parent.
*/
for (xmlNode *child = xml->children; child != NULL; child = child->next) {
FILE *stream = NULL;
if (child->type != XML_CDATA_SECTION_NODE) {
continue;
}
stream = fopen(path, "w+");
if (stream == NULL) {
crm_warn("Could not write schema file %s: %s", path, strerror(errno));
} else {
rc = fprintf(stream, "%s", child->content);
if (rc < 0) {
crm_warn("Could not write schema file %s: %s", path, strerror(errno));
}
fclose(stream);
}
break;
}
free(path);
}
static void
get_schema_files(void)
{
int rc = pcmk_rc_ok;
cib_t *cib = NULL;
xmlNode *reply;
cib = cib_new();
if (cib == NULL) {
- _exit(ENOTCONN);
+ pcmk_common_cleanup();
+ _exit(CRM_EX_OSERR);
}
rc = cib->cmds->signon(cib, crm_system_name, cib_query);
- if (rc != pcmk_ok) {
- crm_err("Could not connect to the CIB manager: %s", pcmk_strerror(rc));
+ rc = pcmk_legacy2rc(rc);
+ if (rc != pcmk_rc_ok) {
+ crm_err("Could not connect to the CIB manager: %s", pcmk_rc_str(rc));
+ pcmk_common_cleanup();
_exit(pcmk_rc2exitc(rc));
}
rc = cib->cmds->fetch_schemas(cib, &reply, pcmk__highest_schema_name(),
cib_sync_call);
if (rc != pcmk_ok) {
crm_err("Could not get schema files: %s", pcmk_strerror(rc));
rc = pcmk_legacy2rc(rc);
} else if (reply->children != NULL) {
/* The returned document looks something like this:
* <cib_command>
* <cib_calldata>
* <schemas>
* <schema version="pacemaker-1.1">
* <file path="foo-1.1">
* </file>
* <file path="bar-1.1">
* </file>
* ...
* </schema>
* <schema version="pacemaker-1.2">
* </schema>
* ...
* </schemas>
* </cib_calldata>
* </cib_command>
*
* All the <schemas> and <schema> tags are really just there for organizing
* the XML a little better. What we really care about are the <file> nodes,
* and specifically the path attributes and the CDATA children (not shown)
* of each. We can use an xpath query to reach down and get all the <file>
* nodes at once.
*
* If we already have the latest schema version, or we asked for one later
* than what the cluster supports, we'll get back an empty <schemas> node,
* so all this will continue to work. It just won't do anything.
*/
crm_foreach_xpath_result(reply, "//" PCMK_XA_FILE,
write_extra_schema_file, NULL);
}
pcmk__xml_free(reply);
cib__clean_up_connection(&cib);
+ pcmk_common_cleanup();
_exit(pcmk_rc2exitc(rc));
}
/* Load any additional schema files when the child is finished fetching and
* saving them to disk.
*/
static void
get_schema_files_complete(mainloop_child_t *p, pid_t pid, int core, int signo, int exitcode)
{
const char *errmsg = "Could not load additional schema files";
if ((signo == 0) && (exitcode == 0)) {
const char *remote_schema_dir = pcmk__remote_schema_dir();
/* Don't just call pcmk__schema_init() here because that will load the
* base schemas again too. Instead just load the things we fetched.
*/
pcmk__load_schemas_from_dir(remote_schema_dir);
pcmk__sort_schemas();
crm_info("Fetching extra schema files completed successfully");
} else {
if (signo == 0) {
crm_err("%s: process %d exited %d", errmsg, (int) pid, exitcode);
} else {
crm_err("%s: process %d terminated with signal %d (%s)%s",
errmsg, (int) pid, signo, strsignal(signo),
(core? " and dumped core" : ""));
}
/* Clean up any incomplete schema data we might have been downloading
* when the process timed out or crashed. We don't need to do any extra
* cleanup because we never loaded the extra schemas, and we don't need
* to call pcmk__schema_init() because that was called in
* remoted_request_cib_schema_files() before this function.
*/
clean_up_extra_schema_files();
}
}
void
remoted_request_cib_schema_files(void)
{
pid_t pid;
int rc;
/* If a previous schema-fetch process is still running when we're called
* again, it's hung. Attempt to kill it before cleaning up the extra
* directory.
*/
if (schema_fetch_pid != 0) {
if (mainloop_child_kill(schema_fetch_pid) == FALSE) {
crm_warn("Unable to kill pre-existing schema-fetch process");
return;
}
schema_fetch_pid = 0;
}
/* Clean up any extra schema files we downloaded from a previous cluster
* connection. After the files are gone, we need to wipe them from
* known_schemas, but there's no opposite operation for add_schema().
*
* Instead, unload all the schemas. This means we'll also forget about all
* installed schemas as well, which means that pcmk__highest_schema_name()
* would fail. So we need to load the base schemas right now.
*/
clean_up_extra_schema_files();
pcmk__schema_cleanup();
pcmk__schema_init();
crm_info("Fetching extra schema files from cluster");
pid = fork();
switch (pid) {
case -1: {
rc = errno;
crm_warn("Could not spawn process to get schema files: %s", pcmk_rc_str(rc));
break;
}
case 0:
/* child */
get_schema_files();
break;
default:
/* parent */
schema_fetch_pid = pid;
mainloop_child_add_with_flags(pid, 5 * 60 * 1000, "schema-fetch", NULL,
mainloop_leave_pid_group,
get_schema_files_complete);
break;
}
}
diff --git a/include/crm/common/actions_internal.h b/include/crm/common/actions_internal.h
index f05e9d59d5..3d025cd5b4 100644
--- a/include/crm/common/actions_internal.h
+++ b/include/crm/common/actions_internal.h
@@ -1,269 +1,269 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#ifndef PCMK__CRM_COMMON_ACTIONS_INTERNAL__H
#define PCMK__CRM_COMMON_ACTIONS_INTERNAL__H
#include <stdbool.h> // bool
#include <stdint.h> // uint32_t, UINT32_C()
#include <glib.h> // guint, GList, GHashTable
#include <libxml/tree.h> // xmlNode
#include <crm/common/actions.h> // PCMK_ACTION_MONITOR
#include <crm/common/roles.h> // enum rsc_role_e
#include <crm/common/scheduler_types.h> // pcmk_resource_t, pcmk_node_t
#include <crm/common/strings_internal.h> // pcmk__str_eq()
#ifdef __cplusplus
extern "C" {
#endif
//! printf-style format to create operation key from resource, action, interval
#define PCMK__OP_FMT "%s_%s_%u"
/*!
* \internal
* \brief Set action flags for an action
*
* \param[in,out] action Action to set flags for
* \param[in] flags_to_set Group of enum pcmk__action_flags to set
*/
#define pcmk__set_action_flags(action, flags_to_set) do { \
(action)->flags = pcmk__set_flags_as(__func__, __LINE__, \
LOG_TRACE, \
"Action", (action)->uuid, \
(action)->flags, \
(flags_to_set), \
#flags_to_set); \
} while (0)
/*!
* \internal
* \brief Clear action flags for an action
*
* \param[in,out] action Action to clear flags for
* \param[in] flags_to_clear Group of enum pcmk__action_flags to clear
*/
#define pcmk__clear_action_flags(action, flags_to_clear) do { \
(action)->flags = pcmk__clear_flags_as(__func__, __LINE__, \
LOG_TRACE, \
"Action", (action)->uuid, \
(action)->flags, \
(flags_to_clear), \
#flags_to_clear); \
} while (0)
/*!
* \internal
* \brief Set action flags for a flag group
*
* \param[in,out] action_flags Flag group to set flags for
* \param[in] action_name Name of action being modified (for logging)
* \param[in] to_set Group of enum pcmk__action_flags to set
*/
#define pcmk__set_raw_action_flags(action_flags, action_name, to_set) do { \
action_flags = pcmk__set_flags_as(__func__, __LINE__, \
LOG_TRACE, "Action", action_name, \
(action_flags), \
(to_set), #to_set); \
} while (0)
/*!
* \internal
* \brief Clear action flags for a flag group
*
* \param[in,out] action_flags Flag group to clear flags for
* \param[in] action_name Name of action being modified (for logging)
* \param[in] to_clear Group of enum pcmk__action_flags to clear
*/
#define pcmk__clear_raw_action_flags(action_flags, action_name, to_clear) \
do { \
action_flags = pcmk__clear_flags_as(__func__, __LINE__, LOG_TRACE, \
"Action", action_name, \
(action_flags), \
(to_clear), #to_clear); \
} while (0)
// Possible actions (including some pseudo-actions)
enum pcmk__action_type {
pcmk__action_unspecified = 0, // Unspecified or unknown action
pcmk__action_monitor, // Monitor
// Each "completed" action must be the regular action plus 1
pcmk__action_stop, // Stop
pcmk__action_stopped, // Stop completed
pcmk__action_start, // Start
pcmk__action_started, // Start completed
pcmk__action_notify, // Notify
pcmk__action_notified, // Notify completed
pcmk__action_promote, // Promote
pcmk__action_promoted, // Promoted
pcmk__action_demote, // Demote
pcmk__action_demoted, // Demoted
pcmk__action_shutdown, // Shut down node
pcmk__action_fence, // Fence node
};
// Action scheduling flags
enum pcmk__action_flags {
// No action flags set (compare with equality rather than bit set)
pcmk__no_action_flags = 0,
// Whether action does not require invoking an agent
pcmk__action_pseudo = (UINT32_C(1) << 0),
// Whether action is runnable
pcmk__action_runnable = (UINT32_C(1) << 1),
// Whether action should not be executed
pcmk__action_optional = (UINT32_C(1) << 2),
// Whether action should be added to transition graph even if optional
pcmk__action_always_in_graph = (UINT32_C(1) << 3),
// Whether operation-specific instance attributes have been unpacked yet
pcmk__action_attrs_evaluated = (UINT32_C(1) << 4),
// Whether action is allowed to be part of a live migration
pcmk__action_migratable = (UINT32_C(1) << 7),
// Whether action has been added to transition graph
pcmk__action_added_to_graph = (UINT32_C(1) << 8),
// Whether action is a stop to abort a dangling migration
pcmk__action_migration_abort = (UINT32_C(1) << 11),
// Whether action is recurring monitor that must be rescheduled if active
pcmk__action_reschedule = (UINT32_C(1) << 13),
// Whether action has already been processed by a recursive procedure
pcmk__action_detect_loop = (UINT32_C(1) << 14),
// Whether action's inputs have been de-duplicated yet
pcmk__action_inputs_deduplicated = (UINT32_C(1) << 15),
// Whether action can be executed on DC rather than own node
pcmk__action_on_dc = (UINT32_C(1) << 16),
};
/* Possible responses to a resource action failure
*
* The order is significant; the values are in order of increasing severity so
* that they can be compared with less than and greater than.
*/
enum pcmk__on_fail {
pcmk__on_fail_ignore, // Act as if failure didn't happen
pcmk__on_fail_demote, // Demote if promotable, else stop
pcmk__on_fail_restart, // Restart resource
/* Fence the remote node created by the resource if fencing is enabled,
* otherwise attempt to restart the resource (used internally for some
* remote connection failures).
*/
pcmk__on_fail_reset_remote,
pcmk__on_fail_restart_container, // Restart resource's container
pcmk__on_fail_ban, // Ban resource from current node
pcmk__on_fail_block, // Treat resource as unmanaged
pcmk__on_fail_stop, // Stop resource and leave stopped
pcmk__on_fail_standby_node, // Put resource's node in standby
pcmk__on_fail_fence_node, // Fence resource's node
};
// What resource needs before it can be recovered from a failed node
enum pcmk__requires {
pcmk__requires_nothing = 0, // Resource can be recovered immediately
pcmk__requires_quorum = 1, // Resource can be recovered if quorate
pcmk__requires_fencing = 2, // Resource can be recovered after fencing
};
// Implementation of pcmk_action_t
struct pcmk__action {
int id; // Counter to identify action
/*
* When the controller aborts a transition graph, it sets an abort priority.
* If this priority is higher, the action will still be executed anyway.
* Pseudo-actions are always allowed, so this is irrelevant for them.
*/
int priority;
pcmk_resource_t *rsc; // Resource to apply action to, if any
- pcmk_node_t *node; // Node to execute action on, if any
+ pcmk_node_t *node; // Copy of node to execute action on, if any
xmlNode *op_entry; // Action XML configuration, if any
char *task; // Action name
char *uuid; // Action key
char *cancel_task; // If task is "cancel", the action being cancelled
char *reason; // Readable description of why action is needed
uint32_t flags; // Group of enum pcmk__action_flags
enum pcmk__requires needs; // Prerequisite for recovery
enum pcmk__on_fail on_fail; // Response to failure
enum rsc_role_e fail_role; // Resource role if action fails
GHashTable *meta; // Meta-attributes relevant to action
GHashTable *extra; // Action-specific instance attributes
pcmk_scheduler_t *scheduler; // Scheduler data this action is part of
/* Current count of runnable instance actions for "first" action in an
* ordering dependency with pcmk__ar_min_runnable set.
*/
int runnable_before;
/*
* Number of instance actions for "first" action in an ordering dependency
* with pcmk__ar_min_runnable set that must be runnable before this action
* can be runnable.
*/
int required_runnable_before;
// Actions in a relation with this one (as pcmk__related_action_t *)
GList *actions_before;
GList *actions_after;
};
char *pcmk__op_key(const char *rsc_id, const char *op_type, guint interval_ms);
char *pcmk__notify_key(const char *rsc_id, const char *notify_type,
const char *op_type);
char *pcmk__transition_key(int transition_id, int action_id, int target_rc,
const char *node);
void pcmk__filter_op_for_digest(xmlNode *param_set);
bool pcmk__is_fencing_action(const char *action);
enum pcmk__action_type pcmk__parse_action(const char *action_name);
const char *pcmk__action_text(enum pcmk__action_type action);
const char *pcmk__on_fail_text(enum pcmk__on_fail on_fail);
/*!
* \internal
* \brief Get a human-friendly action name
*
* \param[in] action_name Actual action name
* \param[in] interval_ms Action interval (in milliseconds)
*
* \return Action name suitable for display
*/
static inline const char *
pcmk__readable_action(const char *action_name, guint interval_ms) {
if ((interval_ms == 0)
&& pcmk__str_eq(action_name, PCMK_ACTION_MONITOR, pcmk__str_none)) {
return "probe";
}
return action_name;
}
#ifdef __cplusplus
}
#endif
#endif // PCMK__CRM_COMMON_ACTIONS_INTERNAL__H
diff --git a/include/crm/common/bundles_internal.h b/include/crm/common/bundles_internal.h
index a0abdeed58..27fe053c0e 100644
--- a/include/crm/common/bundles_internal.h
+++ b/include/crm/common/bundles_internal.h
@@ -1,91 +1,91 @@
/*
* Copyright 2017-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#ifndef PCMK__CRM_COMMON_BUNDLES_INTERNAL__H
#define PCMK__CRM_COMMON_BUNDLES_INTERNAL__H
#include <stdio.h> // NULL
#include <stdbool.h> // bool, false
#include <crm/common/nodes_internal.h> // struct pcmk__node_private
#include <crm/common/remote_internal.h> // pcmk__is_guest_or_bundle_node()
#include <crm/common/resources_internal.h> // pcmk__rsc_variant_bundle etc.
#include <crm/common/scheduler_types.h> // pcmk_resource_t, pcmk_node_t
#ifdef __cplusplus
extern "C" {
#endif
//! A single instance of a bundle
typedef struct {
int offset; //!< 0-origin index of this instance in bundle
char *ipaddr; //!< IP address associated with this instance
- pcmk_node_t *node; //!< Node created for this instance
+ pcmk_node_t *node; //!< Copy of node created for this instance
pcmk_resource_t *ip; //!< IP address resource for ipaddr
pcmk_resource_t *child; //!< Instance of bundled resource
pcmk_resource_t *container; //!< Container associated with this instance
pcmk_resource_t *remote; //!< Pacemaker Remote connection into container
} pcmk__bundle_replica_t;
/*!
* \internal
* \brief Check whether a resource is a bundle resource
*
* \param[in] rsc Resource to check
*
* \return true if \p rsc is a bundle, otherwise false
* \note This does not return true if \p rsc is part of a bundle
* (see pcmk__is_bundled()).
*/
static inline bool
pcmk__is_bundle(const pcmk_resource_t *rsc)
{
return (rsc != NULL) && (rsc->priv->variant == pcmk__rsc_variant_bundle);
}
/*!
* \internal
* \brief Check whether a resource is part of a bundle
*
* \param[in] rsc Resource to check
*
* \return true if \p rsc is part of a bundle, otherwise false
*/
static inline bool
pcmk__is_bundled(const pcmk_resource_t *rsc)
{
if (rsc == NULL) {
return false;
}
while (rsc->priv->parent != NULL) {
rsc = rsc->priv->parent;
}
return rsc->priv->variant == pcmk__rsc_variant_bundle;
}
/*!
* \internal
* \brief Check whether a node is a bundle node
*
* \param[in] node Node to check
*
* \return true if \p node is a bundle node, otherwise false
*/
static inline bool
pcmk__is_bundle_node(const pcmk_node_t *node)
{
return pcmk__is_guest_or_bundle_node(node)
&& pcmk__is_bundled(node->priv->remote);
}
#ifdef __cplusplus
}
#endif
#endif // PCMK__CRM_COMMON_BUNDLES_INTERNAL__H
diff --git a/include/crm/common/location_internal.h b/include/crm/common/location_internal.h
index 5991feea98..39aba91c4b 100644
--- a/include/crm/common/location_internal.h
+++ b/include/crm/common/location_internal.h
@@ -1,36 +1,36 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#ifndef PCMK__CRM_COMMON_LOCATION_INTERNAL__H
#define PCMK__CRM_COMMON_LOCATION_INTERNAL__H
#include <glib.h> // GList
#include <crm/common/nodes_internal.h> // enum pcmk__probe_mode
#include <crm/common/resources.h> // enum rsc_role_e
#include <crm/common/scheduler_types.h> // pcmk_resource_t
#ifdef __cplusplus
extern "C" {
#endif
//! Location constraint object
typedef struct {
char *id; // XML ID of location constraint
pcmk_resource_t *rsc; // Resource with location preference
enum rsc_role_e role_filter; // Limit to instances with this role
enum pcmk__probe_mode probe_mode; // How to probe resource on node
- GList *nodes; // Affected nodes, with preference score
+ GList *nodes; // Copies of affected nodes, with score
} pcmk__location_t;
#ifdef __cplusplus
}
#endif
#endif // PCMK__CRM_COMMON_LOCATION_INTERNAL__H
diff --git a/include/crm/common/nodes_internal.h b/include/crm/common/nodes_internal.h
index 176b0dbaae..4e0dd86691 100644
--- a/include/crm/common/nodes_internal.h
+++ b/include/crm/common/nodes_internal.h
@@ -1,201 +1,202 @@
/*
* Copyright 2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#ifndef PCMK__CRM_COMMON_NODES_INTERNAL__H
#define PCMK__CRM_COMMON_NODES_INTERNAL__H
#include <stdio.h> // NULL
#include <stdbool.h> // bool
#include <stdint.h> // uint32_t, UINT32_C()
#include <glib.h>
#include <crm/common/nodes.h>
#ifdef __cplusplus
extern "C" {
#endif
/*
* Special node attributes
*/
#define PCMK__NODE_ATTR_SHUTDOWN "shutdown"
/* @COMPAT Deprecated since 2.1.8. Use a location constraint with
* PCMK_XA_RSC_PATTERN=".*" and PCMK_XA_RESOURCE_DISCOVERY="never" instead of
* PCMK__NODE_ATTR_RESOURCE_DISCOVERY_ENABLED="false".
*/
#define PCMK__NODE_ATTR_RESOURCE_DISCOVERY_ENABLED "resource-discovery-enabled"
enum pcmk__node_variant { // Possible node types
pcmk__node_variant_cluster = 1, // Cluster layer node
pcmk__node_variant_remote = 2, // Pacemaker Remote node
};
enum pcmk__node_flags {
pcmk__node_none = UINT32_C(0),
// Whether node is in standby mode
pcmk__node_standby = (UINT32_C(1) << 0),
// Whether node is in standby mode due to PCMK_META_ON_FAIL
pcmk__node_fail_standby = (UINT32_C(1) << 1),
// Whether node has ever joined cluster (and thus has node state in CIB)
pcmk__node_seen = (UINT32_C(1) << 2),
// Whether expected join state is member
pcmk__node_expected_up = (UINT32_C(1) << 3),
// Whether probes are allowed on node
pcmk__node_probes_allowed = (UINT32_C(1) << 4),
/* Whether this either is a guest node whose guest resource must be
* recovered or a remote node that must be fenced
*/
pcmk__node_remote_reset = (UINT32_C(1) << 5),
/* Whether this is a Pacemaker Remote node that was fenced since it was last
* connected by the cluster
*/
pcmk__node_remote_fenced = (UINT32_C(1) << 6),
/*
* Whether this is a Pacemaker Remote node previously marked in its
* node state as being in maintenance mode
*/
pcmk__node_remote_maint = (UINT32_C(1) << 7),
// Whether node history has been unpacked
pcmk__node_unpacked = (UINT32_C(1) << 8),
};
// When to probe a resource on a node (as specified in location constraints)
enum pcmk__probe_mode {
pcmk__probe_always = 0, // Always probe resource on node
pcmk__probe_never = 1, // Never probe resource on node
pcmk__probe_exclusive = 2, // Probe only on designated nodes
};
/* Per-node data used in resource assignment
*
* @COMPAT When we can make the pcmk_node_t implementation internal, move these
* there and drop this struct.
*/
struct pcmk__node_assignment {
int score; // Node's score for relevant resource
int count; // Counter reused by assignment and promotion code
enum pcmk__probe_mode probe_mode; // When to probe resource on this node
};
/* Implementation of pcmk__node_private_t (pcmk_node_t objects are shallow
* copies, so all pcmk_node_t objects for the same node will share the same
* private data)
*/
struct pcmk__node_private {
/* Node's XML ID in the CIB (the cluster layer ID for cluster nodes,
* the node name for Pacemaker Remote nodes)
*/
const char *id;
/*
* Sum of priorities of all resources active on node and on any guest nodes
* connected to this node, with +1 for promoted instances (used to compare
* nodes for PCMK_OPT_PRIORITY_FENCING_DELAY)
*/
int priority;
const char *name; // Node name in cluster
enum pcmk__node_variant variant; // Node variant
uint32_t flags; // Group of enum pcmk__node_flags
GHashTable *attrs; // Node attributes
GHashTable *utilization; // Node utilization attributes
int num_resources; // Number of active resources on node
GList *assigned_resources; // List of resources assigned to node
GHashTable *digest_cache; // Cache of calculated resource digests
pcmk_resource_t *remote; // Pacemaker Remote connection (if any)
pcmk_scheduler_t *scheduler; // Scheduler data that node is part of
};
+void pcmk__free_node_copy(void *data);
pcmk_node_t *pcmk__find_node_in_list(const GList *nodes, const char *node_name);
/*!
* \internal
* \brief Set node flags
*
* \param[in,out] node Node to set flags for
* \param[in] flags_to_set Group of enum pcmk_node_flags to set
*/
#define pcmk__set_node_flags(node, flags_to_set) do { \
(node)->priv->flags = pcmk__set_flags_as(__func__, __LINE__, \
LOG_TRACE, "Node", pcmk__node_name(node), \
(node)->priv->flags, (flags_to_set), #flags_to_set); \
} while (0)
/*!
* \internal
* \brief Clear node flags
*
* \param[in,out] node Node to clear flags for
* \param[in] flags_to_clear Group of enum pcmk_node_flags to clear
*/
#define pcmk__clear_node_flags(node, flags_to_clear) do { \
(node)->priv->flags = pcmk__clear_flags_as(__func__, __LINE__, \
LOG_TRACE, "Node", pcmk__node_name(node), \
(node)->priv->flags, (flags_to_clear), #flags_to_clear); \
} while (0)
/*!
* \internal
* \brief Return a string suitable for logging as a node name
*
* \param[in] node Node to return a node name string for
*
* \return Node name if available, otherwise node ID if available,
* otherwise "unspecified node" if node is NULL or "unidentified node"
* if node has neither a name nor ID.
*/
static inline const char *
pcmk__node_name(const pcmk_node_t *node)
{
if (node == NULL) {
return "unspecified node";
} else if (node->priv->name != NULL) {
return node->priv->name;
} else if (node->priv->id != NULL) {
return node->priv->id;
} else {
return "unidentified node";
}
}
/*!
* \internal
* \brief Check whether two node objects refer to the same node
*
* \param[in] node1 First node object to compare
* \param[in] node2 Second node object to compare
*
* \return true if \p node1 and \p node2 refer to the same node
*/
static inline bool
pcmk__same_node(const pcmk_node_t *node1, const pcmk_node_t *node2)
{
return (node1 != NULL) && (node2 != NULL)
&& (node1->priv == node2->priv);
}
#ifdef __cplusplus
}
#endif
#endif // PCMK__CRM_COMMON_NODES_INTERNAL__H
diff --git a/include/crm/common/resources_internal.h b/include/crm/common/resources_internal.h
index b499a808a7..87ac4c609c 100644
--- a/include/crm/common/resources_internal.h
+++ b/include/crm/common/resources_internal.h
@@ -1,462 +1,468 @@
/*
* Copyright 2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#ifndef PCMK__CRM_COMMON_RESOURCES_INTERNAL__H
#define PCMK__CRM_COMMON_RESOURCES_INTERNAL__H
#include <stdint.h> // uint32_t
#include <glib.h> // gboolean, guint, GHashTable, GList
#include <libxml/tree.h> // xmlNode
#include <crm/common/resources.h> // pcmk_resource_t
#include <crm/common/roles.h> // enum rsc_role_e
#include <crm/common/scheduler_types.h> // pcmk_node_t, etc.
#ifdef __cplusplus
extern "C" {
#endif
/*!
* \internal
* \brief Set resource flags
*
* \param[in,out] resource Resource to set flags for
* \param[in] flags_to_set Group of enum pcmk_rsc_flags to set
*/
#define pcmk__set_rsc_flags(resource, flags_to_set) do { \
(resource)->flags = pcmk__set_flags_as(__func__, __LINE__, \
LOG_TRACE, "Resource", (resource)->id, (resource)->flags, \
(flags_to_set), #flags_to_set); \
} while (0)
/*!
* \internal
* \brief Clear resource flags
*
* \param[in,out] resource Resource to clear flags for
* \param[in] flags_to_clear Group of enum pcmk_rsc_flags to clear
*/
#define pcmk__clear_rsc_flags(resource, flags_to_clear) do { \
(resource)->flags = pcmk__clear_flags_as(__func__, __LINE__, \
LOG_TRACE, "Resource", (resource)->id, (resource)->flags, \
(flags_to_clear), #flags_to_clear); \
} while (0)
//! Resource variants supported by Pacemaker
enum pcmk__rsc_variant {
// Order matters: some code compares greater or lesser than
pcmk__rsc_variant_unknown = -1, //!< Unknown resource variant
pcmk__rsc_variant_primitive = 0, //!< Primitive resource
pcmk__rsc_variant_group = 1, //!< Group resource
pcmk__rsc_variant_clone = 2, //!< Clone resource
pcmk__rsc_variant_bundle = 3, //!< Bundle resource
};
//! How to recover a resource that is incorrectly active on multiple nodes
enum pcmk__multiply_active {
pcmk__multiply_active_restart, //!< Stop on all, start on desired
pcmk__multiply_active_stop, //!< Stop on all and leave stopped
pcmk__multiply_active_block, //!< Do nothing to resource
pcmk__multiply_active_unexpected, //!< Stop unexpected instances
};
//! Resource scheduling flags
enum pcmk__rsc_flags {
// No resource flags set (compare with equality rather than bit set)
pcmk__no_rsc_flags = 0ULL,
// Whether resource has been removed from the configuration
pcmk__rsc_removed = (1ULL << 0),
/* NOTE: sbd (at least as of 1.5.2) uses pe_rsc_managed which equates to
* this value, so the value should not be changed
*/
// Whether resource is managed
pcmk__rsc_managed = (1ULL << 1),
// Whether resource is blocked from further action
pcmk__rsc_blocked = (1ULL << 2),
// Whether resource has been removed but was launched
pcmk__rsc_removed_launched = (1ULL << 3),
// Whether resource has clone notifications enabled
pcmk__rsc_notify = (1ULL << 4),
// Whether resource is not an anonymous clone instance
pcmk__rsc_unique = (1ULL << 5),
// Whether resource's class is "stonith"
pcmk__rsc_fence_device = (1ULL << 6),
// Whether resource can be promoted and demoted
pcmk__rsc_promotable = (1ULL << 7),
// Whether resource has not yet been assigned to a node
pcmk__rsc_unassigned = (1ULL << 8),
// Whether resource is in the process of being assigned to a node
pcmk__rsc_assigning = (1ULL << 9),
// Whether resource is in the process of modifying allowed node scores
pcmk__rsc_updating_nodes = (1ULL << 10),
// Whether resource is in the process of scheduling actions to restart
pcmk__rsc_restarting = (1ULL << 11),
// Whether resource must be stopped (instead of demoted) if it is failed
pcmk__rsc_stop_if_failed = (1ULL << 12),
// Whether a reload action has been scheduled for resource
pcmk__rsc_reload = (1ULL << 13),
// Whether resource is a remote connection allowed to run on a remote node
pcmk__rsc_remote_nesting_allowed = (1ULL << 14),
// Whether resource has \c PCMK_META_CRITICAL meta-attribute enabled
pcmk__rsc_critical = (1ULL << 15),
// Whether resource is considered failed
pcmk__rsc_failed = (1ULL << 16),
// Flag for non-scheduler code to use to detect recursion loops
pcmk__rsc_detect_loop = (1ULL << 17),
// Whether resource is a Pacemaker Remote connection
pcmk__rsc_is_remote_connection = (1ULL << 18),
// Whether resource has pending start action in history
pcmk__rsc_start_pending = (1ULL << 19),
// Whether resource is probed only on nodes marked exclusive
pcmk__rsc_exclusive_probes = (1ULL << 20),
/*
* Whether resource is multiply active with recovery set to
* \c PCMK_VALUE_STOP_UNEXPECTED
*/
pcmk__rsc_stop_unexpected = (1ULL << 22),
// Whether resource is allowed to live-migrate
pcmk__rsc_migratable = (1ULL << 23),
// Whether resource has an ignorable failure
pcmk__rsc_ignore_failure = (1ULL << 24),
// Whether resource is an implicit container resource for a bundle replica
pcmk__rsc_replica_container = (1ULL << 25),
// Whether resource, its node, or entire cluster is in maintenance mode
pcmk__rsc_maintenance = (1ULL << 26),
// Whether resource can be started or promoted only on quorate nodes
pcmk__rsc_needs_quorum = (1ULL << 28),
// Whether resource requires fencing before recovery if on unclean node
pcmk__rsc_needs_fencing = (1ULL << 29),
// Whether resource can be started or promoted only on unfenced nodes
pcmk__rsc_needs_unfencing = (1ULL << 30),
};
// Where to look for a resource
enum pcmk__rsc_node {
pcmk__rsc_node_none = 0U, // Nowhere
pcmk__rsc_node_assigned = (1U << 0), // Where resource is assigned
pcmk__rsc_node_current = (1U << 1), // Where resource is running
pcmk__rsc_node_pending = (1U << 2), // Where resource is pending
};
//! Resource assignment methods (implementation defined by libpacemaker)
typedef struct pcmk__assignment_methods pcmk__assignment_methods_t;
//! Resource object methods
typedef struct {
/*!
* \internal
* \brief Parse variant-specific resource XML from CIB into struct members
*
* \param[in,out] rsc Partially unpacked resource
* \param[in,out] scheduler Scheduler data
*
* \return TRUE if resource was unpacked successfully, otherwise FALSE
*/
gboolean (*unpack)(pcmk_resource_t *rsc, pcmk_scheduler_t *scheduler);
/*!
* \internal
* \brief Search for a resource ID in a resource and its children
*
* \param[in] rsc Search this resource and its children
* \param[in] id Search for this resource ID
* \param[in] on_node If not NULL, limit search to resources on this node
* \param[in] flags Group of enum pe_find flags
*
* \return Resource that matches search criteria if any, otherwise NULL
*/
pcmk_resource_t *(*find_rsc)(pcmk_resource_t *rsc, const char *search,
const pcmk_node_t *node, int flags);
/*!
* \internal
* \brief Get value of a resource instance attribute
*
* \param[in,out] rsc Resource to check
* \param[in] node Node to use to evaluate rules
* \param[in] create Ignored
* \param[in] name Name of instance attribute to check
* \param[in,out] scheduler Scheduler data
*
* \return Value of requested attribute if available, otherwise NULL
* \note The caller is responsible for freeing the result using free().
*/
char *(*parameter)(pcmk_resource_t *rsc, pcmk_node_t *node, gboolean create,
const char *name, pcmk_scheduler_t *scheduler);
/*!
* \internal
* \brief Check whether a resource is active
*
* \param[in] rsc Resource to check
* \param[in] all If \p rsc is collective, all instances must be active
*
* \return TRUE if \p rsc is active, otherwise FALSE
*/
gboolean (*active)(pcmk_resource_t *rsc, gboolean all);
/*!
* \internal
* \brief Get resource's current or assigned role
*
* \param[in] rsc Resource to check
* \param[in] current If TRUE, check current role, otherwise assigned role
*
* \return Current or assigned role of \p rsc
*/
enum rsc_role_e (*state)(const pcmk_resource_t *rsc, gboolean current);
/*!
* \internal
* \brief List nodes where a resource (or any of its children) is
*
* \param[in] rsc Resource to check
* \param[out] list List to add result to
* \param[in] target Which resource conditions to target (group of
* enum pcmk__rsc_node flags)
*
* \return If list contains only one node, that node, otherwise NULL
*/
pcmk_node_t *(*location)(const pcmk_resource_t *rsc, GList **list,
uint32_t target);
/*!
* \internal
* \brief Free all memory used by a resource
*
* \param[in,out] rsc Resource to free
*/
void (*free)(pcmk_resource_t *rsc);
/*!
* \internal
* \brief Increment cluster's instance counts for a resource
*
* Given a resource, increment its cluster's ninstances, disabled_resources,
* and blocked_resources counts for the resource and its descendants.
*
* \param[in,out] rsc Resource to count
*/
void (*count)(pcmk_resource_t *rsc);
/*!
* \internal
* \brief Check whether a given resource is in a list of resources
*
* \param[in] rsc Resource ID to check for
* \param[in] only_rsc List of resource IDs to check
* \param[in] check_parent If TRUE, check top ancestor as well
*
* \return TRUE if \p rsc, its top parent if requested, or '*' is in
* \p only_rsc, otherwise FALSE
*/
gboolean (*is_filtered)(const pcmk_resource_t *rsc, GList *only_rsc,
gboolean check_parent);
/*!
* \internal
* \brief Find a node (and optionally count all) where resource is active
*
* \param[in] rsc Resource to check
* \param[out] count_all If not NULL, set this to count of active nodes
* \param[out] count_clean If not NULL, set this to count of clean nodes
*
* \return A node where the resource is active, preferring the source node
* if the resource is involved in a partial migration, or a clean,
* online node if the resource's \c PCMK_META_REQUIRES is
* \c PCMK_VALUE_QUORUM or \c PCMK_VALUE_NOTHING, otherwise \c NULL.
*/
pcmk_node_t *(*active_node)(const pcmk_resource_t *rsc,
unsigned int *count_all,
unsigned int *count_clean);
/*!
* \internal
* \brief Get maximum resource instances per node
*
* \param[in] rsc Resource to check
*
* \return Maximum number of \p rsc instances that can be active on one node
*/
unsigned int (*max_per_node)(const pcmk_resource_t *rsc);
} pcmk__rsc_methods_t;
// Implementation of pcmk__resource_private_t
struct pcmk__resource_private {
enum pcmk__rsc_variant variant; // Resource variant
void *variant_opaque; // Variant-specific data
char *history_id; // Resource instance ID in history
GHashTable *meta; // Resource meta-attributes
GHashTable *utilization; // Resource utilization attributes
int priority; // Priority relative other resources
int promotion_priority; // Promotion priority on assigned node
enum rsc_role_e orig_role; // Resource's role at start of transition
enum rsc_role_e next_role; // Resource's role at end of transition
int stickiness; // Extra preference for current node
guint failure_expiration_ms; // Failures expire after this much time
int ban_after_failures; // Ban from node after this many failures
guint remote_reconnect_ms; // Retry interval for remote connections
char *pending_action; // Pending action in history, if any
const pcmk_node_t *pending_node;// Node on which pending_action is happening
time_t lock_time; // When shutdown lock started
const pcmk_node_t *lock_node; // Node that resource is shutdown-locked to
GList *actions; // Actions scheduled for resource
GList *children; // Resource's child resources, if any
pcmk_resource_t *parent; // Resource's parent resource, if any
pcmk_scheduler_t *scheduler; // Scheduler data containing resource
// Resource configuration (possibly expanded from template)
xmlNode *xml;
// Original resource configuration, if using template
xmlNode *orig_xml;
// Configuration of resource operations (possibly expanded from template)
xmlNode *ops_xml;
/*
* Resource parameters may have node-attribute-based rules, which means the
* values can vary by node. This table has node names as keys and parameter
* name/value tables as values. Use pe_rsc_params() to get the table for a
* given node rather than use this directly.
*/
GHashTable *parameter_cache;
/* A "launcher" is defined in one of these ways:
*
* - A Pacemaker Remote connection for a guest node or bundle node has its
* launcher set to the resource that starts the guest or the bundle
* replica's container.
*
* - If the user configures the PCMK__META_CONTAINER meta-attribute for this
* resource, the launcher is set to that.
*
* If the launcher is a Pacemaker Remote connection resource, this
* resource may run only on the node created by that connection.
*
* Otherwise, this resource will be colocated with and ordered after the
* launcher, and failures of this resource will cause the launcher to be
* recovered instead of this one. This is appropriate for monitoring-only
* resources that represent a service launched by the other resource.
*/
pcmk_resource_t *launcher;
// Resources launched by this one, if any (pcmk_resource_t *)
GList *launched;
// What to do if the resource is incorrectly active on multiple nodes
enum pcmk__multiply_active multiply_active_policy;
/* The assigned node (if not NULL) is the one where the resource *should*
* be active by the end of the current scheduler transition. Only primitive
- * resources have an assigned node.
+ * resources have an assigned node. This is a node copy (created by
+ * pe__copy_node()) and so must be freed using pcmk__free_node_copy().
*
* @TODO This should probably be part of the primitive variant data.
*/
pcmk_node_t *assigned_node;
/* The active nodes are ones where the resource is (or might be, if
* insufficient information is available to be sure) already active at the
* start of the current scheduler transition.
*
* For primitive resources, there should be at most one, but could be more
* if it is (incorrectly) multiply active. For collective resources, this
* combines active nodes of all descendants.
*/
GList *active_nodes;
+ /* The next two tables store node copies (created by pe__copy_node()), which
+ * share some members with the original node objects and must be freed with
+ * pcmk__free_node_copy().
+ */
+
// Nodes where resource has been probed (key is node ID, not name)
GHashTable *probed_nodes;
// Nodes where resource is allowed to run (key is node ID, not name)
GHashTable *allowed_nodes;
// The source node, if migrate_to completed but migrate_from has not
pcmk_node_t *partial_migration_source;
// The destination node, if migrate_to completed but migrate_from has not
pcmk_node_t *partial_migration_target;
// Source nodes where stop is needed after migrate_from and migrate_to
GList *dangling_migration_sources;
/* Pay special attention to whether you want to use with_this_colocations
* and this_with_colocations directly, which include only colocations
* explicitly involving this resource, or call libpacemaker's
* pcmk__with_this_colocations() and pcmk__this_with_colocations()
* functions, which may return relevant colocations involving the resource's
* ancestors as well.
*/
// Colocations of other resources with this one
GList *with_this_colocations;
// Colocations of this resource with others
GList *this_with_colocations;
GList *location_constraints; // Location constraints for resource
GList *ticket_constraints; // Ticket constraints for resource
const pcmk__rsc_methods_t *fns; // Resource object methods
const pcmk__assignment_methods_t *cmds; // Resource assignment methods
};
const char *pcmk__multiply_active_text(const pcmk_resource_t *rsc);
/*!
* \internal
* \brief Get node where resource is currently active (if any)
*
* \param[in] rsc Resource to check
*
* \return Node that \p rsc is active on, if any, otherwise NULL
*/
static inline pcmk_node_t *
pcmk__current_node(const pcmk_resource_t *rsc)
{
if (rsc == NULL) {
return NULL;
}
return rsc->priv->fns->active_node(rsc, NULL, NULL);
}
#ifdef __cplusplus
}
#endif
#endif // PCMK__CRM_COMMON_RESOURCES_INTERNAL__H
diff --git a/include/crm/common/util.h b/include/crm/common/util.h
index 11388b028f..868ab96d0f 100644
--- a/include/crm/common/util.h
+++ b/include/crm/common/util.h
@@ -1,99 +1,97 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#ifndef PCMK__CRM_COMMON_UTIL__H
#define PCMK__CRM_COMMON_UTIL__H
#include <sys/types.h> // gid_t, mode_t, size_t, time_t, uid_t
#include <stdlib.h>
#include <stdbool.h>
#include <stdint.h> // uint32_t
#include <limits.h>
#include <signal.h>
#include <glib.h>
#include <crm/common/acl.h>
#include <crm/common/actions.h>
#include <crm/common/agents.h>
#include <crm/common/results.h>
#include <crm/common/scores.h>
#include <crm/common/strings.h>
#include <crm/common/nvpair.h>
#ifdef __cplusplus
extern "C" {
#endif
/**
* \file
* \brief Utility functions
* \ingroup core
*/
-/* public node attribute functions (from attrd_client.c) */
+/* public node attribute functions (from attrs.c) */
char *pcmk_promotion_score_name(const char *rsc_id);
/* public Pacemaker Remote functions (from remote.c) */
int crm_default_remote_port(void);
int compare_version(const char *version1, const char *version2);
/*!
* \brief Check whether any of specified flags are set in a flag group
*
* \param[in] flag_group The flag group being examined
* \param[in] flags_to_check Which flags in flag_group should be checked
*
* \return true if \p flags_to_check is nonzero and any of its flags are set in
* \p flag_group, or false otherwise
*/
static inline bool
pcmk_any_flags_set(uint64_t flag_group, uint64_t flags_to_check)
{
return (flag_group & flags_to_check) != 0;
}
/*!
* \brief Check whether all of specified flags are set in a flag group
*
* \param[in] flag_group The flag group being examined
* \param[in] flags_to_check Which flags in flag_group should be checked
*
* \return true if \p flags_to_check is zero or all of its flags are set in
* \p flag_group, or false otherwise
*/
static inline bool
pcmk_all_flags_set(uint64_t flag_group, uint64_t flags_to_check)
{
return (flag_group & flags_to_check) == flags_to_check;
}
/*!
* \brief Convenience alias for pcmk_all_flags_set(), to check single flag
*/
#define pcmk_is_set(g, f) pcmk_all_flags_set((g), (f))
+void pcmk_common_cleanup(void);
char *crm_md5sum(const char *buffer);
-
char *crm_generate_uuid(void);
-
int crm_user_lookup(const char *name, uid_t * uid, gid_t * gid);
int pcmk_daemon_user(uid_t *uid, gid_t *gid);
-
void crm_gnutls_global_init(void);
#ifdef __cplusplus
}
#endif
#if !defined(PCMK_ALLOW_DEPRECATED) || (PCMK_ALLOW_DEPRECATED == 1)
#include <crm/common/util_compat.h>
#endif
#endif
diff --git a/lib/common/mainloop.c b/lib/common/mainloop.c
index a65a5bbf1d..9529d523af 100644
--- a/lib/common/mainloop.c
+++ b/lib/common/mainloop.c
@@ -1,1463 +1,1465 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <errno.h>
#include <sys/wait.h>
#include <crm/crm.h>
#include <crm/common/xml.h>
#include <crm/common/mainloop.h>
#include <crm/common/ipc_internal.h>
#include <qb/qbarray.h>
struct mainloop_child_s {
pid_t pid;
char *desc;
unsigned timerid;
gboolean timeout;
void *privatedata;
enum mainloop_child_flags flags;
/* Called when a process dies */
void (*callback) (mainloop_child_t * p, pid_t pid, int core, int signo, int exitcode);
};
struct trigger_s {
GSource source;
gboolean running;
gboolean trigger;
void *user_data;
guint id;
};
struct mainloop_timer_s {
guint id;
guint period_ms;
bool repeat;
char *name;
GSourceFunc cb;
void *userdata;
};
static gboolean
crm_trigger_prepare(GSource * source, gint * timeout)
{
crm_trigger_t *trig = (crm_trigger_t *) source;
/* cluster-glue's FD and IPC related sources make use of
* g_source_add_poll() but do not set a timeout in their prepare
* functions
*
* This means mainloop's poll() will block until an event for one
* of these sources occurs - any /other/ type of source, such as
* this one or g_idle_*, that doesn't use g_source_add_poll() is
* S-O-L and won't be processed until there is something fd-based
* happens.
*
* Luckily the timeout we can set here affects all sources and
* puts an upper limit on how long poll() can take.
*
* So unconditionally set a small-ish timeout, not too small that
* we're in constant motion, which will act as an upper bound on
* how long the signal handling might be delayed for.
*/
*timeout = 500; /* Timeout in ms */
return trig->trigger;
}
static gboolean
crm_trigger_check(GSource * source)
{
crm_trigger_t *trig = (crm_trigger_t *) source;
return trig->trigger;
}
/*!
* \internal
* \brief GSource dispatch function for crm_trigger_t
*
* \param[in] source crm_trigger_t being dispatched
* \param[in] callback Callback passed at source creation
* \param[in,out] userdata User data passed at source creation
*
* \return G_SOURCE_REMOVE to remove source, G_SOURCE_CONTINUE to keep it
*/
static gboolean
crm_trigger_dispatch(GSource *source, GSourceFunc callback, gpointer userdata)
{
gboolean rc = G_SOURCE_CONTINUE;
crm_trigger_t *trig = (crm_trigger_t *) source;
if (trig->running) {
/* Wait until the existing job is complete before starting the next one */
return G_SOURCE_CONTINUE;
}
trig->trigger = FALSE;
if (callback) {
int callback_rc = callback(trig->user_data);
if (callback_rc < 0) {
crm_trace("Trigger handler %p not yet complete", trig);
trig->running = TRUE;
} else if (callback_rc == 0) {
rc = G_SOURCE_REMOVE;
}
}
return rc;
}
static void
crm_trigger_finalize(GSource * source)
{
crm_trace("Trigger %p destroyed", source);
}
static GSourceFuncs crm_trigger_funcs = {
crm_trigger_prepare,
crm_trigger_check,
crm_trigger_dispatch,
crm_trigger_finalize,
};
static crm_trigger_t *
mainloop_setup_trigger(GSource * source, int priority, int (*dispatch) (gpointer user_data),
gpointer userdata)
{
crm_trigger_t *trigger = NULL;
trigger = (crm_trigger_t *) source;
trigger->id = 0;
trigger->trigger = FALSE;
trigger->user_data = userdata;
if (dispatch) {
g_source_set_callback(source, dispatch, trigger, NULL);
}
g_source_set_priority(source, priority);
g_source_set_can_recurse(source, FALSE);
trigger->id = g_source_attach(source, NULL);
return trigger;
}
void
mainloop_trigger_complete(crm_trigger_t * trig)
{
crm_trace("Trigger handler %p complete", trig);
trig->running = FALSE;
}
/*!
* \brief Create a trigger to be used as a mainloop source
*
* \param[in] priority Relative priority of source (lower number is higher priority)
* \param[in] dispatch Trigger dispatch function (should return 0 to remove the
* trigger from the mainloop, -1 if the trigger should be
* kept but the job is still running and not complete, and
* 1 if the trigger should be kept and the job is complete)
* \param[in] userdata Pointer to pass to \p dispatch
*
* \return Newly allocated mainloop source for trigger
*/
crm_trigger_t *
mainloop_add_trigger(int priority, int (*dispatch) (gpointer user_data),
gpointer userdata)
{
GSource *source = NULL;
pcmk__assert(sizeof(crm_trigger_t) > sizeof(GSource));
source = g_source_new(&crm_trigger_funcs, sizeof(crm_trigger_t));
return mainloop_setup_trigger(source, priority, dispatch, userdata);
}
void
mainloop_set_trigger(crm_trigger_t * source)
{
if(source) {
source->trigger = TRUE;
}
}
gboolean
mainloop_destroy_trigger(crm_trigger_t * source)
{
GSource *gs = NULL;
if(source == NULL) {
return TRUE;
}
gs = (GSource *)source;
g_source_destroy(gs); /* Remove from mainloop, ref_count-- */
g_source_unref(gs); /* The caller no longer carries a reference to source
*
* At this point the source should be free'd,
* unless we're currently processing said
* source, in which case mainloop holds an
* additional reference and it will be free'd
* once our processing completes
*/
return TRUE;
}
// Define a custom glib source for signal handling
// Data structure for custom glib source
typedef struct signal_s {
crm_trigger_t trigger; // trigger that invoked source (must be first)
void (*handler) (int sig); // signal handler
int signal; // signal that was received
} crm_signal_t;
// Table to associate signal handlers with signal numbers
static crm_signal_t *crm_signals[NSIG];
/*!
* \internal
* \brief Dispatch an event from custom glib source for signals
*
* Given an signal event, clear the event trigger and call any registered
* signal handler.
*
* \param[in] source glib source that triggered this dispatch
* \param[in] callback (ignored)
* \param[in] userdata (ignored)
*/
static gboolean
crm_signal_dispatch(GSource *source, GSourceFunc callback, gpointer userdata)
{
crm_signal_t *sig = (crm_signal_t *) source;
if(sig->signal != SIGCHLD) {
crm_notice("Caught '%s' signal " QB_XS " %d (%s handler)",
strsignal(sig->signal), sig->signal,
(sig->handler? "invoking" : "no"));
}
sig->trigger.trigger = FALSE;
if (sig->handler) {
sig->handler(sig->signal);
}
return TRUE;
}
/*!
* \internal
* \brief Handle a signal by setting a trigger for signal source
*
* \param[in] sig Signal number that was received
*
* \note This is the true signal handler for the mainloop signal source, and
* must be async-safe.
*/
static void
mainloop_signal_handler(int sig)
{
if (sig > 0 && sig < NSIG && crm_signals[sig] != NULL) {
mainloop_set_trigger((crm_trigger_t *) crm_signals[sig]);
}
}
// Functions implementing our custom glib source for signal handling
static GSourceFuncs crm_signal_funcs = {
crm_trigger_prepare,
crm_trigger_check,
crm_signal_dispatch,
crm_trigger_finalize,
};
/*!
* \internal
* \brief Set a true signal handler
*
* signal()-like interface to sigaction()
*
* \param[in] sig Signal number to register handler for
* \param[in] dispatch Signal handler
*
* \return The previous value of the signal handler, or SIG_ERR on error
* \note The dispatch function must be async-safe.
*/
sighandler_t
crm_signal_handler(int sig, sighandler_t dispatch)
{
sigset_t mask;
struct sigaction sa;
struct sigaction old;
if (sigemptyset(&mask) < 0) {
crm_err("Could not set handler for signal %d: %s",
sig, pcmk_rc_str(errno));
return SIG_ERR;
}
memset(&sa, 0, sizeof(struct sigaction));
sa.sa_handler = dispatch;
sa.sa_flags = SA_RESTART;
sa.sa_mask = mask;
if (sigaction(sig, &sa, &old) < 0) {
crm_err("Could not set handler for signal %d: %s",
sig, pcmk_rc_str(errno));
return SIG_ERR;
}
return old.sa_handler;
}
static void
mainloop_destroy_signal_entry(int sig)
{
crm_signal_t *tmp = crm_signals[sig];
- crm_signals[sig] = NULL;
-
- crm_trace("Destroying signal %d", sig);
- mainloop_destroy_trigger((crm_trigger_t *) tmp);
+ if (tmp != NULL) {
+ crm_signals[sig] = NULL;
+ crm_trace("Unregistering mainloop handler for signal %d", sig);
+ mainloop_destroy_trigger((crm_trigger_t *) tmp);
+ }
}
/*!
* \internal
* \brief Add a signal handler to a mainloop
*
* \param[in] sig Signal number to handle
* \param[in] dispatch Signal handler function
*
* \note The true signal handler merely sets a mainloop trigger to call this
* dispatch function via the mainloop. Therefore, the dispatch function
* does not need to be async-safe.
*/
gboolean
mainloop_add_signal(int sig, void (*dispatch) (int sig))
{
GSource *source = NULL;
int priority = G_PRIORITY_HIGH - 1;
if (sig == SIGTERM) {
/* TERM is higher priority than other signals,
* signals are higher priority than other ipc.
* Yes, minus: smaller is "higher"
*/
priority--;
}
if (sig >= NSIG || sig < 0) {
crm_err("Signal %d is out of range", sig);
return FALSE;
} else if (crm_signals[sig] != NULL && crm_signals[sig]->handler == dispatch) {
crm_trace("Signal handler for %d is already installed", sig);
return TRUE;
} else if (crm_signals[sig] != NULL) {
crm_err("Different signal handler for %d is already installed", sig);
return FALSE;
}
pcmk__assert(sizeof(crm_signal_t) > sizeof(GSource));
source = g_source_new(&crm_signal_funcs, sizeof(crm_signal_t));
crm_signals[sig] = (crm_signal_t *) mainloop_setup_trigger(source, priority, NULL, NULL);
pcmk__assert(crm_signals[sig] != NULL);
crm_signals[sig]->handler = dispatch;
crm_signals[sig]->signal = sig;
if (crm_signal_handler(sig, mainloop_signal_handler) == SIG_ERR) {
mainloop_destroy_signal_entry(sig);
return FALSE;
}
return TRUE;
}
gboolean
mainloop_destroy_signal(int sig)
{
if (sig >= NSIG || sig < 0) {
crm_err("Signal %d is out of range", sig);
return FALSE;
} else if (crm_signal_handler(sig, NULL) == SIG_ERR) {
crm_perror(LOG_ERR, "Could not uninstall signal handler for signal %d", sig);
return FALSE;
} else if (crm_signals[sig] == NULL) {
return TRUE;
}
mainloop_destroy_signal_entry(sig);
return TRUE;
}
static qb_array_t *gio_map = NULL;
void
mainloop_cleanup(void)
{
- if (gio_map) {
+ if (gio_map != NULL) {
qb_array_free(gio_map);
+ gio_map = NULL;
}
for (int sig = 0; sig < NSIG; ++sig) {
mainloop_destroy_signal_entry(sig);
}
}
/*
* libqb...
*/
struct gio_to_qb_poll {
int32_t is_used;
guint source;
int32_t events;
void *data;
qb_ipcs_dispatch_fn_t fn;
enum qb_loop_priority p;
};
static gboolean
gio_read_socket(GIOChannel * gio, GIOCondition condition, gpointer data)
{
struct gio_to_qb_poll *adaptor = (struct gio_to_qb_poll *)data;
gint fd = g_io_channel_unix_get_fd(gio);
crm_trace("%p.%d %d", data, fd, condition);
/* if this assert get's hit, then there is a race condition between
* when we destroy a fd and when mainloop actually gives it up */
pcmk__assert(adaptor->is_used > 0);
return (adaptor->fn(fd, condition, adaptor->data) == 0);
}
static void
gio_poll_destroy(gpointer data)
{
struct gio_to_qb_poll *adaptor = (struct gio_to_qb_poll *)data;
adaptor->is_used--;
pcmk__assert(adaptor->is_used >= 0);
if (adaptor->is_used == 0) {
crm_trace("Marking adaptor %p unused", adaptor);
adaptor->source = 0;
}
}
/*!
* \internal
* \brief Convert libqb's poll priority into GLib's one
*
* \param[in] prio libqb's poll priority (#QB_LOOP_MED assumed as fallback)
*
* \return best matching GLib's priority
*/
static gint
conv_prio_libqb2glib(enum qb_loop_priority prio)
{
switch (prio) {
case QB_LOOP_LOW: return G_PRIORITY_LOW;
case QB_LOOP_HIGH: return G_PRIORITY_HIGH;
default: return G_PRIORITY_DEFAULT; // QB_LOOP_MED
}
}
/*!
* \internal
* \brief Convert libqb's poll priority to rate limiting spec
*
* \param[in] prio libqb's poll priority (#QB_LOOP_MED assumed as fallback)
*
* \return best matching rate limiting spec
* \note This is the inverse of libqb's qb_ipcs_request_rate_limit().
*/
static enum qb_ipcs_rate_limit
conv_libqb_prio2ratelimit(enum qb_loop_priority prio)
{
switch (prio) {
case QB_LOOP_LOW: return QB_IPCS_RATE_SLOW;
case QB_LOOP_HIGH: return QB_IPCS_RATE_FAST;
default: return QB_IPCS_RATE_NORMAL; // QB_LOOP_MED
}
}
static int32_t
gio_poll_dispatch_update(enum qb_loop_priority p, int32_t fd, int32_t evts,
void *data, qb_ipcs_dispatch_fn_t fn, int32_t add)
{
struct gio_to_qb_poll *adaptor;
GIOChannel *channel;
int32_t res = 0;
res = qb_array_index(gio_map, fd, (void **)&adaptor);
if (res < 0) {
crm_err("Array lookup failed for fd=%d: %d", fd, res);
return res;
}
crm_trace("Adding fd=%d to mainloop as adaptor %p", fd, adaptor);
if (add && adaptor->source) {
crm_err("Adaptor for descriptor %d is still in-use", fd);
return -EEXIST;
}
if (!add && !adaptor->is_used) {
crm_err("Adaptor for descriptor %d is not in-use", fd);
return -ENOENT;
}
/* channel is created with ref_count = 1 */
channel = g_io_channel_unix_new(fd);
if (!channel) {
crm_err("No memory left to add fd=%d", fd);
return -ENOMEM;
}
if (adaptor->source) {
g_source_remove(adaptor->source);
adaptor->source = 0;
}
/* Because unlike the poll() API, glib doesn't tell us about HUPs by default */
evts |= (G_IO_HUP | G_IO_NVAL | G_IO_ERR);
adaptor->fn = fn;
adaptor->events = evts;
adaptor->data = data;
adaptor->p = p;
adaptor->is_used++;
adaptor->source =
g_io_add_watch_full(channel, conv_prio_libqb2glib(p), evts,
gio_read_socket, adaptor, gio_poll_destroy);
/* Now that mainloop now holds a reference to channel,
* thanks to g_io_add_watch_full(), drop ours from g_io_channel_unix_new().
*
* This means that channel will be free'd by:
* g_main_context_dispatch()
* -> g_source_destroy_internal()
* -> g_source_callback_unref()
* shortly after gio_poll_destroy() completes
*/
g_io_channel_unref(channel);
crm_trace("Added to mainloop with gsource id=%d", adaptor->source);
if (adaptor->source > 0) {
return 0;
}
return -EINVAL;
}
static int32_t
gio_poll_dispatch_add(enum qb_loop_priority p, int32_t fd, int32_t evts,
void *data, qb_ipcs_dispatch_fn_t fn)
{
return gio_poll_dispatch_update(p, fd, evts, data, fn, QB_TRUE);
}
static int32_t
gio_poll_dispatch_mod(enum qb_loop_priority p, int32_t fd, int32_t evts,
void *data, qb_ipcs_dispatch_fn_t fn)
{
return gio_poll_dispatch_update(p, fd, evts, data, fn, QB_FALSE);
}
static int32_t
gio_poll_dispatch_del(int32_t fd)
{
struct gio_to_qb_poll *adaptor;
crm_trace("Looking for fd=%d", fd);
if (qb_array_index(gio_map, fd, (void **)&adaptor) == 0) {
if (adaptor->source) {
g_source_remove(adaptor->source);
adaptor->source = 0;
}
}
return 0;
}
struct qb_ipcs_poll_handlers gio_poll_funcs = {
.job_add = NULL,
.dispatch_add = gio_poll_dispatch_add,
.dispatch_mod = gio_poll_dispatch_mod,
.dispatch_del = gio_poll_dispatch_del,
};
static enum qb_ipc_type
pick_ipc_type(enum qb_ipc_type requested)
{
const char *env = pcmk__env_option(PCMK__ENV_IPC_TYPE);
if (env && strcmp("shared-mem", env) == 0) {
return QB_IPC_SHM;
} else if (env && strcmp("socket", env) == 0) {
return QB_IPC_SOCKET;
} else if (env && strcmp("posix", env) == 0) {
return QB_IPC_POSIX_MQ;
} else if (env && strcmp("sysv", env) == 0) {
return QB_IPC_SYSV_MQ;
} else if (requested == QB_IPC_NATIVE) {
/* We prefer shared memory because the server never blocks on
* send. If part of a message fits into the socket, libqb
* needs to block until the remainder can be sent also.
* Otherwise the client will wait forever for the remaining
* bytes.
*/
return QB_IPC_SHM;
}
return requested;
}
qb_ipcs_service_t *
mainloop_add_ipc_server(const char *name, enum qb_ipc_type type,
struct qb_ipcs_service_handlers *callbacks)
{
return mainloop_add_ipc_server_with_prio(name, type, callbacks, QB_LOOP_MED);
}
qb_ipcs_service_t *
mainloop_add_ipc_server_with_prio(const char *name, enum qb_ipc_type type,
struct qb_ipcs_service_handlers *callbacks,
enum qb_loop_priority prio)
{
int rc = 0;
qb_ipcs_service_t *server = NULL;
if (gio_map == NULL) {
gio_map = qb_array_create_2(64, sizeof(struct gio_to_qb_poll), 1);
}
server = qb_ipcs_create(name, 0, pick_ipc_type(type), callbacks);
if (server == NULL) {
crm_err("Could not create %s IPC server: %s (%d)",
name, pcmk_rc_str(errno), errno);
return NULL;
}
if (prio != QB_LOOP_MED) {
qb_ipcs_request_rate_limit(server, conv_libqb_prio2ratelimit(prio));
}
// All clients should use at least PCMK_ipc_buffer as their buffer size
qb_ipcs_enforce_buffer_size(server, crm_ipc_default_buffer_size());
qb_ipcs_poll_handlers_set(server, &gio_poll_funcs);
rc = qb_ipcs_run(server);
if (rc < 0) {
crm_err("Could not start %s IPC server: %s (%d)", name, pcmk_strerror(rc), rc);
return NULL; // qb_ipcs_run() destroys server on failure
}
return server;
}
void
mainloop_del_ipc_server(qb_ipcs_service_t * server)
{
if (server) {
qb_ipcs_destroy(server);
}
}
struct mainloop_io_s {
char *name;
void *userdata;
int fd;
guint source;
crm_ipc_t *ipc;
GIOChannel *channel;
int (*dispatch_fn_ipc) (const char *buffer, ssize_t length, gpointer userdata);
int (*dispatch_fn_io) (gpointer userdata);
void (*destroy_fn) (gpointer userdata);
};
/*!
* \internal
* \brief I/O watch callback function (GIOFunc)
*
* \param[in] gio I/O channel being watched
* \param[in] condition I/O condition satisfied
* \param[in] data User data passed when source was created
*
* \return G_SOURCE_REMOVE to remove source, G_SOURCE_CONTINUE to keep it
*/
static gboolean
mainloop_gio_callback(GIOChannel *gio, GIOCondition condition, gpointer data)
{
gboolean rc = G_SOURCE_CONTINUE;
mainloop_io_t *client = data;
pcmk__assert(client->fd == g_io_channel_unix_get_fd(gio));
if (condition & G_IO_IN) {
if (client->ipc) {
long read_rc = 0L;
int max = 10;
do {
read_rc = crm_ipc_read(client->ipc);
if (read_rc <= 0) {
crm_trace("Could not read IPC message from %s: %s (%ld)",
client->name, pcmk_strerror(read_rc), read_rc);
} else if (client->dispatch_fn_ipc) {
const char *buffer = crm_ipc_buffer(client->ipc);
crm_trace("New %ld-byte IPC message from %s "
"after I/O condition %d",
read_rc, client->name, (int) condition);
if (client->dispatch_fn_ipc(buffer, read_rc, client->userdata) < 0) {
crm_trace("Connection to %s no longer required", client->name);
rc = G_SOURCE_REMOVE;
}
}
} while ((rc == G_SOURCE_CONTINUE) && (read_rc > 0) && --max > 0);
} else {
crm_trace("New I/O event for %s after I/O condition %d",
client->name, (int) condition);
if (client->dispatch_fn_io) {
if (client->dispatch_fn_io(client->userdata) < 0) {
crm_trace("Connection to %s no longer required", client->name);
rc = G_SOURCE_REMOVE;
}
}
}
}
if (client->ipc && !crm_ipc_connected(client->ipc)) {
crm_err("Connection to %s closed " QB_XS " client=%p condition=%d",
client->name, client, condition);
rc = G_SOURCE_REMOVE;
} else if (condition & (G_IO_HUP | G_IO_NVAL | G_IO_ERR)) {
crm_trace("The connection %s[%p] has been closed (I/O condition=%d)",
client->name, client, condition);
rc = G_SOURCE_REMOVE;
} else if ((condition & G_IO_IN) == 0) {
/*
#define GLIB_SYSDEF_POLLIN =1
#define GLIB_SYSDEF_POLLPRI =2
#define GLIB_SYSDEF_POLLOUT =4
#define GLIB_SYSDEF_POLLERR =8
#define GLIB_SYSDEF_POLLHUP =16
#define GLIB_SYSDEF_POLLNVAL =32
typedef enum
{
G_IO_IN GLIB_SYSDEF_POLLIN,
G_IO_OUT GLIB_SYSDEF_POLLOUT,
G_IO_PRI GLIB_SYSDEF_POLLPRI,
G_IO_ERR GLIB_SYSDEF_POLLERR,
G_IO_HUP GLIB_SYSDEF_POLLHUP,
G_IO_NVAL GLIB_SYSDEF_POLLNVAL
} GIOCondition;
A bitwise combination representing a condition to watch for on an event source.
G_IO_IN There is data to read.
G_IO_OUT Data can be written (without blocking).
G_IO_PRI There is urgent data to read.
G_IO_ERR Error condition.
G_IO_HUP Hung up (the connection has been broken, usually for pipes and sockets).
G_IO_NVAL Invalid request. The file descriptor is not open.
*/
crm_err("Strange condition: %d", condition);
}
/* G_SOURCE_REMOVE results in mainloop_gio_destroy() being called
* just before the source is removed from mainloop
*/
return rc;
}
static void
mainloop_gio_destroy(gpointer c)
{
mainloop_io_t *client = c;
char *c_name = strdup(client->name);
/* client->source is valid but about to be destroyed (ref_count == 0) in gmain.c
* client->channel will still have ref_count > 0... should be == 1
*/
crm_trace("Destroying client %s[%p]", c_name, c);
if (client->ipc) {
crm_ipc_close(client->ipc);
}
if (client->destroy_fn) {
void (*destroy_fn) (gpointer userdata) = client->destroy_fn;
client->destroy_fn = NULL;
destroy_fn(client->userdata);
}
if (client->ipc) {
crm_ipc_t *ipc = client->ipc;
client->ipc = NULL;
crm_ipc_destroy(ipc);
}
crm_trace("Destroyed client %s[%p]", c_name, c);
free(client->name); client->name = NULL;
free(client);
free(c_name);
}
/*!
* \brief Connect to IPC and add it as a main loop source
*
* \param[in,out] ipc IPC connection to add
* \param[in] priority Event source priority to use for connection
* \param[in] userdata Data to register with callbacks
* \param[in] callbacks Dispatch and destroy callbacks for connection
* \param[out] source Newly allocated event source
*
* \return Standard Pacemaker return code
*
* \note On failure, the caller is still responsible for ipc. On success, the
* caller should call mainloop_del_ipc_client() when source is no longer
* needed, which will lead to the disconnection of the IPC later in the
* main loop if it is connected. However the IPC disconnects,
* mainloop_gio_destroy() will free ipc and source after calling the
* destroy callback.
*/
int
pcmk__add_mainloop_ipc(crm_ipc_t *ipc, int priority, void *userdata,
const struct ipc_client_callbacks *callbacks,
mainloop_io_t **source)
{
int rc = pcmk_rc_ok;
int fd = -1;
const char *ipc_name = NULL;
CRM_CHECK((ipc != NULL) && (callbacks != NULL), return EINVAL);
ipc_name = pcmk__s(crm_ipc_name(ipc), "Pacemaker");
rc = pcmk__connect_generic_ipc(ipc);
if (rc != pcmk_rc_ok) {
crm_debug("Connection to %s failed: %s", ipc_name, pcmk_rc_str(rc));
return rc;
}
rc = pcmk__ipc_fd(ipc, &fd);
if (rc != pcmk_rc_ok) {
crm_debug("Could not obtain file descriptor for %s IPC: %s",
ipc_name, pcmk_rc_str(rc));
crm_ipc_close(ipc);
return rc;
}
*source = mainloop_add_fd(ipc_name, priority, fd, userdata, NULL);
if (*source == NULL) {
rc = errno;
crm_ipc_close(ipc);
return rc;
}
(*source)->ipc = ipc;
(*source)->destroy_fn = callbacks->destroy;
(*source)->dispatch_fn_ipc = callbacks->dispatch;
return pcmk_rc_ok;
}
/*!
* \brief Get period for mainloop timer
*
* \param[in] timer Timer
*
* \return Period in ms
*/
guint
pcmk__mainloop_timer_get_period(const mainloop_timer_t *timer)
{
if (timer) {
return timer->period_ms;
}
return 0;
}
mainloop_io_t *
mainloop_add_ipc_client(const char *name, int priority, size_t max_size,
void *userdata, struct ipc_client_callbacks *callbacks)
{
crm_ipc_t *ipc = crm_ipc_new(name, max_size);
mainloop_io_t *source = NULL;
int rc = pcmk__add_mainloop_ipc(ipc, priority, userdata, callbacks,
&source);
if (rc != pcmk_rc_ok) {
if (crm_log_level == LOG_STDOUT) {
fprintf(stderr, "Connection to %s failed: %s",
name, pcmk_rc_str(rc));
}
crm_ipc_destroy(ipc);
if (rc > 0) {
errno = rc;
} else {
errno = ENOTCONN;
}
return NULL;
}
return source;
}
void
mainloop_del_ipc_client(mainloop_io_t * client)
{
mainloop_del_fd(client);
}
crm_ipc_t *
mainloop_get_ipc_client(mainloop_io_t * client)
{
if (client) {
return client->ipc;
}
return NULL;
}
mainloop_io_t *
mainloop_add_fd(const char *name, int priority, int fd, void *userdata,
struct mainloop_fd_callbacks * callbacks)
{
mainloop_io_t *client = NULL;
if (fd >= 0) {
client = calloc(1, sizeof(mainloop_io_t));
if (client == NULL) {
return NULL;
}
client->name = strdup(name);
client->userdata = userdata;
if (callbacks) {
client->destroy_fn = callbacks->destroy;
client->dispatch_fn_io = callbacks->dispatch;
}
client->fd = fd;
client->channel = g_io_channel_unix_new(fd);
client->source =
g_io_add_watch_full(client->channel, priority,
(G_IO_IN | G_IO_HUP | G_IO_NVAL | G_IO_ERR), mainloop_gio_callback,
client, mainloop_gio_destroy);
/* Now that mainloop now holds a reference to channel,
* thanks to g_io_add_watch_full(), drop ours from g_io_channel_unix_new().
*
* This means that channel will be free'd by:
* g_main_context_dispatch() or g_source_remove()
* -> g_source_destroy_internal()
* -> g_source_callback_unref()
* shortly after mainloop_gio_destroy() completes
*/
g_io_channel_unref(client->channel);
crm_trace("Added connection %d for %s[%p].%d", client->source, client->name, client, fd);
} else {
errno = EINVAL;
}
return client;
}
void
mainloop_del_fd(mainloop_io_t * client)
{
if (client != NULL) {
crm_trace("Removing client %s[%p]", client->name, client);
if (client->source) {
/* Results in mainloop_gio_destroy() being called just
* before the source is removed from mainloop
*/
g_source_remove(client->source);
}
}
}
static GList *child_list = NULL;
pid_t
mainloop_child_pid(mainloop_child_t * child)
{
return child->pid;
}
const char *
mainloop_child_name(mainloop_child_t * child)
{
return child->desc;
}
int
mainloop_child_timeout(mainloop_child_t * child)
{
return child->timeout;
}
void *
mainloop_child_userdata(mainloop_child_t * child)
{
return child->privatedata;
}
void
mainloop_clear_child_userdata(mainloop_child_t * child)
{
child->privatedata = NULL;
}
/* good function name */
static void
child_free(mainloop_child_t *child)
{
if (child->timerid != 0) {
crm_trace("Removing timer %d", child->timerid);
g_source_remove(child->timerid);
child->timerid = 0;
}
free(child->desc);
free(child);
}
/* terrible function name */
static int
child_kill_helper(mainloop_child_t *child)
{
int rc;
if (child->flags & mainloop_leave_pid_group) {
crm_debug("Kill pid %d only. leave group intact.", child->pid);
rc = kill(child->pid, SIGKILL);
} else {
crm_debug("Kill pid %d's group", child->pid);
rc = kill(-child->pid, SIGKILL);
}
if (rc < 0) {
if (errno != ESRCH) {
crm_perror(LOG_ERR, "kill(%d, KILL) failed", child->pid);
}
return -errno;
}
return 0;
}
static gboolean
child_timeout_callback(gpointer p)
{
mainloop_child_t *child = p;
int rc = 0;
child->timerid = 0;
if (child->timeout) {
crm_warn("%s process (PID %d) will not die!", child->desc, (int)child->pid);
return FALSE;
}
rc = child_kill_helper(child);
if (rc == -ESRCH) {
/* Nothing left to do. pid doesn't exist */
return FALSE;
}
child->timeout = TRUE;
crm_debug("%s process (PID %d) timed out", child->desc, (int)child->pid);
child->timerid = pcmk__create_timer(5000, child_timeout_callback, child);
return FALSE;
}
static bool
child_waitpid(mainloop_child_t *child, int flags)
{
int rc = 0;
int core = 0;
int signo = 0;
int status = 0;
int exitcode = 0;
bool callback_needed = true;
rc = waitpid(child->pid, &status, flags);
if (rc == 0) { // WNOHANG in flags, and child status is not available
crm_trace("Child process %d (%s) still active",
child->pid, child->desc);
callback_needed = false;
} else if (rc != child->pid) {
/* According to POSIX, possible conditions:
* - child->pid was non-positive (process group or any child),
* and rc is specific child
* - errno ECHILD (pid does not exist or is not child)
* - errno EINVAL (invalid flags)
* - errno EINTR (caller interrupted by signal)
*
* @TODO Handle these cases more specifically.
*/
signo = SIGCHLD;
exitcode = 1;
crm_notice("Wait for child process %d (%s) interrupted: %s",
child->pid, child->desc, pcmk_rc_str(errno));
} else if (WIFEXITED(status)) {
exitcode = WEXITSTATUS(status);
crm_trace("Child process %d (%s) exited with status %d",
child->pid, child->desc, exitcode);
} else if (WIFSIGNALED(status)) {
signo = WTERMSIG(status);
crm_trace("Child process %d (%s) exited with signal %d (%s)",
child->pid, child->desc, signo, strsignal(signo));
#ifdef WCOREDUMP // AIX, SunOS, maybe others
} else if (WCOREDUMP(status)) {
core = 1;
crm_err("Child process %d (%s) dumped core",
child->pid, child->desc);
#endif
} else { // flags must contain WUNTRACED and/or WCONTINUED to reach this
crm_trace("Child process %d (%s) stopped or continued",
child->pid, child->desc);
callback_needed = false;
}
if (callback_needed && child->callback) {
child->callback(child, child->pid, core, signo, exitcode);
}
return callback_needed;
}
static void
child_death_dispatch(int signal)
{
for (GList *iter = child_list; iter; ) {
GList *saved = iter;
mainloop_child_t *child = iter->data;
iter = iter->next;
if (child_waitpid(child, WNOHANG)) {
crm_trace("Removing completed process %d from child list",
child->pid);
child_list = g_list_remove_link(child_list, saved);
g_list_free(saved);
child_free(child);
}
}
}
static gboolean
child_signal_init(gpointer p)
{
crm_trace("Installed SIGCHLD handler");
/* Do NOT use g_child_watch_add() and friends, they rely on pthreads */
mainloop_add_signal(SIGCHLD, child_death_dispatch);
/* In case they terminated before the signal handler was installed */
child_death_dispatch(SIGCHLD);
return FALSE;
}
gboolean
mainloop_child_kill(pid_t pid)
{
GList *iter;
mainloop_child_t *child = NULL;
mainloop_child_t *match = NULL;
/* It is impossible to block SIGKILL, this allows us to
* call waitpid without WNOHANG flag.*/
int waitflags = 0, rc = 0;
for (iter = child_list; iter != NULL && match == NULL; iter = iter->next) {
child = iter->data;
if (pid == child->pid) {
match = child;
}
}
if (match == NULL) {
return FALSE;
}
rc = child_kill_helper(match);
if(rc == -ESRCH) {
/* It's gone, but hasn't shown up in waitpid() yet. Wait until we get
* SIGCHLD and let handler clean it up as normal (so we get the correct
* return code/status). The blocking alternative would be to call
* child_waitpid(match, 0).
*/
crm_trace("Waiting for signal that child process %d completed",
match->pid);
return TRUE;
} else if(rc != 0) {
/* If KILL for some other reason set the WNOHANG flag since we
* can't be certain what happened.
*/
waitflags = WNOHANG;
}
if (!child_waitpid(match, waitflags)) {
/* not much we can do if this occurs */
return FALSE;
}
child_list = g_list_remove(child_list, match);
child_free(match);
return TRUE;
}
/* Create/Log a new tracked process
* To track a process group, use -pid
*
* @TODO Using a non-positive pid (i.e. any child, or process group) would
* likely not be useful since we will free the child after the first
* completed process.
*/
void
mainloop_child_add_with_flags(pid_t pid, int timeout, const char *desc, void *privatedata, enum mainloop_child_flags flags,
void (*callback) (mainloop_child_t * p, pid_t pid, int core, int signo, int exitcode))
{
static bool need_init = TRUE;
mainloop_child_t *child = pcmk__assert_alloc(1, sizeof(mainloop_child_t));
child->pid = pid;
child->timerid = 0;
child->timeout = FALSE;
child->privatedata = privatedata;
child->callback = callback;
child->flags = flags;
child->desc = pcmk__str_copy(desc);
if (timeout) {
child->timerid = pcmk__create_timer(timeout, child_timeout_callback, child);
}
child_list = g_list_append(child_list, child);
if(need_init) {
need_init = FALSE;
/* SIGCHLD processing has to be invoked from mainloop.
* We do not want it to be possible to both add a child pid
* to mainloop, and have the pid's exit callback invoked within
* the same callstack. */
pcmk__create_timer(1, child_signal_init, NULL);
}
}
void
mainloop_child_add(pid_t pid, int timeout, const char *desc, void *privatedata,
void (*callback) (mainloop_child_t * p, pid_t pid, int core, int signo, int exitcode))
{
mainloop_child_add_with_flags(pid, timeout, desc, privatedata, 0, callback);
}
static gboolean
mainloop_timer_cb(gpointer user_data)
{
int id = 0;
bool repeat = FALSE;
struct mainloop_timer_s *t = user_data;
pcmk__assert(t != NULL);
id = t->id;
t->id = 0; /* Ensure it's unset during callbacks so that
* mainloop_timer_running() works as expected
*/
if(t->cb) {
crm_trace("Invoking callbacks for timer %s", t->name);
repeat = t->repeat;
if(t->cb(t->userdata) == FALSE) {
crm_trace("Timer %s complete", t->name);
repeat = FALSE;
}
}
if(repeat) {
/* Restore if repeating */
t->id = id;
}
return repeat;
}
bool
mainloop_timer_running(mainloop_timer_t *t)
{
if(t && t->id != 0) {
return TRUE;
}
return FALSE;
}
void
mainloop_timer_start(mainloop_timer_t *t)
{
mainloop_timer_stop(t);
if(t && t->period_ms > 0) {
crm_trace("Starting timer %s", t->name);
t->id = pcmk__create_timer(t->period_ms, mainloop_timer_cb, t);
}
}
void
mainloop_timer_stop(mainloop_timer_t *t)
{
if(t && t->id != 0) {
crm_trace("Stopping timer %s", t->name);
g_source_remove(t->id);
t->id = 0;
}
}
guint
mainloop_timer_set_period(mainloop_timer_t *t, guint period_ms)
{
guint last = 0;
if(t) {
last = t->period_ms;
t->period_ms = period_ms;
}
if(t && t->id != 0 && last != t->period_ms) {
mainloop_timer_start(t);
}
return last;
}
mainloop_timer_t *
mainloop_timer_add(const char *name, guint period_ms, bool repeat, GSourceFunc cb, void *userdata)
{
mainloop_timer_t *t = pcmk__assert_alloc(1, sizeof(mainloop_timer_t));
if (name != NULL) {
t->name = crm_strdup_printf("%s-%u-%d", name, period_ms, repeat);
} else {
t->name = crm_strdup_printf("%p-%u-%d", t, period_ms, repeat);
}
t->id = 0;
t->period_ms = period_ms;
t->repeat = repeat;
t->cb = cb;
t->userdata = userdata;
crm_trace("Created timer %s with %p %p", t->name, userdata, t->userdata);
return t;
}
void
mainloop_timer_del(mainloop_timer_t *t)
{
if(t) {
crm_trace("Destroying timer %s", t->name);
mainloop_timer_stop(t);
free(t->name);
free(t);
}
}
/*
* Helpers to make sure certain events aren't lost at shutdown
*/
static gboolean
drain_timeout_cb(gpointer user_data)
{
bool *timeout_popped = (bool*) user_data;
*timeout_popped = TRUE;
return FALSE;
}
/*!
* \brief Drain some remaining main loop events then quit it
*
* \param[in,out] mloop Main loop to drain and quit
* \param[in] n Drain up to this many pending events
*/
void
pcmk_quit_main_loop(GMainLoop *mloop, unsigned int n)
{
if ((mloop != NULL) && g_main_loop_is_running(mloop)) {
GMainContext *ctx = g_main_loop_get_context(mloop);
/* Drain up to n events in case some memory clean-up is pending
* (helpful to reduce noise in valgrind output).
*/
for (int i = 0; (i < n) && g_main_context_pending(ctx); ++i) {
g_main_context_dispatch(ctx);
}
g_main_loop_quit(mloop);
}
}
/*!
* \brief Process main loop events while a certain condition is met
*
* \param[in,out] mloop Main loop to process
* \param[in] timer_ms Don't process longer than this amount of time
* \param[in] check Function that returns true if events should be
* processed
*
* \note This function is intended to be called at shutdown if certain important
* events should not be missed. The caller would likely quit the main loop
* or exit after calling this function. The check() function will be
* passed the remaining timeout in milliseconds.
*/
void
pcmk_drain_main_loop(GMainLoop *mloop, guint timer_ms, bool (*check)(guint))
{
bool timeout_popped = FALSE;
guint timer = 0;
GMainContext *ctx = NULL;
CRM_CHECK(mloop && check, return);
ctx = g_main_loop_get_context(mloop);
if (ctx) {
time_t start_time = time(NULL);
timer = pcmk__create_timer(timer_ms, drain_timeout_cb, &timeout_popped);
while (!timeout_popped
&& check(timer_ms - (time(NULL) - start_time) * 1000)) {
g_main_context_iteration(ctx, TRUE);
}
}
if (!timeout_popped && (timer > 0)) {
g_source_remove(timer);
}
}
diff --git a/lib/common/nodes.c b/lib/common/nodes.c
index 0ef472bb38..fed3159464 100644
--- a/lib/common/nodes.c
+++ b/lib/common/nodes.c
@@ -1,192 +1,212 @@
/*
* Copyright 2022-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <libxml/tree.h> // xmlNode
#include <crm/common/nvpair.h>
+/*!
+ * \internal
+ * \brief Free a copy of a node object
+ *
+ * \param[in] data Node copy (created by pe__copy_node()) to free
+ */
+void
+pcmk__free_node_copy(void *data)
+{
+ if (data != NULL) {
+ pcmk_node_t *node = data;
+
+ if (node->assign != NULL) {
+ // This is the only member allocated separately for a node copy
+ free(node->assign);
+ }
+ free(node);
+ }
+}
+
/*!
* \internal
* \brief Check whether a node is online
*
* \param[in] node Node to check
*
* \return true if \p node is online, otherwise false
*/
bool
pcmk_node_is_online(const pcmk_node_t *node)
{
return (node != NULL) && node->details->online;
}
/*!
* \internal
* \brief Check whether a node is pending
*
* Check whether a node is pending. A node is pending if it is a member of the
* cluster but not the controller group, which means it is in the process of
* either joining or leaving the cluster.
*
* \param[in] node Node to check
*
* \return true if \p node is pending, otherwise false
*/
bool
pcmk_node_is_pending(const pcmk_node_t *node)
{
return (node != NULL) && node->details->pending;
}
/*!
* \internal
* \brief Check whether a node is clean
*
* Check whether a node is clean. A node is clean if it is a cluster node or
* remote node that has been seen by the cluster at least once, or the
* startup-fencing cluster option is false; and the node, and its host if a
* guest or bundle node, are not scheduled to be fenced.
*
* \param[in] node Node to check
*
* \return true if \p node is clean, otherwise false
*/
bool
pcmk_node_is_clean(const pcmk_node_t *node)
{
return (node != NULL) && !(node->details->unclean);
}
/*!
* \internal
* \brief Check whether a node is shutting down
*
* \param[in] node Node to check
*
* \return true if \p node is shutting down, otherwise false
*/
bool
pcmk_node_is_shutting_down(const pcmk_node_t *node)
{
return (node != NULL) && node->details->shutdown;
}
/*!
* \internal
* \brief Check whether a node is in maintenance mode
*
* \param[in] node Node to check
*
* \return true if \p node is in maintenance mode, otherwise false
*/
bool
pcmk_node_is_in_maintenance(const pcmk_node_t *node)
{
return (node != NULL) && node->details->maintenance;
}
/*!
* \internal
* \brief Call a function for each resource active on a node
*
* Call a caller-supplied function with a caller-supplied argument for each
* resource that is active on a given node. If the function returns false, this
* function will return immediately without processing any remaining resources.
*
* \param[in] node Node to check
*
* \return Result of last call of \p fn (or false if none)
*/
bool
pcmk_foreach_active_resource(pcmk_node_t *node,
bool (*fn)(pcmk_resource_t *, void *),
void *user_data)
{
bool result = false;
if ((node != NULL) && (fn != NULL)) {
for (GList *item = node->details->running_rsc; item != NULL;
item = item->next) {
result = fn((pcmk_resource_t *) item->data, user_data);
if (!result) {
break;
}
}
}
return result;
}
void
pcmk__xe_add_node(xmlNode *xml, const char *node, int nodeid)
{
pcmk__assert(xml != NULL);
if (node != NULL) {
crm_xml_add(xml, PCMK__XA_ATTR_HOST, node);
}
if (nodeid > 0) {
crm_xml_add_int(xml, PCMK__XA_ATTR_HOST_ID, nodeid);
}
}
/*!
* \internal
* \brief Find a node by name in a list of nodes
*
* \param[in] nodes List of nodes (as pcmk_node_t*)
* \param[in] node_name Name of node to find
*
* \return Node from \p nodes that matches \p node_name if any, otherwise NULL
*/
pcmk_node_t *
pcmk__find_node_in_list(const GList *nodes, const char *node_name)
{
if (node_name != NULL) {
for (const GList *iter = nodes; iter != NULL; iter = iter->next) {
pcmk_node_t *node = (pcmk_node_t *) iter->data;
if (pcmk__str_eq(node->priv->name, node_name, pcmk__str_casei)) {
return node;
}
}
}
return NULL;
}
#define XP_SHUTDOWN "//" PCMK__XE_NODE_STATE "[@" PCMK_XA_UNAME "='%s']/" \
PCMK__XE_TRANSIENT_ATTRIBUTES "/" PCMK_XE_INSTANCE_ATTRIBUTES "/" \
PCMK_XE_NVPAIR "[@" PCMK_XA_NAME "='" PCMK__NODE_ATTR_SHUTDOWN "']"
/*!
* \brief Get value of a node's shutdown attribute from CIB, if present
*
* \param[in] cib CIB to check
* \param[in] node Name of node to check
*
* \return Value of shutdown attribute for \p node in \p cib if any,
* otherwise NULL
* \note The return value is a pointer into \p cib and so is valid only for the
* lifetime of that object.
*/
const char *
pcmk_cib_node_shutdown(xmlNode *cib, const char *node)
{
if ((cib != NULL) && (node != NULL)) {
char *xpath = crm_strdup_printf(XP_SHUTDOWN, node);
xmlNode *match = get_xpath_object(xpath, cib, LOG_TRACE);
free(xpath);
if (match != NULL) {
return crm_element_value(match, PCMK_XA_VALUE);
}
}
return NULL;
}
diff --git a/lib/common/results.c b/lib/common/results.c
index bad074b6fd..507280492c 100644
--- a/lib/common/results.c
+++ b/lib/common/results.c
@@ -1,1203 +1,1195 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <bzlib.h>
#include <errno.h>
#include <netdb.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <qb/qbdefs.h>
#include <crm/common/mainloop.h>
#include <crm/common/xml.h>
G_DEFINE_QUARK(pcmk-rc-error-quark, pcmk__rc_error)
G_DEFINE_QUARK(pcmk-exitc-error-quark, pcmk__exitc_error)
// General (all result code types)
/*!
* \brief Get the name and description of a given result code
*
* A result code can be interpreted as a member of any one of several families.
*
* \param[in] code The result code to look up
* \param[in] type How \p code should be interpreted
* \param[out] name Where to store the result code's name
* \param[out] desc Where to store the result code's description
*
* \return Standard Pacemaker return code
*/
int
pcmk_result_get_strings(int code, enum pcmk_result_type type, const char **name,
const char **desc)
{
const char *code_name = NULL;
const char *code_desc = NULL;
switch (type) {
case pcmk_result_legacy:
code_name = pcmk_errorname(code);
code_desc = pcmk_strerror(code);
break;
case pcmk_result_rc:
code_name = pcmk_rc_name(code);
code_desc = pcmk_rc_str(code);
break;
case pcmk_result_exitcode:
code_name = crm_exit_name(code);
code_desc = crm_exit_str((crm_exit_t) code);
break;
default:
return pcmk_rc_undetermined;
}
if (name != NULL) {
*name = code_name;
}
if (desc != NULL) {
*desc = code_desc;
}
return pcmk_rc_ok;
}
/*!
* \internal
* \brief Get the lower and upper bounds of a result code family
*
* \param[in] type Type of result code
* \param[out] lower Where to store the lower bound
* \param[out] upper Where to store the upper bound
*
* \return Standard Pacemaker return code
*
* \note There is no true upper bound on standard Pacemaker return codes or
* legacy return codes. All system \p errno values are valid members of
* these result code families, and there is no global upper limit nor a
* constant by which to refer to the highest \p errno value on a given
* system.
*/
int
pcmk__result_bounds(enum pcmk_result_type type, int *lower, int *upper)
{
pcmk__assert((lower != NULL) && (upper != NULL));
switch (type) {
case pcmk_result_legacy:
*lower = pcmk_ok;
*upper = 256; // should be enough for almost any system error code
break;
case pcmk_result_rc:
*lower = pcmk_rc_error - pcmk__n_rc + 1;
*upper = 256;
break;
case pcmk_result_exitcode:
*lower = CRM_EX_OK;
*upper = CRM_EX_MAX;
break;
default:
*lower = 0;
*upper = -1;
return pcmk_rc_undetermined;
}
return pcmk_rc_ok;
}
/*!
* \internal
* \brief Log a failed assertion
*
* \param[in] file File making the assertion
* \param[in] function Function making the assertion
* \param[in] line Line of file making the assertion
* \param[in] assert_condition String representation of assertion
*/
static void
log_assertion_as(const char *file, const char *function, int line,
const char *assert_condition)
{
if (!pcmk__is_daemon) {
crm_enable_stderr(TRUE); // Make sure command-line user sees message
}
crm_err("%s: Triggered fatal assertion at %s:%d : %s",
function, file, line, assert_condition);
}
/* coverity[+kill] */
/*!
* \internal
* \brief Log a failed assertion and abort
*
* \param[in] file File making the assertion
* \param[in] function Function making the assertion
* \param[in] line Line of file making the assertion
* \param[in] assert_condition String representation of assertion
*
* \note This does not return
*/
_Noreturn void
pcmk__abort_as(const char *file, const char *function, int line,
const char *assert_condition)
{
log_assertion_as(file, function, line, assert_condition);
abort();
}
/* coverity[+kill] */
/*!
* \internal
* \brief Handle a failed assertion
*
* When called by a daemon, fork a child that aborts (to dump core), otherwise
* abort the current process.
*
* \param[in] file File making the assertion
* \param[in] function Function making the assertion
* \param[in] line Line of file making the assertion
* \param[in] assert_condition String representation of assertion
*/
static void
fail_assert_as(const char *file, const char *function, int line,
const char *assert_condition)
{
int status = 0;
pid_t pid = 0;
if (!pcmk__is_daemon) {
pcmk__abort_as(file, function, line, assert_condition); // No return
}
pid = fork();
switch (pid) {
case -1: // Fork failed
crm_warn("%s: Cannot dump core for non-fatal assertion at %s:%d "
": %s", function, file, line, assert_condition);
break;
case 0: // Child process: just abort to dump core
abort();
break;
default: // Parent process: wait for child
crm_err("%s: Forked child [%d] to record non-fatal assertion at "
"%s:%d : %s", function, pid, file, line, assert_condition);
crm_write_blackbox(SIGTRAP, NULL);
do {
if (waitpid(pid, &status, 0) == pid) {
return; // Child finished dumping core
}
} while (errno == EINTR);
if (errno == ECHILD) {
// crm_mon ignores SIGCHLD
crm_trace("Cannot wait on forked child [%d] "
"(SIGCHLD is probably ignored)", pid);
} else {
crm_err("Cannot wait on forked child [%d]: %s",
pid, pcmk_rc_str(errno));
}
break;
}
}
/* coverity[+kill] */
void
crm_abort(const char *file, const char *function, int line,
const char *assert_condition, gboolean do_core, gboolean do_fork)
{
if (!do_fork) {
pcmk__abort_as(file, function, line, assert_condition); // No return
} else if (do_core) {
fail_assert_as(file, function, line, assert_condition);
} else {
log_assertion_as(file, function, line, assert_condition);
}
}
// @COMPAT Legacy function return codes
//! \deprecated Use standard return codes and pcmk_rc_name() instead
const char *
pcmk_errorname(int rc)
{
rc = abs(rc);
switch (rc) {
case pcmk_err_generic: return "pcmk_err_generic";
case pcmk_err_no_quorum: return "pcmk_err_no_quorum";
case pcmk_err_schema_validation: return "pcmk_err_schema_validation";
case pcmk_err_transform_failed: return "pcmk_err_transform_failed";
case pcmk_err_old_data: return "pcmk_err_old_data";
case pcmk_err_diff_failed: return "pcmk_err_diff_failed";
case pcmk_err_diff_resync: return "pcmk_err_diff_resync";
case pcmk_err_cib_modified: return "pcmk_err_cib_modified";
case pcmk_err_cib_backup: return "pcmk_err_cib_backup";
case pcmk_err_cib_save: return "pcmk_err_cib_save";
case pcmk_err_cib_corrupt: return "pcmk_err_cib_corrupt";
case pcmk_err_multiple: return "pcmk_err_multiple";
case pcmk_err_node_unknown: return "pcmk_err_node_unknown";
case pcmk_err_already: return "pcmk_err_already";
case pcmk_err_bad_nvpair: return "pcmk_err_bad_nvpair";
case pcmk_err_unknown_format: return "pcmk_err_unknown_format";
default: return pcmk_rc_name(rc); // system errno
}
}
//! \deprecated Use standard return codes and pcmk_rc_str() instead
const char *
pcmk_strerror(int rc)
{
return pcmk_rc_str(pcmk_legacy2rc(rc));
}
// Standard Pacemaker API return codes
/* This array is used only for nonzero values of pcmk_rc_e. Its values must be
* kept in the exact reverse order of the enum value numbering (i.e. add new
* values to the end of the array).
*/
static const struct pcmk__rc_info {
const char *name;
const char *desc;
int legacy_rc;
} pcmk__rcs[] = {
{ "pcmk_rc_error",
"Error",
-pcmk_err_generic,
},
{ "pcmk_rc_unknown_format",
"Unknown output format",
-pcmk_err_unknown_format,
},
{ "pcmk_rc_bad_nvpair",
"Bad name/value pair given",
-pcmk_err_bad_nvpair,
},
{ "pcmk_rc_already",
"Already in requested state",
-pcmk_err_already,
},
{ "pcmk_rc_node_unknown",
"Node not found",
-pcmk_err_node_unknown,
},
{ "pcmk_rc_multiple",
"Resource active on multiple nodes",
-pcmk_err_multiple,
},
{ "pcmk_rc_cib_corrupt",
"Could not parse on-disk configuration",
-pcmk_err_cib_corrupt,
},
{ "pcmk_rc_cib_save",
"Could not save new configuration to disk",
-pcmk_err_cib_save,
},
{ "pcmk_rc_cib_backup",
"Could not archive previous configuration",
-pcmk_err_cib_backup,
},
{ "pcmk_rc_cib_modified",
"On-disk configuration was manually modified",
-pcmk_err_cib_modified,
},
{ "pcmk_rc_diff_resync",
"Application of update diff failed, requesting full refresh",
-pcmk_err_diff_resync,
},
{ "pcmk_rc_diff_failed",
"Application of update diff failed",
-pcmk_err_diff_failed,
},
{ "pcmk_rc_old_data",
"Update was older than existing configuration",
-pcmk_err_old_data,
},
{ "pcmk_rc_transform_failed",
"Schema transform failed",
-pcmk_err_transform_failed,
},
{ "pcmk_rc_schema_unchanged",
"Schema is already the latest available",
-pcmk_err_schema_unchanged,
},
{ "pcmk_rc_schema_validation",
"Update does not conform to the configured schema",
-pcmk_err_schema_validation,
},
{ "pcmk_rc_no_quorum",
"Operation requires quorum",
-pcmk_err_no_quorum,
},
{ "pcmk_rc_ipc_unauthorized",
"IPC server is blocked by unauthorized process",
-pcmk_err_generic,
},
{ "pcmk_rc_ipc_unresponsive",
"IPC server is unresponsive",
-pcmk_err_generic,
},
{ "pcmk_rc_ipc_pid_only",
"IPC server process is active but not accepting connections",
-pcmk_err_generic,
},
{ "pcmk_rc_op_unsatisfied",
"Not applicable under current conditions",
-pcmk_err_generic,
},
{ "pcmk_rc_undetermined",
"Result undetermined",
-pcmk_err_generic,
},
{ "pcmk_rc_before_range",
"Result occurs before given range",
-pcmk_err_generic,
},
{ "pcmk_rc_within_range",
"Result occurs within given range",
-pcmk_err_generic,
},
{ "pcmk_rc_after_range",
"Result occurs after given range",
-pcmk_err_generic,
},
{ "pcmk_rc_no_output",
"Output message produced no output",
-pcmk_err_generic,
},
{ "pcmk_rc_no_input",
"Input file not available",
-pcmk_err_generic,
},
{ "pcmk_rc_underflow",
"Value too small to be stored in data type",
-pcmk_err_generic,
},
{ "pcmk_rc_dot_error",
"Error writing dot(1) file",
-pcmk_err_generic,
},
{ "pcmk_rc_graph_error",
"Error writing graph file",
-pcmk_err_generic,
},
{ "pcmk_rc_invalid_transition",
"Cluster simulation produced invalid transition",
-pcmk_err_generic,
},
{ "pcmk_rc_unpack_error",
"Unable to parse CIB XML",
-pcmk_err_generic,
},
{ "pcmk_rc_duplicate_id",
"Two or more XML elements have the same ID",
-pcmk_err_generic,
},
{ "pcmk_rc_disabled",
"Disabled",
-pcmk_err_generic,
},
{ "pcmk_rc_bad_input",
"Bad input value provided",
-pcmk_err_generic,
},
{ "pcmk_rc_bad_xml_patch",
"Bad XML patch format",
-pcmk_err_generic,
},
{ "pcmk_rc_no_transaction",
"No active transaction found",
-pcmk_err_generic,
},
{ "pcmk_rc_ns_resolution",
"Nameserver resolution error",
-pcmk_err_generic,
},
{ "pcmk_rc_compression",
"Compression/decompression error",
-pcmk_err_generic,
},
};
/*!
* \internal
* \brief The number of <tt>enum pcmk_rc_e</tt> values, excluding \c pcmk_rc_ok
*
* This constant stores the number of negative standard Pacemaker return codes.
* These represent Pacemaker-custom error codes. The count does not include
* positive system error numbers, nor does it include \c pcmk_rc_ok (success).
*/
const size_t pcmk__n_rc = PCMK__NELEM(pcmk__rcs);
/*!
* \brief Get a return code constant name as a string
*
* \param[in] rc Integer return code to convert
*
* \return String of constant name corresponding to rc
*/
const char *
pcmk_rc_name(int rc)
{
if ((rc <= pcmk_rc_error) && ((pcmk_rc_error - rc) < pcmk__n_rc)) {
return pcmk__rcs[pcmk_rc_error - rc].name;
}
switch (rc) {
case pcmk_rc_ok: return "pcmk_rc_ok";
case E2BIG: return "E2BIG";
case EACCES: return "EACCES";
case EADDRINUSE: return "EADDRINUSE";
case EADDRNOTAVAIL: return "EADDRNOTAVAIL";
case EAFNOSUPPORT: return "EAFNOSUPPORT";
case EAGAIN: return "EAGAIN";
case EALREADY: return "EALREADY";
case EBADF: return "EBADF";
case EBADMSG: return "EBADMSG";
case EBUSY: return "EBUSY";
case ECANCELED: return "ECANCELED";
case ECHILD: return "ECHILD";
case ECOMM: return "ECOMM";
case ECONNABORTED: return "ECONNABORTED";
case ECONNREFUSED: return "ECONNREFUSED";
case ECONNRESET: return "ECONNRESET";
/* case EDEADLK: return "EDEADLK"; */
case EDESTADDRREQ: return "EDESTADDRREQ";
case EDOM: return "EDOM";
case EDQUOT: return "EDQUOT";
case EEXIST: return "EEXIST";
case EFAULT: return "EFAULT";
case EFBIG: return "EFBIG";
case EHOSTDOWN: return "EHOSTDOWN";
case EHOSTUNREACH: return "EHOSTUNREACH";
case EIDRM: return "EIDRM";
case EILSEQ: return "EILSEQ";
case EINPROGRESS: return "EINPROGRESS";
case EINTR: return "EINTR";
case EINVAL: return "EINVAL";
case EIO: return "EIO";
case EISCONN: return "EISCONN";
case EISDIR: return "EISDIR";
case ELIBACC: return "ELIBACC";
case ELOOP: return "ELOOP";
case EMFILE: return "EMFILE";
case EMLINK: return "EMLINK";
case EMSGSIZE: return "EMSGSIZE";
#ifdef EMULTIHOP // Not available on OpenBSD
case EMULTIHOP: return "EMULTIHOP";
#endif
case ENAMETOOLONG: return "ENAMETOOLONG";
case ENETDOWN: return "ENETDOWN";
case ENETRESET: return "ENETRESET";
case ENETUNREACH: return "ENETUNREACH";
case ENFILE: return "ENFILE";
case ENOBUFS: return "ENOBUFS";
case ENODATA: return "ENODATA";
case ENODEV: return "ENODEV";
case ENOENT: return "ENOENT";
case ENOEXEC: return "ENOEXEC";
case ENOKEY: return "ENOKEY";
case ENOLCK: return "ENOLCK";
#ifdef ENOLINK // Not available on OpenBSD
case ENOLINK: return "ENOLINK";
#endif
case ENOMEM: return "ENOMEM";
case ENOMSG: return "ENOMSG";
case ENOPROTOOPT: return "ENOPROTOOPT";
case ENOSPC: return "ENOSPC";
#ifdef ENOSR
case ENOSR: return "ENOSR";
#endif
#ifdef ENOSTR
case ENOSTR: return "ENOSTR";
#endif
case ENOSYS: return "ENOSYS";
case ENOTBLK: return "ENOTBLK";
case ENOTCONN: return "ENOTCONN";
case ENOTDIR: return "ENOTDIR";
case ENOTEMPTY: return "ENOTEMPTY";
case ENOTSOCK: return "ENOTSOCK";
#if ENOTSUP != EOPNOTSUPP
case ENOTSUP: return "ENOTSUP";
#endif
case ENOTTY: return "ENOTTY";
case ENOTUNIQ: return "ENOTUNIQ";
case ENXIO: return "ENXIO";
case EOPNOTSUPP: return "EOPNOTSUPP";
case EOVERFLOW: return "EOVERFLOW";
case EPERM: return "EPERM";
case EPFNOSUPPORT: return "EPFNOSUPPORT";
case EPIPE: return "EPIPE";
case EPROTO: return "EPROTO";
case EPROTONOSUPPORT: return "EPROTONOSUPPORT";
case EPROTOTYPE: return "EPROTOTYPE";
case ERANGE: return "ERANGE";
case EREMOTE: return "EREMOTE";
case EREMOTEIO: return "EREMOTEIO";
case EROFS: return "EROFS";
case ESHUTDOWN: return "ESHUTDOWN";
case ESPIPE: return "ESPIPE";
case ESOCKTNOSUPPORT: return "ESOCKTNOSUPPORT";
case ESRCH: return "ESRCH";
case ESTALE: return "ESTALE";
case ETIME: return "ETIME";
case ETIMEDOUT: return "ETIMEDOUT";
case ETXTBSY: return "ETXTBSY";
#ifdef EUNATCH
case EUNATCH: return "EUNATCH";
#endif
case EUSERS: return "EUSERS";
/* case EWOULDBLOCK: return "EWOULDBLOCK"; */
case EXDEV: return "EXDEV";
#ifdef EBADE // Not available on OS X
case EBADE: return "EBADE";
case EBADFD: return "EBADFD";
case EBADSLT: return "EBADSLT";
case EDEADLOCK: return "EDEADLOCK";
case EBADR: return "EBADR";
case EBADRQC: return "EBADRQC";
case ECHRNG: return "ECHRNG";
#ifdef EISNAM // Not available on OS X, Illumos, Solaris
case EISNAM: return "EISNAM";
case EKEYEXPIRED: return "EKEYEXPIRED";
case EKEYREVOKED: return "EKEYREVOKED";
#endif
case EKEYREJECTED: return "EKEYREJECTED";
case EL2HLT: return "EL2HLT";
case EL2NSYNC: return "EL2NSYNC";
case EL3HLT: return "EL3HLT";
case EL3RST: return "EL3RST";
case ELIBBAD: return "ELIBBAD";
case ELIBMAX: return "ELIBMAX";
case ELIBSCN: return "ELIBSCN";
case ELIBEXEC: return "ELIBEXEC";
#ifdef ENOMEDIUM // Not available on OS X, Illumos, Solaris
case ENOMEDIUM: return "ENOMEDIUM";
case EMEDIUMTYPE: return "EMEDIUMTYPE";
#endif
case ENONET: return "ENONET";
case ENOPKG: return "ENOPKG";
case EREMCHG: return "EREMCHG";
case ERESTART: return "ERESTART";
case ESTRPIPE: return "ESTRPIPE";
#ifdef EUCLEAN // Not available on OS X, Illumos, Solaris
case EUCLEAN: return "EUCLEAN";
#endif
case EXFULL: return "EXFULL";
#endif // EBADE
default: return "Unknown";
}
}
/*!
* \brief Get a user-friendly description of a return code
*
* \param[in] rc Integer return code to convert
*
* \return String description of rc
*/
const char *
pcmk_rc_str(int rc)
{
if (rc == pcmk_rc_ok) {
return "OK";
}
if ((rc <= pcmk_rc_error) && ((pcmk_rc_error - rc) < pcmk__n_rc)) {
return pcmk__rcs[pcmk_rc_error - rc].desc;
}
if (rc < 0) {
return "Error";
}
// Handle values that could be defined by system or by portability.h
switch (rc) {
#ifdef PCMK__ENOTUNIQ
case ENOTUNIQ: return "Name not unique on network";
#endif
#ifdef PCMK__ECOMM
case ECOMM: return "Communication error on send";
#endif
#ifdef PCMK__ELIBACC
case ELIBACC: return "Can not access a needed shared library";
#endif
#ifdef PCMK__EREMOTEIO
case EREMOTEIO: return "Remote I/O error";
#endif
#ifdef PCMK__ENOKEY
case ENOKEY: return "Required key not available";
#endif
#ifdef PCMK__ENODATA
case ENODATA: return "No data available";
#endif
#ifdef PCMK__ETIME
case ETIME: return "Timer expired";
#endif
#ifdef PCMK__EKEYREJECTED
case EKEYREJECTED: return "Key was rejected by service";
#endif
default: return strerror(rc);
}
}
// This returns negative values for errors
//! \deprecated Use standard return codes instead
int
pcmk_rc2legacy(int rc)
{
if (rc >= 0) {
return -rc; // OK or system errno
}
if ((rc <= pcmk_rc_error) && ((pcmk_rc_error - rc) < pcmk__n_rc)) {
return pcmk__rcs[pcmk_rc_error - rc].legacy_rc;
}
return -pcmk_err_generic;
}
//! \deprecated Use standard return codes instead
int
pcmk_legacy2rc(int legacy_rc)
{
legacy_rc = abs(legacy_rc);
switch (legacy_rc) {
case pcmk_err_no_quorum: return pcmk_rc_no_quorum;
case pcmk_err_schema_validation: return pcmk_rc_schema_validation;
case pcmk_err_schema_unchanged: return pcmk_rc_schema_unchanged;
case pcmk_err_transform_failed: return pcmk_rc_transform_failed;
case pcmk_err_old_data: return pcmk_rc_old_data;
case pcmk_err_diff_failed: return pcmk_rc_diff_failed;
case pcmk_err_diff_resync: return pcmk_rc_diff_resync;
case pcmk_err_cib_modified: return pcmk_rc_cib_modified;
case pcmk_err_cib_backup: return pcmk_rc_cib_backup;
case pcmk_err_cib_save: return pcmk_rc_cib_save;
case pcmk_err_cib_corrupt: return pcmk_rc_cib_corrupt;
case pcmk_err_multiple: return pcmk_rc_multiple;
case pcmk_err_node_unknown: return pcmk_rc_node_unknown;
case pcmk_err_already: return pcmk_rc_already;
case pcmk_err_bad_nvpair: return pcmk_rc_bad_nvpair;
case pcmk_err_unknown_format: return pcmk_rc_unknown_format;
case pcmk_err_generic: return pcmk_rc_error;
case pcmk_ok: return pcmk_rc_ok;
default: return legacy_rc; // system errno
}
}
// Exit status codes
const char *
crm_exit_name(crm_exit_t exit_code)
{
switch (exit_code) {
case CRM_EX_OK: return "CRM_EX_OK";
case CRM_EX_ERROR: return "CRM_EX_ERROR";
case CRM_EX_INVALID_PARAM: return "CRM_EX_INVALID_PARAM";
case CRM_EX_UNIMPLEMENT_FEATURE: return "CRM_EX_UNIMPLEMENT_FEATURE";
case CRM_EX_INSUFFICIENT_PRIV: return "CRM_EX_INSUFFICIENT_PRIV";
case CRM_EX_NOT_INSTALLED: return "CRM_EX_NOT_INSTALLED";
case CRM_EX_NOT_CONFIGURED: return "CRM_EX_NOT_CONFIGURED";
case CRM_EX_NOT_RUNNING: return "CRM_EX_NOT_RUNNING";
case CRM_EX_PROMOTED: return "CRM_EX_PROMOTED";
case CRM_EX_FAILED_PROMOTED: return "CRM_EX_FAILED_PROMOTED";
case CRM_EX_USAGE: return "CRM_EX_USAGE";
case CRM_EX_DATAERR: return "CRM_EX_DATAERR";
case CRM_EX_NOINPUT: return "CRM_EX_NOINPUT";
case CRM_EX_NOUSER: return "CRM_EX_NOUSER";
case CRM_EX_NOHOST: return "CRM_EX_NOHOST";
case CRM_EX_UNAVAILABLE: return "CRM_EX_UNAVAILABLE";
case CRM_EX_SOFTWARE: return "CRM_EX_SOFTWARE";
case CRM_EX_OSERR: return "CRM_EX_OSERR";
case CRM_EX_OSFILE: return "CRM_EX_OSFILE";
case CRM_EX_CANTCREAT: return "CRM_EX_CANTCREAT";
case CRM_EX_IOERR: return "CRM_EX_IOERR";
case CRM_EX_TEMPFAIL: return "CRM_EX_TEMPFAIL";
case CRM_EX_PROTOCOL: return "CRM_EX_PROTOCOL";
case CRM_EX_NOPERM: return "CRM_EX_NOPERM";
case CRM_EX_CONFIG: return "CRM_EX_CONFIG";
case CRM_EX_FATAL: return "CRM_EX_FATAL";
case CRM_EX_PANIC: return "CRM_EX_PANIC";
case CRM_EX_DISCONNECT: return "CRM_EX_DISCONNECT";
case CRM_EX_DIGEST: return "CRM_EX_DIGEST";
case CRM_EX_NOSUCH: return "CRM_EX_NOSUCH";
case CRM_EX_QUORUM: return "CRM_EX_QUORUM";
case CRM_EX_UNSAFE: return "CRM_EX_UNSAFE";
case CRM_EX_EXISTS: return "CRM_EX_EXISTS";
case CRM_EX_MULTIPLE: return "CRM_EX_MULTIPLE";
case CRM_EX_EXPIRED: return "CRM_EX_EXPIRED";
case CRM_EX_NOT_YET_IN_EFFECT: return "CRM_EX_NOT_YET_IN_EFFECT";
case CRM_EX_INDETERMINATE: return "CRM_EX_INDETERMINATE";
case CRM_EX_UNSATISFIED: return "CRM_EX_UNSATISFIED";
case CRM_EX_OLD: return "CRM_EX_OLD";
case CRM_EX_TIMEOUT: return "CRM_EX_TIMEOUT";
case CRM_EX_DEGRADED: return "CRM_EX_DEGRADED";
case CRM_EX_DEGRADED_PROMOTED: return "CRM_EX_DEGRADED_PROMOTED";
case CRM_EX_NONE: return "CRM_EX_NONE";
case CRM_EX_MAX: return "CRM_EX_UNKNOWN";
}
return "CRM_EX_UNKNOWN";
}
const char *
crm_exit_str(crm_exit_t exit_code)
{
switch (exit_code) {
case CRM_EX_OK: return "OK";
case CRM_EX_ERROR: return "Error occurred";
case CRM_EX_INVALID_PARAM: return "Invalid parameter";
case CRM_EX_UNIMPLEMENT_FEATURE: return "Unimplemented";
case CRM_EX_INSUFFICIENT_PRIV: return "Insufficient privileges";
case CRM_EX_NOT_INSTALLED: return "Not installed";
case CRM_EX_NOT_CONFIGURED: return "Not configured";
case CRM_EX_NOT_RUNNING: return "Not running";
case CRM_EX_PROMOTED: return "Promoted";
case CRM_EX_FAILED_PROMOTED: return "Failed in promoted role";
case CRM_EX_USAGE: return "Incorrect usage";
case CRM_EX_DATAERR: return "Invalid data given";
case CRM_EX_NOINPUT: return "Input file not available";
case CRM_EX_NOUSER: return "User does not exist";
case CRM_EX_NOHOST: return "Host does not exist";
case CRM_EX_UNAVAILABLE: return "Necessary service unavailable";
case CRM_EX_SOFTWARE: return "Internal software bug";
case CRM_EX_OSERR: return "Operating system error occurred";
case CRM_EX_OSFILE: return "System file not available";
case CRM_EX_CANTCREAT: return "Cannot create output file";
case CRM_EX_IOERR: return "I/O error occurred";
case CRM_EX_TEMPFAIL: return "Temporary failure, try again";
case CRM_EX_PROTOCOL: return "Protocol violated";
case CRM_EX_NOPERM: return "Insufficient privileges";
case CRM_EX_CONFIG: return "Invalid configuration";
case CRM_EX_FATAL: return "Fatal error occurred, will not respawn";
case CRM_EX_PANIC: return "System panic required";
case CRM_EX_DISCONNECT: return "Not connected";
case CRM_EX_DIGEST: return "Digest mismatch";
case CRM_EX_NOSUCH: return "No such object";
case CRM_EX_QUORUM: return "Quorum required";
case CRM_EX_UNSAFE: return "Operation not safe";
case CRM_EX_EXISTS: return "Requested item already exists";
case CRM_EX_MULTIPLE: return "Multiple items match request";
case CRM_EX_EXPIRED: return "Requested item has expired";
case CRM_EX_NOT_YET_IN_EFFECT: return "Requested item is not yet in effect";
case CRM_EX_INDETERMINATE: return "Could not determine status";
case CRM_EX_UNSATISFIED: return "Not applicable under current conditions";
case CRM_EX_OLD: return "Update was older than existing configuration";
case CRM_EX_TIMEOUT: return "Timeout occurred";
case CRM_EX_DEGRADED: return "Service is active but might fail soon";
case CRM_EX_DEGRADED_PROMOTED: return "Service is promoted but might fail soon";
case CRM_EX_NONE: return "No exit status available";
case CRM_EX_MAX: return "Error occurred";
}
if ((exit_code > 128) && (exit_code < CRM_EX_MAX)) {
return "Interrupted by signal";
}
return "Unknown exit status";
}
/*!
* \brief Map a function return code to the most similar exit code
*
* \param[in] rc Function return code
*
* \return Most similar exit code
*/
crm_exit_t
pcmk_rc2exitc(int rc)
{
switch (rc) {
case pcmk_rc_ok:
case pcmk_rc_no_output: // quiet mode, or nothing to output
return CRM_EX_OK;
case pcmk_rc_no_quorum:
return CRM_EX_QUORUM;
case pcmk_rc_old_data:
return CRM_EX_OLD;
case pcmk_rc_schema_validation:
case pcmk_rc_transform_failed:
case pcmk_rc_unpack_error:
return CRM_EX_CONFIG;
case pcmk_rc_bad_nvpair:
return CRM_EX_INVALID_PARAM;
case EACCES:
return CRM_EX_INSUFFICIENT_PRIV;
case EBADF:
case EINVAL:
case EFAULT:
case ENOSYS:
case EOVERFLOW:
case pcmk_rc_underflow:
case pcmk_rc_compression:
return CRM_EX_SOFTWARE;
case EBADMSG:
case EMSGSIZE:
case ENOMSG:
case ENOPROTOOPT:
case EPROTO:
case EPROTONOSUPPORT:
case EPROTOTYPE:
return CRM_EX_PROTOCOL;
case ECOMM:
case ENOMEM:
return CRM_EX_OSERR;
case ECONNABORTED:
case ECONNREFUSED:
case ECONNRESET:
case ENOTCONN:
return CRM_EX_DISCONNECT;
case EEXIST:
case pcmk_rc_already:
return CRM_EX_EXISTS;
case EIO:
case pcmk_rc_dot_error:
case pcmk_rc_graph_error:
return CRM_EX_IOERR;
case ENOTSUP:
#if EOPNOTSUPP != ENOTSUP
case EOPNOTSUPP:
#endif
return CRM_EX_UNIMPLEMENT_FEATURE;
case ENOTUNIQ:
case pcmk_rc_multiple:
return CRM_EX_MULTIPLE;
case ENODEV:
case ENOENT:
case ENXIO:
case pcmk_rc_no_transaction:
case pcmk_rc_unknown_format:
return CRM_EX_NOSUCH;
case pcmk_rc_node_unknown:
case pcmk_rc_ns_resolution:
return CRM_EX_NOHOST;
case ETIME:
case ETIMEDOUT:
return CRM_EX_TIMEOUT;
case EAGAIN:
case EBUSY:
return CRM_EX_UNSATISFIED;
case pcmk_rc_before_range:
return CRM_EX_NOT_YET_IN_EFFECT;
case pcmk_rc_after_range:
return CRM_EX_EXPIRED;
case pcmk_rc_undetermined:
return CRM_EX_INDETERMINATE;
case pcmk_rc_op_unsatisfied:
return CRM_EX_UNSATISFIED;
case pcmk_rc_within_range:
return CRM_EX_OK;
case pcmk_rc_no_input:
return CRM_EX_NOINPUT;
case pcmk_rc_duplicate_id:
return CRM_EX_MULTIPLE;
case pcmk_rc_bad_input:
case pcmk_rc_bad_xml_patch:
return CRM_EX_DATAERR;
default:
return CRM_EX_ERROR;
}
}
/*!
* \brief Map a function return code to the most similar OCF exit code
*
* \param[in] rc Function return code
*
* \return Most similar OCF exit code
*/
enum ocf_exitcode
pcmk_rc2ocf(int rc)
{
switch (rc) {
case pcmk_rc_ok:
return PCMK_OCF_OK;
case pcmk_rc_bad_nvpair:
return PCMK_OCF_INVALID_PARAM;
case EACCES:
return PCMK_OCF_INSUFFICIENT_PRIV;
case ENOTSUP:
#if EOPNOTSUPP != ENOTSUP
case EOPNOTSUPP:
#endif
return PCMK_OCF_UNIMPLEMENT_FEATURE;
default:
return PCMK_OCF_UNKNOWN_ERROR;
}
}
// Other functions
/*!
* \brief Map a getaddrinfo() return code to the most similar Pacemaker
* return code
*
* \param[in] gai getaddrinfo() return code
*
* \return Most similar Pacemaker return code
*/
int
pcmk__gaierror2rc(int gai)
{
switch (gai) {
case 0:
return pcmk_rc_ok;
case EAI_AGAIN:
return EAGAIN;
case EAI_BADFLAGS:
case EAI_SERVICE:
return EINVAL;
case EAI_FAMILY:
return EAFNOSUPPORT;
case EAI_MEMORY:
return ENOMEM;
case EAI_NONAME:
return pcmk_rc_node_unknown;
case EAI_SOCKTYPE:
return ESOCKTNOSUPPORT;
case EAI_SYSTEM:
return errno;
default:
return pcmk_rc_ns_resolution;
}
}
/*!
* \brief Map a bz2 return code to the most similar Pacemaker return code
*
* \param[in] bz2 bz2 return code
*
* \return Most similar Pacemaker return code
*/
int
pcmk__bzlib2rc(int bz2)
{
switch (bz2) {
case BZ_OK:
case BZ_RUN_OK:
case BZ_FLUSH_OK:
case BZ_FINISH_OK:
case BZ_STREAM_END:
return pcmk_rc_ok;
case BZ_MEM_ERROR:
return ENOMEM;
case BZ_DATA_ERROR:
case BZ_DATA_ERROR_MAGIC:
case BZ_UNEXPECTED_EOF:
return pcmk_rc_bad_input;
case BZ_IO_ERROR:
return EIO;
case BZ_OUTBUFF_FULL:
return EFBIG;
default:
return pcmk_rc_compression;
}
}
crm_exit_t
-crm_exit(crm_exit_t rc)
+crm_exit(crm_exit_t exit_status)
{
/* A compiler could theoretically use any type for crm_exit_t, but an int
* should always hold it, so cast to int to keep static analysis happy.
*/
- if ((((int) rc) < 0) || (((int) rc) > CRM_EX_MAX)) {
- rc = CRM_EX_ERROR;
+ if ((((int) exit_status) < 0) || (((int) exit_status) > CRM_EX_MAX)) {
+ exit_status = CRM_EX_ERROR;
}
- mainloop_cleanup();
- pcmk__xml_cleanup();
-
- if (crm_system_name) {
- crm_info("Exiting %s " QB_XS " with status %d", crm_system_name, rc);
- free(crm_system_name);
- } else {
- crm_trace("Exiting with status %d", rc);
- }
- pcmk__free_common_logger();
- qb_log_fini(); // Don't log anything after this point
-
- exit(rc);
+ crm_info("Exiting %s " QB_XS " with status %d (%s: %s)",
+ pcmk__s(crm_system_name, "process"), exit_status,
+ crm_exit_name(exit_status), crm_exit_str(exit_status));
+ pcmk_common_cleanup();
+ exit(exit_status);
}
/*
* External action results
*/
/*!
* \internal
* \brief Set the result of an action
*
* \param[out] result Where to set action result
* \param[in] exit_status OCF exit status to set
* \param[in] exec_status Execution status to set
* \param[in] exit_reason Human-friendly description of event to set
*/
void
pcmk__set_result(pcmk__action_result_t *result, int exit_status,
enum pcmk_exec_status exec_status, const char *exit_reason)
{
if (result == NULL) {
return;
}
result->exit_status = exit_status;
result->execution_status = exec_status;
if (!pcmk__str_eq(result->exit_reason, exit_reason, pcmk__str_none)) {
free(result->exit_reason);
result->exit_reason = (exit_reason == NULL)? NULL : strdup(exit_reason);
}
}
/*!
* \internal
* \brief Set the result of an action, with a formatted exit reason
*
* \param[out] result Where to set action result
* \param[in] exit_status OCF exit status to set
* \param[in] exec_status Execution status to set
* \param[in] format printf-style format for a human-friendly
* description of reason for result
* \param[in] ... arguments for \p format
*/
G_GNUC_PRINTF(4, 5)
void
pcmk__format_result(pcmk__action_result_t *result, int exit_status,
enum pcmk_exec_status exec_status,
const char *format, ...)
{
va_list ap;
int len = 0;
char *reason = NULL;
if (result == NULL) {
return;
}
result->exit_status = exit_status;
result->execution_status = exec_status;
if (format != NULL) {
va_start(ap, format);
len = vasprintf(&reason, format, ap);
pcmk__assert(len > 0);
va_end(ap);
}
free(result->exit_reason);
result->exit_reason = reason;
}
/*!
* \internal
* \brief Set the output of an action
*
* \param[out] result Action result to set output for
* \param[in] out Action output to set (must be dynamically
* allocated)
* \param[in] err Action error output to set (must be dynamically
* allocated)
*
* \note \p result will take ownership of \p out and \p err, so the caller
* should not free them.
*/
void
pcmk__set_result_output(pcmk__action_result_t *result, char *out, char *err)
{
if (result == NULL) {
return;
}
free(result->action_stdout);
result->action_stdout = out;
free(result->action_stderr);
result->action_stderr = err;
}
/*!
* \internal
* \brief Clear a result's exit reason, output, and error output
*
* \param[in,out] result Result to reset
*/
void
pcmk__reset_result(pcmk__action_result_t *result)
{
if (result == NULL) {
return;
}
free(result->exit_reason);
result->exit_reason = NULL;
free(result->action_stdout);
result->action_stdout = NULL;
free(result->action_stderr);
result->action_stderr = NULL;
}
/*!
* \internal
* \brief Copy the result of an action
*
* \param[in] src Result to copy
* \param[out] dst Where to copy \p src to
*/
void
pcmk__copy_result(const pcmk__action_result_t *src, pcmk__action_result_t *dst)
{
CRM_CHECK((src != NULL) && (dst != NULL), return);
dst->exit_status = src->exit_status;
dst->execution_status = src->execution_status;
dst->exit_reason = pcmk__str_copy(src->exit_reason);
dst->action_stdout = pcmk__str_copy(src->action_stdout);
dst->action_stderr = pcmk__str_copy(src->action_stderr);
}
diff --git a/lib/common/utils.c b/lib/common/utils.c
index 16f543711a..c137330a95 100644
--- a/lib/common/utils.c
+++ b/lib/common/utils.c
@@ -1,472 +1,492 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <sys/stat.h>
#include <sys/utsname.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
#include <limits.h>
#include <pwd.h>
#include <time.h>
#include <libgen.h>
#include <signal.h>
#include <grp.h>
#include <qb/qbdefs.h>
#include <crm/crm.h>
#include <crm/services.h>
#include <crm/cib/internal.h>
#include <crm/common/xml.h>
#include <crm/common/util.h>
#include <crm/common/ipc.h>
#include <crm/common/iso8601.h>
#include <crm/common/mainloop.h>
#include <libxml2/libxml/relaxng.h>
#include "crmcommon_private.h"
CRM_TRACE_INIT_DATA(common);
bool pcmk__config_has_error = false;
bool pcmk__config_has_warning = false;
char *crm_system_name = NULL;
+/*!
+ * \brief Free all memory used by libcrmcommon
+ *
+ * Free all global memory allocated by the libcrmcommon library. This should be
+ * called before exiting a process that uses the library, and the process should
+ * not call any libcrmcommon or libxml2 APIs after calling this one.
+ */
+void
+pcmk_common_cleanup(void)
+{
+ // @TODO This isn't really everything, move all cleanup here
+ mainloop_cleanup();
+ pcmk__xml_cleanup();
+ pcmk__free_common_logger();
+ qb_log_fini(); // Don't log anything after this point
+
+ free(crm_system_name);
+ crm_system_name = NULL;
+}
+
bool
pcmk__is_user_in_group(const char *user, const char *group)
{
struct group *grent;
char **gr_mem;
if (user == NULL || group == NULL) {
return false;
}
setgrent();
while ((grent = getgrent()) != NULL) {
if (grent->gr_mem == NULL) {
continue;
}
if(strcmp(group, grent->gr_name) != 0) {
continue;
}
gr_mem = grent->gr_mem;
while (*gr_mem != NULL) {
if (!strcmp(user, *gr_mem++)) {
endgrent();
return true;
}
}
}
endgrent();
return false;
}
int
crm_user_lookup(const char *name, uid_t * uid, gid_t * gid)
{
int rc = pcmk_ok;
char *buffer = NULL;
struct passwd pwd;
struct passwd *pwentry = NULL;
buffer = calloc(1, PCMK__PW_BUFFER_LEN);
if (buffer == NULL) {
return -ENOMEM;
}
rc = getpwnam_r(name, &pwd, buffer, PCMK__PW_BUFFER_LEN, &pwentry);
if (pwentry) {
if (uid) {
*uid = pwentry->pw_uid;
}
if (gid) {
*gid = pwentry->pw_gid;
}
crm_trace("User %s has uid=%d gid=%d", name, pwentry->pw_uid, pwentry->pw_gid);
} else {
rc = rc? -rc : -EINVAL;
crm_info("User %s lookup: %s", name, pcmk_strerror(rc));
}
free(buffer);
return rc;
}
/*!
* \brief Get user and group IDs of pacemaker daemon user
*
* \param[out] uid If non-NULL, where to store daemon user ID
* \param[out] gid If non-NULL, where to store daemon group ID
*
* \return pcmk_ok on success, -errno otherwise
*/
int
pcmk_daemon_user(uid_t *uid, gid_t *gid)
{
static uid_t daemon_uid;
static gid_t daemon_gid;
static bool found = false;
int rc = pcmk_ok;
if (!found) {
rc = crm_user_lookup(CRM_DAEMON_USER, &daemon_uid, &daemon_gid);
if (rc == pcmk_ok) {
found = true;
}
}
if (found) {
if (uid) {
*uid = daemon_uid;
}
if (gid) {
*gid = daemon_gid;
}
}
return rc;
}
/*!
* \internal
* \brief Return the integer equivalent of a portion of a string
*
* \param[in] text Pointer to beginning of string portion
* \param[out] end_text This will point to next character after integer
*/
static int
version_helper(const char *text, const char **end_text)
{
int atoi_result = -1;
pcmk__assert(end_text != NULL);
errno = 0;
if (text != NULL && text[0] != 0) {
/* seemingly sacrificing const-correctness -- because while strtol
doesn't modify the input, it doesn't want to artificially taint the
"end_text" pointer-to-pointer-to-first-char-in-string with constness
in case the input wasn't actually constant -- by semantic definition
not a single character will get modified so it shall be perfectly
safe to make compiler happy with dropping "const" qualifier here */
atoi_result = (int) strtol(text, (char **) end_text, 10);
if (errno == EINVAL) {
crm_err("Conversion of '%s' %c failed", text, text[0]);
atoi_result = -1;
}
}
return atoi_result;
}
/*
* version1 < version2 : -1
* version1 = version2 : 0
* version1 > version2 : 1
*/
int
compare_version(const char *version1, const char *version2)
{
int rc = 0;
int lpc = 0;
const char *ver1_iter, *ver2_iter;
if (version1 == NULL && version2 == NULL) {
return 0;
} else if (version1 == NULL) {
return -1;
} else if (version2 == NULL) {
return 1;
}
ver1_iter = version1;
ver2_iter = version2;
while (1) {
int digit1 = 0;
int digit2 = 0;
lpc++;
if (ver1_iter == ver2_iter) {
break;
}
if (ver1_iter != NULL) {
digit1 = version_helper(ver1_iter, &ver1_iter);
}
if (ver2_iter != NULL) {
digit2 = version_helper(ver2_iter, &ver2_iter);
}
if (digit1 < digit2) {
rc = -1;
break;
} else if (digit1 > digit2) {
rc = 1;
break;
}
if (ver1_iter != NULL && *ver1_iter == '.') {
ver1_iter++;
}
if (ver1_iter != NULL && *ver1_iter == '\0') {
ver1_iter = NULL;
}
if (ver2_iter != NULL && *ver2_iter == '.') {
ver2_iter++;
}
if (ver2_iter != NULL && *ver2_iter == 0) {
ver2_iter = NULL;
}
}
if (rc == 0) {
crm_trace("%s == %s (%d)", version1, version2, lpc);
} else if (rc < 0) {
crm_trace("%s < %s (%d)", version1, version2, lpc);
} else if (rc > 0) {
crm_trace("%s > %s (%d)", version1, version2, lpc);
}
return rc;
}
/*!
* \internal
* \brief Convert the current process to a daemon process
*
* Fork a child process, exit the parent, create a PID file with the current
* process ID, and close the standard input/output/error file descriptors.
* Exit instead if a daemon is already running and using the PID file.
*
* \param[in] name Daemon executable name
* \param[in] pidfile File name to use as PID file
*/
void
pcmk__daemonize(const char *name, const char *pidfile)
{
int rc;
pid_t pid;
/* Check before we even try... */
rc = pcmk__pidfile_matches(pidfile, 1, name, &pid);
if ((rc != pcmk_rc_ok) && (rc != ENOENT)) {
crm_err("%s: already running [pid %lld in %s]",
name, (long long) pid, pidfile);
printf("%s: already running [pid %lld in %s]\n",
name, (long long) pid, pidfile);
crm_exit(CRM_EX_ERROR);
}
pid = fork();
if (pid < 0) {
fprintf(stderr, "%s: could not start daemon\n", name);
crm_perror(LOG_ERR, "fork");
crm_exit(CRM_EX_OSERR);
} else if (pid > 0) {
crm_exit(CRM_EX_OK);
}
rc = pcmk__lock_pidfile(pidfile, name);
if (rc != pcmk_rc_ok) {
crm_err("Could not lock '%s' for %s: %s " QB_XS " rc=%d",
pidfile, name, pcmk_rc_str(rc), rc);
printf("Could not lock '%s' for %s: %s (%d)\n",
pidfile, name, pcmk_rc_str(rc), rc);
crm_exit(CRM_EX_ERROR);
}
umask(S_IWGRP | S_IWOTH | S_IROTH);
close(STDIN_FILENO);
pcmk__open_devnull(O_RDONLY); // stdin (fd 0)
close(STDOUT_FILENO);
pcmk__open_devnull(O_WRONLY); // stdout (fd 1)
close(STDERR_FILENO);
pcmk__open_devnull(O_WRONLY); // stderr (fd 2)
}
#ifdef HAVE_UUID_UUID_H
# include <uuid/uuid.h>
#endif
char *
crm_generate_uuid(void)
{
unsigned char uuid[16];
char *buffer = malloc(37); /* Including NUL byte */
pcmk__mem_assert(buffer);
uuid_generate(uuid);
uuid_unparse(uuid, buffer);
return buffer;
}
void
crm_gnutls_global_init(void)
{
signal(SIGPIPE, SIG_IGN);
gnutls_global_init();
}
/*!
* \internal
* \brief Sleep for given milliseconds
*
* \param[in] ms Time to sleep
*
* \note The full time might not be slept if a signal is received.
*/
void
pcmk__sleep_ms(unsigned int ms)
{
// @TODO Impose a sane maximum sleep to avoid hanging a process for long
//CRM_CHECK(ms <= MAX_SLEEP, ms = MAX_SLEEP);
// Use sleep() for any whole seconds
if (ms >= 1000) {
sleep(ms / 1000);
ms -= ms / 1000;
}
if (ms == 0) {
return;
}
#if defined(HAVE_NANOSLEEP)
// nanosleep() is POSIX-2008, so prefer that
{
struct timespec req = { .tv_sec = 0, .tv_nsec = (long) (ms * 1000000) };
nanosleep(&req, NULL);
}
#elif defined(HAVE_USLEEP)
// usleep() is widely available, though considered obsolete
usleep((useconds_t) ms);
#else
// Otherwise use a trick with select() timeout
{
struct timeval tv = { .tv_sec = 0, .tv_usec = (suseconds_t) ms };
select(0, NULL, NULL, NULL, &tv);
}
#endif
}
/*!
* \internal
* \brief Add a timer
*
* \param[in] interval_ms The interval for the function to be called, in ms
* \param[in] fn The function to be called
* \param[in] data Data to be passed to fn (can be NULL)
*
* \return The ID of the event source
*/
guint
pcmk__create_timer(guint interval_ms, GSourceFunc fn, gpointer data)
{
pcmk__assert(interval_ms != 0 && fn != NULL);
if (interval_ms % 1000 == 0) {
/* In case interval_ms is 0, the call to pcmk__timeout_ms2s ensures
* an interval of one second.
*/
return g_timeout_add_seconds(pcmk__timeout_ms2s(interval_ms), fn, data);
} else {
return g_timeout_add(interval_ms, fn, data);
}
}
/*!
* \internal
* \brief Convert milliseconds to seconds
*
* \param[in] timeout_ms The interval, in ms
*
* \return If \p timeout_ms is 0, return 0. Otherwise, return the number of
* seconds, rounded to the nearest integer, with a minimum of 1.
*/
guint
pcmk__timeout_ms2s(guint timeout_ms)
{
guint quot, rem;
if (timeout_ms == 0) {
return 0;
} else if (timeout_ms < 1000) {
return 1;
}
quot = timeout_ms / 1000;
rem = timeout_ms % 1000;
if (rem >= 500) {
quot += 1;
}
return quot;
}
// Deprecated functions kept only for backward API compatibility
// LCOV_EXCL_START
#include <crm/common/util_compat.h>
/*!
* \brief Check whether string represents a client name used by cluster daemons
*
* \param[in] name String to check
*
* \return true if name is standard client name used by daemons, false otherwise
*
* \note This is provided by the client, and so cannot be used by itself as a
* secure means of authentication.
*/
bool
crm_is_daemon_name(const char *name)
{
return pcmk__str_any_of(name,
"attrd",
CRM_SYSTEM_CIB,
CRM_SYSTEM_CRMD,
CRM_SYSTEM_DC,
CRM_SYSTEM_LRMD,
CRM_SYSTEM_MCP,
CRM_SYSTEM_PENGINE,
CRM_SYSTEM_TENGINE,
"pacemaker-attrd",
"pacemaker-based",
"pacemaker-controld",
"pacemaker-execd",
"pacemaker-fenced",
"pacemaker-remoted",
"pacemaker-schedulerd",
"stonith-ng",
"stonithd",
NULL);
}
// LCOV_EXCL_STOP
// End deprecated API
diff --git a/lib/pacemaker/pcmk_sched_instances.c b/lib/pacemaker/pcmk_sched_instances.c
index 1c95a1333e..a204a65525 100644
--- a/lib/pacemaker/pcmk_sched_instances.c
+++ b/lib/pacemaker/pcmk_sched_instances.c
@@ -1,1714 +1,1714 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU General Public License version 2
* or later (GPLv2+) WITHOUT ANY WARRANTY.
*/
/* This file is intended for code usable with both clone instances and bundle
* replica containers.
*/
#include <crm_internal.h>
#include <crm/common/xml.h>
#include <pacemaker-internal.h>
#include "libpacemaker_private.h"
/*!
* \internal
* \brief Check whether a node is allowed to run an instance
*
* \param[in] instance Clone instance or bundle container to check
* \param[in] node Node to check
* \param[in] max_per_node Maximum number of instances allowed to run on a node
*
* \return true if \p node is allowed to run \p instance, otherwise false
*/
static bool
can_run_instance(const pcmk_resource_t *instance, const pcmk_node_t *node,
int max_per_node)
{
pcmk_node_t *allowed_node = NULL;
if (pcmk_is_set(instance->flags, pcmk__rsc_removed)) {
pcmk__rsc_trace(instance, "%s cannot run on %s: orphaned",
instance->id, pcmk__node_name(node));
return false;
}
if (!pcmk__node_available(node, false, false)) {
pcmk__rsc_trace(instance,
"%s cannot run on %s: node cannot run resources",
instance->id, pcmk__node_name(node));
return false;
}
allowed_node = pcmk__top_allowed_node(instance, node);
if (allowed_node == NULL) {
crm_warn("%s cannot run on %s: node not allowed",
instance->id, pcmk__node_name(node));
return false;
}
if (allowed_node->assign->score < 0) {
pcmk__rsc_trace(instance,
"%s cannot run on %s: parent score is %s there",
instance->id, pcmk__node_name(node),
pcmk_readable_score(allowed_node->assign->score));
return false;
}
if (allowed_node->assign->count >= max_per_node) {
pcmk__rsc_trace(instance,
"%s cannot run on %s: node already has %d instance%s",
instance->id, pcmk__node_name(node), max_per_node,
pcmk__plural_s(max_per_node));
return false;
}
pcmk__rsc_trace(instance, "%s can run on %s (%d already running)",
instance->id, pcmk__node_name(node),
allowed_node->assign->count);
return true;
}
/*!
* \internal
* \brief Ban a clone instance or bundle replica from unavailable allowed nodes
*
* \param[in,out] instance Clone instance or bundle replica to ban
* \param[in] max_per_node Maximum instances allowed to run on a node
*/
static void
ban_unavailable_allowed_nodes(pcmk_resource_t *instance, int max_per_node)
{
if (instance->priv->allowed_nodes != NULL) {
GHashTableIter iter;
pcmk_node_t *node = NULL;
g_hash_table_iter_init(&iter, instance->priv->allowed_nodes);
while (g_hash_table_iter_next(&iter, NULL, (void **) &node)) {
if (!can_run_instance(instance, node, max_per_node)) {
pcmk__rsc_trace(instance, "Banning %s from unavailable node %s",
instance->id, pcmk__node_name(node));
node->assign->score = -PCMK_SCORE_INFINITY;
for (GList *child_iter = instance->priv->children;
child_iter != NULL; child_iter = child_iter->next) {
pcmk_resource_t *child = child_iter->data;
pcmk_node_t *child_node = NULL;
child_node =
g_hash_table_lookup(child->priv->allowed_nodes,
node->priv->id);
if (child_node != NULL) {
pcmk__rsc_trace(instance,
"Banning %s child %s "
"from unavailable node %s",
instance->id, child->id,
pcmk__node_name(node));
child_node->assign->score = -PCMK_SCORE_INFINITY;
}
}
}
}
}
}
/*!
* \internal
* \brief Create a hash table with a single node in it
*
* \param[in] node Node to copy into new table
*
* \return Newly created hash table containing a copy of \p node
* \note The caller is responsible for freeing the result with
* g_hash_table_destroy().
*/
static GHashTable *
new_node_table(pcmk_node_t *node)
{
- GHashTable *table = pcmk__strkey_table(NULL, free);
+ GHashTable *table = pcmk__strkey_table(NULL, pcmk__free_node_copy);
node = pe__copy_node(node);
g_hash_table_insert(table, (gpointer) node->priv->id, node);
return table;
}
/*!
* \internal
* \brief Apply a resource's parent's colocation scores to a node table
*
* \param[in] rsc Resource whose colocations should be applied
* \param[in,out] nodes Node table to apply colocations to
*/
static void
apply_parent_colocations(const pcmk_resource_t *rsc, GHashTable **nodes)
{
GList *colocations = pcmk__this_with_colocations(rsc);
for (const GList *iter = colocations; iter != NULL; iter = iter->next) {
const pcmk__colocation_t *colocation = iter->data;
pcmk_resource_t *other = colocation->primary;
float factor = colocation->score / (float) PCMK_SCORE_INFINITY;
other->priv->cmds->add_colocated_node_scores(other, rsc, rsc->id,
nodes, colocation, factor,
pcmk__coloc_select_default);
}
g_list_free(colocations);
colocations = pcmk__with_this_colocations(rsc);
for (const GList *iter = colocations; iter != NULL; iter = iter->next) {
const pcmk__colocation_t *colocation = iter->data;
pcmk_resource_t *other = colocation->dependent;
float factor = colocation->score / (float) PCMK_SCORE_INFINITY;
if (!pcmk__colocation_has_influence(colocation, rsc)) {
continue;
}
other->priv->cmds->add_colocated_node_scores(other, rsc, rsc->id,
nodes, colocation, factor,
pcmk__coloc_select_nonnegative);
}
g_list_free(colocations);
}
/*!
* \internal
* \brief Compare clone or bundle instances based on colocation scores
*
* Determine the relative order in which two clone or bundle instances should be
* assigned to nodes, considering the scores of colocation constraints directly
* or indirectly involving them.
*
* \param[in] instance1 First instance to compare
* \param[in] instance2 Second instance to compare
*
* \return A negative number if \p instance1 should be assigned first,
* a positive number if \p instance2 should be assigned first,
* or 0 if assignment order doesn't matter
*/
static int
cmp_instance_by_colocation(const pcmk_resource_t *instance1,
const pcmk_resource_t *instance2)
{
int rc = 0;
pcmk_node_t *node1 = NULL;
pcmk_node_t *node2 = NULL;
pcmk_node_t *current_node1 = pcmk__current_node(instance1);
pcmk_node_t *current_node2 = pcmk__current_node(instance2);
GHashTable *colocated_scores1 = NULL;
GHashTable *colocated_scores2 = NULL;
pcmk__assert((instance1 != NULL) && (instance1->priv->parent != NULL)
&& (instance2 != NULL) && (instance2->priv->parent != NULL)
&& (current_node1 != NULL) && (current_node2 != NULL));
// Create node tables initialized with each node
colocated_scores1 = new_node_table(current_node1);
colocated_scores2 = new_node_table(current_node2);
// Apply parental colocations
apply_parent_colocations(instance1, &colocated_scores1);
apply_parent_colocations(instance2, &colocated_scores2);
// Find original nodes again, with scores updated for colocations
node1 = g_hash_table_lookup(colocated_scores1, current_node1->priv->id);
node2 = g_hash_table_lookup(colocated_scores2, current_node2->priv->id);
// Compare nodes by updated scores
if (node1->assign->score < node2->assign->score) {
crm_trace("Assign %s (%d on %s) after %s (%d on %s)",
instance1->id, node1->assign->score, pcmk__node_name(node1),
instance2->id, node2->assign->score, pcmk__node_name(node2));
rc = 1;
} else if (node1->assign->score > node2->assign->score) {
crm_trace("Assign %s (%d on %s) before %s (%d on %s)",
instance1->id, node1->assign->score, pcmk__node_name(node1),
instance2->id, node2->assign->score, pcmk__node_name(node2));
rc = -1;
}
g_hash_table_destroy(colocated_scores1);
g_hash_table_destroy(colocated_scores2);
return rc;
}
/*!
* \internal
* \brief Check whether a resource or any of its children are failed
*
* \param[in] rsc Resource to check
*
* \return true if \p rsc or any of its children are failed, otherwise false
*/
static bool
did_fail(const pcmk_resource_t *rsc)
{
if (pcmk_is_set(rsc->flags, pcmk__rsc_failed)) {
return true;
}
for (GList *iter = rsc->priv->children;
iter != NULL; iter = iter->next) {
if (did_fail((const pcmk_resource_t *) iter->data)) {
return true;
}
}
return false;
}
/*!
* \internal
* \brief Check whether a node is allowed to run a resource
*
* \param[in] rsc Resource to check
* \param[in,out] node Node to check (will be set NULL if not allowed)
*
* \return true if *node is either NULL or allowed for \p rsc, otherwise false
*/
static bool
node_is_allowed(const pcmk_resource_t *rsc, pcmk_node_t **node)
{
if (*node != NULL) {
pcmk_node_t *allowed = g_hash_table_lookup(rsc->priv->allowed_nodes,
(*node)->priv->id);
if ((allowed == NULL) || (allowed->assign->score < 0)) {
pcmk__rsc_trace(rsc, "%s: current location (%s) is unavailable",
rsc->id, pcmk__node_name(*node));
*node = NULL;
return false;
}
}
return true;
}
/*!
* \internal
* \brief Compare two clone or bundle instances' instance numbers
*
* \param[in] a First instance to compare
* \param[in] b Second instance to compare
*
* \return A negative number if \p a's instance number is lower,
* a positive number if \p b's instance number is lower,
* or 0 if their instance numbers are the same
*/
gint
pcmk__cmp_instance_number(gconstpointer a, gconstpointer b)
{
const pcmk_resource_t *instance1 = (const pcmk_resource_t *) a;
const pcmk_resource_t *instance2 = (const pcmk_resource_t *) b;
char *div1 = NULL;
char *div2 = NULL;
pcmk__assert((instance1 != NULL) && (instance2 != NULL));
// Clone numbers are after a colon, bundle numbers after a dash
div1 = strrchr(instance1->id, ':');
if (div1 == NULL) {
div1 = strrchr(instance1->id, '-');
}
div2 = strrchr(instance2->id, ':');
if (div2 == NULL) {
div2 = strrchr(instance2->id, '-');
}
pcmk__assert((div1 != NULL) && (div2 != NULL));
return (gint) (strtol(div1 + 1, NULL, 10) - strtol(div2 + 1, NULL, 10));
}
/*!
* \internal
* \brief Compare clone or bundle instances according to assignment order
*
* Compare two clone or bundle instances according to the order they should be
* assigned to nodes, preferring (in order):
*
* - Active instance that is less multiply active
* - Instance that is not active on a disallowed node
* - Instance with higher configured priority
* - Active instance whose current node can run resources
* - Active instance whose parent is allowed on current node
* - Active instance whose current node has fewer other instances
* - Active instance
* - Instance that isn't failed
* - Instance whose colocations result in higher score on current node
* - Instance with lower ID in lexicographic order
*
* \param[in] a First instance to compare
* \param[in] b Second instance to compare
*
* \return A negative number if \p a should be assigned first,
* a positive number if \p b should be assigned first,
* or 0 if assignment order doesn't matter
*/
gint
pcmk__cmp_instance(gconstpointer a, gconstpointer b)
{
int rc = 0;
pcmk_node_t *node1 = NULL;
pcmk_node_t *node2 = NULL;
unsigned int nnodes1 = 0;
unsigned int nnodes2 = 0;
bool can1 = true;
bool can2 = true;
const pcmk_resource_t *instance1 = (const pcmk_resource_t *) a;
const pcmk_resource_t *instance2 = (const pcmk_resource_t *) b;
pcmk__assert((instance1 != NULL) && (instance2 != NULL));
node1 = instance1->priv->fns->active_node(instance1, &nnodes1, NULL);
node2 = instance2->priv->fns->active_node(instance2, &nnodes2, NULL);
/* If both instances are running and at least one is multiply
* active, prefer instance that's running on fewer nodes.
*/
if ((nnodes1 > 0) && (nnodes2 > 0)) {
if (nnodes1 < nnodes2) {
crm_trace("Assign %s (active on %d) before %s (active on %d): "
"less multiply active",
instance1->id, nnodes1, instance2->id, nnodes2);
return -1;
} else if (nnodes1 > nnodes2) {
crm_trace("Assign %s (active on %d) after %s (active on %d): "
"more multiply active",
instance1->id, nnodes1, instance2->id, nnodes2);
return 1;
}
}
/* An instance that is either inactive or active on an allowed node is
* preferred over an instance that is active on a no-longer-allowed node.
*/
can1 = node_is_allowed(instance1, &node1);
can2 = node_is_allowed(instance2, &node2);
if (can1 && !can2) {
crm_trace("Assign %s before %s: not active on a disallowed node",
instance1->id, instance2->id);
return -1;
} else if (!can1 && can2) {
crm_trace("Assign %s after %s: active on a disallowed node",
instance1->id, instance2->id);
return 1;
}
// Prefer instance with higher configured priority
if (instance1->priv->priority > instance2->priv->priority) {
crm_trace("Assign %s before %s: priority (%d > %d)",
instance1->id, instance2->id,
instance1->priv->priority, instance2->priv->priority);
return -1;
} else if (instance1->priv->priority < instance2->priv->priority) {
crm_trace("Assign %s after %s: priority (%d < %d)",
instance1->id, instance2->id,
instance1->priv->priority, instance2->priv->priority);
return 1;
}
// Prefer active instance
if ((node1 == NULL) && (node2 == NULL)) {
crm_trace("No assignment preference for %s vs. %s: inactive",
instance1->id, instance2->id);
return 0;
} else if (node1 == NULL) {
crm_trace("Assign %s after %s: active", instance1->id, instance2->id);
return 1;
} else if (node2 == NULL) {
crm_trace("Assign %s before %s: active", instance1->id, instance2->id);
return -1;
}
// Prefer instance whose current node can run resources
can1 = pcmk__node_available(node1, false, false);
can2 = pcmk__node_available(node2, false, false);
if (can1 && !can2) {
crm_trace("Assign %s before %s: current node can run resources",
instance1->id, instance2->id);
return -1;
} else if (!can1 && can2) {
crm_trace("Assign %s after %s: current node can't run resources",
instance1->id, instance2->id);
return 1;
}
// Prefer instance whose parent is allowed to run on instance's current node
node1 = pcmk__top_allowed_node(instance1, node1);
node2 = pcmk__top_allowed_node(instance2, node2);
if ((node1 == NULL) && (node2 == NULL)) {
crm_trace("No assignment preference for %s vs. %s: "
"parent not allowed on either instance's current node",
instance1->id, instance2->id);
return 0;
} else if (node1 == NULL) {
crm_trace("Assign %s after %s: parent not allowed on current node",
instance1->id, instance2->id);
return 1;
} else if (node2 == NULL) {
crm_trace("Assign %s before %s: parent allowed on current node",
instance1->id, instance2->id);
return -1;
}
// Prefer instance whose current node is running fewer other instances
if (node1->assign->count < node2->assign->count) {
crm_trace("Assign %s before %s: fewer active instances on current node",
instance1->id, instance2->id);
return -1;
} else if (node1->assign->count > node2->assign->count) {
crm_trace("Assign %s after %s: more active instances on current node",
instance1->id, instance2->id);
return 1;
}
// Prefer instance that isn't failed
can1 = did_fail(instance1);
can2 = did_fail(instance2);
if (!can1 && can2) {
crm_trace("Assign %s before %s: not failed",
instance1->id, instance2->id);
return -1;
} else if (can1 && !can2) {
crm_trace("Assign %s after %s: failed",
instance1->id, instance2->id);
return 1;
}
// Prefer instance with higher cumulative colocation score on current node
rc = cmp_instance_by_colocation(instance1, instance2);
if (rc != 0) {
return rc;
}
// Prefer instance with lower instance number
rc = pcmk__cmp_instance_number(instance1, instance2);
if (rc < 0) {
crm_trace("Assign %s before %s: instance number",
instance1->id, instance2->id);
} else if (rc > 0) {
crm_trace("Assign %s after %s: instance number",
instance1->id, instance2->id);
} else {
crm_trace("No assignment preference for %s vs. %s",
instance1->id, instance2->id);
}
return rc;
}
/*!
* \internal
* \brief Increment the parent's instance count after assigning an instance
*
* An instance's parent tracks how many instances have been assigned to each
* node via its pcmk_node_t:count member. After assigning an instance to a node,
* find the corresponding node in the parent's allowed table and increment it.
*
* \param[in,out] instance Instance whose parent to update
* \param[in] assigned_to Node to which the instance was assigned
*/
static void
increment_parent_count(pcmk_resource_t *instance,
const pcmk_node_t *assigned_to)
{
pcmk_node_t *allowed = NULL;
if (assigned_to == NULL) {
return;
}
allowed = pcmk__top_allowed_node(instance, assigned_to);
if (allowed == NULL) {
/* The instance is allowed on the node, but its parent isn't. This
* shouldn't be possible if the resource is managed, and we won't be
* able to limit the number of instances assigned to the node.
*/
CRM_LOG_ASSERT(!pcmk_is_set(instance->flags, pcmk__rsc_managed));
} else {
allowed->assign->count++;
}
}
/*!
* \internal
* \brief Assign an instance to a node
*
* \param[in,out] instance Clone instance or bundle replica container
* \param[in] prefer If not NULL, attempt early assignment to this
* node, if still the best choice; otherwise,
* perform final assignment
* \param[in] max_per_node Assign at most this many instances to one node
*
* \return Node to which \p instance is assigned
*/
static const pcmk_node_t *
assign_instance(pcmk_resource_t *instance, const pcmk_node_t *prefer,
int max_per_node)
{
pcmk_node_t *chosen = NULL;
pcmk__rsc_trace(instance, "Assigning %s (preferring %s)", instance->id,
((prefer == NULL)? "no node" : prefer->priv->name));
if (pcmk_is_set(instance->flags, pcmk__rsc_assigning)) {
pcmk__rsc_debug(instance,
"Assignment loop detected involving %s colocations",
instance->id);
return NULL;
}
ban_unavailable_allowed_nodes(instance, max_per_node);
// Failed early assignments are reversible (stop_if_fail=false)
chosen = instance->priv->cmds->assign(instance, prefer, (prefer == NULL));
increment_parent_count(instance, chosen);
return chosen;
}
/*!
* \internal
* \brief Try to assign an instance to its current node early
*
* \param[in] rsc Clone or bundle being assigned (for logs only)
* \param[in] instance Clone instance or bundle replica container
* \param[in] current Instance's current node
* \param[in] max_per_node Maximum number of instances per node
* \param[in] available Number of instances still available for assignment
*
* \return \c true if \p instance was successfully assigned to its current node,
* or \c false otherwise
*/
static bool
assign_instance_early(const pcmk_resource_t *rsc, pcmk_resource_t *instance,
const pcmk_node_t *current, int max_per_node,
int available)
{
const pcmk_node_t *chosen = NULL;
int reserved = 0;
pcmk_resource_t *parent = instance->priv->parent;
GHashTable *allowed_orig = NULL;
GHashTable *allowed_orig_parent = parent->priv->allowed_nodes;
const pcmk_node_t *allowed_node = NULL;
pcmk__rsc_trace(instance, "Trying to assign %s to its current node %s",
instance->id, pcmk__node_name(current));
allowed_node = g_hash_table_lookup(instance->priv->allowed_nodes,
current->priv->id);
if (!pcmk__node_available(allowed_node, true, false)) {
pcmk__rsc_info(instance,
"Not assigning %s to current node %s: unavailable",
instance->id, pcmk__node_name(current));
return false;
}
/* On each iteration, if instance gets assigned to a node other than its
* current one, we reserve one instance for the chosen node, unassign
* instance, restore instance's original node tables, and try again. This
* way, instances are proportionally assigned to nodes based on preferences,
* but shuffling of specific instances is minimized. If a node will be
* assigned instances at all, it preferentially receives instances that are
* currently active there.
*
* parent->private->allowed_nodes tracks the number of instances assigned to
* each node. If a node already has max_per_node instances assigned,
* ban_unavailable_allowed_nodes() marks it as unavailable.
*
* In the end, we restore the original parent->private->allowed_nodes to
* undo the changes to counts during tentative assignments. If we
* successfully assigned an instance to its current node, we increment that
* node's counter.
*/
// Back up the allowed node tables of instance and its children recursively
pcmk__copy_node_tables(instance, &allowed_orig);
// Update instances-per-node counts in a scratch table
parent->priv->allowed_nodes = pcmk__copy_node_table(allowed_orig_parent);
while (reserved < available) {
chosen = assign_instance(instance, current, max_per_node);
if (pcmk__same_node(chosen, current)) {
// Successfully assigned to current node
break;
}
// Assignment updates scores, so restore to original state
pcmk__rsc_debug(instance, "Rolling back node scores for %s",
instance->id);
pcmk__restore_node_tables(instance, allowed_orig);
if (chosen == NULL) {
// Assignment failed, so give up
pcmk__rsc_info(instance,
"Not assigning %s to current node %s: unavailable",
instance->id, pcmk__node_name(current));
pcmk__set_rsc_flags(instance, pcmk__rsc_unassigned);
break;
}
// We prefer more strongly to assign an instance to the chosen node
pcmk__rsc_debug(instance,
"Not assigning %s to current node %s: %s is better",
instance->id, pcmk__node_name(current),
pcmk__node_name(chosen));
// Reserve one instance for the chosen node and try again
if (++reserved >= available) {
pcmk__rsc_info(instance,
"Not assigning %s to current node %s: "
"other assignments are more important",
instance->id, pcmk__node_name(current));
} else {
pcmk__rsc_debug(instance,
"Reserved an instance of %s for %s. Retrying "
"assignment of %s to %s",
rsc->id, pcmk__node_name(chosen), instance->id,
pcmk__node_name(current));
}
// Clear this assignment (frees chosen); leave instance counts in parent
pcmk__unassign_resource(instance);
chosen = NULL;
}
g_hash_table_destroy(allowed_orig);
// Restore original instances-per-node counts
g_hash_table_destroy(parent->priv->allowed_nodes);
parent->priv->allowed_nodes = allowed_orig_parent;
if (chosen == NULL) {
// Couldn't assign instance to current node
return false;
}
pcmk__rsc_trace(instance, "Assigned %s to current node %s",
instance->id, pcmk__node_name(current));
increment_parent_count(instance, chosen);
return true;
}
/*!
* \internal
* \brief Reset the node counts of a resource's allowed nodes to zero
*
* \param[in,out] rsc Resource to reset
*
* \return Number of nodes that are available to run resources
*/
static unsigned int
reset_allowed_node_counts(pcmk_resource_t *rsc)
{
unsigned int available_nodes = 0;
pcmk_node_t *node = NULL;
GHashTableIter iter;
g_hash_table_iter_init(&iter, rsc->priv->allowed_nodes);
while (g_hash_table_iter_next(&iter, NULL, (gpointer *) &node)) {
node->assign->count = 0;
if (pcmk__node_available(node, false, false)) {
available_nodes++;
}
}
return available_nodes;
}
/*!
* \internal
* \brief Check whether an instance has a preferred node
*
* \param[in] instance Clone instance or bundle replica container
* \param[in] optimal_per_node Optimal number of instances per node
*
* \return Instance's current node if still available, otherwise NULL
*/
static const pcmk_node_t *
preferred_node(const pcmk_resource_t *instance, int optimal_per_node)
{
const pcmk_node_t *node = NULL;
const pcmk_node_t *parent_node = NULL;
// Check whether instance is active, healthy, and not yet assigned
if ((instance->priv->active_nodes == NULL)
|| !pcmk_is_set(instance->flags, pcmk__rsc_unassigned)
|| pcmk_is_set(instance->flags, pcmk__rsc_failed)) {
return NULL;
}
// Check whether instance's current node can run resources
node = pcmk__current_node(instance);
if (!pcmk__node_available(node, true, false)) {
pcmk__rsc_trace(instance, "Not assigning %s to %s early (unavailable)",
instance->id, pcmk__node_name(node));
return NULL;
}
// Check whether node already has optimal number of instances assigned
parent_node = pcmk__top_allowed_node(instance, node);
if ((parent_node != NULL)
&& (parent_node->assign->count >= optimal_per_node)) {
pcmk__rsc_trace(instance,
"Not assigning %s to %s early "
"(optimal instances already assigned)",
instance->id, pcmk__node_name(node));
return NULL;
}
return node;
}
/*!
* \internal
* \brief Assign collective instances to nodes
*
* \param[in,out] collective Clone or bundle resource being assigned
* \param[in,out] instances List of clone instances or bundle containers
* \param[in] max_total Maximum instances to assign in total
* \param[in] max_per_node Maximum instances to assign to any one node
*/
void
pcmk__assign_instances(pcmk_resource_t *collective, GList *instances,
int max_total, int max_per_node)
{
// Reuse node count to track number of assigned instances
unsigned int available_nodes = reset_allowed_node_counts(collective);
int optimal_per_node = 0;
int assigned = 0;
GList *iter = NULL;
pcmk_resource_t *instance = NULL;
const pcmk_node_t *current = NULL;
if (available_nodes > 0) {
optimal_per_node = max_total / available_nodes;
}
if (optimal_per_node < 1) {
optimal_per_node = 1;
}
pcmk__rsc_debug(collective,
"Assigning up to %d %s instance%s to up to %u node%s "
"(at most %d per host, %d optimal)",
max_total, collective->id, pcmk__plural_s(max_total),
available_nodes, pcmk__plural_s(available_nodes),
max_per_node, optimal_per_node);
// Assign as many instances as possible to their current location
for (iter = instances; (iter != NULL) && (assigned < max_total);
iter = iter->next) {
int available = max_total - assigned;
instance = iter->data;
if (!pcmk_is_set(instance->flags, pcmk__rsc_unassigned)) {
continue; // Already assigned
}
current = preferred_node(instance, optimal_per_node);
if ((current != NULL)
&& assign_instance_early(collective, instance, current,
max_per_node, available)) {
assigned++;
}
}
pcmk__rsc_trace(collective, "Assigned %d of %d instance%s to current node",
assigned, max_total, pcmk__plural_s(max_total));
for (iter = instances; iter != NULL; iter = iter->next) {
instance = (pcmk_resource_t *) iter->data;
if (!pcmk_is_set(instance->flags, pcmk__rsc_unassigned)) {
continue; // Already assigned
}
if (instance->priv->active_nodes != NULL) {
current = pcmk__current_node(instance);
if (pcmk__top_allowed_node(instance, current) == NULL) {
const char *unmanaged = "";
if (!pcmk_is_set(instance->flags, pcmk__rsc_managed)) {
unmanaged = "Unmanaged resource ";
}
crm_notice("%s%s is running on %s which is no longer allowed",
unmanaged, instance->id, pcmk__node_name(current));
}
}
if (assigned >= max_total) {
pcmk__rsc_debug(collective,
"Not assigning %s because maximum %d instances "
"already assigned",
instance->id, max_total);
resource_location(instance, NULL, -PCMK_SCORE_INFINITY,
"collective_limit_reached",
collective->priv->scheduler);
} else if (assign_instance(instance, NULL, max_per_node) != NULL) {
assigned++;
}
}
pcmk__rsc_debug(collective, "Assigned %d of %d possible instance%s of %s",
assigned, max_total, pcmk__plural_s(max_total),
collective->id);
}
enum instance_state {
instance_starting = (1 << 0),
instance_stopping = (1 << 1),
/* This indicates that some instance is restarting. It's not the same as
* instance_starting|instance_stopping, which would indicate that some
* instance is starting, and some instance (not necessarily the same one) is
* stopping.
*/
instance_restarting = (1 << 2),
instance_active = (1 << 3),
instance_all = instance_starting|instance_stopping
|instance_restarting|instance_active,
};
/*!
* \internal
* \brief Check whether an instance is active, starting, and/or stopping
*
* \param[in] instance Clone instance or bundle replica container
* \param[in,out] state Whether any instance is starting, stopping, etc.
*/
static void
check_instance_state(const pcmk_resource_t *instance, uint32_t *state)
{
const GList *iter = NULL;
uint32_t instance_state = 0; // State of just this instance
// No need to check further if all conditions have already been detected
if (pcmk_all_flags_set(*state, instance_all)) {
return;
}
// If instance is a collective (a cloned group), check its children instead
if (instance->priv->variant > pcmk__rsc_variant_primitive) {
for (iter = instance->priv->children;
(iter != NULL) && !pcmk_all_flags_set(*state, instance_all);
iter = iter->next) {
check_instance_state((const pcmk_resource_t *) iter->data, state);
}
return;
}
// If we get here, instance is a primitive
if (instance->priv->active_nodes != NULL) {
instance_state |= instance_active;
}
// Check each of the instance's actions for runnable start or stop
for (iter = instance->priv->actions;
(iter != NULL) && !pcmk_all_flags_set(instance_state,
instance_starting
|instance_stopping);
iter = iter->next) {
const pcmk_action_t *action = (const pcmk_action_t *) iter->data;
const bool optional = pcmk_is_set(action->flags, pcmk__action_optional);
if (pcmk__str_eq(PCMK_ACTION_START, action->task, pcmk__str_none)) {
if (!optional
&& pcmk_is_set(action->flags, pcmk__action_runnable)) {
pcmk__rsc_trace(instance, "Instance is starting due to %s",
action->uuid);
instance_state |= instance_starting;
} else {
pcmk__rsc_trace(instance, "%s doesn't affect %s state (%s)",
action->uuid, instance->id,
(optional? "optional" : "unrunnable"));
}
} else if (pcmk__str_eq(PCMK_ACTION_STOP, action->task,
pcmk__str_none)) {
/* Only stop actions can be pseudo-actions for primitives. That
* indicates that the node they are on is being fenced, so the stop
* is implied rather than actually executed.
*/
if (!optional
&& pcmk_any_flags_set(action->flags, pcmk__action_pseudo
|pcmk__action_runnable)) {
pcmk__rsc_trace(instance, "Instance is stopping due to %s",
action->uuid);
instance_state |= instance_stopping;
} else {
pcmk__rsc_trace(instance, "%s doesn't affect %s state (%s)",
action->uuid, instance->id,
(optional? "optional" : "unrunnable"));
}
}
}
if (pcmk_all_flags_set(instance_state,
instance_starting|instance_stopping)) {
instance_state |= instance_restarting;
}
*state |= instance_state;
}
/*!
* \internal
* \brief Create actions for collective resource instances
*
* \param[in,out] collective Clone or bundle resource to create actions for
* \param[in,out] instances List of clone instances or bundle containers
*/
void
pcmk__create_instance_actions(pcmk_resource_t *collective, GList *instances)
{
uint32_t state = 0;
pcmk_action_t *stop = NULL;
pcmk_action_t *stopped = NULL;
pcmk_action_t *start = NULL;
pcmk_action_t *started = NULL;
pcmk__rsc_trace(collective, "Creating collective instance actions for %s",
collective->id);
// Create actions for each instance appropriate to its variant
for (GList *iter = instances; iter != NULL; iter = iter->next) {
pcmk_resource_t *instance = (pcmk_resource_t *) iter->data;
instance->priv->cmds->create_actions(instance);
check_instance_state(instance, &state);
}
// Create pseudo-actions for rsc start and started
start = pe__new_rsc_pseudo_action(collective, PCMK_ACTION_START,
!pcmk_is_set(state, instance_starting),
true);
started = pe__new_rsc_pseudo_action(collective, PCMK_ACTION_RUNNING,
!pcmk_is_set(state, instance_starting),
false);
started->priority = PCMK_SCORE_INFINITY;
if (pcmk_any_flags_set(state, instance_active|instance_starting)) {
pcmk__set_action_flags(started, pcmk__action_runnable);
}
// Create pseudo-actions for rsc stop and stopped
stop = pe__new_rsc_pseudo_action(collective, PCMK_ACTION_STOP,
!pcmk_is_set(state, instance_stopping),
true);
stopped = pe__new_rsc_pseudo_action(collective, PCMK_ACTION_STOPPED,
!pcmk_is_set(state, instance_stopping),
true);
stopped->priority = PCMK_SCORE_INFINITY;
if (!pcmk_is_set(state, instance_restarting)) {
pcmk__set_action_flags(stop, pcmk__action_migratable);
}
if (pcmk__is_clone(collective)) {
pe__create_clone_notif_pseudo_ops(collective, start, started, stop,
stopped);
}
}
/*!
* \internal
* \brief Get a list of clone instances or bundle replica containers
*
* \param[in] rsc Clone or bundle resource
*
* \return Clone instances if \p rsc is a clone, or a newly created list of
* \p rsc's replica containers if \p rsc is a bundle
* \note The caller must call free_instance_list() on the result when the list
* is no longer needed.
*/
static inline GList *
get_instance_list(const pcmk_resource_t *rsc)
{
if (pcmk__is_bundle(rsc)) {
return pe__bundle_containers(rsc);
} else {
return rsc->priv->children;
}
}
/*!
* \internal
* \brief Free any memory created by get_instance_list()
*
* \param[in] rsc Clone or bundle resource passed to get_instance_list()
* \param[in,out] list Return value of get_instance_list() for \p rsc
*/
static inline void
free_instance_list(const pcmk_resource_t *rsc, GList *list)
{
if (list != rsc->priv->children) {
g_list_free(list);
}
}
/*!
* \internal
* \brief Check whether an instance is compatible with a role and node
*
* \param[in] instance Clone instance or bundle replica container
* \param[in] node Instance must match this node
* \param[in] role If not pcmk_role_unknown, instance must match this role
* \param[in] current If true, compare instance's original node and role,
* otherwise compare assigned next node and role
*
* \return true if \p instance is compatible with \p node and \p role,
* otherwise false
*/
bool
pcmk__instance_matches(const pcmk_resource_t *instance, const pcmk_node_t *node,
enum rsc_role_e role, bool current)
{
pcmk_node_t *instance_node = NULL;
CRM_CHECK((instance != NULL) && (node != NULL), return false);
if ((role != pcmk_role_unknown)
&& (role != instance->priv->fns->state(instance, current))) {
pcmk__rsc_trace(instance,
"%s is not a compatible instance (role is not %s)",
instance->id, pcmk_role_text(role));
return false;
}
if (!is_set_recursive(instance, pcmk__rsc_blocked, true)) {
uint32_t target = pcmk__rsc_node_assigned;
if (current) {
target = pcmk__rsc_node_current;
}
// We only want instances that haven't failed
instance_node = instance->priv->fns->location(instance, NULL, target);
}
if (instance_node == NULL) {
pcmk__rsc_trace(instance,
"%s is not a compatible instance "
"(not assigned to a node)",
instance->id);
return false;
}
if (!pcmk__same_node(instance_node, node)) {
pcmk__rsc_trace(instance,
"%s is not a compatible instance "
"(assigned to %s not %s)",
instance->id, pcmk__node_name(instance_node),
pcmk__node_name(node));
return false;
}
return true;
}
#define display_role(r) \
(((r) == pcmk_role_unknown)? "matching" : pcmk_role_text(r))
/*!
* \internal
* \brief Find an instance that matches a given resource by node and role
*
* \param[in] match_rsc Resource that instance must match (for logging only)
* \param[in] rsc Clone or bundle resource to check for matching instance
* \param[in] node Instance must match this node
* \param[in] role If not pcmk_role_unknown, instance must match this role
* \param[in] current If true, compare instance's original node and role,
* otherwise compare assigned next node and role
*
* \return \p rsc instance matching \p node and \p role if any, otherwise NULL
*/
static pcmk_resource_t *
find_compatible_instance_on_node(const pcmk_resource_t *match_rsc,
const pcmk_resource_t *rsc,
const pcmk_node_t *node, enum rsc_role_e role,
bool current)
{
GList *instances = NULL;
instances = get_instance_list(rsc);
for (GList *iter = instances; iter != NULL; iter = iter->next) {
pcmk_resource_t *instance = (pcmk_resource_t *) iter->data;
if (pcmk__instance_matches(instance, node, role, current)) {
pcmk__rsc_trace(match_rsc,
"Found %s %s instance %s compatible with %s on %s",
display_role(role), rsc->id, instance->id,
match_rsc->id, pcmk__node_name(node));
free_instance_list(rsc, instances); // Only frees list, not contents
return instance;
}
}
free_instance_list(rsc, instances);
pcmk__rsc_trace(match_rsc,
"No %s %s instance found compatible with %s on %s",
display_role(role), rsc->id, match_rsc->id,
pcmk__node_name(node));
return NULL;
}
/*!
* \internal
* \brief Find a clone instance or bundle container compatible with a resource
*
* \param[in] match_rsc Resource that instance must match
* \param[in] rsc Clone or bundle resource to check for matching instance
* \param[in] role If not pcmk_role_unknown, instance must match this role
* \param[in] current If true, compare instance's original node and role,
* otherwise compare assigned next node and role
*
* \return Compatible (by \p role and \p match_rsc location) instance of \p rsc
* if any, otherwise NULL
*/
pcmk_resource_t *
pcmk__find_compatible_instance(const pcmk_resource_t *match_rsc,
const pcmk_resource_t *rsc, enum rsc_role_e role,
bool current)
{
pcmk_resource_t *instance = NULL;
GList *nodes = NULL;
const pcmk_node_t *node = NULL;
GHashTable *allowed_nodes = match_rsc->priv->allowed_nodes;
uint32_t target = pcmk__rsc_node_assigned;
if (current) {
target = pcmk__rsc_node_current;
}
// If match_rsc has a node, check only that node
node = match_rsc->priv->fns->location(match_rsc, NULL, target);
if (node != NULL) {
return find_compatible_instance_on_node(match_rsc, rsc, node, role,
current);
}
// Otherwise check for an instance matching any of match_rsc's allowed nodes
nodes = pcmk__sort_nodes(g_hash_table_get_values(allowed_nodes), NULL);
for (GList *iter = nodes; (iter != NULL) && (instance == NULL);
iter = iter->next) {
instance = find_compatible_instance_on_node(match_rsc, rsc,
(pcmk_node_t *) iter->data,
role, current);
}
if (instance == NULL) {
pcmk__rsc_debug(rsc, "No %s instance found compatible with %s",
rsc->id, match_rsc->id);
}
g_list_free(nodes);
return instance;
}
/*!
* \internal
* \brief Unassign an instance if mandatory ordering has no interleave match
*
* \param[in] first 'First' action in an ordering
* \param[in] then 'Then' action in an ordering
* \param[in,out] then_instance 'Then' instance that has no interleave match
* \param[in] type Group of enum pcmk__action_relation_flags
* \param[in] current If true, "then" action is stopped or demoted
*
* \return true if \p then_instance was unassigned, otherwise false
*/
static bool
unassign_if_mandatory(const pcmk_action_t *first, const pcmk_action_t *then,
pcmk_resource_t *then_instance, uint32_t type,
bool current)
{
// Allow "then" instance to go down even without an interleave match
if (current) {
pcmk__rsc_trace(then->rsc,
"%s has no instance to order before stopping "
"or demoting %s",
first->rsc->id, then_instance->id);
/* If the "first" action must be runnable, but there is no "first"
* instance, the "then" instance must not be allowed to come up.
*/
} else if (pcmk_any_flags_set(type, pcmk__ar_unrunnable_first_blocks
|pcmk__ar_first_implies_then)) {
pcmk__rsc_info(then->rsc,
"Inhibiting %s from being active "
"because there is no %s instance to interleave",
then_instance->id, first->rsc->id);
return pcmk__assign_resource(then_instance, NULL, true, true);
}
return false;
}
/*!
* \internal
* \brief Find first matching action for a clone instance or bundle container
*
* \param[in] action Action in an interleaved ordering
* \param[in] instance Clone instance or bundle container being interleaved
* \param[in] action_name Action to look for
* \param[in] node If not NULL, require action to be on this node
* \param[in] for_first If true, \p instance is the 'first' resource in the
* ordering, otherwise it is the 'then' resource
*
* \return First action for \p instance (or in some cases if \p instance is a
* bundle container, its containerized resource) that matches
* \p action_name and \p node if any, otherwise NULL
*/
static pcmk_action_t *
find_instance_action(const pcmk_action_t *action, const pcmk_resource_t *instance,
const char *action_name, const pcmk_node_t *node,
bool for_first)
{
const pcmk_resource_t *rsc = NULL;
pcmk_action_t *matching_action = NULL;
/* If instance is a bundle container, sometimes we should interleave the
* action for the container itself, and sometimes for the containerized
* resource.
*
* For example, given "start bundle A then bundle B", B likely requires the
* service inside A's container to be active, rather than just the
* container, so we should interleave the action for A's containerized
* resource. On the other hand, it's possible B's container itself requires
* something from A, so we should interleave the action for B's container.
*
* Essentially, for 'first', we should use the containerized resource for
* everything except stop, and for 'then', we should use the container for
* everything except promote and demote (which can only be performed on the
* containerized resource).
*/
if ((for_first && !pcmk__str_any_of(action->task, PCMK_ACTION_STOP,
PCMK_ACTION_STOPPED, NULL))
|| (!for_first && pcmk__str_any_of(action->task, PCMK_ACTION_PROMOTE,
PCMK_ACTION_PROMOTED,
PCMK_ACTION_DEMOTE,
PCMK_ACTION_DEMOTED, NULL))) {
rsc = pe__get_rsc_in_container(instance);
}
if (rsc == NULL) {
rsc = instance; // No containerized resource, use instance itself
} else {
node = NULL; // Containerized actions are on bundle-created guest
}
matching_action = find_first_action(rsc->priv->actions, NULL,
action_name, node);
if (matching_action != NULL) {
return matching_action;
}
if (pcmk_is_set(instance->flags, pcmk__rsc_removed)
|| pcmk__str_any_of(action_name, PCMK_ACTION_STOP, PCMK_ACTION_DEMOTE,
NULL)) {
crm_trace("No %s action found for %s%s",
action_name,
pcmk_is_set(instance->flags, pcmk__rsc_removed)? "orphan " : "",
instance->id);
} else {
crm_err("No %s action found for %s to interleave (bug?)",
action_name, instance->id);
}
return NULL;
}
/*!
* \internal
* \brief Get the original action name of a bundle or clone action
*
* Given an action for a bundle or clone, get the original action name,
* mapping notify to the action being notified, and if the instances are
* primitives, mapping completion actions to the action that was completed
* (for example, stopped to stop).
*
* \param[in] action Clone or bundle action to check
*
* \return Original action name for \p action
*/
static const char *
orig_action_name(const pcmk_action_t *action)
{
// Any instance will do
const pcmk_resource_t *instance = action->rsc->priv->children->data;
char *action_type = NULL;
const char *action_name = action->task;
enum pcmk__action_type orig_task = pcmk__action_unspecified;
if (pcmk__strcase_any_of(action->task, PCMK_ACTION_NOTIFY,
PCMK_ACTION_NOTIFIED, NULL)) {
// action->uuid is RSC_(confirmed-){pre,post}_notify_ACTION_INTERVAL
CRM_CHECK(parse_op_key(action->uuid, NULL, &action_type, NULL),
return pcmk__action_text(pcmk__action_unspecified));
action_name = strstr(action_type, "_notify_");
CRM_CHECK(action_name != NULL,
return pcmk__action_text(pcmk__action_unspecified));
action_name += strlen("_notify_");
}
orig_task = get_complex_task(instance, action_name);
free(action_type);
return pcmk__action_text(orig_task);
}
/*!
* \internal
* \brief Update two interleaved actions according to an ordering between them
*
* Given information about an ordering of two interleaved actions, update the
* actions' flags (and runnable_before members if appropriate) as appropriate
* for the ordering. Effects may cascade to other orderings involving the
* actions as well.
*
* \param[in,out] first 'First' action in an ordering
* \param[in,out] then 'Then' action in an ordering
* \param[in] node If not NULL, limit scope of ordering to this node
* \param[in] filter Action flags to limit scope of certain updates (may
* include pcmk__action_optional to affect only
* mandatory actions, and pcmk__action_runnable to
* affect only runnable actions)
* \param[in] type Group of enum pcmk__action_relation_flags to apply
*
* \return Group of enum pcmk__updated flags indicating what was updated
*/
static uint32_t
update_interleaved_actions(pcmk_action_t *first, pcmk_action_t *then,
const pcmk_node_t *node, uint32_t filter,
uint32_t type)
{
GList *instances = NULL;
uint32_t changed = pcmk__updated_none;
const char *orig_first_task = orig_action_name(first);
// Stops and demotes must be interleaved with instance on current node
bool current = pcmk__ends_with(first->uuid, "_" PCMK_ACTION_STOPPED "_0")
|| pcmk__ends_with(first->uuid,
"_" PCMK_ACTION_DEMOTED "_0");
// Update the specified actions for each "then" instance individually
instances = get_instance_list(then->rsc);
for (GList *iter = instances; iter != NULL; iter = iter->next) {
pcmk_resource_t *first_instance = NULL;
pcmk_resource_t *then_instance = iter->data;
pcmk_action_t *first_action = NULL;
pcmk_action_t *then_action = NULL;
// Find a "first" instance to interleave with this "then" instance
first_instance = pcmk__find_compatible_instance(then_instance,
first->rsc,
pcmk_role_unknown,
current);
if (first_instance == NULL) { // No instance can be interleaved
if (unassign_if_mandatory(first, then, then_instance, type,
current)) {
pcmk__set_updated_flags(changed, first, pcmk__updated_then);
}
continue;
}
first_action = find_instance_action(first, first_instance,
orig_first_task, node, true);
if (first_action == NULL) {
continue;
}
then_action = find_instance_action(then, then_instance, then->task,
node, false);
if (then_action == NULL) {
continue;
}
if (order_actions(first_action, then_action, type)) {
pcmk__set_updated_flags(changed, first,
pcmk__updated_first|pcmk__updated_then);
}
changed |= then_instance->priv->cmds->update_ordered_actions(
first_action, then_action, node,
first_instance->priv->cmds->action_flags(first_action, node),
filter, type, then->rsc->priv->scheduler);
}
free_instance_list(then->rsc, instances);
return changed;
}
/*!
* \internal
* \brief Check whether two actions in an ordering can be interleaved
*
* \param[in] first 'First' action in the ordering
* \param[in] then 'Then' action in the ordering
*
* \return true if \p first and \p then can be interleaved, otherwise false
*/
static bool
can_interleave_actions(const pcmk_action_t *first, const pcmk_action_t *then)
{
bool interleave = false;
pcmk_resource_t *rsc = NULL;
if ((first->rsc == NULL) || (then->rsc == NULL)) {
crm_trace("Not interleaving %s with %s: not resource actions",
first->uuid, then->uuid);
return false;
}
if (first->rsc == then->rsc) {
crm_trace("Not interleaving %s with %s: same resource",
first->uuid, then->uuid);
return false;
}
if ((first->rsc->priv->variant < pcmk__rsc_variant_clone)
|| (then->rsc->priv->variant < pcmk__rsc_variant_clone)) {
crm_trace("Not interleaving %s with %s: not clones or bundles",
first->uuid, then->uuid);
return false;
}
if (pcmk__ends_with(then->uuid, "_stop_0")
|| pcmk__ends_with(then->uuid, "_demote_0")) {
rsc = first->rsc;
} else {
rsc = then->rsc;
}
interleave = crm_is_true(g_hash_table_lookup(rsc->priv->meta,
PCMK_META_INTERLEAVE));
pcmk__rsc_trace(rsc, "'%s then %s' will %sbe interleaved (based on %s)",
first->uuid, then->uuid, (interleave? "" : "not "),
rsc->id);
return interleave;
}
/*!
* \internal
* \brief Update non-interleaved instance actions according to an ordering
*
* Given information about an ordering of two non-interleaved actions, update
* the actions' flags (and runnable_before members if appropriate) as
* appropriate for the ordering. Effects may cascade to other orderings
* involving the actions as well.
*
* \param[in,out] instance Clone instance or bundle container
* \param[in,out] first "First" action in ordering
* \param[in] then "Then" action in ordering (for \p instance's parent)
* \param[in] node If not NULL, limit scope of ordering to this node
* \param[in] flags Action flags for \p first for ordering purposes
* \param[in] filter Action flags to limit scope of certain updates (may
* include pcmk__action_optional to affect only
* mandatory actions, and pcmk__action_runnable to
* affect only runnable actions)
* \param[in] type Group of enum pcmk__action_relation_flags to apply
*
* \return Group of enum pcmk__updated flags indicating what was updated
*/
static uint32_t
update_noninterleaved_actions(pcmk_resource_t *instance, pcmk_action_t *first,
const pcmk_action_t *then, const pcmk_node_t *node,
uint32_t flags, uint32_t filter, uint32_t type)
{
pcmk_action_t *instance_action = NULL;
pcmk_scheduler_t *scheduler = instance->priv->scheduler;
uint32_t instance_flags = 0;
uint32_t changed = pcmk__updated_none;
// Check whether instance has an equivalent of "then" action
instance_action = find_first_action(instance->priv->actions, NULL,
then->task, node);
if (instance_action == NULL) {
return changed;
}
// Check whether action is runnable
instance_flags = instance->priv->cmds->action_flags(instance_action, node);
if (!pcmk_is_set(instance_flags, pcmk__action_runnable)) {
return changed;
}
// If so, update actions for the instance
changed = instance->priv->cmds->update_ordered_actions(first,
instance_action,
node, flags, filter,
type, scheduler);
// Propagate any changes to later actions
if (pcmk_is_set(changed, pcmk__updated_then)) {
for (GList *after_iter = instance_action->actions_after;
after_iter != NULL; after_iter = after_iter->next) {
pcmk__related_action_t *after = after_iter->data;
pcmk__update_action_for_orderings(after->action, scheduler);
}
}
return changed;
}
/*!
* \internal
* \brief Update two actions according to an ordering between them
*
* Given information about an ordering of two clone or bundle actions, update
* the actions' flags (and runnable_before members if appropriate) as
* appropriate for the ordering. Effects may cascade to other orderings
* involving the actions as well.
*
* \param[in,out] first 'First' action in an ordering
* \param[in,out] then 'Then' action in an ordering
* \param[in] node If not NULL, limit scope of ordering to this node
* (only used when interleaving instances)
* \param[in] flags Action flags for \p first for ordering purposes
* \param[in] filter Action flags to limit scope of certain updates (may
* include pcmk__action_optional to affect only
* mandatory actions, and pcmk__action_runnable to
* affect only runnable actions)
* \param[in] type Group of enum pcmk__action_relation_flags to apply
* \param[in,out] scheduler Scheduler data
*
* \return Group of enum pcmk__updated flags indicating what was updated
*/
uint32_t
pcmk__instance_update_ordered_actions(pcmk_action_t *first, pcmk_action_t *then,
const pcmk_node_t *node, uint32_t flags,
uint32_t filter, uint32_t type,
pcmk_scheduler_t *scheduler)
{
pcmk__assert((first != NULL) && (then != NULL) && (scheduler != NULL));
if (then->rsc == NULL) {
return pcmk__updated_none;
} else if (can_interleave_actions(first, then)) {
return update_interleaved_actions(first, then, node, filter, type);
} else {
uint32_t changed = pcmk__updated_none;
GList *instances = get_instance_list(then->rsc);
// Update actions for the clone or bundle resource itself
changed |= pcmk__update_ordered_actions(first, then, node, flags,
filter, type, scheduler);
// Update the 'then' clone instances or bundle containers individually
for (GList *iter = instances; iter != NULL; iter = iter->next) {
pcmk_resource_t *instance = iter->data;
changed |= update_noninterleaved_actions(instance, first, then,
node, flags, filter, type);
}
free_instance_list(then->rsc, instances);
return changed;
}
}
#define pe__clear_action_summary_flags(flags, action, flag) do { \
flags = pcmk__clear_flags_as(__func__, __LINE__, LOG_TRACE, \
"Action summary", action->rsc->id, \
flags, flag, #flag); \
} while (0)
/*!
* \internal
* \brief Return action flags for a given clone or bundle action
*
* \param[in,out] action Action for a clone or bundle
* \param[in] instances Clone instances or bundle containers
* \param[in] node If not NULL, limit effects to this node
*
* \return Flags appropriate to \p action on \p node
*/
uint32_t
pcmk__collective_action_flags(pcmk_action_t *action, const GList *instances,
const pcmk_node_t *node)
{
bool any_runnable = false;
const char *action_name = orig_action_name(action);
// Set original assumptions (optional and runnable may be cleared below)
uint32_t flags = pcmk__action_optional
|pcmk__action_runnable
|pcmk__action_pseudo;
for (const GList *iter = instances; iter != NULL; iter = iter->next) {
const pcmk_resource_t *instance = iter->data;
const pcmk_node_t *instance_node = NULL;
pcmk_action_t *instance_action = NULL;
uint32_t instance_flags;
// Node is relevant only to primitive instances
if (pcmk__is_primitive(instance)) {
instance_node = node;
}
instance_action = find_first_action(instance->priv->actions, NULL,
action_name, instance_node);
if (instance_action == NULL) {
pcmk__rsc_trace(action->rsc, "%s has no %s action on %s",
instance->id, action_name, pcmk__node_name(node));
continue;
}
pcmk__rsc_trace(action->rsc, "%s has %s for %s on %s",
instance->id, instance_action->uuid, action_name,
pcmk__node_name(node));
instance_flags = instance->priv->cmds->action_flags(instance_action,
node);
// If any instance action is mandatory, so is the collective action
if (pcmk_is_set(flags, pcmk__action_optional)
&& !pcmk_is_set(instance_flags, pcmk__action_optional)) {
pcmk__rsc_trace(instance, "%s is mandatory because %s is",
action->uuid, instance_action->uuid);
pe__clear_action_summary_flags(flags, action,
pcmk__action_optional);
pcmk__clear_action_flags(action, pcmk__action_optional);
}
// If any instance action is runnable, so is the collective action
if (pcmk_is_set(instance_flags, pcmk__action_runnable)) {
any_runnable = true;
}
}
if (!any_runnable) {
pcmk__rsc_trace(action->rsc,
"%s is not runnable because no instance can run %s",
action->uuid, action_name);
pe__clear_action_summary_flags(flags, action, pcmk__action_runnable);
if (node == NULL) {
pcmk__clear_action_flags(action, pcmk__action_runnable);
}
}
return flags;
}
diff --git a/lib/pacemaker/pcmk_sched_nodes.c b/lib/pacemaker/pcmk_sched_nodes.c
index 503215c22b..544a8ced6b 100644
--- a/lib/pacemaker/pcmk_sched_nodes.c
+++ b/lib/pacemaker/pcmk_sched_nodes.c
@@ -1,445 +1,446 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU General Public License version 2
* or later (GPLv2+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <crm/common/xml.h>
#include <crm/common/xml_internal.h>
#include <pacemaker-internal.h>
#include <pacemaker.h>
#include "libpacemaker_private.h"
/*!
* \internal
* \brief Check whether a node is available to run resources
*
* \param[in] node Node to check
* \param[in] consider_score If true, consider a negative score unavailable
* \param[in] consider_guest If true, consider a guest node unavailable whose
* resource will not be active
*
* \return true if node is online and not shutting down, unclean, or in standby
* or maintenance mode, otherwise false
*/
bool
pcmk__node_available(const pcmk_node_t *node, bool consider_score,
bool consider_guest)
{
if ((node == NULL) || (node->details == NULL) || !node->details->online
|| node->details->shutdown || node->details->unclean
|| pcmk_is_set(node->priv->flags, pcmk__node_standby)
|| node->details->maintenance) {
return false;
}
if (consider_score && (node->assign->score < 0)) {
return false;
}
// @TODO Go through all callers to see which should set consider_guest
if (consider_guest && pcmk__is_guest_or_bundle_node(node)) {
pcmk_resource_t *guest = node->priv->remote->priv->launcher;
if (guest->priv->fns->location(guest, NULL,
pcmk__rsc_node_assigned) == NULL) {
return false;
}
}
return true;
}
/*!
* \internal
- * \brief Copy a hash table of node objects
+ * \brief Create a hash table with copies of another table's nodes
*
* \param[in] nodes Hash table to copy
*
- * \return New copy of nodes (or NULL if nodes is NULL)
+ * \return New table with copies of nodes in \p nodes, or \c NULL if \p nodes is
+ * \c NULL
*/
GHashTable *
pcmk__copy_node_table(GHashTable *nodes)
{
GHashTable *new_table = NULL;
GHashTableIter iter;
pcmk_node_t *node = NULL;
if (nodes == NULL) {
return NULL;
}
- new_table = pcmk__strkey_table(NULL, free);
+ new_table = pcmk__strkey_table(NULL, pcmk__free_node_copy);
g_hash_table_iter_init(&iter, nodes);
while (g_hash_table_iter_next(&iter, NULL, (gpointer *) &node)) {
pcmk_node_t *new_node = pe__copy_node(node);
g_hash_table_insert(new_table, (gpointer) new_node->priv->id,
new_node);
}
return new_table;
}
/*!
* \internal
* \brief Free a table of node tables
*
* \param[in,out] data Table to free
*
* \note This is a \c GDestroyNotify wrapper for \c g_hash_table_destroy().
*/
static void
destroy_node_tables(gpointer data)
{
g_hash_table_destroy((GHashTable *) data);
}
/*!
* \internal
* \brief Recursively copy the node tables of a resource
*
* Build a hash table containing copies of the allowed nodes tables of \p rsc
* and its entire tree of descendants. The key is the resource ID, and the value
* is a copy of the resource's node table.
*
* \param[in] rsc Resource whose node table to copy
* \param[in,out] copy Where to store the copied node tables
*
* \note \p *copy should be \c NULL for the top-level call.
* \note The caller is responsible for freeing \p copy using
* \c g_hash_table_destroy().
*/
void
pcmk__copy_node_tables(const pcmk_resource_t *rsc, GHashTable **copy)
{
pcmk__assert((rsc != NULL) && (copy != NULL));
if (*copy == NULL) {
*copy = pcmk__strkey_table(NULL, destroy_node_tables);
}
g_hash_table_insert(*copy, rsc->id,
pcmk__copy_node_table(rsc->priv->allowed_nodes));
for (const GList *iter = rsc->priv->children;
iter != NULL; iter = iter->next) {
pcmk__copy_node_tables((const pcmk_resource_t *) iter->data, copy);
}
}
/*!
* \internal
* \brief Recursively restore the node tables of a resource from backup
*
* Given a hash table containing backup copies of the allowed nodes tables of
* \p rsc and its entire tree of descendants, replace the resources' current
* node tables with the backed-up copies.
*
* \param[in,out] rsc Resource whose node tables to restore
* \param[in] backup Table of backup node tables (created by
* \c pcmk__copy_node_tables())
*
* \note This function frees the resources' current node tables.
*/
void
pcmk__restore_node_tables(pcmk_resource_t *rsc, GHashTable *backup)
{
pcmk__assert((rsc != NULL) && (backup != NULL));
g_hash_table_destroy(rsc->priv->allowed_nodes);
// Copy to avoid danger with multiple restores
rsc->priv->allowed_nodes =
pcmk__copy_node_table(g_hash_table_lookup(backup, rsc->id));
for (GList *iter = rsc->priv->children;
iter != NULL; iter = iter->next) {
pcmk__restore_node_tables((pcmk_resource_t *) iter->data, backup);
}
}
/*!
* \internal
* \brief Copy a list of node objects
*
* \param[in] list List to copy
* \param[in] reset Set copies' scores to 0
*
* \return New list of shallow copies of nodes in original list
*/
GList *
pcmk__copy_node_list(const GList *list, bool reset)
{
GList *result = NULL;
for (const GList *iter = list; iter != NULL; iter = iter->next) {
pcmk_node_t *new_node = NULL;
pcmk_node_t *this_node = iter->data;
new_node = pe__copy_node(this_node);
if (reset) {
new_node->assign->score = 0;
}
result = g_list_prepend(result, new_node);
}
return result;
}
/*!
* \internal
* \brief Compare two nodes for assignment preference
*
* Given two nodes, check which one is more preferred by assignment criteria
* such as node score and utilization.
*
* \param[in] a First node to compare
* \param[in] b Second node to compare
* \param[in] data Node to prefer if all else equal
*
* \return -1 if \p a is preferred, +1 if \p b is preferred, or 0 if they are
* equally preferred
*/
static gint
compare_nodes(gconstpointer a, gconstpointer b, gpointer data)
{
const pcmk_node_t *node1 = (const pcmk_node_t *) a;
const pcmk_node_t *node2 = (const pcmk_node_t *) b;
const pcmk_node_t *preferred = (const pcmk_node_t *) data;
int node1_score = -PCMK_SCORE_INFINITY;
int node2_score = -PCMK_SCORE_INFINITY;
int result = 0;
if (a == NULL) {
return 1;
}
if (b == NULL) {
return -1;
}
// Compare node scores
if (pcmk__node_available(node1, false, false)) {
node1_score = node1->assign->score;
}
if (pcmk__node_available(node2, false, false)) {
node2_score = node2->assign->score;
}
if (node1_score > node2_score) {
crm_trace("%s before %s (score %d > %d)",
pcmk__node_name(node1), pcmk__node_name(node2),
node1_score, node2_score);
return -1;
}
if (node1_score < node2_score) {
crm_trace("%s after %s (score %d < %d)",
pcmk__node_name(node1), pcmk__node_name(node2),
node1_score, node2_score);
return 1;
}
// If appropriate, compare node utilization
if (pcmk__str_eq(node1->priv->scheduler->priv->placement_strategy,
PCMK_VALUE_MINIMAL, pcmk__str_casei)) {
goto equal;
}
if (pcmk__str_eq(node1->priv->scheduler->priv->placement_strategy,
PCMK_VALUE_BALANCED, pcmk__str_casei)) {
result = pcmk__compare_node_capacities(node1, node2);
if (result < 0) {
crm_trace("%s before %s (greater capacity by %d attributes)",
pcmk__node_name(node1), pcmk__node_name(node2),
result * -1);
return -1;
} else if (result > 0) {
crm_trace("%s after %s (lower capacity by %d attributes)",
pcmk__node_name(node1), pcmk__node_name(node2), result);
return 1;
}
}
// Compare number of resources already assigned to node
if (node1->priv->num_resources < node2->priv->num_resources) {
crm_trace("%s before %s (%d resources < %d)",
pcmk__node_name(node1), pcmk__node_name(node2),
node1->priv->num_resources, node2->priv->num_resources);
return -1;
} else if (node1->priv->num_resources > node2->priv->num_resources) {
crm_trace("%s after %s (%d resources > %d)",
pcmk__node_name(node1), pcmk__node_name(node2),
node1->priv->num_resources, node2->priv->num_resources);
return 1;
}
// Check whether one node is already running desired resource
if (preferred != NULL) {
if (pcmk__same_node(preferred, node1)) {
crm_trace("%s before %s (preferred node)",
pcmk__node_name(node1), pcmk__node_name(node2));
return -1;
} else if (pcmk__same_node(preferred, node2)) {
crm_trace("%s after %s (not preferred node)",
pcmk__node_name(node1), pcmk__node_name(node2));
return 1;
}
}
// If all else is equal, prefer node with lowest-sorting name
equal:
result = strcmp(node1->priv->name, node2->priv->name);
if (result < 0) {
crm_trace("%s before %s (name)",
pcmk__node_name(node1), pcmk__node_name(node2));
return -1;
} else if (result > 0) {
crm_trace("%s after %s (name)",
pcmk__node_name(node1), pcmk__node_name(node2));
return 1;
}
crm_trace("%s == %s", pcmk__node_name(node1), pcmk__node_name(node2));
return 0;
}
/*!
* \internal
* \brief Sort a list of nodes by assigment preference
*
* \param[in,out] nodes Node list to sort
* \param[in] active_node Node where resource being assigned is active
*
* \return New head of sorted list
*/
GList *
pcmk__sort_nodes(GList *nodes, pcmk_node_t *active_node)
{
return g_list_sort_with_data(nodes, compare_nodes, active_node);
}
/*!
* \internal
* \brief Check whether any node is available to run resources
*
* \param[in] nodes Nodes to check
*
* \return true if any node in \p nodes is available to run resources,
* otherwise false
*/
bool
pcmk__any_node_available(GHashTable *nodes)
{
GHashTableIter iter;
const pcmk_node_t *node = NULL;
if (nodes == NULL) {
return false;
}
g_hash_table_iter_init(&iter, nodes);
while (g_hash_table_iter_next(&iter, NULL, (void **) &node)) {
if (pcmk__node_available(node, true, false)) {
return true;
}
}
return false;
}
/*!
* \internal
* \brief Apply node health values for all nodes in cluster
*
* \param[in,out] scheduler Scheduler data
*/
void
pcmk__apply_node_health(pcmk_scheduler_t *scheduler)
{
int base_health = 0;
enum pcmk__health_strategy strategy;
const char *strategy_str =
pcmk__cluster_option(scheduler->priv->options,
PCMK_OPT_NODE_HEALTH_STRATEGY);
strategy = pcmk__parse_health_strategy(strategy_str);
if (strategy == pcmk__health_strategy_none) {
return;
}
crm_info("Applying node health strategy '%s'", strategy_str);
// The progressive strategy can use a base health score
if (strategy == pcmk__health_strategy_progressive) {
base_health = pcmk__health_score(PCMK_OPT_NODE_HEALTH_BASE, scheduler);
}
for (GList *iter = scheduler->nodes; iter != NULL; iter = iter->next) {
pcmk_node_t *node = (pcmk_node_t *) iter->data;
int health = pe__sum_node_health_scores(node, base_health);
// An overall health score of 0 has no effect
if (health == 0) {
continue;
}
crm_info("Overall system health of %s is %d",
pcmk__node_name(node), health);
// Use node health as a location score for each resource on the node
for (GList *r = scheduler->priv->resources; r != NULL; r = r->next) {
pcmk_resource_t *rsc = (pcmk_resource_t *) r->data;
bool constrain = true;
if (health < 0) {
/* Negative health scores do not apply to resources with
* PCMK_META_ALLOW_UNHEALTHY_NODES=true.
*/
constrain = !crm_is_true(g_hash_table_lookup(rsc->priv->meta,
PCMK_META_ALLOW_UNHEALTHY_NODES));
}
if (constrain) {
pcmk__new_location(strategy_str, rsc, health, NULL, node);
} else {
pcmk__rsc_trace(rsc, "%s is immune from health ban on %s",
rsc->id, pcmk__node_name(node));
}
}
}
}
/*!
* \internal
* \brief Check for a node in a resource's parent's allowed nodes
*
* \param[in] rsc Resource whose parent should be checked
* \param[in] node Node to check for
*
* \return Equivalent of \p node from \p rsc's parent's allowed nodes if any,
* otherwise NULL
*/
pcmk_node_t *
pcmk__top_allowed_node(const pcmk_resource_t *rsc, const pcmk_node_t *node)
{
GHashTable *allowed_nodes = NULL;
if ((rsc == NULL) || (node == NULL)) {
return NULL;
}
if (rsc->priv->parent == NULL) {
allowed_nodes = rsc->priv->allowed_nodes;
} else {
allowed_nodes = rsc->priv->parent->priv->allowed_nodes;
}
return g_hash_table_lookup(allowed_nodes, node->priv->id);
}
diff --git a/lib/pacemaker/pcmk_sched_resource.c b/lib/pacemaker/pcmk_sched_resource.c
index 17487f5095..2bc843f05f 100644
--- a/lib/pacemaker/pcmk_sched_resource.c
+++ b/lib/pacemaker/pcmk_sched_resource.c
@@ -1,800 +1,800 @@
/*
* Copyright 2014-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU General Public License version 2
* or later (GPLv2+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <stdlib.h>
#include <string.h>
#include <crm/common/xml.h>
#include <pacemaker-internal.h>
#include "libpacemaker_private.h"
// Resource assignment methods by resource variant
static pcmk__assignment_methods_t assignment_methods[] = {
{
pcmk__primitive_assign,
pcmk__primitive_create_actions,
pcmk__probe_rsc_on_node,
pcmk__primitive_internal_constraints,
pcmk__primitive_apply_coloc_score,
pcmk__colocated_resources,
pcmk__with_primitive_colocations,
pcmk__primitive_with_colocations,
pcmk__add_colocated_node_scores,
pcmk__apply_location,
pcmk__primitive_action_flags,
pcmk__update_ordered_actions,
pcmk__output_resource_actions,
pcmk__add_rsc_actions_to_graph,
pcmk__primitive_add_graph_meta,
pcmk__primitive_add_utilization,
pcmk__primitive_shutdown_lock,
},
{
pcmk__group_assign,
pcmk__group_create_actions,
pcmk__probe_rsc_on_node,
pcmk__group_internal_constraints,
pcmk__group_apply_coloc_score,
pcmk__group_colocated_resources,
pcmk__with_group_colocations,
pcmk__group_with_colocations,
pcmk__group_add_colocated_node_scores,
pcmk__group_apply_location,
pcmk__group_action_flags,
pcmk__group_update_ordered_actions,
pcmk__output_resource_actions,
pcmk__add_rsc_actions_to_graph,
pcmk__noop_add_graph_meta,
pcmk__group_add_utilization,
pcmk__group_shutdown_lock,
},
{
pcmk__clone_assign,
pcmk__clone_create_actions,
pcmk__clone_create_probe,
pcmk__clone_internal_constraints,
pcmk__clone_apply_coloc_score,
pcmk__colocated_resources,
pcmk__with_clone_colocations,
pcmk__clone_with_colocations,
pcmk__add_colocated_node_scores,
pcmk__clone_apply_location,
pcmk__clone_action_flags,
pcmk__instance_update_ordered_actions,
pcmk__output_resource_actions,
pcmk__clone_add_actions_to_graph,
pcmk__clone_add_graph_meta,
pcmk__clone_add_utilization,
pcmk__clone_shutdown_lock,
},
{
pcmk__bundle_assign,
pcmk__bundle_create_actions,
pcmk__bundle_create_probe,
pcmk__bundle_internal_constraints,
pcmk__bundle_apply_coloc_score,
pcmk__colocated_resources,
pcmk__with_bundle_colocations,
pcmk__bundle_with_colocations,
pcmk__add_colocated_node_scores,
pcmk__bundle_apply_location,
pcmk__bundle_action_flags,
pcmk__instance_update_ordered_actions,
pcmk__output_bundle_actions,
pcmk__bundle_add_actions_to_graph,
pcmk__noop_add_graph_meta,
pcmk__bundle_add_utilization,
pcmk__bundle_shutdown_lock,
}
};
/*!
* \internal
* \brief Check whether a resource's agent standard, provider, or type changed
*
* \param[in,out] rsc Resource to check
* \param[in,out] node Node needing unfencing if agent changed
* \param[in] rsc_entry XML with previously known agent information
* \param[in] active_on_node Whether \p rsc is active on \p node
*
* \return true if agent for \p rsc changed, otherwise false
*/
bool
pcmk__rsc_agent_changed(pcmk_resource_t *rsc, pcmk_node_t *node,
const xmlNode *rsc_entry, bool active_on_node)
{
bool changed = false;
const char *attr_list[] = {
PCMK_XA_TYPE,
PCMK_XA_CLASS,
PCMK_XA_PROVIDER,
};
for (int i = 0; i < PCMK__NELEM(attr_list); i++) {
const char *value = crm_element_value(rsc->priv->xml, attr_list[i]);
const char *old_value = crm_element_value(rsc_entry, attr_list[i]);
if (!pcmk__str_eq(value, old_value, pcmk__str_none)) {
changed = true;
trigger_unfencing(rsc, node, "Device definition changed", NULL,
rsc->priv->scheduler);
if (active_on_node) {
crm_notice("Forcing restart of %s on %s "
"because %s changed from '%s' to '%s'",
rsc->id, pcmk__node_name(node), attr_list[i],
pcmk__s(old_value, ""), pcmk__s(value, ""));
}
}
}
if (changed && active_on_node) {
// Make sure the resource is restarted
custom_action(rsc, stop_key(rsc), PCMK_ACTION_STOP, node, FALSE,
rsc->priv->scheduler);
pcmk__set_rsc_flags(rsc, pcmk__rsc_start_pending);
}
return changed;
}
/*!
* \internal
* \brief Add resource (and any matching children) to list if it matches ID
*
* \param[in] result List to add resource to
* \param[in] rsc Resource to check
* \param[in] id ID to match
*
* \return (Possibly new) head of list
*/
static GList *
add_rsc_if_matching(GList *result, pcmk_resource_t *rsc, const char *id)
{
if (pcmk__str_eq(id, rsc->id, pcmk__str_none)
|| pcmk__str_eq(id, rsc->priv->history_id, pcmk__str_none)) {
result = g_list_prepend(result, rsc);
}
for (GList *iter = rsc->priv->children;
iter != NULL; iter = iter->next) {
pcmk_resource_t *child = (pcmk_resource_t *) iter->data;
result = add_rsc_if_matching(result, child, id);
}
return result;
}
/*!
* \internal
* \brief Find all resources matching a given ID by either ID or clone name
*
* \param[in] id Resource ID to check
* \param[in] scheduler Scheduler data
*
* \return List of all resources that match \p id
* \note The caller is responsible for freeing the return value with
* g_list_free().
*/
GList *
pcmk__rscs_matching_id(const char *id, const pcmk_scheduler_t *scheduler)
{
GList *result = NULL;
CRM_CHECK((id != NULL) && (scheduler != NULL), return NULL);
for (GList *iter = scheduler->priv->resources;
iter != NULL; iter = iter->next) {
result = add_rsc_if_matching(result, (pcmk_resource_t *) iter->data,
id);
}
return result;
}
/*!
* \internal
* \brief Set the variant-appropriate assignment methods for a resource
*
* \param[in,out] data Resource to set assignment methods for
* \param[in] user_data Ignored
*/
static void
set_assignment_methods_for_rsc(gpointer data, gpointer user_data)
{
pcmk_resource_t *rsc = data;
rsc->priv->cmds = &assignment_methods[rsc->priv->variant];
g_list_foreach(rsc->priv->children, set_assignment_methods_for_rsc,
NULL);
}
/*!
* \internal
* \brief Set the variant-appropriate assignment methods for all resources
*
* \param[in,out] scheduler Scheduler data
*/
void
pcmk__set_assignment_methods(pcmk_scheduler_t *scheduler)
{
g_list_foreach(scheduler->priv->resources, set_assignment_methods_for_rsc,
NULL);
}
/*!
* \internal
* \brief Wrapper for colocated_resources() method for readability
*
* \param[in] rsc Resource to add to colocated list
* \param[in] orig_rsc Resource originally requested
* \param[in,out] list Pointer to list to add to
*
* \return (Possibly new) head of list
*/
static inline void
add_colocated_resources(const pcmk_resource_t *rsc,
const pcmk_resource_t *orig_rsc, GList **list)
{
*list = rsc->priv->cmds->colocated_resources(rsc, orig_rsc, *list);
}
// Shared implementation of pcmk__assignment_methods_t:colocated_resources()
GList *
pcmk__colocated_resources(const pcmk_resource_t *rsc,
const pcmk_resource_t *orig_rsc,
GList *colocated_rscs)
{
const GList *iter = NULL;
GList *colocations = NULL;
if (orig_rsc == NULL) {
orig_rsc = rsc;
}
if ((rsc == NULL) || (g_list_find(colocated_rscs, rsc) != NULL)) {
return colocated_rscs;
}
pcmk__rsc_trace(orig_rsc, "%s is in colocation chain with %s",
rsc->id, orig_rsc->id);
colocated_rscs = g_list_prepend(colocated_rscs, (gpointer) rsc);
// Follow colocations where this resource is the dependent resource
colocations = pcmk__this_with_colocations(rsc);
for (iter = colocations; iter != NULL; iter = iter->next) {
const pcmk__colocation_t *constraint = iter->data;
const pcmk_resource_t *primary = constraint->primary;
if (primary == orig_rsc) {
continue; // Break colocation loop
}
if ((constraint->score == PCMK_SCORE_INFINITY) &&
(pcmk__colocation_affects(rsc, primary, constraint,
true) == pcmk__coloc_affects_location)) {
add_colocated_resources(primary, orig_rsc, &colocated_rscs);
}
}
g_list_free(colocations);
// Follow colocations where this resource is the primary resource
colocations = pcmk__with_this_colocations(rsc);
for (iter = colocations; iter != NULL; iter = iter->next) {
const pcmk__colocation_t *constraint = iter->data;
const pcmk_resource_t *dependent = constraint->dependent;
if (dependent == orig_rsc) {
continue; // Break colocation loop
}
if (pcmk__is_clone(rsc) && !pcmk__is_clone(dependent)) {
continue; // We can't be sure whether dependent will be colocated
}
if ((constraint->score == PCMK_SCORE_INFINITY) &&
(pcmk__colocation_affects(dependent, rsc, constraint,
true) == pcmk__coloc_affects_location)) {
add_colocated_resources(dependent, orig_rsc, &colocated_rscs);
}
}
g_list_free(colocations);
return colocated_rscs;
}
// No-op function for variants that don't need to implement add_graph_meta()
void
pcmk__noop_add_graph_meta(const pcmk_resource_t *rsc, xmlNode *xml)
{
}
/*!
* \internal
* \brief Output a summary of scheduled actions for a resource
*
* \param[in,out] rsc Resource to output actions for
*/
void
pcmk__output_resource_actions(pcmk_resource_t *rsc)
{
pcmk_node_t *next = NULL;
pcmk_node_t *current = NULL;
pcmk__output_t *out = NULL;
pcmk__assert(rsc != NULL);
out = rsc->priv->scheduler->priv->out;
if (rsc->priv->children != NULL) {
for (GList *iter = rsc->priv->children;
iter != NULL; iter = iter->next) {
pcmk_resource_t *child = (pcmk_resource_t *) iter->data;
child->priv->cmds->output_actions(child);
}
return;
}
next = rsc->priv->assigned_node;
if (rsc->priv->active_nodes != NULL) {
current = pcmk__current_node(rsc);
if (rsc->priv->orig_role == pcmk_role_stopped) {
/* This can occur when resources are being recovered because
* the current role can change in pcmk__primitive_create_actions()
*/
rsc->priv->orig_role = pcmk_role_started;
}
}
if ((current == NULL) && pcmk_is_set(rsc->flags, pcmk__rsc_removed)) {
/* Don't log stopped orphans */
return;
}
out->message(out, "rsc-action", rsc, current, next);
}
/*!
* \internal
* \brief Add a resource to a node's list of assigned resources
*
* \param[in,out] node Node to add resource to
* \param[in] rsc Resource to add
*/
static inline void
add_assigned_resource(pcmk_node_t *node, pcmk_resource_t *rsc)
{
node->priv->assigned_resources =
g_list_prepend(node->priv->assigned_resources, rsc);
}
/*!
* \internal
* \brief Assign a specified resource (of any variant) to a node
*
* Assign a specified resource and its children (if any) to a specified node, if
* the node can run the resource (or unconditionally, if \p force is true). Mark
* the resources as no longer provisional.
*
* If a resource can't be assigned (or \p node is \c NULL), unassign any
* previous assignment. If \p stop_if_fail is \c true, set next role to stopped
* and update any existing actions scheduled for the resource.
*
* \param[in,out] rsc Resource to assign
* \param[in,out] node Node to assign \p rsc to
* \param[in] force If true, assign to \p node even if unavailable
* \param[in] stop_if_fail If \c true and either \p rsc can't be assigned
* or \p chosen is \c NULL, set next role to
* stopped and update existing actions (if \p rsc
* is not a primitive, this applies to its
* primitive descendants instead)
*
* \return \c true if the assignment of \p rsc changed, or \c false otherwise
*
* \note Assigning a resource to the NULL node using this function is different
* from calling pcmk__unassign_resource(), in that it may also update any
* actions created for the resource.
* \note The \c pcmk__assignment_methods_t:assign() method is preferred, unless
* a resource should be assigned to the \c NULL node or every resource in
* a tree should be assigned to the same node.
* \note If \p stop_if_fail is \c false, then \c pcmk__unassign_resource() can
* completely undo the assignment. A successful assignment can be either
* undone or left alone as final. A failed assignment has the same effect
* as calling pcmk__unassign_resource(); there are no side effects on
* roles or actions.
*/
bool
pcmk__assign_resource(pcmk_resource_t *rsc, pcmk_node_t *node, bool force,
bool stop_if_fail)
{
bool changed = false;
pcmk_scheduler_t *scheduler = NULL;
pcmk__assert(rsc != NULL);
scheduler = rsc->priv->scheduler;
if (rsc->priv->children != NULL) {
for (GList *iter = rsc->priv->children;
iter != NULL; iter = iter->next) {
pcmk_resource_t *child_rsc = iter->data;
changed |= pcmk__assign_resource(child_rsc, node, force,
stop_if_fail);
}
return changed;
}
// Assigning a primitive
if (!force && (node != NULL)
&& ((node->assign->score < 0)
// Allow graph to assume that guest node connections will come up
|| (!pcmk__node_available(node, true, false)
&& !pcmk__is_guest_or_bundle_node(node)))) {
pcmk__rsc_debug(rsc,
"All nodes for resource %s are unavailable, unclean or "
"shutting down (%s can%s run resources, with score %s)",
rsc->id, pcmk__node_name(node),
(pcmk__node_available(node, true, false)? "" : "not"),
pcmk_readable_score(node->assign->score));
if (stop_if_fail) {
pe__set_next_role(rsc, pcmk_role_stopped, "node availability");
}
node = NULL;
}
if (rsc->priv->assigned_node != NULL) {
changed = !pcmk__same_node(rsc->priv->assigned_node, node);
} else {
changed = (node != NULL);
}
pcmk__unassign_resource(rsc);
pcmk__clear_rsc_flags(rsc, pcmk__rsc_unassigned);
if (node == NULL) {
char *rc_stopped = NULL;
pcmk__rsc_debug(rsc, "Could not assign %s to a node", rsc->id);
if (!stop_if_fail) {
return changed;
}
pe__set_next_role(rsc, pcmk_role_stopped, "unable to assign");
for (GList *iter = rsc->priv->actions;
iter != NULL; iter = iter->next) {
pcmk_action_t *op = (pcmk_action_t *) iter->data;
pcmk__rsc_debug(rsc, "Updating %s for %s assignment failure",
op->uuid, rsc->id);
if (pcmk__str_eq(op->task, PCMK_ACTION_STOP, pcmk__str_none)) {
pcmk__clear_action_flags(op, pcmk__action_optional);
} else if (pcmk__str_eq(op->task, PCMK_ACTION_START,
pcmk__str_none)) {
pcmk__clear_action_flags(op, pcmk__action_runnable);
} else {
// Cancel recurring actions, unless for stopped state
const char *interval_ms_s = NULL;
const char *target_rc_s = NULL;
interval_ms_s = g_hash_table_lookup(op->meta,
PCMK_META_INTERVAL);
target_rc_s = g_hash_table_lookup(op->meta,
PCMK__META_OP_TARGET_RC);
if (rc_stopped == NULL) {
rc_stopped = pcmk__itoa(PCMK_OCF_NOT_RUNNING);
}
if (!pcmk__str_eq(interval_ms_s, "0", pcmk__str_null_matches)
&& !pcmk__str_eq(rc_stopped, target_rc_s, pcmk__str_none)) {
pcmk__clear_action_flags(op, pcmk__action_runnable);
}
}
}
free(rc_stopped);
return changed;
}
pcmk__rsc_debug(rsc, "Assigning %s to %s", rsc->id, pcmk__node_name(node));
rsc->priv->assigned_node = pe__copy_node(node);
add_assigned_resource(node, rsc);
node->priv->num_resources++;
node->assign->count++;
pcmk__consume_node_capacity(node->priv->utilization, rsc);
if (pcmk_is_set(scheduler->flags, pcmk__sched_show_utilization)) {
pcmk__output_t *out = scheduler->priv->out;
out->message(out, "resource-util", rsc, node, __func__);
}
return changed;
}
/*!
* \internal
* \brief Remove any node assignment from a specified resource and its children
*
* If a specified resource has been assigned to a node, remove that assignment
* and mark the resource as provisional again.
*
* \param[in,out] rsc Resource to unassign
*
* \note This function is called recursively on \p rsc and its children.
*/
void
pcmk__unassign_resource(pcmk_resource_t *rsc)
{
pcmk_node_t *old = rsc->priv->assigned_node;
if (old == NULL) {
crm_info("Unassigning %s", rsc->id);
} else {
crm_info("Unassigning %s from %s", rsc->id, pcmk__node_name(old));
}
pcmk__set_rsc_flags(rsc, pcmk__rsc_unassigned);
if (rsc->priv->children == NULL) {
if (old == NULL) {
return;
}
rsc->priv->assigned_node = NULL;
- /* We're going to free the pcmk_node_t, but its details member is shared
- * and will remain, so update that appropriately first.
+ /* We're going to free the pcmk_node_t copy, but its priv member is
+ * shared and will remain, so update that appropriately first.
*/
old->priv->assigned_resources =
g_list_remove(old->priv->assigned_resources, rsc);
old->priv->num_resources--;
pcmk__release_node_capacity(old->priv->utilization, rsc);
- free(old);
+ pcmk__free_node_copy(old);
return;
}
for (GList *iter = rsc->priv->children;
iter != NULL; iter = iter->next) {
pcmk__unassign_resource((pcmk_resource_t *) iter->data);
}
}
/*!
* \internal
* \brief Check whether a resource has reached its migration threshold on a node
*
* \param[in,out] rsc Resource to check
* \param[in] node Node to check
* \param[out] failed If threshold has been reached, this will be set to
* resource that failed (possibly a parent of \p rsc)
*
* \return true if the migration threshold has been reached, false otherwise
*/
bool
pcmk__threshold_reached(pcmk_resource_t *rsc, const pcmk_node_t *node,
pcmk_resource_t **failed)
{
int fail_count, remaining_tries;
pcmk_resource_t *rsc_to_ban = rsc;
// Migration threshold of 0 means never force away
if (rsc->priv->ban_after_failures == 0) {
return false;
}
// If we're ignoring failures, also ignore the migration threshold
if (pcmk_is_set(rsc->flags, pcmk__rsc_ignore_failure)) {
return false;
}
// If there are no failures, there's no need to force away
fail_count = pe_get_failcount(node, rsc, NULL,
pcmk__fc_effective|pcmk__fc_launched, NULL);
if (fail_count <= 0) {
return false;
}
// If failed resource is anonymous clone instance, we'll force clone away
if (!pcmk_is_set(rsc->flags, pcmk__rsc_unique)) {
rsc_to_ban = uber_parent(rsc);
}
// How many more times recovery will be tried on this node
remaining_tries = rsc->priv->ban_after_failures - fail_count;
if (remaining_tries <= 0) {
pcmk__sched_warn(rsc->priv->scheduler,
"%s cannot run on %s due to reaching migration "
"threshold (clean up resource to allow again) "
QB_XS " failures=%d "
PCMK_META_MIGRATION_THRESHOLD "=%d",
rsc_to_ban->id, pcmk__node_name(node), fail_count,
rsc->priv->ban_after_failures);
if (failed != NULL) {
*failed = rsc_to_ban;
}
return true;
}
crm_info("%s can fail %d more time%s on "
"%s before reaching migration threshold (%d)",
rsc_to_ban->id, remaining_tries, pcmk__plural_s(remaining_tries),
pcmk__node_name(node), rsc->priv->ban_after_failures);
return false;
}
/*!
* \internal
* \brief Get a node's score
*
* \param[in] node Node with ID to check
* \param[in] nodes List of nodes to look for \p node score in
*
* \return Node's score, or -INFINITY if not found
*/
static int
get_node_score(const pcmk_node_t *node, GHashTable *nodes)
{
pcmk_node_t *found_node = NULL;
if ((node != NULL) && (nodes != NULL)) {
found_node = g_hash_table_lookup(nodes, node->priv->id);
}
if (found_node == NULL) {
return -PCMK_SCORE_INFINITY;
}
return found_node->assign->score;
}
/*!
* \internal
* \brief Compare two resources according to which should be assigned first
*
* \param[in] a First resource to compare
* \param[in] b Second resource to compare
* \param[in] data Sorted list of all nodes in cluster
*
* \return -1 if \p a should be assigned before \b, 0 if they are equal,
* or +1 if \p a should be assigned after \b
*/
static gint
cmp_resources(gconstpointer a, gconstpointer b, gpointer data)
{
/* GLib insists that this function require gconstpointer arguments, but we
* make a small, temporary change to each argument (setting the
* pe_rsc_merging flag) during comparison
*/
pcmk_resource_t *resource1 = (pcmk_resource_t *) a;
pcmk_resource_t *resource2 = (pcmk_resource_t *) b;
const GList *nodes = data;
int rc = 0;
int r1_score = -PCMK_SCORE_INFINITY;
int r2_score = -PCMK_SCORE_INFINITY;
pcmk_node_t *r1_node = NULL;
pcmk_node_t *r2_node = NULL;
GHashTable *r1_nodes = NULL;
GHashTable *r2_nodes = NULL;
const char *reason = NULL;
// Resources with highest priority should be assigned first
reason = "priority";
r1_score = resource1->priv->priority;
r2_score = resource2->priv->priority;
if (r1_score > r2_score) {
rc = -1;
goto done;
}
if (r1_score < r2_score) {
rc = 1;
goto done;
}
// We need nodes to make any other useful comparisons
reason = "no node list";
if (nodes == NULL) {
goto done;
}
// Calculate and log node scores
resource1->priv->cmds->add_colocated_node_scores(resource1, NULL,
resource1->id,
&r1_nodes, NULL, 1,
pcmk__coloc_select_this_with);
resource2->priv->cmds->add_colocated_node_scores(resource2, NULL,
resource2->id,
&r2_nodes, NULL, 1,
pcmk__coloc_select_this_with);
pe__show_node_scores(true, NULL, resource1->id, r1_nodes,
resource1->priv->scheduler);
pe__show_node_scores(true, NULL, resource2->id, r2_nodes,
resource2->priv->scheduler);
// The resource with highest score on its current node goes first
reason = "current location";
if (resource1->priv->active_nodes != NULL) {
r1_node = pcmk__current_node(resource1);
}
if (resource2->priv->active_nodes != NULL) {
r2_node = pcmk__current_node(resource2);
}
r1_score = get_node_score(r1_node, r1_nodes);
r2_score = get_node_score(r2_node, r2_nodes);
if (r1_score > r2_score) {
rc = -1;
goto done;
}
if (r1_score < r2_score) {
rc = 1;
goto done;
}
// Otherwise a higher score on any node will do
reason = "score";
for (const GList *iter = nodes; iter != NULL; iter = iter->next) {
const pcmk_node_t *node = (const pcmk_node_t *) iter->data;
r1_score = get_node_score(node, r1_nodes);
r2_score = get_node_score(node, r2_nodes);
if (r1_score > r2_score) {
rc = -1;
goto done;
}
if (r1_score < r2_score) {
rc = 1;
goto done;
}
}
done:
crm_trace("%s (%d)%s%s %c %s (%d)%s%s: %s",
resource1->id, r1_score,
((r1_node == NULL)? "" : " on "),
((r1_node == NULL)? "" : r1_node->priv->id),
((rc < 0)? '>' : ((rc > 0)? '<' : '=')),
resource2->id, r2_score,
((r2_node == NULL)? "" : " on "),
((r2_node == NULL)? "" : r2_node->priv->id),
reason);
if (r1_nodes != NULL) {
g_hash_table_destroy(r1_nodes);
}
if (r2_nodes != NULL) {
g_hash_table_destroy(r2_nodes);
}
return rc;
}
/*!
* \internal
* \brief Sort resources in the order they should be assigned to nodes
*
* \param[in,out] scheduler Scheduler data
*/
void
pcmk__sort_resources(pcmk_scheduler_t *scheduler)
{
GList *nodes = g_list_copy(scheduler->nodes);
nodes = pcmk__sort_nodes(nodes, NULL);
scheduler->priv->resources =
g_list_sort_with_data(scheduler->priv->resources, cmp_resources, nodes);
g_list_free(nodes);
}
diff --git a/lib/pengine/bundle.c b/lib/pengine/bundle.c
index dd2752ebca..43aef213f6 100644
--- a/lib/pengine/bundle.c
+++ b/lib/pengine/bundle.c
@@ -1,2093 +1,2091 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <ctype.h>
#include <stdint.h>
#include <crm/pengine/rules.h>
#include <crm/pengine/status.h>
#include <crm/pengine/internal.h>
#include <crm/common/xml.h>
#include <crm/common/output.h>
#include <crm/common/xml_internal.h>
#include <pe_status_private.h>
enum pe__bundle_mount_flags {
pe__bundle_mount_none = 0x00,
// mount instance-specific subdirectory rather than source directly
pe__bundle_mount_subdir = 0x01
};
typedef struct {
char *source;
char *target;
char *options;
uint32_t flags; // bitmask of pe__bundle_mount_flags
} pe__bundle_mount_t;
typedef struct {
char *source;
char *target;
} pe__bundle_port_t;
enum pe__container_agent {
PE__CONTAINER_AGENT_UNKNOWN,
PE__CONTAINER_AGENT_DOCKER,
PE__CONTAINER_AGENT_PODMAN,
};
#define PE__CONTAINER_AGENT_UNKNOWN_S "unknown"
#define PE__CONTAINER_AGENT_DOCKER_S "docker"
#define PE__CONTAINER_AGENT_PODMAN_S "podman"
typedef struct pe__bundle_variant_data_s {
int promoted_max;
int nreplicas;
int nreplicas_per_host;
char *prefix;
char *image;
const char *ip_last;
char *host_network;
char *host_netmask;
char *control_port;
char *container_network;
char *ip_range_start;
gboolean add_host;
gchar *container_host_options;
char *container_command;
char *launcher_options;
const char *attribute_target;
pcmk_resource_t *child;
GList *replicas; // pcmk__bundle_replica_t *
GList *ports; // pe__bundle_port_t *
GList *mounts; // pe__bundle_mount_t *
enum pe__container_agent agent_type;
} pe__bundle_variant_data_t;
#define get_bundle_variant_data(data, rsc) do { \
pcmk__assert(pcmk__is_bundle(rsc)); \
data = rsc->priv->variant_opaque; \
} while (0)
/*!
* \internal
* \brief Get maximum number of bundle replicas allowed to run
*
* \param[in] rsc Bundle or bundled resource to check
*
* \return Maximum replicas for bundle corresponding to \p rsc
*/
int
pe__bundle_max(const pcmk_resource_t *rsc)
{
const pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, pe__const_top_resource(rsc, true));
return bundle_data->nreplicas;
}
/*!
* \internal
* \brief Get the resource inside a bundle
*
* \param[in] bundle Bundle to check
*
* \return Resource inside \p bundle if any, otherwise NULL
*/
pcmk_resource_t *
pe__bundled_resource(const pcmk_resource_t *rsc)
{
const pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, pe__const_top_resource(rsc, true));
return bundle_data->child;
}
/*!
* \internal
* \brief Get containerized resource corresponding to a given bundle container
*
* \param[in] instance Collective instance that might be a bundle container
*
* \return Bundled resource instance inside \p instance if it is a bundle
* container instance, otherwise NULL
*/
const pcmk_resource_t *
pe__get_rsc_in_container(const pcmk_resource_t *instance)
{
const pe__bundle_variant_data_t *data = NULL;
const pcmk_resource_t *top = pe__const_top_resource(instance, true);
if (!pcmk__is_bundle(top)) {
return NULL;
}
get_bundle_variant_data(data, top);
for (const GList *iter = data->replicas; iter != NULL; iter = iter->next) {
const pcmk__bundle_replica_t *replica = iter->data;
if (instance == replica->container) {
return replica->child;
}
}
return NULL;
}
/*!
* \internal
* \brief Check whether a given node is created by a bundle
*
* \param[in] bundle Bundle resource to check
* \param[in] node Node to check
*
* \return true if \p node is an instance of \p bundle, otherwise false
*/
bool
pe__node_is_bundle_instance(const pcmk_resource_t *bundle,
const pcmk_node_t *node)
{
pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, bundle);
for (GList *iter = bundle_data->replicas; iter != NULL; iter = iter->next) {
pcmk__bundle_replica_t *replica = iter->data;
if (pcmk__same_node(node, replica->node)) {
return true;
}
}
return false;
}
/*!
* \internal
* \brief Get the container of a bundle's first replica
*
* \param[in] bundle Bundle resource to get container for
*
* \return Container resource from first replica of \p bundle if any,
* otherwise NULL
*/
pcmk_resource_t *
pe__first_container(const pcmk_resource_t *bundle)
{
const pe__bundle_variant_data_t *bundle_data = NULL;
const pcmk__bundle_replica_t *replica = NULL;
get_bundle_variant_data(bundle_data, bundle);
if (bundle_data->replicas == NULL) {
return NULL;
}
replica = bundle_data->replicas->data;
return replica->container;
}
/*!
* \internal
* \brief Iterate over bundle replicas
*
* \param[in,out] bundle Bundle to iterate over
* \param[in] fn Function to call for each replica (its return value
* indicates whether to continue iterating)
* \param[in,out] user_data Pointer to pass to \p fn
*/
void
pe__foreach_bundle_replica(pcmk_resource_t *bundle,
bool (*fn)(pcmk__bundle_replica_t *, void *),
void *user_data)
{
const pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, bundle);
for (GList *iter = bundle_data->replicas; iter != NULL; iter = iter->next) {
if (!fn((pcmk__bundle_replica_t *) iter->data, user_data)) {
break;
}
}
}
/*!
* \internal
* \brief Iterate over const bundle replicas
*
* \param[in] bundle Bundle to iterate over
* \param[in] fn Function to call for each replica (its return value
* indicates whether to continue iterating)
* \param[in,out] user_data Pointer to pass to \p fn
*/
void
pe__foreach_const_bundle_replica(const pcmk_resource_t *bundle,
bool (*fn)(const pcmk__bundle_replica_t *,
void *),
void *user_data)
{
const pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, bundle);
for (const GList *iter = bundle_data->replicas; iter != NULL;
iter = iter->next) {
if (!fn((const pcmk__bundle_replica_t *) iter->data, user_data)) {
break;
}
}
}
static char *
next_ip(const char *last_ip)
{
unsigned int oct1 = 0;
unsigned int oct2 = 0;
unsigned int oct3 = 0;
unsigned int oct4 = 0;
int rc = sscanf(last_ip, "%u.%u.%u.%u", &oct1, &oct2, &oct3, &oct4);
if (rc != 4) {
/*@ TODO check for IPv6 */
return NULL;
} else if (oct3 > 253) {
return NULL;
} else if (oct4 > 253) {
++oct3;
oct4 = 1;
} else {
++oct4;
}
return crm_strdup_printf("%u.%u.%u.%u", oct1, oct2, oct3, oct4);
}
static void
allocate_ip(pe__bundle_variant_data_t *data, pcmk__bundle_replica_t *replica,
GString *buffer)
{
if(data->ip_range_start == NULL) {
return;
} else if(data->ip_last) {
replica->ipaddr = next_ip(data->ip_last);
} else {
replica->ipaddr = strdup(data->ip_range_start);
}
data->ip_last = replica->ipaddr;
switch (data->agent_type) {
case PE__CONTAINER_AGENT_DOCKER:
case PE__CONTAINER_AGENT_PODMAN:
if (data->add_host) {
g_string_append_printf(buffer, " --add-host=%s-%d:%s",
data->prefix, replica->offset,
replica->ipaddr);
} else {
g_string_append_printf(buffer, " --hosts-entry=%s=%s-%d",
replica->ipaddr, data->prefix,
replica->offset);
}
break;
default: // PE__CONTAINER_AGENT_UNKNOWN
break;
}
}
static xmlNode *
create_resource(const char *name, const char *provider, const char *kind)
{
xmlNode *rsc = pcmk__xe_create(NULL, PCMK_XE_PRIMITIVE);
crm_xml_add(rsc, PCMK_XA_ID, name);
crm_xml_add(rsc, PCMK_XA_CLASS, PCMK_RESOURCE_CLASS_OCF);
crm_xml_add(rsc, PCMK_XA_PROVIDER, provider);
crm_xml_add(rsc, PCMK_XA_TYPE, kind);
return rsc;
}
/*!
* \internal
* \brief Check whether cluster can manage resource inside container
*
* \param[in,out] data Container variant data
*
* \return TRUE if networking configuration is acceptable, FALSE otherwise
*
* \note The resource is manageable if an IP range or control port has been
* specified. If a control port is used without an IP range, replicas per
* host must be 1.
*/
static bool
valid_network(pe__bundle_variant_data_t *data)
{
if(data->ip_range_start) {
return TRUE;
}
if(data->control_port) {
if(data->nreplicas_per_host > 1) {
pcmk__config_err("Specifying the '" PCMK_XA_CONTROL_PORT "' for %s "
"requires '" PCMK_XA_REPLICAS_PER_HOST "=1'",
data->prefix);
data->nreplicas_per_host = 1;
// @TODO to be sure:
// pcmk__clear_rsc_flags(rsc, pcmk__rsc_unique);
}
return TRUE;
}
return FALSE;
}
static int
create_ip_resource(pcmk_resource_t *parent, pe__bundle_variant_data_t *data,
pcmk__bundle_replica_t *replica)
{
if(data->ip_range_start) {
char *id = NULL;
xmlNode *xml_ip = NULL;
xmlNode *xml_obj = NULL;
id = crm_strdup_printf("%s-ip-%s", data->prefix, replica->ipaddr);
pcmk__xml_sanitize_id(id);
xml_ip = create_resource(id, "heartbeat", "IPaddr2");
free(id);
xml_obj = pcmk__xe_create(xml_ip, PCMK_XE_INSTANCE_ATTRIBUTES);
pcmk__xe_set_id(xml_obj, "%s-attributes-%d",
data->prefix, replica->offset);
crm_create_nvpair_xml(xml_obj, NULL, "ip", replica->ipaddr);
if(data->host_network) {
crm_create_nvpair_xml(xml_obj, NULL, "nic", data->host_network);
}
if(data->host_netmask) {
crm_create_nvpair_xml(xml_obj, NULL,
"cidr_netmask", data->host_netmask);
} else {
crm_create_nvpair_xml(xml_obj, NULL, "cidr_netmask", "32");
}
xml_obj = pcmk__xe_create(xml_ip, PCMK_XE_OPERATIONS);
crm_create_op_xml(xml_obj, pcmk__xe_id(xml_ip), PCMK_ACTION_MONITOR,
"60s", NULL);
// TODO: Other ops? Timeouts and intervals from underlying resource?
if (pe__unpack_resource(xml_ip, &replica->ip, parent,
parent->priv->scheduler) != pcmk_rc_ok) {
return pcmk_rc_unpack_error;
}
parent->priv->children = g_list_append(parent->priv->children,
replica->ip);
}
return pcmk_rc_ok;
}
static const char*
container_agent_str(enum pe__container_agent t)
{
switch (t) {
case PE__CONTAINER_AGENT_DOCKER: return PE__CONTAINER_AGENT_DOCKER_S;
case PE__CONTAINER_AGENT_PODMAN: return PE__CONTAINER_AGENT_PODMAN_S;
default: // PE__CONTAINER_AGENT_UNKNOWN
break;
}
return PE__CONTAINER_AGENT_UNKNOWN_S;
}
static int
create_container_resource(pcmk_resource_t *parent,
const pe__bundle_variant_data_t *data,
pcmk__bundle_replica_t *replica)
{
char *id = NULL;
xmlNode *xml_container = NULL;
xmlNode *xml_obj = NULL;
// Agent-specific
const char *hostname_opt = NULL;
const char *env_opt = NULL;
const char *agent_str = NULL;
GString *buffer = NULL;
GString *dbuffer = NULL;
// Where syntax differences are drop-in replacements, set them now
switch (data->agent_type) {
case PE__CONTAINER_AGENT_DOCKER:
case PE__CONTAINER_AGENT_PODMAN:
hostname_opt = "-h ";
env_opt = "-e ";
break;
default: // PE__CONTAINER_AGENT_UNKNOWN
return pcmk_rc_unpack_error;
}
agent_str = container_agent_str(data->agent_type);
buffer = g_string_sized_new(4096);
id = crm_strdup_printf("%s-%s-%d", data->prefix, agent_str,
replica->offset);
pcmk__xml_sanitize_id(id);
xml_container = create_resource(id, "heartbeat", agent_str);
free(id);
xml_obj = pcmk__xe_create(xml_container, PCMK_XE_INSTANCE_ATTRIBUTES);
pcmk__xe_set_id(xml_obj, "%s-attributes-%d", data->prefix, replica->offset);
crm_create_nvpair_xml(xml_obj, NULL, "image", data->image);
crm_create_nvpair_xml(xml_obj, NULL, "allow_pull", PCMK_VALUE_TRUE);
crm_create_nvpair_xml(xml_obj, NULL, "force_kill", PCMK_VALUE_FALSE);
crm_create_nvpair_xml(xml_obj, NULL, "reuse", PCMK_VALUE_FALSE);
if (data->agent_type == PE__CONTAINER_AGENT_DOCKER) {
g_string_append(buffer, " --restart=no");
}
/* Set a container hostname only if we have an IP to map it to. The user can
* set -h or --uts=host themselves if they want a nicer name for logs, but
* this makes applications happy who need their hostname to match the IP
* they bind to.
*/
if (data->ip_range_start != NULL) {
g_string_append_printf(buffer, " %s%s-%d", hostname_opt, data->prefix,
replica->offset);
}
pcmk__g_strcat(buffer, " ", env_opt, "PCMK_stderr=1", NULL);
if (data->container_network != NULL) {
pcmk__g_strcat(buffer, " --net=", data->container_network, NULL);
}
if (data->control_port != NULL) {
pcmk__g_strcat(buffer, " ", env_opt, "PCMK_" PCMK__ENV_REMOTE_PORT "=",
data->control_port, NULL);
} else {
g_string_append_printf(buffer, " %sPCMK_" PCMK__ENV_REMOTE_PORT "=%d",
env_opt, DEFAULT_REMOTE_PORT);
}
for (GList *iter = data->mounts; iter != NULL; iter = iter->next) {
pe__bundle_mount_t *mount = (pe__bundle_mount_t *) iter->data;
char *source = NULL;
if (pcmk_is_set(mount->flags, pe__bundle_mount_subdir)) {
source = crm_strdup_printf("%s/%s-%d", mount->source, data->prefix,
replica->offset);
pcmk__add_separated_word(&dbuffer, 1024, source, ",");
}
switch (data->agent_type) {
case PE__CONTAINER_AGENT_DOCKER:
case PE__CONTAINER_AGENT_PODMAN:
pcmk__g_strcat(buffer,
" -v ", pcmk__s(source, mount->source),
":", mount->target, NULL);
if (mount->options != NULL) {
pcmk__g_strcat(buffer, ":", mount->options, NULL);
}
break;
default:
break;
}
free(source);
}
for (GList *iter = data->ports; iter != NULL; iter = iter->next) {
pe__bundle_port_t *port = (pe__bundle_port_t *) iter->data;
switch (data->agent_type) {
case PE__CONTAINER_AGENT_DOCKER:
case PE__CONTAINER_AGENT_PODMAN:
if (replica->ipaddr != NULL) {
pcmk__g_strcat(buffer,
" -p ", replica->ipaddr, ":", port->source,
":", port->target, NULL);
} else if (!pcmk__str_eq(data->container_network,
PCMK_VALUE_HOST, pcmk__str_none)) {
// No need to do port mapping if net == host
pcmk__g_strcat(buffer,
" -p ", port->source, ":", port->target,
NULL);
}
break;
default:
break;
}
}
/* @COMPAT: We should use pcmk__add_word() here, but we can't yet, because
* it would cause restarts during rolling upgrades.
*
* In a previous version of the container resource creation logic, if
* data->launcher_options is not NULL, we append
* (" %s", data->launcher_options) even if data->launcher_options is an
* empty string. Likewise for data->container_host_options. Using
*
* pcmk__add_word(buffer, 0, data->launcher_options)
*
* removes that extra trailing space, causing a resource definition change.
*/
if (data->launcher_options != NULL) {
pcmk__g_strcat(buffer, " ", data->launcher_options, NULL);
}
if (data->container_host_options != NULL) {
pcmk__g_strcat(buffer, " ", data->container_host_options, NULL);
}
crm_create_nvpair_xml(xml_obj, NULL, "run_opts",
(const char *) buffer->str);
g_string_free(buffer, TRUE);
crm_create_nvpair_xml(xml_obj, NULL, "mount_points",
(dbuffer != NULL)? (const char *) dbuffer->str : "");
if (dbuffer != NULL) {
g_string_free(dbuffer, TRUE);
}
if (replica->child != NULL) {
if (data->container_command != NULL) {
crm_create_nvpair_xml(xml_obj, NULL, "run_cmd",
data->container_command);
} else {
crm_create_nvpair_xml(xml_obj, NULL, "run_cmd",
SBIN_DIR "/" PCMK__SERVER_REMOTED);
}
/* TODO: Allow users to specify their own?
*
* We just want to know if the container is alive; we'll monitor the
* child independently.
*/
crm_create_nvpair_xml(xml_obj, NULL, "monitor_cmd", "/bin/true");
#if 0
/* @TODO Consider supporting the use case where we can start and stop
* resources, but not proxy local commands (such as setting node
* attributes), by running the local executor in stand-alone mode.
* However, this would probably be better done via ACLs as with other
* Pacemaker Remote nodes.
*/
} else if ((child != NULL) && data->untrusted) {
crm_create_nvpair_xml(xml_obj, NULL, "run_cmd",
CRM_DAEMON_DIR "/" PCMK__SERVER_EXECD);
crm_create_nvpair_xml(xml_obj, NULL, "monitor_cmd",
CRM_DAEMON_DIR "/pacemaker/cts-exec-helper -c poke");
#endif
} else {
if (data->container_command != NULL) {
crm_create_nvpair_xml(xml_obj, NULL, "run_cmd",
data->container_command);
}
/* TODO: Allow users to specify their own?
*
* We don't know what's in the container, so we just want to know if it
* is alive.
*/
crm_create_nvpair_xml(xml_obj, NULL, "monitor_cmd", "/bin/true");
}
xml_obj = pcmk__xe_create(xml_container, PCMK_XE_OPERATIONS);
crm_create_op_xml(xml_obj, pcmk__xe_id(xml_container), PCMK_ACTION_MONITOR,
"60s", NULL);
// TODO: Other ops? Timeouts and intervals from underlying resource?
if (pe__unpack_resource(xml_container, &replica->container, parent,
parent->priv->scheduler) != pcmk_rc_ok) {
return pcmk_rc_unpack_error;
}
pcmk__set_rsc_flags(replica->container, pcmk__rsc_replica_container);
parent->priv->children = g_list_append(parent->priv->children,
replica->container);
return pcmk_rc_ok;
}
/*!
* \brief Ban a node from a resource's (and its children's) allowed nodes list
*
* \param[in,out] rsc Resource to modify
* \param[in] uname Name of node to ban
*/
static void
disallow_node(pcmk_resource_t *rsc, const char *uname)
{
gpointer match = g_hash_table_lookup(rsc->priv->allowed_nodes, uname);
if (match) {
((pcmk_node_t *) match)->assign->score = -PCMK_SCORE_INFINITY;
((pcmk_node_t *) match)->assign->probe_mode = pcmk__probe_never;
}
g_list_foreach(rsc->priv->children, (GFunc) disallow_node,
(gpointer) uname);
}
static int
create_remote_resource(pcmk_resource_t *parent, pe__bundle_variant_data_t *data,
pcmk__bundle_replica_t *replica)
{
if (replica->child && valid_network(data)) {
GHashTableIter gIter;
pcmk_node_t *node = NULL;
xmlNode *xml_remote = NULL;
char *id = crm_strdup_printf("%s-%d", data->prefix, replica->offset);
char *port_s = NULL;
const char *uname = NULL;
const char *connect_name = NULL;
pcmk_scheduler_t *scheduler = parent->priv->scheduler;
if (pe_find_resource(scheduler->priv->resources, id) != NULL) {
free(id);
// The biggest hammer we have
id = crm_strdup_printf("pcmk-internal-%s-remote-%d",
replica->child->id, replica->offset);
//@TODO return error instead of asserting?
pcmk__assert(pe_find_resource(scheduler->priv->resources,
id) == NULL);
}
/* REMOTE_CONTAINER_HACK: Using "#uname" as the server name when the
* connection does not have its own IP is a magic string that we use to
* support nested remotes (i.e. a bundle running on a remote node).
*/
connect_name = (replica->ipaddr? replica->ipaddr : "#uname");
if (data->control_port == NULL) {
port_s = pcmk__itoa(DEFAULT_REMOTE_PORT);
}
/* This sets replica->container as replica->remote's container, which is
* similar to what happens with guest nodes. This is how the scheduler
* knows that the bundle node is fenced by recovering the container, and
* that remote should be ordered relative to the container.
*/
xml_remote = pe_create_remote_xml(NULL, id, replica->container->id,
NULL, NULL, NULL,
connect_name, (data->control_port?
data->control_port : port_s));
free(port_s);
/* Abandon our created ID, and pull the copy from the XML, because we
* need something that will get freed during scheduler data cleanup to
* use as the node ID and uname.
*/
free(id);
id = NULL;
uname = pcmk__xe_id(xml_remote);
/* Ensure a node has been created for the guest (it may have already
* been, if it has a permanent node attribute), and ensure its weight is
* -INFINITY so no other resources can run on it.
*/
node = pcmk_find_node(scheduler, uname);
if (node == NULL) {
node = pe_create_node(uname, uname, PCMK_VALUE_REMOTE,
-PCMK_SCORE_INFINITY, scheduler);
} else {
node->assign->score = -PCMK_SCORE_INFINITY;
}
node->assign->probe_mode = pcmk__probe_never;
/* unpack_remote_nodes() ensures that each remote node and guest node
* has a pcmk_node_t entry. Ideally, it would do the same for bundle
* nodes. Unfortunately, a bundle has to be mostly unpacked before it's
* obvious what nodes will be needed, so we do it just above.
*
* Worse, that means that the node may have been utilized while
* unpacking other resources, without our weight correction. The most
* likely place for this to happen is when pe__unpack_resource() calls
* resource_location() to set a default score in symmetric clusters.
* This adds a node *copy* to each resource's allowed nodes, and these
* copies will have the wrong weight.
*
* As a hacky workaround, fix those copies here.
*
* @TODO Possible alternative: ensure bundles are unpacked before other
* resources, so the weight is correct before any copies are made.
*/
g_list_foreach(scheduler->priv->resources,
(GFunc) disallow_node, (gpointer) uname);
replica->node = pe__copy_node(node);
replica->node->assign->score = 500;
replica->node->assign->probe_mode = pcmk__probe_exclusive;
/* Ensure the node shows up as allowed and with the correct discovery set */
if (replica->child->priv->allowed_nodes != NULL) {
g_hash_table_destroy(replica->child->priv->allowed_nodes);
}
replica->child->priv->allowed_nodes = pcmk__strkey_table(NULL, free);
g_hash_table_insert(replica->child->priv->allowed_nodes,
(gpointer) replica->node->priv->id,
pe__copy_node(replica->node));
{
const pcmk_resource_t *parent = replica->child->priv->parent;
pcmk_node_t *copy = pe__copy_node(replica->node);
copy->assign->score = -PCMK_SCORE_INFINITY;
g_hash_table_insert(parent->priv->allowed_nodes,
(gpointer) replica->node->priv->id, copy);
}
if (pe__unpack_resource(xml_remote, &replica->remote, parent,
scheduler) != pcmk_rc_ok) {
return pcmk_rc_unpack_error;
}
g_hash_table_iter_init(&gIter, replica->remote->priv->allowed_nodes);
while (g_hash_table_iter_next(&gIter, NULL, (void **)&node)) {
if (pcmk__is_pacemaker_remote_node(node)) {
/* Remote resources can only run on 'normal' cluster node */
node->assign->score = -PCMK_SCORE_INFINITY;
}
}
replica->node->priv->remote = replica->remote;
// Ensure pcmk__is_guest_or_bundle_node() functions correctly
replica->remote->priv->launcher = replica->container;
/* A bundle's #kind is closer to "container" (guest node) than the
* "remote" set by pe_create_node().
*/
pcmk__insert_dup(replica->node->priv->attrs,
CRM_ATTR_KIND, "container");
/* One effect of this is that unpack_launcher() will add
* replica->remote to replica->container's launched resources, which
* will make pe__resource_contains_guest_node() true for
* replica->container.
*
* replica->child does NOT get added to replica->container's launched
* resources. The only noticeable effect if it did would be for its
* fail count to be taken into account when checking
* replica->container's migration threshold.
*/
parent->priv->children = g_list_append(parent->priv->children,
replica->remote);
}
return pcmk_rc_ok;
}
static int
create_replica_resources(pcmk_resource_t *parent,
pe__bundle_variant_data_t *data,
pcmk__bundle_replica_t *replica)
{
int rc = pcmk_rc_ok;
rc = create_container_resource(parent, data, replica);
if (rc != pcmk_rc_ok) {
return rc;
}
rc = create_ip_resource(parent, data, replica);
if (rc != pcmk_rc_ok) {
return rc;
}
rc = create_remote_resource(parent, data, replica);
if (rc != pcmk_rc_ok) {
return rc;
}
if ((replica->child != NULL) && (replica->ipaddr != NULL)) {
pcmk__insert_meta(replica->child->priv, "external-ip", replica->ipaddr);
}
if (replica->remote != NULL) {
/*
* Allow the remote connection resource to be allocated to a
* different node than the one on which the container is active.
*
* This makes it possible to have Pacemaker Remote nodes running
* containers with the remote executor inside in order to start
* services inside those containers.
*/
pcmk__set_rsc_flags(replica->remote, pcmk__rsc_remote_nesting_allowed);
}
return rc;
}
static void
mount_add(pe__bundle_variant_data_t *bundle_data, const char *source,
const char *target, const char *options, uint32_t flags)
{
pe__bundle_mount_t *mount = pcmk__assert_alloc(1,
sizeof(pe__bundle_mount_t));
mount->source = pcmk__str_copy(source);
mount->target = pcmk__str_copy(target);
mount->options = pcmk__str_copy(options);
mount->flags = flags;
bundle_data->mounts = g_list_append(bundle_data->mounts, mount);
}
static void
mount_free(pe__bundle_mount_t *mount)
{
free(mount->source);
free(mount->target);
free(mount->options);
free(mount);
}
static void
port_free(pe__bundle_port_t *port)
{
free(port->source);
free(port->target);
free(port);
}
static pcmk__bundle_replica_t *
replica_for_remote(pcmk_resource_t *remote)
{
pcmk_resource_t *top = remote;
pe__bundle_variant_data_t *bundle_data = NULL;
if (top == NULL) {
return NULL;
}
while (top->priv->parent != NULL) {
top = top->priv->parent;
}
get_bundle_variant_data(bundle_data, top);
for (GList *gIter = bundle_data->replicas; gIter != NULL;
gIter = gIter->next) {
pcmk__bundle_replica_t *replica = gIter->data;
if (replica->remote == remote) {
return replica;
}
}
CRM_LOG_ASSERT(FALSE);
return NULL;
}
bool
pe__bundle_needs_remote_name(pcmk_resource_t *rsc)
{
const char *value;
GHashTable *params = NULL;
if (rsc == NULL) {
return false;
}
// Use NULL node since pcmk__bundle_expand() uses that to set value
params = pe_rsc_params(rsc, NULL, rsc->priv->scheduler);
value = g_hash_table_lookup(params, PCMK_REMOTE_RA_ADDR);
return pcmk__str_eq(value, "#uname", pcmk__str_casei)
&& xml_contains_remote_node(rsc->priv->xml);
}
const char *
pe__add_bundle_remote_name(pcmk_resource_t *rsc, xmlNode *xml,
const char *field)
{
// REMOTE_CONTAINER_HACK: Allow remote nodes that start containers with pacemaker remote inside
pcmk_node_t *node = NULL;
pcmk__bundle_replica_t *replica = NULL;
if (!pe__bundle_needs_remote_name(rsc)) {
return NULL;
}
replica = replica_for_remote(rsc);
if (replica == NULL) {
return NULL;
}
node = replica->container->priv->assigned_node;
if (node == NULL) {
/* If it won't be running anywhere after the
* transition, go with where it's running now.
*/
node = pcmk__current_node(replica->container);
}
if(node == NULL) {
crm_trace("Cannot determine address for bundle connection %s", rsc->id);
return NULL;
}
crm_trace("Setting address for bundle connection %s to bundle host %s",
rsc->id, pcmk__node_name(node));
if(xml != NULL && field != NULL) {
crm_xml_add(xml, field, node->priv->name);
}
return node->priv->name;
}
#define pe__set_bundle_mount_flags(mount_xml, flags, flags_to_set) do { \
flags = pcmk__set_flags_as(__func__, __LINE__, LOG_TRACE, \
"Bundle mount", pcmk__xe_id(mount_xml), \
flags, (flags_to_set), #flags_to_set); \
} while (0)
gboolean
pe__unpack_bundle(pcmk_resource_t *rsc, pcmk_scheduler_t *scheduler)
{
const char *value = NULL;
xmlNode *xml_obj = NULL;
const xmlNode *xml_child = NULL;
xmlNode *xml_resource = NULL;
pe__bundle_variant_data_t *bundle_data = NULL;
bool need_log_mount = TRUE;
pcmk__assert(rsc != NULL);
pcmk__rsc_trace(rsc, "Processing resource %s...", rsc->id);
bundle_data = pcmk__assert_alloc(1, sizeof(pe__bundle_variant_data_t));
rsc->priv->variant_opaque = bundle_data;
bundle_data->prefix = strdup(rsc->id);
xml_obj = pcmk__xe_first_child(rsc->priv->xml, PCMK_XE_DOCKER, NULL,
NULL);
if (xml_obj != NULL) {
bundle_data->agent_type = PE__CONTAINER_AGENT_DOCKER;
}
if (xml_obj == NULL) {
xml_obj = pcmk__xe_first_child(rsc->priv->xml, PCMK_XE_PODMAN, NULL,
NULL);
if (xml_obj != NULL) {
bundle_data->agent_type = PE__CONTAINER_AGENT_PODMAN;
}
}
if (xml_obj == NULL) {
return FALSE;
}
// Use 0 for default, minimum, and invalid PCMK_XA_PROMOTED_MAX
value = crm_element_value(xml_obj, PCMK_XA_PROMOTED_MAX);
pcmk__scan_min_int(value, &bundle_data->promoted_max, 0);
/* Default replicas to PCMK_XA_PROMOTED_MAX if it was specified and 1
* otherwise
*/
value = crm_element_value(xml_obj, PCMK_XA_REPLICAS);
if ((value == NULL) && (bundle_data->promoted_max > 0)) {
bundle_data->nreplicas = bundle_data->promoted_max;
} else {
pcmk__scan_min_int(value, &bundle_data->nreplicas, 1);
}
/*
* Communication between containers on the same host via the
* floating IPs only works if the container is started with:
* --userland-proxy=false --ip-masq=false
*/
value = crm_element_value(xml_obj, PCMK_XA_REPLICAS_PER_HOST);
pcmk__scan_min_int(value, &bundle_data->nreplicas_per_host, 1);
if (bundle_data->nreplicas_per_host == 1) {
pcmk__clear_rsc_flags(rsc, pcmk__rsc_unique);
}
bundle_data->container_command =
crm_element_value_copy(xml_obj, PCMK_XA_RUN_COMMAND);
bundle_data->launcher_options = crm_element_value_copy(xml_obj,
PCMK_XA_OPTIONS);
bundle_data->image = crm_element_value_copy(xml_obj, PCMK_XA_IMAGE);
bundle_data->container_network = crm_element_value_copy(xml_obj,
PCMK_XA_NETWORK);
xml_obj = pcmk__xe_first_child(rsc->priv->xml, PCMK_XE_NETWORK, NULL,
NULL);
if(xml_obj) {
bundle_data->ip_range_start =
crm_element_value_copy(xml_obj, PCMK_XA_IP_RANGE_START);
bundle_data->host_netmask =
crm_element_value_copy(xml_obj, PCMK_XA_HOST_NETMASK);
bundle_data->host_network =
crm_element_value_copy(xml_obj, PCMK_XA_HOST_INTERFACE);
bundle_data->control_port =
crm_element_value_copy(xml_obj, PCMK_XA_CONTROL_PORT);
value = crm_element_value(xml_obj, PCMK_XA_ADD_HOST);
if (crm_str_to_boolean(value, &bundle_data->add_host) != 1) {
bundle_data->add_host = TRUE;
}
for (xml_child = pcmk__xe_first_child(xml_obj, PCMK_XE_PORT_MAPPING,
NULL, NULL);
xml_child != NULL;
xml_child = pcmk__xe_next(xml_child, PCMK_XE_PORT_MAPPING)) {
pe__bundle_port_t *port =
pcmk__assert_alloc(1, sizeof(pe__bundle_port_t));
port->source = crm_element_value_copy(xml_child, PCMK_XA_PORT);
if(port->source == NULL) {
port->source = crm_element_value_copy(xml_child, PCMK_XA_RANGE);
} else {
port->target = crm_element_value_copy(xml_child,
PCMK_XA_INTERNAL_PORT);
}
if(port->source != NULL && strlen(port->source) > 0) {
if(port->target == NULL) {
port->target = strdup(port->source);
}
bundle_data->ports = g_list_append(bundle_data->ports, port);
} else {
pcmk__config_err("Invalid " PCMK_XA_PORT " directive %s",
pcmk__xe_id(xml_child));
port_free(port);
}
}
}
xml_obj = pcmk__xe_first_child(rsc->priv->xml, PCMK_XE_STORAGE, NULL,
NULL);
for (xml_child = pcmk__xe_first_child(xml_obj, PCMK_XE_STORAGE_MAPPING,
NULL, NULL);
xml_child != NULL;
xml_child = pcmk__xe_next(xml_child, PCMK_XE_STORAGE_MAPPING)) {
const char *source = crm_element_value(xml_child, PCMK_XA_SOURCE_DIR);
const char *target = crm_element_value(xml_child, PCMK_XA_TARGET_DIR);
const char *options = crm_element_value(xml_child, PCMK_XA_OPTIONS);
int flags = pe__bundle_mount_none;
if (source == NULL) {
source = crm_element_value(xml_child, PCMK_XA_SOURCE_DIR_ROOT);
pe__set_bundle_mount_flags(xml_child, flags,
pe__bundle_mount_subdir);
}
if (source && target) {
mount_add(bundle_data, source, target, options, flags);
if (strcmp(target, "/var/log") == 0) {
need_log_mount = FALSE;
}
} else {
pcmk__config_err("Invalid mount directive %s",
pcmk__xe_id(xml_child));
}
}
xml_obj = pcmk__xe_first_child(rsc->priv->xml, PCMK_XE_PRIMITIVE, NULL,
NULL);
if (xml_obj && valid_network(bundle_data)) {
const char *suffix = NULL;
char *value = NULL;
xmlNode *xml_set = NULL;
xml_resource = pcmk__xe_create(NULL, PCMK_XE_CLONE);
/* @COMPAT We no longer use the <master> tag, but we need to keep it as
* part of the resource name, so that bundles don't restart in a rolling
* upgrade. (It also avoids needing to change regression tests.)
*/
suffix = (const char *) xml_resource->name;
if (bundle_data->promoted_max > 0) {
suffix = "master";
}
pcmk__xe_set_id(xml_resource, "%s-%s", bundle_data->prefix, suffix);
xml_set = pcmk__xe_create(xml_resource, PCMK_XE_META_ATTRIBUTES);
pcmk__xe_set_id(xml_set, "%s-%s-meta",
bundle_data->prefix, xml_resource->name);
crm_create_nvpair_xml(xml_set, NULL,
PCMK_META_ORDERED, PCMK_VALUE_TRUE);
value = pcmk__itoa(bundle_data->nreplicas);
crm_create_nvpair_xml(xml_set, NULL, PCMK_META_CLONE_MAX, value);
free(value);
value = pcmk__itoa(bundle_data->nreplicas_per_host);
crm_create_nvpair_xml(xml_set, NULL, PCMK_META_CLONE_NODE_MAX, value);
free(value);
crm_create_nvpair_xml(xml_set, NULL, PCMK_META_GLOBALLY_UNIQUE,
pcmk__btoa(bundle_data->nreplicas_per_host > 1));
if (bundle_data->promoted_max) {
crm_create_nvpair_xml(xml_set, NULL,
PCMK_META_PROMOTABLE, PCMK_VALUE_TRUE);
value = pcmk__itoa(bundle_data->promoted_max);
crm_create_nvpair_xml(xml_set, NULL, PCMK_META_PROMOTED_MAX, value);
free(value);
}
//crm_xml_add(xml_obj, PCMK_XA_ID, bundle_data->prefix);
pcmk__xml_copy(xml_resource, xml_obj);
} else if(xml_obj) {
pcmk__config_err("Cannot control %s inside %s without either "
PCMK_XA_IP_RANGE_START " or " PCMK_XA_CONTROL_PORT,
rsc->id, pcmk__xe_id(xml_obj));
return FALSE;
}
if(xml_resource) {
int lpc = 0;
GList *childIter = NULL;
pe__bundle_port_t *port = NULL;
GString *buffer = NULL;
if (pe__unpack_resource(xml_resource, &(bundle_data->child), rsc,
scheduler) != pcmk_rc_ok) {
return FALSE;
}
/* Currently, we always map the default authentication key location
* into the same location inside the container.
*
* Ideally, we would respect the host's PCMK_authkey_location, but:
* - it may be different on different nodes;
* - the actual connection will do extra checking to make sure the key
* file exists and is readable, that we can't do here on the DC
* - tools such as crm_resource and crm_simulate may not have the same
* environment variables as the cluster, causing operation digests to
* differ
*
* Always using the default location inside the container is fine,
* because we control the pacemaker_remote environment, and it avoids
* having to pass another environment variable to the container.
*
* @TODO A better solution may be to have only pacemaker_remote use the
* environment variable, and have the cluster nodes use a new
* cluster option for key location. This would introduce the limitation
* of the location being the same on all cluster nodes, but that's
* reasonable.
*/
mount_add(bundle_data, DEFAULT_REMOTE_KEY_LOCATION,
DEFAULT_REMOTE_KEY_LOCATION, NULL, pe__bundle_mount_none);
if (need_log_mount) {
mount_add(bundle_data, CRM_BUNDLE_DIR, "/var/log", NULL,
pe__bundle_mount_subdir);
}
port = pcmk__assert_alloc(1, sizeof(pe__bundle_port_t));
if(bundle_data->control_port) {
port->source = strdup(bundle_data->control_port);
} else {
/* If we wanted to respect PCMK_remote_port, we could use
* crm_default_remote_port() here and elsewhere in this file instead
* of DEFAULT_REMOTE_PORT.
*
* However, it gains nothing, since we control both the container
* environment and the connection resource parameters, and the user
* can use a different port if desired by setting
* PCMK_XA_CONTROL_PORT.
*/
port->source = pcmk__itoa(DEFAULT_REMOTE_PORT);
}
port->target = strdup(port->source);
bundle_data->ports = g_list_append(bundle_data->ports, port);
buffer = g_string_sized_new(1024);
for (childIter = bundle_data->child->priv->children;
childIter != NULL; childIter = childIter->next) {
pcmk__bundle_replica_t *replica = NULL;
replica = pcmk__assert_alloc(1, sizeof(pcmk__bundle_replica_t));
replica->child = childIter->data;
pcmk__set_rsc_flags(replica->child, pcmk__rsc_exclusive_probes);
replica->offset = lpc++;
// Ensure the child's notify gets set based on the underlying primitive's value
if (pcmk_is_set(replica->child->flags, pcmk__rsc_notify)) {
pcmk__set_rsc_flags(bundle_data->child, pcmk__rsc_notify);
}
allocate_ip(bundle_data, replica, buffer);
bundle_data->replicas = g_list_append(bundle_data->replicas,
replica);
bundle_data->attribute_target =
g_hash_table_lookup(replica->child->priv->meta,
PCMK_META_CONTAINER_ATTRIBUTE_TARGET);
}
bundle_data->container_host_options = g_string_free(buffer, FALSE);
if (bundle_data->attribute_target) {
pcmk__insert_dup(rsc->priv->meta,
PCMK_META_CONTAINER_ATTRIBUTE_TARGET,
bundle_data->attribute_target);
pcmk__insert_dup(bundle_data->child->priv->meta,
PCMK_META_CONTAINER_ATTRIBUTE_TARGET,
bundle_data->attribute_target);
}
} else {
// Just a naked container, no pacemaker-remote
GString *buffer = g_string_sized_new(1024);
for (int lpc = 0; lpc < bundle_data->nreplicas; lpc++) {
pcmk__bundle_replica_t *replica = NULL;
replica = pcmk__assert_alloc(1, sizeof(pcmk__bundle_replica_t));
replica->offset = lpc;
allocate_ip(bundle_data, replica, buffer);
bundle_data->replicas = g_list_append(bundle_data->replicas,
replica);
}
bundle_data->container_host_options = g_string_free(buffer, FALSE);
}
for (GList *gIter = bundle_data->replicas; gIter != NULL;
gIter = gIter->next) {
pcmk__bundle_replica_t *replica = gIter->data;
if (create_replica_resources(rsc, bundle_data, replica) != pcmk_rc_ok) {
pcmk__config_err("Failed unpacking resource %s", rsc->id);
rsc->priv->fns->free(rsc);
return FALSE;
}
/* Utilization needs special handling for bundles. It makes no sense for
* the inner primitive to have utilization, because it is tied
* one-to-one to the guest node created by the container resource -- and
* there's no way to set capacities for that guest node anyway.
*
* What the user really wants is to configure utilization for the
* container. However, the schema only allows utilization for
* primitives, and the container resource is implicit anyway, so the
* user can *only* configure utilization for the inner primitive. If
* they do, move the primitive's utilization values to the container.
*
* @TODO This means that bundles without an inner primitive can't have
* utilization. An alternative might be to allow utilization values in
* the top-level bundle XML in the schema, and copy those to each
* container.
*/
if (replica->child != NULL) {
GHashTable *empty = replica->container->priv->utilization;
replica->container->priv->utilization =
replica->child->priv->utilization;
replica->child->priv->utilization = empty;
}
}
if (bundle_data->child) {
rsc->priv->children = g_list_append(rsc->priv->children,
bundle_data->child);
}
return TRUE;
}
static int
replica_resource_active(pcmk_resource_t *rsc, gboolean all)
{
if (rsc) {
gboolean child_active = rsc->priv->fns->active(rsc, all);
if (child_active && !all) {
return TRUE;
} else if (!child_active && all) {
return FALSE;
}
}
return -1;
}
gboolean
pe__bundle_active(pcmk_resource_t *rsc, gboolean all)
{
pe__bundle_variant_data_t *bundle_data = NULL;
GList *iter = NULL;
get_bundle_variant_data(bundle_data, rsc);
for (iter = bundle_data->replicas; iter != NULL; iter = iter->next) {
pcmk__bundle_replica_t *replica = iter->data;
int rsc_active;
rsc_active = replica_resource_active(replica->ip, all);
if (rsc_active >= 0) {
return (gboolean) rsc_active;
}
rsc_active = replica_resource_active(replica->child, all);
if (rsc_active >= 0) {
return (gboolean) rsc_active;
}
rsc_active = replica_resource_active(replica->container, all);
if (rsc_active >= 0) {
return (gboolean) rsc_active;
}
rsc_active = replica_resource_active(replica->remote, all);
if (rsc_active >= 0) {
return (gboolean) rsc_active;
}
}
/* If "all" is TRUE, we've already checked that no resources were inactive,
* so return TRUE; if "all" is FALSE, we didn't find any active resources,
* so return FALSE.
*/
return all;
}
/*!
* \internal
* \brief Find the bundle replica corresponding to a given node
*
* \param[in] bundle Top-level bundle resource
* \param[in] node Node to search for
*
* \return Bundle replica if found, NULL otherwise
*/
pcmk_resource_t *
pe__find_bundle_replica(const pcmk_resource_t *bundle, const pcmk_node_t *node)
{
pe__bundle_variant_data_t *bundle_data = NULL;
pcmk__assert((bundle != NULL) && (node != NULL));
get_bundle_variant_data(bundle_data, bundle);
for (GList *gIter = bundle_data->replicas; gIter != NULL;
gIter = gIter->next) {
pcmk__bundle_replica_t *replica = gIter->data;
pcmk__assert((replica != NULL) && (replica->node != NULL));
if (pcmk__same_node(replica->node, node)) {
return replica->child;
}
}
return NULL;
}
PCMK__OUTPUT_ARGS("bundle", "uint32_t", "pcmk_resource_t *", "GList *",
"GList *")
int
pe__bundle_xml(pcmk__output_t *out, va_list args)
{
uint32_t show_opts = va_arg(args, uint32_t);
pcmk_resource_t *rsc = va_arg(args, pcmk_resource_t *);
GList *only_node = va_arg(args, GList *);
GList *only_rsc = va_arg(args, GList *);
pe__bundle_variant_data_t *bundle_data = NULL;
int rc = pcmk_rc_no_output;
gboolean printed_header = FALSE;
gboolean print_everything = TRUE;
const char *desc = NULL;
pcmk__assert(rsc != NULL);
get_bundle_variant_data(bundle_data, rsc);
if (rsc->priv->fns->is_filtered(rsc, only_rsc, TRUE)) {
return rc;
}
print_everything = pcmk__str_in_list(rsc->id, only_rsc, pcmk__str_star_matches);
for (GList *gIter = bundle_data->replicas; gIter != NULL;
gIter = gIter->next) {
pcmk__bundle_replica_t *replica = gIter->data;
pcmk_resource_t *ip = replica->ip;
pcmk_resource_t *child = replica->child;
pcmk_resource_t *container = replica->container;
pcmk_resource_t *remote = replica->remote;
char *id = NULL;
gboolean print_ip, print_child, print_ctnr, print_remote;
pcmk__assert(replica != NULL);
if (pcmk__rsc_filtered_by_node(container, only_node)) {
continue;
}
print_ip = (ip != NULL)
&& !ip->priv->fns->is_filtered(ip, only_rsc,
print_everything);
print_child = (child != NULL)
&& !child->priv->fns->is_filtered(child, only_rsc,
print_everything);
print_ctnr = !container->priv->fns->is_filtered(container, only_rsc,
print_everything);
print_remote = (remote != NULL)
&& !remote->priv->fns->is_filtered(remote, only_rsc,
print_everything);
if (!print_everything && !print_ip && !print_child && !print_ctnr && !print_remote) {
continue;
}
if (!printed_header) {
const char *type = container_agent_str(bundle_data->agent_type);
const char *unique = pcmk__flag_text(rsc->flags, pcmk__rsc_unique);
const char *maintenance = pcmk__flag_text(rsc->flags,
pcmk__rsc_maintenance);
const char *managed = pcmk__flag_text(rsc->flags,
pcmk__rsc_managed);
const char *failed = pcmk__flag_text(rsc->flags, pcmk__rsc_failed);
printed_header = TRUE;
desc = pe__resource_description(rsc, show_opts);
rc = pe__name_and_nvpairs_xml(out, true, PCMK_XE_BUNDLE,
PCMK_XA_ID, rsc->id,
PCMK_XA_TYPE, type,
PCMK_XA_IMAGE, bundle_data->image,
PCMK_XA_UNIQUE, unique,
PCMK_XA_MAINTENANCE, maintenance,
PCMK_XA_MANAGED, managed,
PCMK_XA_FAILED, failed,
PCMK_XA_DESCRIPTION, desc,
NULL);
pcmk__assert(rc == pcmk_rc_ok);
}
id = pcmk__itoa(replica->offset);
rc = pe__name_and_nvpairs_xml(out, true, PCMK_XE_REPLICA,
PCMK_XA_ID, id,
NULL);
free(id);
pcmk__assert(rc == pcmk_rc_ok);
if (print_ip) {
out->message(out, (const char *) ip->priv->xml->name, show_opts,
ip, only_node, only_rsc);
}
if (print_child) {
out->message(out, (const char *) child->priv->xml->name,
show_opts, child, only_node, only_rsc);
}
if (print_ctnr) {
out->message(out, (const char *) container->priv->xml->name,
show_opts, container, only_node, only_rsc);
}
if (print_remote) {
out->message(out, (const char *) remote->priv->xml->name,
show_opts, remote, only_node, only_rsc);
}
pcmk__output_xml_pop_parent(out); // replica
}
if (printed_header) {
pcmk__output_xml_pop_parent(out); // bundle
}
return rc;
}
static void
pe__bundle_replica_output_html(pcmk__output_t *out,
pcmk__bundle_replica_t *replica,
pcmk_node_t *node, uint32_t show_opts)
{
pcmk_resource_t *rsc = replica->child;
int offset = 0;
char buffer[LINE_MAX];
if(rsc == NULL) {
rsc = replica->container;
}
if (replica->remote) {
offset += snprintf(buffer + offset, LINE_MAX - offset, "%s",
rsc_printable_id(replica->remote));
} else {
offset += snprintf(buffer + offset, LINE_MAX - offset, "%s",
rsc_printable_id(replica->container));
}
if (replica->ipaddr) {
offset += snprintf(buffer + offset, LINE_MAX - offset, " (%s)",
replica->ipaddr);
}
pe__common_output_html(out, rsc, buffer, node, show_opts);
}
/*!
* \internal
* \brief Get a string describing a resource's unmanaged state or lack thereof
*
* \param[in] rsc Resource to describe
*
* \return A string indicating that a resource is in maintenance mode or
* otherwise unmanaged, or an empty string otherwise
*/
static const char *
get_unmanaged_str(const pcmk_resource_t *rsc)
{
if (pcmk_is_set(rsc->flags, pcmk__rsc_maintenance)) {
return " (maintenance)";
}
if (!pcmk_is_set(rsc->flags, pcmk__rsc_managed)) {
return " (unmanaged)";
}
return "";
}
PCMK__OUTPUT_ARGS("bundle", "uint32_t", "pcmk_resource_t *", "GList *",
"GList *")
int
pe__bundle_html(pcmk__output_t *out, va_list args)
{
uint32_t show_opts = va_arg(args, uint32_t);
pcmk_resource_t *rsc = va_arg(args, pcmk_resource_t *);
GList *only_node = va_arg(args, GList *);
GList *only_rsc = va_arg(args, GList *);
const char *desc = NULL;
pe__bundle_variant_data_t *bundle_data = NULL;
int rc = pcmk_rc_no_output;
gboolean print_everything = TRUE;
pcmk__assert(rsc != NULL);
get_bundle_variant_data(bundle_data, rsc);
desc = pe__resource_description(rsc, show_opts);
if (rsc->priv->fns->is_filtered(rsc, only_rsc, TRUE)) {
return rc;
}
print_everything = pcmk__str_in_list(rsc->id, only_rsc, pcmk__str_star_matches);
for (GList *gIter = bundle_data->replicas; gIter != NULL;
gIter = gIter->next) {
pcmk__bundle_replica_t *replica = gIter->data;
pcmk_resource_t *ip = replica->ip;
pcmk_resource_t *child = replica->child;
pcmk_resource_t *container = replica->container;
pcmk_resource_t *remote = replica->remote;
gboolean print_ip, print_child, print_ctnr, print_remote;
pcmk__assert(replica != NULL);
if (pcmk__rsc_filtered_by_node(container, only_node)) {
continue;
}
print_ip = (ip != NULL)
&& !ip->priv->fns->is_filtered(ip, only_rsc,
print_everything);
print_child = (child != NULL)
&& !child->priv->fns->is_filtered(child, only_rsc,
print_everything);
print_ctnr = !container->priv->fns->is_filtered(container, only_rsc,
print_everything);
print_remote = (remote != NULL)
&& !remote->priv->fns->is_filtered(remote, only_rsc,
print_everything);
if (pcmk_is_set(show_opts, pcmk_show_implicit_rscs) ||
(print_everything == FALSE && (print_ip || print_child || print_ctnr || print_remote))) {
/* The text output messages used below require pe_print_implicit to
* be set to do anything.
*/
uint32_t new_show_opts = show_opts | pcmk_show_implicit_rscs;
PCMK__OUTPUT_LIST_HEADER(out, FALSE, rc, "Container bundle%s: %s [%s]%s%s%s%s%s",
(bundle_data->nreplicas > 1)? " set" : "",
rsc->id, bundle_data->image,
pcmk_is_set(rsc->flags, pcmk__rsc_unique)? " (unique)" : "",
desc ? " (" : "", desc ? desc : "", desc ? ")" : "",
get_unmanaged_str(rsc));
if (pcmk__list_of_multiple(bundle_data->replicas)) {
out->begin_list(out, NULL, NULL, "Replica[%d]", replica->offset);
}
if (print_ip) {
out->message(out, (const char *) ip->priv->xml->name,
new_show_opts, ip, only_node, only_rsc);
}
if (print_child) {
out->message(out, (const char *) child->priv->xml->name,
new_show_opts, child, only_node, only_rsc);
}
if (print_ctnr) {
out->message(out, (const char *) container->priv->xml->name,
new_show_opts, container, only_node, only_rsc);
}
if (print_remote) {
out->message(out, (const char *) remote->priv->xml->name,
new_show_opts, remote, only_node, only_rsc);
}
if (pcmk__list_of_multiple(bundle_data->replicas)) {
out->end_list(out);
}
} else if (print_everything == FALSE && !(print_ip || print_child || print_ctnr || print_remote)) {
continue;
} else {
PCMK__OUTPUT_LIST_HEADER(out, FALSE, rc, "Container bundle%s: %s [%s]%s%s%s%s%s",
(bundle_data->nreplicas > 1)? " set" : "",
rsc->id, bundle_data->image,
pcmk_is_set(rsc->flags, pcmk__rsc_unique)? " (unique)" : "",
desc ? " (" : "", desc ? desc : "", desc ? ")" : "",
get_unmanaged_str(rsc));
pe__bundle_replica_output_html(out, replica,
pcmk__current_node(container),
show_opts);
}
}
PCMK__OUTPUT_LIST_FOOTER(out, rc);
return rc;
}
static void
pe__bundle_replica_output_text(pcmk__output_t *out,
pcmk__bundle_replica_t *replica,
pcmk_node_t *node, uint32_t show_opts)
{
const pcmk_resource_t *rsc = replica->child;
int offset = 0;
char buffer[LINE_MAX];
if(rsc == NULL) {
rsc = replica->container;
}
if (replica->remote) {
offset += snprintf(buffer + offset, LINE_MAX - offset, "%s",
rsc_printable_id(replica->remote));
} else {
offset += snprintf(buffer + offset, LINE_MAX - offset, "%s",
rsc_printable_id(replica->container));
}
if (replica->ipaddr) {
offset += snprintf(buffer + offset, LINE_MAX - offset, " (%s)",
replica->ipaddr);
}
pe__common_output_text(out, rsc, buffer, node, show_opts);
}
PCMK__OUTPUT_ARGS("bundle", "uint32_t", "pcmk_resource_t *", "GList *",
"GList *")
int
pe__bundle_text(pcmk__output_t *out, va_list args)
{
uint32_t show_opts = va_arg(args, uint32_t);
pcmk_resource_t *rsc = va_arg(args, pcmk_resource_t *);
GList *only_node = va_arg(args, GList *);
GList *only_rsc = va_arg(args, GList *);
const char *desc = NULL;
pe__bundle_variant_data_t *bundle_data = NULL;
int rc = pcmk_rc_no_output;
gboolean print_everything = TRUE;
desc = pe__resource_description(rsc, show_opts);
pcmk__assert(rsc != NULL);
get_bundle_variant_data(bundle_data, rsc);
if (rsc->priv->fns->is_filtered(rsc, only_rsc, TRUE)) {
return rc;
}
print_everything = pcmk__str_in_list(rsc->id, only_rsc, pcmk__str_star_matches);
for (GList *gIter = bundle_data->replicas; gIter != NULL;
gIter = gIter->next) {
pcmk__bundle_replica_t *replica = gIter->data;
pcmk_resource_t *ip = replica->ip;
pcmk_resource_t *child = replica->child;
pcmk_resource_t *container = replica->container;
pcmk_resource_t *remote = replica->remote;
gboolean print_ip, print_child, print_ctnr, print_remote;
pcmk__assert(replica != NULL);
if (pcmk__rsc_filtered_by_node(container, only_node)) {
continue;
}
print_ip = (ip != NULL)
&& !ip->priv->fns->is_filtered(ip, only_rsc,
print_everything);
print_child = (child != NULL)
&& !child->priv->fns->is_filtered(child, only_rsc,
print_everything);
print_ctnr = !container->priv->fns->is_filtered(container, only_rsc,
print_everything);
print_remote = (remote != NULL)
&& !remote->priv->fns->is_filtered(remote, only_rsc,
print_everything);
if (pcmk_is_set(show_opts, pcmk_show_implicit_rscs) ||
(print_everything == FALSE && (print_ip || print_child || print_ctnr || print_remote))) {
/* The text output messages used below require pe_print_implicit to
* be set to do anything.
*/
uint32_t new_show_opts = show_opts | pcmk_show_implicit_rscs;
PCMK__OUTPUT_LIST_HEADER(out, FALSE, rc, "Container bundle%s: %s [%s]%s%s%s%s%s",
(bundle_data->nreplicas > 1)? " set" : "",
rsc->id, bundle_data->image,
pcmk_is_set(rsc->flags, pcmk__rsc_unique)? " (unique)" : "",
desc ? " (" : "", desc ? desc : "", desc ? ")" : "",
get_unmanaged_str(rsc));
if (pcmk__list_of_multiple(bundle_data->replicas)) {
out->list_item(out, NULL, "Replica[%d]", replica->offset);
}
out->begin_list(out, NULL, NULL, NULL);
if (print_ip) {
out->message(out, (const char *) ip->priv->xml->name,
new_show_opts, ip, only_node, only_rsc);
}
if (print_child) {
out->message(out, (const char *) child->priv->xml->name,
new_show_opts, child, only_node, only_rsc);
}
if (print_ctnr) {
out->message(out, (const char *) container->priv->xml->name,
new_show_opts, container, only_node, only_rsc);
}
if (print_remote) {
out->message(out, (const char *) remote->priv->xml->name,
new_show_opts, remote, only_node, only_rsc);
}
out->end_list(out);
} else if (print_everything == FALSE && !(print_ip || print_child || print_ctnr || print_remote)) {
continue;
} else {
PCMK__OUTPUT_LIST_HEADER(out, FALSE, rc, "Container bundle%s: %s [%s]%s%s%s%s%s",
(bundle_data->nreplicas > 1)? " set" : "",
rsc->id, bundle_data->image,
pcmk_is_set(rsc->flags, pcmk__rsc_unique)? " (unique)" : "",
desc ? " (" : "", desc ? desc : "", desc ? ")" : "",
get_unmanaged_str(rsc));
pe__bundle_replica_output_text(out, replica,
pcmk__current_node(container),
show_opts);
}
}
PCMK__OUTPUT_LIST_FOOTER(out, rc);
return rc;
}
static void
free_bundle_replica(pcmk__bundle_replica_t *replica)
{
if (replica == NULL) {
return;
}
- if (replica->node) {
- free(replica->node);
- replica->node = NULL;
- }
+ pcmk__free_node_copy(replica->node);
+ replica->node = NULL;
if (replica->ip) {
pcmk__xml_free(replica->ip->priv->xml);
replica->ip->priv->xml = NULL;
replica->ip->priv->fns->free(replica->ip);
}
if (replica->container) {
pcmk__xml_free(replica->container->priv->xml);
replica->container->priv->xml = NULL;
replica->container->priv->fns->free(replica->container);
}
if (replica->remote) {
pcmk__xml_free(replica->remote->priv->xml);
replica->remote->priv->xml = NULL;
replica->remote->priv->fns->free(replica->remote);
}
free(replica->ipaddr);
free(replica);
}
void
pe__free_bundle(pcmk_resource_t *rsc)
{
pe__bundle_variant_data_t *bundle_data = NULL;
CRM_CHECK(rsc != NULL, return);
get_bundle_variant_data(bundle_data, rsc);
pcmk__rsc_trace(rsc, "Freeing %s", rsc->id);
free(bundle_data->prefix);
free(bundle_data->image);
free(bundle_data->control_port);
free(bundle_data->host_network);
free(bundle_data->host_netmask);
free(bundle_data->ip_range_start);
free(bundle_data->container_network);
free(bundle_data->launcher_options);
free(bundle_data->container_command);
g_free(bundle_data->container_host_options);
g_list_free_full(bundle_data->replicas,
(GDestroyNotify) free_bundle_replica);
g_list_free_full(bundle_data->mounts, (GDestroyNotify)mount_free);
g_list_free_full(bundle_data->ports, (GDestroyNotify)port_free);
g_list_free(rsc->priv->children);
if(bundle_data->child) {
pcmk__xml_free(bundle_data->child->priv->xml);
bundle_data->child->priv->xml = NULL;
bundle_data->child->priv->fns->free(bundle_data->child);
}
common_free(rsc);
}
enum rsc_role_e
pe__bundle_resource_state(const pcmk_resource_t *rsc, gboolean current)
{
enum rsc_role_e container_role = pcmk_role_unknown;
return container_role;
}
/*!
* \brief Get the number of configured replicas in a bundle
*
* \param[in] rsc Bundle resource
*
* \return Number of configured replicas, or 0 on error
*/
int
pe_bundle_replicas(const pcmk_resource_t *rsc)
{
if (pcmk__is_bundle(rsc)) {
pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, rsc);
return bundle_data->nreplicas;
}
return 0;
}
void
pe__count_bundle(pcmk_resource_t *rsc)
{
pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, rsc);
for (GList *item = bundle_data->replicas; item != NULL; item = item->next) {
pcmk__bundle_replica_t *replica = item->data;
if (replica->ip) {
replica->ip->priv->fns->count(replica->ip);
}
if (replica->child) {
replica->child->priv->fns->count(replica->child);
}
if (replica->container) {
replica->container->priv->fns->count(replica->container);
}
if (replica->remote) {
replica->remote->priv->fns->count(replica->remote);
}
}
}
gboolean
pe__bundle_is_filtered(const pcmk_resource_t *rsc, GList *only_rsc,
gboolean check_parent)
{
gboolean passes = FALSE;
pe__bundle_variant_data_t *bundle_data = NULL;
if (pcmk__str_in_list(rsc_printable_id(rsc), only_rsc, pcmk__str_star_matches)) {
passes = TRUE;
} else {
get_bundle_variant_data(bundle_data, rsc);
for (GList *gIter = bundle_data->replicas; gIter != NULL; gIter = gIter->next) {
pcmk__bundle_replica_t *replica = gIter->data;
pcmk_resource_t *ip = replica->ip;
pcmk_resource_t *child = replica->child;
pcmk_resource_t *container = replica->container;
pcmk_resource_t *remote = replica->remote;
if ((ip != NULL)
&& !ip->priv->fns->is_filtered(ip, only_rsc, FALSE)) {
passes = TRUE;
break;
}
if ((child != NULL)
&& !child->priv->fns->is_filtered(child, only_rsc, FALSE)) {
passes = TRUE;
break;
}
if (!container->priv->fns->is_filtered(container, only_rsc,
FALSE)) {
passes = TRUE;
break;
}
if ((remote != NULL)
&& !remote->priv->fns->is_filtered(remote, only_rsc, FALSE)) {
passes = TRUE;
break;
}
}
}
return !passes;
}
/*!
* \internal
* \brief Get a list of a bundle's containers
*
* \param[in] bundle Bundle resource
*
* \return Newly created list of \p bundle's containers
* \note It is the caller's responsibility to free the result with
* g_list_free().
*/
GList *
pe__bundle_containers(const pcmk_resource_t *bundle)
{
GList *containers = NULL;
const pe__bundle_variant_data_t *data = NULL;
get_bundle_variant_data(data, bundle);
for (GList *iter = data->replicas; iter != NULL; iter = iter->next) {
pcmk__bundle_replica_t *replica = iter->data;
containers = g_list_append(containers, replica->container);
}
return containers;
}
// Bundle implementation of pcmk__rsc_methods_t:active_node()
pcmk_node_t *
pe__bundle_active_node(const pcmk_resource_t *rsc, unsigned int *count_all,
unsigned int *count_clean)
{
pcmk_node_t *active = NULL;
pcmk_node_t *node = NULL;
pcmk_resource_t *container = NULL;
GList *containers = NULL;
GList *iter = NULL;
GHashTable *nodes = NULL;
const pe__bundle_variant_data_t *data = NULL;
if (count_all != NULL) {
*count_all = 0;
}
if (count_clean != NULL) {
*count_clean = 0;
}
if (rsc == NULL) {
return NULL;
}
/* For the purposes of this method, we only care about where the bundle's
* containers are active, so build a list of active containers.
*/
get_bundle_variant_data(data, rsc);
for (iter = data->replicas; iter != NULL; iter = iter->next) {
pcmk__bundle_replica_t *replica = iter->data;
if (replica->container->priv->active_nodes != NULL) {
containers = g_list_append(containers, replica->container);
}
}
if (containers == NULL) {
return NULL;
}
/* If the bundle has only a single active container, just use that
* container's method. If live migration is ever supported for bundle
* containers, this will allow us to prefer the migration source when there
* is only one container and it is migrating. For now, this just lets us
* avoid creating the nodes table.
*/
if (pcmk__list_of_1(containers)) {
container = containers->data;
node = container->priv->fns->active_node(container, count_all,
count_clean);
g_list_free(containers);
return node;
}
// Add all containers' active nodes to a hash table (for uniqueness)
nodes = g_hash_table_new(NULL, NULL);
for (iter = containers; iter != NULL; iter = iter->next) {
container = iter->data;
for (GList *node_iter = container->priv->active_nodes;
node_iter != NULL; node_iter = node_iter->next) {
node = node_iter->data;
// If insert returns true, we haven't counted this node yet
if (g_hash_table_insert(nodes, (gpointer) node->details,
(gpointer) node)
&& !pe__count_active_node(rsc, node, &active, count_all,
count_clean)) {
goto done;
}
}
}
done:
g_list_free(containers);
g_hash_table_destroy(nodes);
return active;
}
/*!
* \internal
* \brief Get maximum bundle resource instances per node
*
* \param[in] rsc Bundle resource to check
*
* \return Maximum number of \p rsc instances that can be active on one node
*/
unsigned int
pe__bundle_max_per_node(const pcmk_resource_t *rsc)
{
pe__bundle_variant_data_t *bundle_data = NULL;
get_bundle_variant_data(bundle_data, rsc);
pcmk__assert(bundle_data->nreplicas_per_host >= 0);
return (unsigned int) bundle_data->nreplicas_per_host;
}
diff --git a/lib/pengine/complex.c b/lib/pengine/complex.c
index d47a3ad8c7..2e9ec18574 100644
--- a/lib/pengine/complex.c
+++ b/lib/pengine/complex.c
@@ -1,1274 +1,1274 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <crm/pengine/rules.h>
#include <crm/pengine/internal.h>
#include <crm/common/xml.h>
#include <crm/common/xml_internal.h>
#include <crm/common/scheduler_internal.h>
#include "pe_status_private.h"
void populate_hash(xmlNode * nvpair_list, GHashTable * hash, const char **attrs, int attrs_length);
static pcmk_node_t *active_node(const pcmk_resource_t *rsc,
unsigned int *count_all,
unsigned int *count_clean);
static pcmk__rsc_methods_t resource_class_functions[] = {
{
native_unpack,
native_find_rsc,
native_parameter,
native_active,
native_resource_state,
native_location,
native_free,
pe__count_common,
pe__native_is_filtered,
active_node,
pe__primitive_max_per_node,
},
{
group_unpack,
native_find_rsc,
native_parameter,
group_active,
group_resource_state,
native_location,
group_free,
pe__count_common,
pe__group_is_filtered,
active_node,
pe__group_max_per_node,
},
{
clone_unpack,
native_find_rsc,
native_parameter,
clone_active,
clone_resource_state,
native_location,
clone_free,
pe__count_common,
pe__clone_is_filtered,
active_node,
pe__clone_max_per_node,
},
{
pe__unpack_bundle,
native_find_rsc,
native_parameter,
pe__bundle_active,
pe__bundle_resource_state,
native_location,
pe__free_bundle,
pe__count_bundle,
pe__bundle_is_filtered,
pe__bundle_active_node,
pe__bundle_max_per_node,
}
};
static enum pcmk__rsc_variant
get_resource_type(const char *name)
{
if (pcmk__str_eq(name, PCMK_XE_PRIMITIVE, pcmk__str_casei)) {
return pcmk__rsc_variant_primitive;
} else if (pcmk__str_eq(name, PCMK_XE_GROUP, pcmk__str_casei)) {
return pcmk__rsc_variant_group;
} else if (pcmk__str_eq(name, PCMK_XE_CLONE, pcmk__str_casei)) {
return pcmk__rsc_variant_clone;
} else if (pcmk__str_eq(name, PCMK_XE_BUNDLE, pcmk__str_casei)) {
return pcmk__rsc_variant_bundle;
}
return pcmk__rsc_variant_unknown;
}
/*!
* \internal
* \brief Insert a meta-attribute if not already present
*
* \param[in] key Meta-attribute name
* \param[in] value Meta-attribute value to add if not already present
* \param[in,out] table Meta-attribute hash table to insert into
*
* \note This is like pcmk__insert_meta() except it won't overwrite existing
* values.
*/
static void
dup_attr(gpointer key, gpointer value, gpointer user_data)
{
GHashTable *table = user_data;
CRM_CHECK((key != NULL) && (table != NULL), return);
if (pcmk__str_eq((const char *) value, "#default", pcmk__str_casei)) {
// @COMPAT Deprecated since 2.1.8
pcmk__config_warn("Support for setting meta-attributes (such as %s) to "
"the explicit value '#default' is deprecated and "
"will be removed in a future release",
(const char *) key);
} else if ((value != NULL) && (g_hash_table_lookup(table, key) == NULL)) {
pcmk__insert_dup(table, (const char *) key, (const char *) value);
}
}
static void
expand_parents_fixed_nvpairs(pcmk_resource_t *rsc,
pe_rule_eval_data_t *rule_data,
GHashTable *meta_hash, pcmk_scheduler_t *scheduler)
{
GHashTable *parent_orig_meta = pcmk__strkey_table(free, free);
pcmk_resource_t *p = rsc->priv->parent;
if (p == NULL) {
return ;
}
/* Search all parent resources, get the fixed value of
* PCMK_XE_META_ATTRIBUTES set only in the original xml, and stack it in the
* hash table. The fixed value of the lower parent resource takes precedence
* and is not overwritten.
*/
while(p != NULL) {
/* A hash table for comparison is generated, including the id-ref. */
pe__unpack_dataset_nvpairs(p->priv->xml, PCMK_XE_META_ATTRIBUTES,
rule_data, parent_orig_meta, NULL,
scheduler);
p = p->priv->parent;
}
if (parent_orig_meta != NULL) {
// This will not overwrite any values already existing for child
g_hash_table_foreach(parent_orig_meta, dup_attr, meta_hash);
}
if (parent_orig_meta != NULL) {
g_hash_table_destroy(parent_orig_meta);
}
return ;
}
/*
* \brief Get fully evaluated resource meta-attributes
*
* \param[in,out] meta_hash Where to store evaluated meta-attributes
* \param[in] rsc Resource to get meta-attributes for
* \param[in] node Ignored
* \param[in,out] scheduler Scheduler data
*/
void
get_meta_attributes(GHashTable * meta_hash, pcmk_resource_t * rsc,
pcmk_node_t *node, pcmk_scheduler_t *scheduler)
{
pe_rsc_eval_data_t rsc_rule_data = {
.standard = crm_element_value(rsc->priv->xml, PCMK_XA_CLASS),
.provider = crm_element_value(rsc->priv->xml, PCMK_XA_PROVIDER),
.agent = crm_element_value(rsc->priv->xml, PCMK_XA_TYPE)
};
pe_rule_eval_data_t rule_data = {
.node_hash = NULL,
.now = scheduler->priv->now,
.match_data = NULL,
.rsc_data = &rsc_rule_data,
.op_data = NULL
};
for (xmlAttrPtr a = pcmk__xe_first_attr(rsc->priv->xml);
a != NULL; a = a->next) {
if (a->children != NULL) {
dup_attr((gpointer) a->name, (gpointer) a->children->content,
meta_hash);
}
}
pe__unpack_dataset_nvpairs(rsc->priv->xml, PCMK_XE_META_ATTRIBUTES,
&rule_data, meta_hash, NULL, scheduler);
/* Set the PCMK_XE_META_ATTRIBUTES explicitly set in the parent resource to
* the hash table of the child resource. If it is already explicitly set as
* a child, it will not be overwritten.
*/
if (rsc->priv->parent != NULL) {
expand_parents_fixed_nvpairs(rsc, &rule_data, meta_hash, scheduler);
}
/* check the defaults */
pe__unpack_dataset_nvpairs(scheduler->priv->rsc_defaults,
PCMK_XE_META_ATTRIBUTES, &rule_data, meta_hash,
NULL, scheduler);
/* If there is PCMK_XE_META_ATTRIBUTES that the parent resource has not
* explicitly set, set a value that is not set from PCMK_XE_RSC_DEFAULTS
* either. The values already set up to this point will not be overwritten.
*/
if (rsc->priv->parent != NULL) {
g_hash_table_foreach(rsc->priv->parent->priv->meta, dup_attr,
meta_hash);
}
}
/*!
* \brief Get final values of a resource's instance attributes
*
* \param[in,out] instance_attrs Where to store the instance attributes
* \param[in] rsc Resource to get instance attributes for
* \param[in] node If not NULL, evaluate rules for this node
* \param[in,out] scheduler Scheduler data
*/
void
get_rsc_attributes(GHashTable *instance_attrs, const pcmk_resource_t *rsc,
const pcmk_node_t *node, pcmk_scheduler_t *scheduler)
{
pe_rule_eval_data_t rule_data = {
.node_hash = NULL,
.now = NULL,
.match_data = NULL,
.rsc_data = NULL,
.op_data = NULL
};
CRM_CHECK((instance_attrs != NULL) && (rsc != NULL) && (scheduler != NULL),
return);
rule_data.now = scheduler->priv->now;
if (node != NULL) {
rule_data.node_hash = node->priv->attrs;
}
// Evaluate resource's own values, then its ancestors' values
pe__unpack_dataset_nvpairs(rsc->priv->xml, PCMK_XE_INSTANCE_ATTRIBUTES,
&rule_data, instance_attrs, NULL, scheduler);
if (rsc->priv->parent != NULL) {
get_rsc_attributes(instance_attrs, rsc->priv->parent, node, scheduler);
}
}
static char *
template_op_key(xmlNode * op)
{
const char *name = crm_element_value(op, PCMK_XA_NAME);
const char *role = crm_element_value(op, PCMK_XA_ROLE);
char *key = NULL;
if ((role == NULL)
|| pcmk__strcase_any_of(role, PCMK_ROLE_STARTED, PCMK_ROLE_UNPROMOTED,
PCMK__ROLE_UNPROMOTED_LEGACY, NULL)) {
role = PCMK__ROLE_UNKNOWN;
}
key = crm_strdup_printf("%s-%s", name, role);
return key;
}
static gboolean
unpack_template(xmlNode *xml_obj, xmlNode **expanded_xml,
pcmk_scheduler_t *scheduler)
{
xmlNode *cib_resources = NULL;
xmlNode *template = NULL;
xmlNode *new_xml = NULL;
xmlNode *child_xml = NULL;
xmlNode *rsc_ops = NULL;
xmlNode *template_ops = NULL;
const char *template_ref = NULL;
const char *id = NULL;
if (xml_obj == NULL) {
pcmk__config_err("No resource object for template unpacking");
return FALSE;
}
template_ref = crm_element_value(xml_obj, PCMK_XA_TEMPLATE);
if (template_ref == NULL) {
return TRUE;
}
id = pcmk__xe_id(xml_obj);
if (id == NULL) {
pcmk__config_err("'%s' object must have a id", xml_obj->name);
return FALSE;
}
if (pcmk__str_eq(template_ref, id, pcmk__str_none)) {
pcmk__config_err("The resource object '%s' should not reference itself",
id);
return FALSE;
}
cib_resources = get_xpath_object("//" PCMK_XE_RESOURCES, scheduler->input,
LOG_TRACE);
if (cib_resources == NULL) {
pcmk__config_err("No resources configured");
return FALSE;
}
template = pcmk__xe_first_child(cib_resources, PCMK_XE_TEMPLATE,
PCMK_XA_ID, template_ref);
if (template == NULL) {
pcmk__config_err("No template named '%s'", template_ref);
return FALSE;
}
new_xml = pcmk__xml_copy(NULL, template);
xmlNodeSetName(new_xml, xml_obj->name);
crm_xml_add(new_xml, PCMK_XA_ID, id);
crm_xml_add(new_xml, PCMK__META_CLONE,
crm_element_value(xml_obj, PCMK__META_CLONE));
template_ops = pcmk__xe_first_child(new_xml, PCMK_XE_OPERATIONS, NULL,
NULL);
for (child_xml = pcmk__xe_first_child(xml_obj, NULL, NULL, NULL);
child_xml != NULL; child_xml = pcmk__xe_next(child_xml, NULL)) {
xmlNode *new_child = pcmk__xml_copy(new_xml, child_xml);
if (pcmk__xe_is(new_child, PCMK_XE_OPERATIONS)) {
rsc_ops = new_child;
}
}
if (template_ops && rsc_ops) {
xmlNode *op = NULL;
GHashTable *rsc_ops_hash = pcmk__strkey_table(free, NULL);
for (op = pcmk__xe_first_child(rsc_ops, NULL, NULL, NULL); op != NULL;
op = pcmk__xe_next(op, NULL)) {
char *key = template_op_key(op);
g_hash_table_insert(rsc_ops_hash, key, op);
}
for (op = pcmk__xe_first_child(template_ops, NULL, NULL, NULL);
op != NULL; op = pcmk__xe_next(op, NULL)) {
char *key = template_op_key(op);
if (g_hash_table_lookup(rsc_ops_hash, key) == NULL) {
pcmk__xml_copy(rsc_ops, op);
}
free(key);
}
if (rsc_ops_hash) {
g_hash_table_destroy(rsc_ops_hash);
}
pcmk__xml_free(template_ops);
}
/*pcmk__xml_free(*expanded_xml); */
*expanded_xml = new_xml;
#if 0 /* Disable multi-level templates for now */
if (!unpack_template(new_xml, expanded_xml, scheduler)) {
pcmk__xml_free(*expanded_xml);
*expanded_xml = NULL;
return FALSE;
}
#endif
return TRUE;
}
static gboolean
add_template_rsc(xmlNode *xml_obj, pcmk_scheduler_t *scheduler)
{
const char *template_ref = NULL;
const char *id = NULL;
if (xml_obj == NULL) {
pcmk__config_err("No resource object for processing resource list "
"of template");
return FALSE;
}
template_ref = crm_element_value(xml_obj, PCMK_XA_TEMPLATE);
if (template_ref == NULL) {
return TRUE;
}
id = pcmk__xe_id(xml_obj);
if (id == NULL) {
pcmk__config_err("'%s' object must have a id", xml_obj->name);
return FALSE;
}
if (pcmk__str_eq(template_ref, id, pcmk__str_none)) {
pcmk__config_err("The resource object '%s' should not reference itself",
id);
return FALSE;
}
pcmk__add_idref(scheduler->priv->templates, template_ref, id);
return TRUE;
}
/*!
* \internal
* \brief Check whether a clone or instance being unpacked is globally unique
*
* \param[in] rsc Clone or clone instance to check
*
* \return \c true if \p rsc is globally unique according to its
* meta-attributes, otherwise \c false
*/
static bool
detect_unique(const pcmk_resource_t *rsc)
{
const char *value = g_hash_table_lookup(rsc->priv->meta,
PCMK_META_GLOBALLY_UNIQUE);
if (value == NULL) { // Default to true if clone-node-max > 1
value = g_hash_table_lookup(rsc->priv->meta,
PCMK_META_CLONE_NODE_MAX);
if (value != NULL) {
int node_max = 1;
if ((pcmk__scan_min_int(value, &node_max, 0) == pcmk_rc_ok)
&& (node_max > 1)) {
return true;
}
}
return false;
}
return crm_is_true(value);
}
static void
free_params_table(gpointer data)
{
g_hash_table_destroy((GHashTable *) data);
}
/*!
* \brief Get a table of resource parameters
*
* \param[in,out] rsc Resource to query
* \param[in] node Node for evaluating rules (NULL for defaults)
* \param[in,out] scheduler Scheduler data
*
* \return Hash table containing resource parameter names and values
* (or NULL if \p rsc or \p scheduler is NULL)
* \note The returned table will be destroyed when the resource is freed, so
* callers should not destroy it.
*/
GHashTable *
pe_rsc_params(pcmk_resource_t *rsc, const pcmk_node_t *node,
pcmk_scheduler_t *scheduler)
{
GHashTable *params_on_node = NULL;
/* A NULL node is used to request the resource's default parameters
* (not evaluated for node), but we always want something non-NULL
* as a hash table key.
*/
const char *node_name = "";
// Sanity check
if ((rsc == NULL) || (scheduler == NULL)) {
return NULL;
}
if ((node != NULL) && (node->priv->name != NULL)) {
node_name = node->priv->name;
}
// Find the parameter table for given node
if (rsc->priv->parameter_cache == NULL) {
rsc->priv->parameter_cache = pcmk__strikey_table(free,
free_params_table);
} else {
params_on_node = g_hash_table_lookup(rsc->priv->parameter_cache,
node_name);
}
// If none exists yet, create one with parameters evaluated for node
if (params_on_node == NULL) {
params_on_node = pcmk__strkey_table(free, free);
get_rsc_attributes(params_on_node, rsc, node, scheduler);
g_hash_table_insert(rsc->priv->parameter_cache, strdup(node_name),
params_on_node);
}
return params_on_node;
}
/*!
* \internal
* \brief Unpack a resource's \c PCMK_META_REQUIRES meta-attribute
*
* \param[in,out] rsc Resource being unpacked
* \param[in] value Value of \c PCMK_META_REQUIRES meta-attribute
* \param[in] is_default Whether \p value was selected by default
*/
static void
unpack_requires(pcmk_resource_t *rsc, const char *value, bool is_default)
{
const pcmk_scheduler_t *scheduler = rsc->priv->scheduler;
if (pcmk__str_eq(value, PCMK_VALUE_NOTHING, pcmk__str_casei)) {
} else if (pcmk__str_eq(value, PCMK_VALUE_QUORUM, pcmk__str_casei)) {
pcmk__set_rsc_flags(rsc, pcmk__rsc_needs_quorum);
} else if (pcmk__str_eq(value, PCMK_VALUE_FENCING, pcmk__str_casei)) {
pcmk__set_rsc_flags(rsc, pcmk__rsc_needs_fencing);
if (!pcmk_is_set(scheduler->flags, pcmk__sched_fencing_enabled)) {
pcmk__config_warn("%s requires fencing but fencing is disabled",
rsc->id);
}
} else if (pcmk__str_eq(value, PCMK_VALUE_UNFENCING, pcmk__str_casei)) {
if (pcmk_is_set(rsc->flags, pcmk__rsc_fence_device)) {
pcmk__config_warn("Resetting \"" PCMK_META_REQUIRES "\" for %s "
"to \"" PCMK_VALUE_QUORUM "\" because fencing "
"devices cannot require unfencing", rsc->id);
unpack_requires(rsc, PCMK_VALUE_QUORUM, true);
return;
} else if (!pcmk_is_set(scheduler->flags, pcmk__sched_fencing_enabled)) {
pcmk__config_warn("Resetting \"" PCMK_META_REQUIRES "\" for %s "
"to \"" PCMK_VALUE_QUORUM "\" because fencing is "
"disabled", rsc->id);
unpack_requires(rsc, PCMK_VALUE_QUORUM, true);
return;
} else {
pcmk__set_rsc_flags(rsc, pcmk__rsc_needs_fencing
|pcmk__rsc_needs_unfencing);
}
} else {
const char *orig_value = value;
if (pcmk_is_set(rsc->flags, pcmk__rsc_fence_device)) {
value = PCMK_VALUE_QUORUM;
} else if (pcmk__is_primitive(rsc)
&& xml_contains_remote_node(rsc->priv->xml)) {
value = PCMK_VALUE_QUORUM;
} else if (pcmk_is_set(scheduler->flags, pcmk__sched_enable_unfencing)) {
value = PCMK_VALUE_UNFENCING;
} else if (pcmk_is_set(scheduler->flags, pcmk__sched_fencing_enabled)) {
value = PCMK_VALUE_FENCING;
} else if (scheduler->no_quorum_policy == pcmk_no_quorum_ignore) {
value = PCMK_VALUE_NOTHING;
} else {
value = PCMK_VALUE_QUORUM;
}
if (orig_value != NULL) {
pcmk__config_err("Resetting '" PCMK_META_REQUIRES "' for %s "
"to '%s' because '%s' is not valid",
rsc->id, value, orig_value);
}
unpack_requires(rsc, value, true);
return;
}
pcmk__rsc_trace(rsc, "\tRequired to start: %s%s", value,
(is_default? " (default)" : ""));
}
/*!
* \internal
* \brief Parse resource priority from meta-attribute
*
* \param[in,out] rsc Resource being unpacked
*/
static void
unpack_priority(pcmk_resource_t *rsc)
{
const char *value = g_hash_table_lookup(rsc->priv->meta,
PCMK_META_PRIORITY);
int rc = pcmk_parse_score(value, &(rsc->priv->priority), 0);
if (rc != pcmk_rc_ok) {
pcmk__config_warn("Using default (0) for resource %s "
PCMK_META_PRIORITY
" because '%s' is not a valid value: %s",
rsc->id, value, pcmk_rc_str(rc));
}
}
/*!
* \internal
* \brief Parse resource stickiness from meta-attribute
*
* \param[in,out] rsc Resource being unpacked
*/
static void
unpack_stickiness(pcmk_resource_t *rsc)
{
const char *value = g_hash_table_lookup(rsc->priv->meta,
PCMK_META_RESOURCE_STICKINESS);
if (pcmk__str_eq(value, PCMK_VALUE_DEFAULT, pcmk__str_casei)) {
// @COMPAT Deprecated since 2.1.8
pcmk__config_warn("Support for setting "
PCMK_META_RESOURCE_STICKINESS
" to the explicit value '" PCMK_VALUE_DEFAULT
"' is deprecated and will be removed in a "
"future release (just leave it unset)");
} else {
int rc = pcmk_parse_score(value, &(rsc->priv->stickiness), 0);
if (rc != pcmk_rc_ok) {
pcmk__config_warn("Using default (0) for resource %s "
PCMK_META_RESOURCE_STICKINESS
" because '%s' is not a valid value: %s",
rsc->id, value, pcmk_rc_str(rc));
}
}
}
/*!
* \internal
* \brief Parse resource migration threshold from meta-attribute
*
* \param[in,out] rsc Resource being unpacked
*/
static void
unpack_migration_threshold(pcmk_resource_t *rsc)
{
const char *value = g_hash_table_lookup(rsc->priv->meta,
PCMK_META_MIGRATION_THRESHOLD);
if (pcmk__str_eq(value, PCMK_VALUE_DEFAULT, pcmk__str_casei)) {
// @COMPAT Deprecated since 2.1.8
pcmk__config_warn("Support for setting "
PCMK_META_MIGRATION_THRESHOLD
" to the explicit value '" PCMK_VALUE_DEFAULT
"' is deprecated and will be removed in a "
"future release (just leave it unset)");
rsc->priv->ban_after_failures = PCMK_SCORE_INFINITY;
} else {
int rc = pcmk_parse_score(value, &(rsc->priv->ban_after_failures),
PCMK_SCORE_INFINITY);
if ((rc != pcmk_rc_ok) || (rsc->priv->ban_after_failures < 0)) {
pcmk__config_warn("Using default (" PCMK_VALUE_INFINITY
") for resource %s meta-attribute "
PCMK_META_MIGRATION_THRESHOLD
" because '%s' is not a valid value: %s",
rsc->id, value, pcmk_rc_str(rc));
rsc->priv->ban_after_failures = PCMK_SCORE_INFINITY;
}
}
}
/*!
* \internal
* \brief Unpack configuration XML for a given resource
*
* Unpack the XML object containing a resource's configuration into a new
* \c pcmk_resource_t object.
*
* \param[in] xml_obj XML node containing the resource's configuration
* \param[out] rsc Where to store the unpacked resource information
* \param[in] parent Resource's parent, if any
* \param[in,out] scheduler Scheduler data
*
* \return Standard Pacemaker return code
* \note If pcmk_rc_ok is returned, \p *rsc is guaranteed to be non-NULL, and
* the caller is responsible for freeing it using its variant-specific
* free() method. Otherwise, \p *rsc is guaranteed to be NULL.
*/
int
pe__unpack_resource(xmlNode *xml_obj, pcmk_resource_t **rsc,
pcmk_resource_t *parent, pcmk_scheduler_t *scheduler)
{
xmlNode *expanded_xml = NULL;
xmlNode *ops = NULL;
const char *value = NULL;
const char *id = NULL;
bool guest_node = false;
bool remote_node = false;
pcmk__resource_private_t *rsc_private = NULL;
pe_rule_eval_data_t rule_data = {
.node_hash = NULL,
.now = NULL,
.match_data = NULL,
.rsc_data = NULL,
.op_data = NULL
};
CRM_CHECK(rsc != NULL, return EINVAL);
CRM_CHECK((xml_obj != NULL) && (scheduler != NULL),
*rsc = NULL;
return EINVAL);
rule_data.now = scheduler->priv->now;
crm_log_xml_trace(xml_obj, "[raw XML]");
id = crm_element_value(xml_obj, PCMK_XA_ID);
if (id == NULL) {
pcmk__config_err("Ignoring <%s> configuration without " PCMK_XA_ID,
xml_obj->name);
return pcmk_rc_unpack_error;
}
if (unpack_template(xml_obj, &expanded_xml, scheduler) == FALSE) {
return pcmk_rc_unpack_error;
}
*rsc = calloc(1, sizeof(pcmk_resource_t));
if (*rsc == NULL) {
pcmk__sched_err(scheduler,
"Unable to allocate memory for resource '%s'", id);
return ENOMEM;
}
(*rsc)->priv = calloc(1, sizeof(pcmk__resource_private_t));
if ((*rsc)->priv == NULL) {
pcmk__sched_err(scheduler,
"Unable to allocate memory for resource '%s'", id);
free(*rsc);
return ENOMEM;
}
rsc_private = (*rsc)->priv;
rsc_private->scheduler = scheduler;
if (expanded_xml) {
crm_log_xml_trace(expanded_xml, "[expanded XML]");
rsc_private->xml = expanded_xml;
rsc_private->orig_xml = xml_obj;
} else {
rsc_private->xml = xml_obj;
rsc_private->orig_xml = NULL;
}
/* Do not use xml_obj from here on, use (*rsc)->xml in case templates are involved */
rsc_private->parent = parent;
ops = pcmk__xe_first_child(rsc_private->xml, PCMK_XE_OPERATIONS, NULL,
NULL);
rsc_private->ops_xml = pcmk__xe_resolve_idref(ops, scheduler->input);
rsc_private->variant = get_resource_type((const char *)
rsc_private->xml->name);
if (rsc_private->variant == pcmk__rsc_variant_unknown) {
pcmk__config_err("Ignoring resource '%s' of unknown type '%s'",
id, rsc_private->xml->name);
common_free(*rsc);
*rsc = NULL;
return pcmk_rc_unpack_error;
}
rsc_private->meta = pcmk__strkey_table(free, free);
rsc_private->utilization = pcmk__strkey_table(free, free);
- rsc_private->probed_nodes = pcmk__strkey_table(NULL, free);
- rsc_private->allowed_nodes = pcmk__strkey_table(NULL, free);
+ rsc_private->probed_nodes = pcmk__strkey_table(NULL, pcmk__free_node_copy);
+ rsc_private->allowed_nodes = pcmk__strkey_table(NULL, pcmk__free_node_copy);
value = crm_element_value(rsc_private->xml, PCMK__META_CLONE);
if (value) {
(*rsc)->id = crm_strdup_printf("%s:%s", id, value);
pcmk__insert_meta(rsc_private, PCMK__META_CLONE, value);
} else {
(*rsc)->id = strdup(id);
}
rsc_private->fns = &resource_class_functions[rsc_private->variant];
get_meta_attributes(rsc_private->meta, *rsc, NULL, scheduler);
(*rsc)->flags = 0;
pcmk__set_rsc_flags(*rsc, pcmk__rsc_unassigned);
if (!pcmk_is_set(scheduler->flags, pcmk__sched_in_maintenance)) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_managed);
}
rsc_private->orig_role = pcmk_role_stopped;
rsc_private->next_role = pcmk_role_unknown;
unpack_priority(*rsc);
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_CRITICAL);
if ((value == NULL) || crm_is_true(value)) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_critical);
}
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_NOTIFY);
if (crm_is_true(value)) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_notify);
}
if (xml_contains_remote_node(rsc_private->xml)) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_is_remote_connection);
if (g_hash_table_lookup(rsc_private->meta, PCMK__META_CONTAINER)) {
guest_node = true;
} else {
remote_node = true;
}
}
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_ALLOW_MIGRATE);
if (crm_is_true(value)) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_migratable);
} else if ((value == NULL) && remote_node) {
/* By default, we want remote nodes to be able
* to float around the cluster without having to stop all the
* resources within the remote-node before moving. Allowing
* migration support enables this feature. If this ever causes
* problems, migration support can be explicitly turned off with
* PCMK_META_ALLOW_MIGRATE=false.
*/
pcmk__set_rsc_flags(*rsc, pcmk__rsc_migratable);
}
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_IS_MANAGED);
if (value != NULL) {
if (pcmk__str_eq(PCMK_VALUE_DEFAULT, value, pcmk__str_casei)) {
// @COMPAT Deprecated since 2.1.8
pcmk__config_warn("Support for setting " PCMK_META_IS_MANAGED
" to the explicit value '" PCMK_VALUE_DEFAULT
"' is deprecated and will be removed in a "
"future release (just leave it unset)");
} else if (crm_is_true(value)) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_managed);
} else {
pcmk__clear_rsc_flags(*rsc, pcmk__rsc_managed);
}
}
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_MAINTENANCE);
if (crm_is_true(value)) {
pcmk__clear_rsc_flags(*rsc, pcmk__rsc_managed);
pcmk__set_rsc_flags(*rsc, pcmk__rsc_maintenance);
}
if (pcmk_is_set(scheduler->flags, pcmk__sched_in_maintenance)) {
pcmk__clear_rsc_flags(*rsc, pcmk__rsc_managed);
pcmk__set_rsc_flags(*rsc, pcmk__rsc_maintenance);
}
if (pcmk__is_clone(pe__const_top_resource(*rsc, false))) {
if (detect_unique(*rsc)) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_unique);
}
if (crm_is_true(g_hash_table_lookup((*rsc)->priv->meta,
PCMK_META_PROMOTABLE))) {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_promotable);
}
} else {
pcmk__set_rsc_flags(*rsc, pcmk__rsc_unique);
}
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_MULTIPLE_ACTIVE);
if (pcmk__str_eq(value, PCMK_VALUE_STOP_ONLY, pcmk__str_casei)) {
rsc_private->multiply_active_policy = pcmk__multiply_active_stop;
pcmk__rsc_trace(*rsc, "%s multiple running resource recovery: stop only",
(*rsc)->id);
} else if (pcmk__str_eq(value, PCMK_VALUE_BLOCK, pcmk__str_casei)) {
rsc_private->multiply_active_policy = pcmk__multiply_active_block;
pcmk__rsc_trace(*rsc, "%s multiple running resource recovery: block",
(*rsc)->id);
} else if (pcmk__str_eq(value, PCMK_VALUE_STOP_UNEXPECTED,
pcmk__str_casei)) {
rsc_private->multiply_active_policy = pcmk__multiply_active_unexpected;
pcmk__rsc_trace(*rsc,
"%s multiple running resource recovery: "
"stop unexpected instances",
(*rsc)->id);
} else { // PCMK_VALUE_STOP_START
if (!pcmk__str_eq(value, PCMK_VALUE_STOP_START,
pcmk__str_casei|pcmk__str_null_matches)) {
pcmk__config_warn("%s is not a valid value for "
PCMK_META_MULTIPLE_ACTIVE
", using default of "
"\"" PCMK_VALUE_STOP_START "\"",
value);
}
rsc_private->multiply_active_policy = pcmk__multiply_active_restart;
pcmk__rsc_trace(*rsc,
"%s multiple running resource recovery: stop/start",
(*rsc)->id);
}
unpack_stickiness(*rsc);
unpack_migration_threshold(*rsc);
if (pcmk__str_eq(crm_element_value(rsc_private->xml, PCMK_XA_CLASS),
PCMK_RESOURCE_CLASS_STONITH, pcmk__str_casei)) {
pcmk__set_scheduler_flags(scheduler, pcmk__sched_have_fencing);
pcmk__set_rsc_flags(*rsc, pcmk__rsc_fence_device);
}
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_REQUIRES);
unpack_requires(*rsc, value, false);
value = g_hash_table_lookup(rsc_private->meta, PCMK_META_FAILURE_TIMEOUT);
if (value != NULL) {
pcmk_parse_interval_spec(value, &(rsc_private->failure_expiration_ms));
}
if (remote_node) {
GHashTable *params = pe_rsc_params(*rsc, NULL, scheduler);
/* Grabbing the value now means that any rules based on node attributes
* will evaluate to false, so such rules should not be used with
* PCMK_REMOTE_RA_RECONNECT_INTERVAL.
*
* @TODO Evaluate per node before using
*/
value = g_hash_table_lookup(params, PCMK_REMOTE_RA_RECONNECT_INTERVAL);
if (value) {
/* reconnect delay works by setting failure_timeout and preventing the
* connection from starting until the failure is cleared. */
pcmk_parse_interval_spec(value,
&(rsc_private->remote_reconnect_ms));
/* We want to override any default failure_timeout in use when remote
* PCMK_REMOTE_RA_RECONNECT_INTERVAL is in use.
*/
rsc_private->failure_expiration_ms =
rsc_private->remote_reconnect_ms;
}
}
get_target_role(*rsc, &(rsc_private->next_role));
pcmk__rsc_trace(*rsc, "%s desired next state: %s", (*rsc)->id,
(rsc_private->next_role == pcmk_role_unknown)?
"default" : pcmk_role_text(rsc_private->next_role));
if (rsc_private->fns->unpack(*rsc, scheduler) == FALSE) {
rsc_private->fns->free(*rsc);
*rsc = NULL;
return pcmk_rc_unpack_error;
}
if (pcmk_is_set(scheduler->flags, pcmk__sched_symmetric_cluster)) {
// This tag must stay exactly the same because it is tested elsewhere
resource_location(*rsc, NULL, 0, "symmetric_default", scheduler);
} else if (guest_node) {
/* remote resources tied to a container resource must always be allowed
* to opt-in to the cluster. Whether the connection resource is actually
* allowed to be placed on a node is dependent on the container resource */
resource_location(*rsc, NULL, 0, "remote_connection_default",
scheduler);
}
pcmk__rsc_trace(*rsc, "%s action notification: %s", (*rsc)->id,
pcmk_is_set((*rsc)->flags, pcmk__rsc_notify)? "required" : "not required");
pe__unpack_dataset_nvpairs(rsc_private->xml, PCMK_XE_UTILIZATION,
&rule_data, rsc_private->utilization, NULL,
scheduler);
if (expanded_xml) {
if (add_template_rsc(xml_obj, scheduler) == FALSE) {
rsc_private->fns->free(*rsc);
*rsc = NULL;
return pcmk_rc_unpack_error;
}
}
return pcmk_rc_ok;
}
gboolean
is_parent(pcmk_resource_t *child, pcmk_resource_t *rsc)
{
pcmk_resource_t *parent = child;
if (parent == NULL || rsc == NULL) {
return FALSE;
}
while (parent->priv->parent != NULL) {
if (parent->priv->parent == rsc) {
return TRUE;
}
parent = parent->priv->parent;
}
return FALSE;
}
pcmk_resource_t *
uber_parent(pcmk_resource_t *rsc)
{
pcmk_resource_t *parent = rsc;
if (parent == NULL) {
return NULL;
}
while ((parent->priv->parent != NULL)
&& !pcmk__is_bundle(parent->priv->parent)) {
parent = parent->priv->parent;
}
return parent;
}
/*!
* \internal
* \brief Get the topmost parent of a resource as a const pointer
*
* \param[in] rsc Resource to check
* \param[in] include_bundle If true, go all the way to bundle
*
* \return \p NULL if \p rsc is NULL, \p rsc if \p rsc has no parent,
* the bundle if \p rsc is bundled and \p include_bundle is true,
* otherwise the topmost parent of \p rsc up to a clone
*/
const pcmk_resource_t *
pe__const_top_resource(const pcmk_resource_t *rsc, bool include_bundle)
{
const pcmk_resource_t *parent = rsc;
if (parent == NULL) {
return NULL;
}
while (parent->priv->parent != NULL) {
if (!include_bundle && pcmk__is_bundle(parent->priv->parent)) {
break;
}
parent = parent->priv->parent;
}
return parent;
}
void
common_free(pcmk_resource_t * rsc)
{
if (rsc == NULL) {
return;
}
pcmk__rsc_trace(rsc, "Freeing %s", rsc->id);
if (rsc->priv->parameter_cache != NULL) {
g_hash_table_destroy(rsc->priv->parameter_cache);
}
if ((rsc->priv->parent == NULL)
&& pcmk_is_set(rsc->flags, pcmk__rsc_removed)) {
pcmk__xml_free(rsc->priv->xml);
rsc->priv->xml = NULL;
pcmk__xml_free(rsc->priv->orig_xml);
rsc->priv->orig_xml = NULL;
} else if (rsc->priv->orig_xml != NULL) {
// rsc->private->xml was expanded from a template
pcmk__xml_free(rsc->priv->xml);
rsc->priv->xml = NULL;
}
free(rsc->id);
free(rsc->priv->variant_opaque);
free(rsc->priv->history_id);
free(rsc->priv->pending_action);
- free(rsc->priv->assigned_node);
+ pcmk__free_node_copy(rsc->priv->assigned_node);
g_list_free(rsc->priv->actions);
g_list_free(rsc->priv->active_nodes);
g_list_free(rsc->priv->launched);
g_list_free(rsc->priv->dangling_migration_sources);
g_list_free(rsc->priv->with_this_colocations);
g_list_free(rsc->priv->this_with_colocations);
g_list_free(rsc->priv->location_constraints);
g_list_free(rsc->priv->ticket_constraints);
if (rsc->priv->meta != NULL) {
g_hash_table_destroy(rsc->priv->meta);
}
if (rsc->priv->utilization != NULL) {
g_hash_table_destroy(rsc->priv->utilization);
}
if (rsc->priv->probed_nodes != NULL) {
g_hash_table_destroy(rsc->priv->probed_nodes);
}
if (rsc->priv->allowed_nodes != NULL) {
g_hash_table_destroy(rsc->priv->allowed_nodes);
}
free(rsc->priv);
free(rsc);
}
/*!
* \internal
* \brief Count a node and update most preferred to it as appropriate
*
* \param[in] rsc An active resource
* \param[in] node A node that \p rsc is active on
* \param[in,out] active This will be set to \p node if \p node is more
* preferred than the current value
* \param[in,out] count_all If not NULL, this will be incremented
* \param[in,out] count_clean If not NULL, this will be incremented if \p node
* is online and clean
*
* \return true if the count should continue, or false if sufficiently known
*/
bool
pe__count_active_node(const pcmk_resource_t *rsc, pcmk_node_t *node,
pcmk_node_t **active, unsigned int *count_all,
unsigned int *count_clean)
{
bool keep_looking = false;
bool is_happy = false;
CRM_CHECK((rsc != NULL) && (node != NULL) && (active != NULL),
return false);
is_happy = node->details->online && !node->details->unclean;
if (count_all != NULL) {
++*count_all;
}
if ((count_clean != NULL) && is_happy) {
++*count_clean;
}
if ((count_all != NULL) || (count_clean != NULL)) {
keep_looking = true; // We're counting, so go through entire list
}
if (rsc->priv->partial_migration_source != NULL) {
if (pcmk__same_node(node, rsc->priv->partial_migration_source)) {
*active = node; // This is the migration source
} else {
keep_looking = true;
}
} else if (!pcmk_is_set(rsc->flags, pcmk__rsc_needs_fencing)) {
if (is_happy && ((*active == NULL) || !(*active)->details->online
|| (*active)->details->unclean)) {
*active = node; // This is the first clean node
} else {
keep_looking = true;
}
}
if (*active == NULL) {
*active = node; // This is the first node checked
}
return keep_looking;
}
// Shared implementation of pcmk__rsc_methods_t:active_node()
static pcmk_node_t *
active_node(const pcmk_resource_t *rsc, unsigned int *count_all,
unsigned int *count_clean)
{
pcmk_node_t *active = NULL;
if (count_all != NULL) {
*count_all = 0;
}
if (count_clean != NULL) {
*count_clean = 0;
}
if (rsc == NULL) {
return NULL;
}
for (GList *iter = rsc->priv->active_nodes;
iter != NULL; iter = iter->next) {
if (!pe__count_active_node(rsc, (pcmk_node_t *) iter->data, &active,
count_all, count_clean)) {
break; // Don't waste time iterating if we don't have to
}
}
return active;
}
/*!
* \brief
* \internal Find and count active nodes according to \c PCMK_META_REQUIRES
*
* \param[in] rsc Resource to check
* \param[out] count If not NULL, will be set to count of active nodes
*
* \return An active node (or NULL if resource is not active anywhere)
*
* \note This is a convenience wrapper for active_node() where the count of all
* active nodes or only clean active nodes is desired according to the
* \c PCMK_META_REQUIRES meta-attribute.
*/
pcmk_node_t *
pe__find_active_requires(const pcmk_resource_t *rsc, unsigned int *count)
{
if (rsc == NULL) {
if (count != NULL) {
*count = 0;
}
return NULL;
}
if (pcmk_is_set(rsc->flags, pcmk__rsc_needs_fencing)) {
return rsc->priv->fns->active_node(rsc, count, NULL);
} else {
return rsc->priv->fns->active_node(rsc, NULL, count);
}
}
void
pe__count_common(pcmk_resource_t *rsc)
{
if (rsc->priv->children != NULL) {
for (GList *item = rsc->priv->children;
item != NULL; item = item->next) {
pcmk_resource_t *child = item->data;
child->priv->fns->count(item->data);
}
} else if (!pcmk_is_set(rsc->flags, pcmk__rsc_removed)
|| (rsc->priv->orig_role > pcmk_role_stopped)) {
rsc->priv->scheduler->priv->ninstances++;
if (pe__resource_is_disabled(rsc)) {
rsc->priv->scheduler->priv->disabled_resources++;
}
if (pcmk_is_set(rsc->flags, pcmk__rsc_blocked)) {
rsc->priv->scheduler->priv->blocked_resources++;
}
}
}
/*!
* \internal
* \brief Update a resource's next role
*
* \param[in,out] rsc Resource to be updated
* \param[in] role Resource's new next role
* \param[in] why Human-friendly reason why role is changing (for logs)
*/
void
pe__set_next_role(pcmk_resource_t *rsc, enum rsc_role_e role, const char *why)
{
pcmk__assert((rsc != NULL) && (why != NULL));
if (rsc->priv->next_role != role) {
pcmk__rsc_trace(rsc, "Resetting next role for %s from %s to %s (%s)",
rsc->id, pcmk_role_text(rsc->priv->next_role),
pcmk_role_text(role), why);
rsc->priv->next_role = role;
}
}
diff --git a/lib/pengine/pe_actions.c b/lib/pengine/pe_actions.c
index cbe7bda25f..eab9304879 100644
--- a/lib/pengine/pe_actions.c
+++ b/lib/pengine/pe_actions.c
@@ -1,1782 +1,1782 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <glib.h>
#include <stdbool.h>
#include <crm/crm.h>
#include <crm/common/xml.h>
#include <crm/common/scheduler_internal.h>
#include <crm/pengine/internal.h>
#include <crm/common/xml_internal.h>
#include "pe_status_private.h"
static void unpack_operation(pcmk_action_t *action, const xmlNode *xml_obj,
guint interval_ms);
static void
add_singleton(pcmk_scheduler_t *scheduler, pcmk_action_t *action)
{
if (scheduler->priv->singletons == NULL) {
scheduler->priv->singletons = pcmk__strkey_table(NULL, NULL);
}
g_hash_table_insert(scheduler->priv->singletons, action->uuid, action);
}
static pcmk_action_t *
lookup_singleton(pcmk_scheduler_t *scheduler, const char *action_uuid)
{
/* @TODO This is the only use of the pcmk_scheduler_t:singletons hash table.
* Compare the performance of this approach to keeping the
* pcmk_scheduler_t:actions list sorted by action key and just searching
* that instead.
*/
if (scheduler->priv->singletons == NULL) {
return NULL;
}
return g_hash_table_lookup(scheduler->priv->singletons, action_uuid);
}
/*!
* \internal
* \brief Find an existing action that matches arguments
*
* \param[in] key Action key to match
* \param[in] rsc Resource to match (if any)
* \param[in] node Node to match (if any)
* \param[in] scheduler Scheduler data
*
* \return Existing action that matches arguments (or NULL if none)
*/
static pcmk_action_t *
find_existing_action(const char *key, const pcmk_resource_t *rsc,
const pcmk_node_t *node, const pcmk_scheduler_t *scheduler)
{
/* When rsc is NULL, it would be quicker to check
* scheduler->priv->singletons, but checking all scheduler->priv->actions
* takes the node into account.
*/
GList *actions = (rsc == NULL)? scheduler->priv->actions : rsc->priv->actions;
GList *matches = find_actions(actions, key, node);
pcmk_action_t *action = NULL;
if (matches == NULL) {
return NULL;
}
CRM_LOG_ASSERT(!pcmk__list_of_multiple(matches));
action = matches->data;
g_list_free(matches);
return action;
}
/*!
* \internal
* \brief Find the XML configuration corresponding to a specific action key
*
* \param[in] rsc Resource to find action configuration for
* \param[in] key "RSC_ACTION_INTERVAL" of action to find
* \param[in] include_disabled If false, do not return disabled actions
*
* \return XML configuration of desired action if any, otherwise NULL
*/
static xmlNode *
find_exact_action_config(const pcmk_resource_t *rsc, const char *action_name,
guint interval_ms, bool include_disabled)
{
for (xmlNode *operation = pcmk__xe_first_child(rsc->priv->ops_xml,
PCMK_XE_OP, NULL, NULL);
operation != NULL; operation = pcmk__xe_next(operation, PCMK_XE_OP)) {
bool enabled = false;
const char *config_name = NULL;
const char *interval_spec = NULL;
guint tmp_ms = 0U;
// @TODO This does not consider meta-attributes, rules, defaults, etc.
if (!include_disabled
&& (pcmk__xe_get_bool_attr(operation, PCMK_META_ENABLED,
&enabled) == pcmk_rc_ok) && !enabled) {
continue;
}
interval_spec = crm_element_value(operation, PCMK_META_INTERVAL);
pcmk_parse_interval_spec(interval_spec, &tmp_ms);
if (tmp_ms != interval_ms) {
continue;
}
config_name = crm_element_value(operation, PCMK_XA_NAME);
if (pcmk__str_eq(action_name, config_name, pcmk__str_none)) {
return operation;
}
}
return NULL;
}
/*!
* \internal
* \brief Find the XML configuration of a resource action
*
* \param[in] rsc Resource to find action configuration for
* \param[in] action_name Action name to search for
* \param[in] interval_ms Action interval (in milliseconds) to search for
* \param[in] include_disabled If false, do not return disabled actions
*
* \return XML configuration of desired action if any, otherwise NULL
*/
xmlNode *
pcmk__find_action_config(const pcmk_resource_t *rsc, const char *action_name,
guint interval_ms, bool include_disabled)
{
xmlNode *action_config = NULL;
// Try requested action first
action_config = find_exact_action_config(rsc, action_name, interval_ms,
include_disabled);
// For migrate_to and migrate_from actions, retry with "migrate"
// @TODO This should be either documented or deprecated
if ((action_config == NULL)
&& pcmk__str_any_of(action_name, PCMK_ACTION_MIGRATE_TO,
PCMK_ACTION_MIGRATE_FROM, NULL)) {
action_config = find_exact_action_config(rsc, "migrate", 0,
include_disabled);
}
return action_config;
}
/*!
* \internal
* \brief Create a new action object
*
* \param[in] key Action key
* \param[in] task Action name
* \param[in,out] rsc Resource that action is for (if any)
* \param[in] node Node that action is on (if any)
* \param[in] optional Whether action should be considered optional
* \param[in,out] scheduler Scheduler data
*
* \return Newly allocated action
* \note This function takes ownership of \p key. It is the caller's
* responsibility to free the return value with pe_free_action().
*/
static pcmk_action_t *
new_action(char *key, const char *task, pcmk_resource_t *rsc,
const pcmk_node_t *node, bool optional, pcmk_scheduler_t *scheduler)
{
pcmk_action_t *action = pcmk__assert_alloc(1, sizeof(pcmk_action_t));
action->rsc = rsc;
action->task = pcmk__str_copy(task);
action->uuid = key;
action->scheduler = scheduler;
if (node) {
action->node = pe__copy_node(node);
}
if (pcmk__str_eq(task, PCMK_ACTION_LRM_DELETE, pcmk__str_casei)) {
// Resource history deletion for a node can be done on the DC
pcmk__set_action_flags(action, pcmk__action_on_dc);
}
pcmk__set_action_flags(action, pcmk__action_runnable);
if (optional) {
pcmk__set_action_flags(action, pcmk__action_optional);
} else {
pcmk__clear_action_flags(action, pcmk__action_optional);
}
if (rsc == NULL) {
action->meta = pcmk__strkey_table(free, free);
} else {
guint interval_ms = 0;
parse_op_key(key, NULL, NULL, &interval_ms);
action->op_entry = pcmk__find_action_config(rsc, task, interval_ms,
true);
/* If the given key is for one of the many notification pseudo-actions
* (pre_notify_promote, etc.), the actual action name is "notify"
*/
if ((action->op_entry == NULL) && (strstr(key, "_notify_") != NULL)) {
action->op_entry = find_exact_action_config(rsc, PCMK_ACTION_NOTIFY,
0, true);
}
unpack_operation(action, action->op_entry, interval_ms);
}
pcmk__rsc_trace(rsc, "Created %s action %d (%s): %s for %s on %s",
(optional? "optional" : "required"),
scheduler->priv->next_action_id, key, task,
((rsc == NULL)? "no resource" : rsc->id),
pcmk__node_name(node));
action->id = scheduler->priv->next_action_id++;
scheduler->priv->actions = g_list_prepend(scheduler->priv->actions, action);
if (rsc == NULL) {
add_singleton(scheduler, action);
} else {
rsc->priv->actions = g_list_prepend(rsc->priv->actions, action);
}
return action;
}
/*!
* \internal
* \brief Unpack a resource's action-specific instance parameters
*
* \param[in] action_xml XML of action's configuration in CIB (if any)
* \param[in,out] node_attrs Table of node attributes (for rule evaluation)
* \param[in,out] scheduler Cluster working set (for rule evaluation)
*
* \return Newly allocated hash table of action-specific instance parameters
*/
GHashTable *
pcmk__unpack_action_rsc_params(const xmlNode *action_xml,
GHashTable *node_attrs,
pcmk_scheduler_t *scheduler)
{
GHashTable *params = pcmk__strkey_table(free, free);
pe_rule_eval_data_t rule_data = {
.node_hash = node_attrs,
.now = scheduler->priv->now,
.match_data = NULL,
.rsc_data = NULL,
.op_data = NULL
};
pe__unpack_dataset_nvpairs(action_xml, PCMK_XE_INSTANCE_ATTRIBUTES,
&rule_data, params, NULL, scheduler);
return params;
}
/*!
* \internal
* \brief Update an action's optional flag
*
* \param[in,out] action Action to update
* \param[in] optional Requested optional status
*/
static void
update_action_optional(pcmk_action_t *action, gboolean optional)
{
// Force a non-recurring action to be optional if its resource is unmanaged
if ((action->rsc != NULL) && (action->node != NULL)
&& !pcmk_is_set(action->flags, pcmk__action_pseudo)
&& !pcmk_is_set(action->rsc->flags, pcmk__rsc_managed)
&& (g_hash_table_lookup(action->meta, PCMK_META_INTERVAL) == NULL)) {
pcmk__rsc_debug(action->rsc,
"%s on %s is optional (%s is unmanaged)",
action->uuid, pcmk__node_name(action->node),
action->rsc->id);
pcmk__set_action_flags(action, pcmk__action_optional);
// We shouldn't clear runnable here because ... something
// Otherwise require the action if requested
} else if (!optional) {
pcmk__clear_action_flags(action, pcmk__action_optional);
}
}
static enum pe_quorum_policy
effective_quorum_policy(pcmk_resource_t *rsc, pcmk_scheduler_t *scheduler)
{
enum pe_quorum_policy policy = scheduler->no_quorum_policy;
if (pcmk_is_set(scheduler->flags, pcmk__sched_quorate)) {
policy = pcmk_no_quorum_ignore;
} else if (scheduler->no_quorum_policy == pcmk_no_quorum_demote) {
switch (rsc->priv->orig_role) {
case pcmk_role_promoted:
case pcmk_role_unpromoted:
if (rsc->priv->next_role > pcmk_role_unpromoted) {
pe__set_next_role(rsc, pcmk_role_unpromoted,
PCMK_OPT_NO_QUORUM_POLICY "=demote");
}
policy = pcmk_no_quorum_ignore;
break;
default:
policy = pcmk_no_quorum_stop;
break;
}
}
return policy;
}
/*!
* \internal
* \brief Update a resource action's runnable flag
*
* \param[in,out] action Action to update
* \param[in,out] scheduler Scheduler data
*
* \note This may also schedule fencing if a stop is unrunnable.
*/
static void
update_resource_action_runnable(pcmk_action_t *action,
pcmk_scheduler_t *scheduler)
{
pcmk_resource_t *rsc = action->rsc;
if (pcmk_is_set(action->flags, pcmk__action_pseudo)) {
return;
}
if (action->node == NULL) {
pcmk__rsc_trace(rsc, "%s is unrunnable (unallocated)", action->uuid);
pcmk__clear_action_flags(action, pcmk__action_runnable);
} else if (!pcmk_is_set(action->flags, pcmk__action_on_dc)
&& !(action->node->details->online)
&& (!pcmk__is_guest_or_bundle_node(action->node)
|| pcmk_is_set(action->node->priv->flags,
pcmk__node_remote_reset))) {
pcmk__clear_action_flags(action, pcmk__action_runnable);
do_crm_log(LOG_WARNING, "%s on %s is unrunnable (node is offline)",
action->uuid, pcmk__node_name(action->node));
if (pcmk_is_set(rsc->flags, pcmk__rsc_managed)
&& pcmk__str_eq(action->task, PCMK_ACTION_STOP, pcmk__str_casei)
&& !(action->node->details->unclean)) {
pe_fence_node(scheduler, action->node, "stop is unrunnable", false);
}
} else if (!pcmk_is_set(action->flags, pcmk__action_on_dc)
&& action->node->details->pending) {
pcmk__clear_action_flags(action, pcmk__action_runnable);
do_crm_log(LOG_WARNING,
"Action %s on %s is unrunnable (node is pending)",
action->uuid, pcmk__node_name(action->node));
} else if (action->needs == pcmk__requires_nothing) {
pe_action_set_reason(action, NULL, TRUE);
if (pcmk__is_guest_or_bundle_node(action->node)
&& !pe_can_fence(scheduler, action->node)) {
/* An action that requires nothing usually does not require any
* fencing in order to be runnable. However, there is an exception:
* such an action cannot be completed if it is on a guest node whose
* host is unclean and cannot be fenced.
*/
pcmk__rsc_debug(rsc,
"%s on %s is unrunnable "
"(node's host cannot be fenced)",
action->uuid, pcmk__node_name(action->node));
pcmk__clear_action_flags(action, pcmk__action_runnable);
} else {
pcmk__rsc_trace(rsc,
"%s on %s does not require fencing or quorum",
action->uuid, pcmk__node_name(action->node));
pcmk__set_action_flags(action, pcmk__action_runnable);
}
} else {
switch (effective_quorum_policy(rsc, scheduler)) {
case pcmk_no_quorum_stop:
pcmk__rsc_debug(rsc, "%s on %s is unrunnable (no quorum)",
action->uuid, pcmk__node_name(action->node));
pcmk__clear_action_flags(action, pcmk__action_runnable);
pe_action_set_reason(action, "no quorum", true);
break;
case pcmk_no_quorum_freeze:
if (!rsc->priv->fns->active(rsc, TRUE)
|| (rsc->priv->next_role > rsc->priv->orig_role)) {
pcmk__rsc_debug(rsc, "%s on %s is unrunnable (no quorum)",
action->uuid,
pcmk__node_name(action->node));
pcmk__clear_action_flags(action, pcmk__action_runnable);
pe_action_set_reason(action, "quorum freeze", true);
}
break;
default:
//pe_action_set_reason(action, NULL, TRUE);
pcmk__set_action_flags(action, pcmk__action_runnable);
break;
}
}
}
static bool
valid_stop_on_fail(const char *value)
{
return !pcmk__strcase_any_of(value,
PCMK_VALUE_STANDBY, PCMK_VALUE_DEMOTE,
PCMK_VALUE_STOP, NULL);
}
/*!
* \internal
* \brief Validate (and possibly reset) resource action's on_fail meta-attribute
*
* \param[in] rsc Resource that action is for
* \param[in] action_name Action name
* \param[in] action_config Action configuration XML from CIB (if any)
* \param[in,out] meta Table of action meta-attributes
*/
static void
validate_on_fail(const pcmk_resource_t *rsc, const char *action_name,
const xmlNode *action_config, GHashTable *meta)
{
const char *name = NULL;
const char *role = NULL;
const char *interval_spec = NULL;
const char *value = g_hash_table_lookup(meta, PCMK_META_ON_FAIL);
guint interval_ms = 0U;
// Stop actions can only use certain on-fail values
if (pcmk__str_eq(action_name, PCMK_ACTION_STOP, pcmk__str_none)
&& !valid_stop_on_fail(value)) {
pcmk__config_err("Resetting '" PCMK_META_ON_FAIL "' for %s stop "
"action to default value because '%s' is not "
"allowed for stop", rsc->id, value);
g_hash_table_remove(meta, PCMK_META_ON_FAIL);
return;
}
/* Demote actions default on-fail to the on-fail value for the first
* recurring monitor for the promoted role (if any).
*/
if (pcmk__str_eq(action_name, PCMK_ACTION_DEMOTE, pcmk__str_none)
&& (value == NULL)) {
/* @TODO This does not consider promote options set in a meta-attribute
* block (which may have rules that need to be evaluated) rather than
* XML properties.
*/
for (xmlNode *operation = pcmk__xe_first_child(rsc->priv->ops_xml,
PCMK_XE_OP, NULL, NULL);
operation != NULL;
operation = pcmk__xe_next(operation, PCMK_XE_OP)) {
bool enabled = false;
const char *promote_on_fail = NULL;
/* We only care about explicit on-fail (if promote uses default, so
* can demote)
*/
promote_on_fail = crm_element_value(operation, PCMK_META_ON_FAIL);
if (promote_on_fail == NULL) {
continue;
}
// We only care about recurring monitors for the promoted role
name = crm_element_value(operation, PCMK_XA_NAME);
role = crm_element_value(operation, PCMK_XA_ROLE);
if (!pcmk__str_eq(name, PCMK_ACTION_MONITOR, pcmk__str_none)
|| !pcmk__strcase_any_of(role, PCMK_ROLE_PROMOTED,
PCMK__ROLE_PROMOTED_LEGACY, NULL)) {
continue;
}
interval_spec = crm_element_value(operation, PCMK_META_INTERVAL);
pcmk_parse_interval_spec(interval_spec, &interval_ms);
if (interval_ms == 0U) {
continue;
}
// We only care about enabled monitors
if ((pcmk__xe_get_bool_attr(operation, PCMK_META_ENABLED,
&enabled) == pcmk_rc_ok) && !enabled) {
continue;
}
/* Demote actions can't default to
* PCMK_META_ON_FAIL=PCMK_VALUE_DEMOTE
*/
if (pcmk__str_eq(promote_on_fail, PCMK_VALUE_DEMOTE,
pcmk__str_casei)) {
continue;
}
// Use value from first applicable promote action found
pcmk__insert_dup(meta, PCMK_META_ON_FAIL, promote_on_fail);
}
return;
}
if (pcmk__str_eq(action_name, PCMK_ACTION_LRM_DELETE, pcmk__str_none)
&& !pcmk__str_eq(value, PCMK_VALUE_IGNORE, pcmk__str_casei)) {
pcmk__insert_dup(meta, PCMK_META_ON_FAIL, PCMK_VALUE_IGNORE);
return;
}
// PCMK_META_ON_FAIL=PCMK_VALUE_DEMOTE is allowed only for certain actions
if (pcmk__str_eq(value, PCMK_VALUE_DEMOTE, pcmk__str_casei)) {
name = crm_element_value(action_config, PCMK_XA_NAME);
role = crm_element_value(action_config, PCMK_XA_ROLE);
interval_spec = crm_element_value(action_config, PCMK_META_INTERVAL);
pcmk_parse_interval_spec(interval_spec, &interval_ms);
if (!pcmk__str_eq(name, PCMK_ACTION_PROMOTE, pcmk__str_none)
&& ((interval_ms == 0U)
|| !pcmk__str_eq(name, PCMK_ACTION_MONITOR, pcmk__str_none)
|| !pcmk__strcase_any_of(role, PCMK_ROLE_PROMOTED,
PCMK__ROLE_PROMOTED_LEGACY, NULL))) {
pcmk__config_err("Resetting '" PCMK_META_ON_FAIL "' for %s %s "
"action to default value because 'demote' is not "
"allowed for it", rsc->id, name);
g_hash_table_remove(meta, PCMK_META_ON_FAIL);
return;
}
}
}
static int
unpack_timeout(const char *value)
{
long long timeout_ms = crm_get_msec(value);
if (timeout_ms <= 0) {
timeout_ms = PCMK_DEFAULT_ACTION_TIMEOUT_MS;
}
return (int) QB_MIN(timeout_ms, INT_MAX);
}
// true if value contains valid, non-NULL interval origin for recurring op
static bool
unpack_interval_origin(const char *value, const xmlNode *xml_obj,
guint interval_ms, const crm_time_t *now,
long long *start_delay)
{
long long result = 0;
guint interval_sec = pcmk__timeout_ms2s(interval_ms);
crm_time_t *origin = NULL;
// Ignore unspecified values and non-recurring operations
if ((value == NULL) || (interval_ms == 0) || (now == NULL)) {
return false;
}
// Parse interval origin from text
origin = crm_time_new(value);
if (origin == NULL) {
pcmk__config_err("Ignoring '" PCMK_META_INTERVAL_ORIGIN "' for "
"operation '%s' because '%s' is not valid",
pcmk__s(pcmk__xe_id(xml_obj), "(missing ID)"), value);
return false;
}
// Get seconds since origin (negative if origin is in the future)
result = crm_time_get_seconds(now) - crm_time_get_seconds(origin);
crm_time_free(origin);
// Calculate seconds from closest interval to now
result = result % interval_sec;
// Calculate seconds remaining until next interval
result = ((result <= 0)? 0 : interval_sec) - result;
crm_info("Calculated a start delay of %llds for operation '%s'",
result, pcmk__s(pcmk__xe_id(xml_obj), "(unspecified)"));
if (start_delay != NULL) {
*start_delay = result * 1000; // milliseconds
}
return true;
}
static int
unpack_start_delay(const char *value, GHashTable *meta)
{
long long start_delay_ms = 0;
if (value == NULL) {
return 0;
}
start_delay_ms = crm_get_msec(value);
start_delay_ms = QB_MIN(start_delay_ms, INT_MAX);
if (start_delay_ms < 0) {
start_delay_ms = 0;
}
if (meta != NULL) {
g_hash_table_replace(meta, strdup(PCMK_META_START_DELAY),
pcmk__itoa(start_delay_ms));
}
return (int) start_delay_ms;
}
/*!
* \internal
* \brief Find a resource's most frequent recurring monitor
*
* \param[in] rsc Resource to check
*
* \return Operation XML configured for most frequent recurring monitor for
* \p rsc (if any)
*/
static xmlNode *
most_frequent_monitor(const pcmk_resource_t *rsc)
{
guint min_interval_ms = G_MAXUINT;
xmlNode *op = NULL;
for (xmlNode *operation = pcmk__xe_first_child(rsc->priv->ops_xml,
PCMK_XE_OP, NULL, NULL);
operation != NULL; operation = pcmk__xe_next(operation, PCMK_XE_OP)) {
bool enabled = false;
guint interval_ms = 0U;
const char *interval_spec = crm_element_value(operation,
PCMK_META_INTERVAL);
// We only care about enabled recurring monitors
if (!pcmk__str_eq(crm_element_value(operation, PCMK_XA_NAME),
PCMK_ACTION_MONITOR, pcmk__str_none)) {
continue;
}
pcmk_parse_interval_spec(interval_spec, &interval_ms);
if (interval_ms == 0U) {
continue;
}
// @TODO This does not consider meta-attributes, rules, defaults, etc.
if ((pcmk__xe_get_bool_attr(operation, PCMK_META_ENABLED,
&enabled) == pcmk_rc_ok) && !enabled) {
continue;
}
if (interval_ms < min_interval_ms) {
min_interval_ms = interval_ms;
op = operation;
}
}
return op;
}
/*!
* \internal
* \brief Unpack action meta-attributes
*
* \param[in,out] rsc Resource that action is for
* \param[in] node Node that action is on
* \param[in] action_name Action name
* \param[in] interval_ms Action interval (in milliseconds)
* \param[in] action_config Action XML configuration from CIB (if any)
*
* Unpack a resource action's meta-attributes (normalizing the interval,
* timeout, and start delay values as integer milliseconds) from its CIB XML
* configuration (including defaults).
*
* \return Newly allocated hash table with normalized action meta-attributes
*/
GHashTable *
pcmk__unpack_action_meta(pcmk_resource_t *rsc, const pcmk_node_t *node,
const char *action_name, guint interval_ms,
const xmlNode *action_config)
{
GHashTable *meta = NULL;
const char *timeout_spec = NULL;
const char *str = NULL;
pe_rsc_eval_data_t rsc_rule_data = {
.standard = crm_element_value(rsc->priv->xml, PCMK_XA_CLASS),
.provider = crm_element_value(rsc->priv->xml, PCMK_XA_PROVIDER),
.agent = crm_element_value(rsc->priv->xml, PCMK_XA_TYPE),
};
pe_op_eval_data_t op_rule_data = {
.op_name = action_name,
.interval = interval_ms,
};
pe_rule_eval_data_t rule_data = {
/* Node attributes are not set because node expressions are not allowed
* for meta-attributes
*/
.now = rsc->priv->scheduler->priv->now,
.match_data = NULL,
.rsc_data = &rsc_rule_data,
.op_data = &op_rule_data,
};
meta = pcmk__strkey_table(free, free);
if (action_config != NULL) {
// <op> <meta_attributes> take precedence over defaults
pe__unpack_dataset_nvpairs(action_config, PCMK_XE_META_ATTRIBUTES,
&rule_data, meta, NULL,
rsc->priv->scheduler);
/* Anything set as an <op> XML property has highest precedence.
* This ensures we use the name and interval from the <op> tag.
* (See below for the only exception, fence device start/probe timeout.)
*/
for (xmlAttrPtr attr = action_config->properties;
attr != NULL; attr = attr->next) {
pcmk__insert_dup(meta, (const char *) attr->name,
pcmk__xml_attr_value(attr));
}
}
// Derive default timeout for probes from recurring monitor timeouts
if (pcmk_is_probe(action_name, interval_ms)
&& (g_hash_table_lookup(meta, PCMK_META_TIMEOUT) == NULL)) {
xmlNode *min_interval_mon = most_frequent_monitor(rsc);
if (min_interval_mon != NULL) {
/* @TODO This does not consider timeouts set in
* PCMK_XE_META_ATTRIBUTES blocks (which may also have rules that
* need to be evaluated).
*/
timeout_spec = crm_element_value(min_interval_mon,
PCMK_META_TIMEOUT);
if (timeout_spec != NULL) {
pcmk__rsc_trace(rsc,
"Setting default timeout for %s probe to "
"most frequent monitor's timeout '%s'",
rsc->id, timeout_spec);
pcmk__insert_dup(meta, PCMK_META_TIMEOUT, timeout_spec);
}
}
}
// Cluster-wide <op_defaults> <meta_attributes>
pe__unpack_dataset_nvpairs(rsc->priv->scheduler->priv->op_defaults,
PCMK_XE_META_ATTRIBUTES, &rule_data, meta, NULL,
rsc->priv->scheduler);
g_hash_table_remove(meta, PCMK_XA_ID);
// Normalize interval to milliseconds
if (interval_ms > 0) {
g_hash_table_insert(meta, pcmk__str_copy(PCMK_META_INTERVAL),
crm_strdup_printf("%u", interval_ms));
} else {
g_hash_table_remove(meta, PCMK_META_INTERVAL);
}
/* Timeout order of precedence (highest to lowest):
* 1. pcmk_monitor_timeout resource parameter (only for starts and probes
* when rsc has pcmk_ra_cap_fence_params; this gets used for recurring
* monitors via the executor instead)
* 2. timeout configured in <op> (with <op timeout> taking precedence over
* <op> <meta_attributes>)
* 3. timeout configured in <op_defaults> <meta_attributes>
* 4. PCMK_DEFAULT_ACTION_TIMEOUT_MS
*/
// Check for pcmk_monitor_timeout
if (pcmk_is_set(pcmk_get_ra_caps(rsc_rule_data.standard),
pcmk_ra_cap_fence_params)
&& (pcmk__str_eq(action_name, PCMK_ACTION_START, pcmk__str_none)
|| pcmk_is_probe(action_name, interval_ms))) {
GHashTable *params = pe_rsc_params(rsc, node, rsc->priv->scheduler);
timeout_spec = g_hash_table_lookup(params, "pcmk_monitor_timeout");
if (timeout_spec != NULL) {
pcmk__rsc_trace(rsc,
"Setting timeout for %s %s to "
"pcmk_monitor_timeout (%s)",
rsc->id, action_name, timeout_spec);
pcmk__insert_dup(meta, PCMK_META_TIMEOUT, timeout_spec);
}
}
// Normalize timeout to positive milliseconds
timeout_spec = g_hash_table_lookup(meta, PCMK_META_TIMEOUT);
g_hash_table_insert(meta, pcmk__str_copy(PCMK_META_TIMEOUT),
pcmk__itoa(unpack_timeout(timeout_spec)));
// Ensure on-fail has a valid value
validate_on_fail(rsc, action_name, action_config, meta);
// Normalize PCMK_META_START_DELAY
str = g_hash_table_lookup(meta, PCMK_META_START_DELAY);
if (str != NULL) {
unpack_start_delay(str, meta);
} else {
long long start_delay = 0;
str = g_hash_table_lookup(meta, PCMK_META_INTERVAL_ORIGIN);
if (unpack_interval_origin(str, action_config, interval_ms,
rsc->priv->scheduler->priv->now,
&start_delay)) {
g_hash_table_insert(meta, pcmk__str_copy(PCMK_META_START_DELAY),
crm_strdup_printf("%lld", start_delay));
}
}
return meta;
}
/*!
* \internal
* \brief Determine an action's quorum and fencing dependency
*
* \param[in] rsc Resource that action is for
* \param[in] action_name Name of action being unpacked
*
* \return Quorum and fencing dependency appropriate to action
*/
enum pcmk__requires
pcmk__action_requires(const pcmk_resource_t *rsc, const char *action_name)
{
const char *value = NULL;
enum pcmk__requires requires = pcmk__requires_nothing;
CRM_CHECK((rsc != NULL) && (action_name != NULL), return requires);
if (!pcmk__strcase_any_of(action_name, PCMK_ACTION_START,
PCMK_ACTION_PROMOTE, NULL)) {
value = "nothing (not start or promote)";
} else if (pcmk_is_set(rsc->flags, pcmk__rsc_needs_fencing)) {
requires = pcmk__requires_fencing;
value = "fencing";
} else if (pcmk_is_set(rsc->flags, pcmk__rsc_needs_quorum)) {
requires = pcmk__requires_quorum;
value = "quorum";
} else {
value = "nothing";
}
pcmk__rsc_trace(rsc, "%s of %s requires %s", action_name, rsc->id, value);
return requires;
}
/*!
* \internal
* \brief Parse action failure response from a user-provided string
*
* \param[in] rsc Resource that action is for
* \param[in] action_name Name of action
* \param[in] interval_ms Action interval (in milliseconds)
* \param[in] value User-provided configuration value for on-fail
*
* \return Action failure response parsed from \p text
*/
enum pcmk__on_fail
pcmk__parse_on_fail(const pcmk_resource_t *rsc, const char *action_name,
guint interval_ms, const char *value)
{
const char *desc = NULL;
bool needs_remote_reset = false;
enum pcmk__on_fail on_fail = pcmk__on_fail_ignore;
const pcmk_scheduler_t *scheduler = NULL;
// There's no enum value for unknown or invalid, so assert
pcmk__assert((rsc != NULL) && (action_name != NULL));
scheduler = rsc->priv->scheduler;
if (value == NULL) {
// Use default
} else if (pcmk__str_eq(value, PCMK_VALUE_BLOCK, pcmk__str_casei)) {
on_fail = pcmk__on_fail_block;
desc = "block";
} else if (pcmk__str_eq(value, PCMK_VALUE_FENCE, pcmk__str_casei)) {
if (pcmk_is_set(scheduler->flags, pcmk__sched_fencing_enabled)) {
on_fail = pcmk__on_fail_fence_node;
desc = "node fencing";
} else {
pcmk__config_err("Resetting '" PCMK_META_ON_FAIL "' for "
"%s of %s to 'stop' because 'fence' is not "
"valid when fencing is disabled",
action_name, rsc->id);
on_fail = pcmk__on_fail_stop;
desc = "stop resource";
}
} else if (pcmk__str_eq(value, PCMK_VALUE_STANDBY, pcmk__str_casei)) {
on_fail = pcmk__on_fail_standby_node;
desc = "node standby";
} else if (pcmk__strcase_any_of(value,
PCMK_VALUE_IGNORE, PCMK_VALUE_NOTHING,
NULL)) {
desc = "ignore";
} else if (pcmk__str_eq(value, "migrate", pcmk__str_casei)) {
on_fail = pcmk__on_fail_ban;
desc = "force migration";
} else if (pcmk__str_eq(value, PCMK_VALUE_STOP, pcmk__str_casei)) {
on_fail = pcmk__on_fail_stop;
desc = "stop resource";
} else if (pcmk__str_eq(value, PCMK_VALUE_RESTART, pcmk__str_casei)) {
on_fail = pcmk__on_fail_restart;
desc = "restart (and possibly migrate)";
} else if (pcmk__str_eq(value, PCMK_VALUE_RESTART_CONTAINER,
pcmk__str_casei)) {
if (rsc->priv->launcher == NULL) {
pcmk__rsc_debug(rsc,
"Using default " PCMK_META_ON_FAIL " for %s "
"of %s because it does not have a launcher",
action_name, rsc->id);
} else {
on_fail = pcmk__on_fail_restart_container;
desc = "restart container (and possibly migrate)";
}
} else if (pcmk__str_eq(value, PCMK_VALUE_DEMOTE, pcmk__str_casei)) {
on_fail = pcmk__on_fail_demote;
desc = "demote instance";
} else {
pcmk__config_err("Using default '" PCMK_META_ON_FAIL "' for "
"%s of %s because '%s' is not valid",
action_name, rsc->id, value);
}
/* Remote node connections are handled specially. Failures that result
* in dropping an active connection must result in fencing. The only
* failures that don't are probes and starts. The user can explicitly set
* PCMK_META_ON_FAIL=PCMK_VALUE_FENCE to fence after start failures.
*/
if (pcmk_is_set(rsc->flags, pcmk__rsc_is_remote_connection)
&& pcmk__is_remote_node(pcmk_find_node(scheduler, rsc->id))
&& !pcmk_is_probe(action_name, interval_ms)
&& !pcmk__str_eq(action_name, PCMK_ACTION_START, pcmk__str_none)) {
needs_remote_reset = true;
if (!pcmk_is_set(rsc->flags, pcmk__rsc_managed)) {
desc = NULL; // Force default for unmanaged connections
}
}
if (desc != NULL) {
// Explicit value used, default not needed
} else if (rsc->priv->launcher != NULL) {
on_fail = pcmk__on_fail_restart_container;
desc = "restart container (and possibly migrate) (default)";
} else if (needs_remote_reset) {
if (pcmk_is_set(rsc->flags, pcmk__rsc_managed)) {
if (pcmk_is_set(scheduler->flags, pcmk__sched_fencing_enabled)) {
desc = "fence remote node (default)";
} else {
desc = "recover remote node connection (default)";
}
on_fail = pcmk__on_fail_reset_remote;
} else {
on_fail = pcmk__on_fail_stop;
desc = "stop unmanaged remote node (enforcing default)";
}
} else if (pcmk__str_eq(action_name, PCMK_ACTION_STOP, pcmk__str_none)) {
if (pcmk_is_set(scheduler->flags, pcmk__sched_fencing_enabled)) {
on_fail = pcmk__on_fail_fence_node;
desc = "resource fence (default)";
} else {
on_fail = pcmk__on_fail_block;
desc = "resource block (default)";
}
} else {
on_fail = pcmk__on_fail_restart;
desc = "restart (and possibly migrate) (default)";
}
pcmk__rsc_trace(rsc, "Failure handling for %s-interval %s of %s: %s",
pcmk__readable_interval(interval_ms), action_name,
rsc->id, desc);
return on_fail;
}
/*!
* \internal
* \brief Determine a resource's role after failure of an action
*
* \param[in] rsc Resource that action is for
* \param[in] action_name Action name
* \param[in] on_fail Failure handling for action
* \param[in] meta Unpacked action meta-attributes
*
* \return Resource role that results from failure of action
*/
enum rsc_role_e
pcmk__role_after_failure(const pcmk_resource_t *rsc, const char *action_name,
enum pcmk__on_fail on_fail, GHashTable *meta)
{
enum rsc_role_e role = pcmk_role_unknown;
// Set default for role after failure specially in certain circumstances
switch (on_fail) {
case pcmk__on_fail_stop:
role = pcmk_role_stopped;
break;
case pcmk__on_fail_reset_remote:
if (rsc->priv->remote_reconnect_ms != 0U) {
role = pcmk_role_stopped;
}
break;
default:
break;
}
if (role == pcmk_role_unknown) {
// Use default
if (pcmk__str_eq(action_name, PCMK_ACTION_PROMOTE, pcmk__str_none)) {
role = pcmk_role_unpromoted;
} else {
role = pcmk_role_started;
}
}
pcmk__rsc_trace(rsc, "Role after %s %s failure is: %s",
rsc->id, action_name, pcmk_role_text(role));
return role;
}
/*!
* \internal
* \brief Unpack action configuration
*
* Unpack a resource action's meta-attributes (normalizing the interval,
* timeout, and start delay values as integer milliseconds), requirements, and
* failure policy from its CIB XML configuration (including defaults).
*
* \param[in,out] action Resource action to unpack into
* \param[in] xml_obj Action configuration XML (NULL for defaults only)
* \param[in] interval_ms How frequently to perform the operation
*/
static void
unpack_operation(pcmk_action_t *action, const xmlNode *xml_obj,
guint interval_ms)
{
const char *value = NULL;
action->meta = pcmk__unpack_action_meta(action->rsc, action->node,
action->task, interval_ms, xml_obj);
action->needs = pcmk__action_requires(action->rsc, action->task);
value = g_hash_table_lookup(action->meta, PCMK_META_ON_FAIL);
action->on_fail = pcmk__parse_on_fail(action->rsc, action->task,
interval_ms, value);
action->fail_role = pcmk__role_after_failure(action->rsc, action->task,
action->on_fail, action->meta);
}
/*!
* \brief Create or update an action object
*
* \param[in,out] rsc Resource that action is for (if any)
* \param[in,out] key Action key (must be non-NULL)
* \param[in] task Action name (must be non-NULL)
* \param[in] on_node Node that action is on (if any)
* \param[in] optional Whether action should be considered optional
* \param[in,out] scheduler Scheduler data
*
* \return Action object corresponding to arguments (guaranteed not to be
* \c NULL)
* \note This function takes ownership of (and might free) \p key, and
* \p scheduler takes ownership of the returned action (the caller should
* not free it).
*/
pcmk_action_t *
custom_action(pcmk_resource_t *rsc, char *key, const char *task,
const pcmk_node_t *on_node, gboolean optional,
pcmk_scheduler_t *scheduler)
{
pcmk_action_t *action = NULL;
pcmk__assert((key != NULL) && (task != NULL) && (scheduler != NULL));
action = find_existing_action(key, rsc, on_node, scheduler);
if (action == NULL) {
action = new_action(key, task, rsc, on_node, optional, scheduler);
} else {
free(key);
}
update_action_optional(action, optional);
if (rsc != NULL) {
/* An action can be initially created with a NULL node, and later have
* the node added via find_existing_action() (above) -> find_actions().
* That is why the extra parameters are unpacked here rather than in
* new_action().
*/
if ((action->node != NULL) && (action->op_entry != NULL)
&& !pcmk_is_set(action->flags, pcmk__action_attrs_evaluated)) {
GHashTable *attrs = action->node->priv->attrs;
if (action->extra != NULL) {
g_hash_table_destroy(action->extra);
}
action->extra = pcmk__unpack_action_rsc_params(action->op_entry,
attrs, scheduler);
pcmk__set_action_flags(action, pcmk__action_attrs_evaluated);
}
update_resource_action_runnable(action, scheduler);
}
if (action->extra == NULL) {
action->extra = pcmk__strkey_table(free, free);
}
return action;
}
pcmk_action_t *
get_pseudo_op(const char *name, pcmk_scheduler_t *scheduler)
{
pcmk_action_t *op = lookup_singleton(scheduler, name);
if (op == NULL) {
op = custom_action(NULL, strdup(name), name, NULL, TRUE, scheduler);
pcmk__set_action_flags(op, pcmk__action_pseudo|pcmk__action_runnable);
}
return op;
}
static GList *
find_unfencing_devices(GList *candidates, GList *matches)
{
for (GList *gIter = candidates; gIter != NULL; gIter = gIter->next) {
pcmk_resource_t *candidate = gIter->data;
if (candidate->priv->children != NULL) {
matches = find_unfencing_devices(candidate->priv->children,
matches);
} else if (!pcmk_is_set(candidate->flags, pcmk__rsc_fence_device)) {
continue;
} else if (pcmk_is_set(candidate->flags, pcmk__rsc_needs_unfencing)) {
matches = g_list_prepend(matches, candidate);
} else if (pcmk__str_eq(g_hash_table_lookup(candidate->priv->meta,
PCMK_STONITH_PROVIDES),
PCMK_VALUE_UNFENCING, pcmk__str_casei)) {
matches = g_list_prepend(matches, candidate);
}
}
return matches;
}
static int
node_priority_fencing_delay(const pcmk_node_t *node,
const pcmk_scheduler_t *scheduler)
{
int member_count = 0;
int online_count = 0;
int top_priority = 0;
int lowest_priority = 0;
GList *gIter = NULL;
// PCMK_OPT_PRIORITY_FENCING_DELAY is disabled
if (scheduler->priv->priority_fencing_ms == 0U) {
return 0;
}
/* No need to request a delay if the fencing target is not a normal cluster
* member, for example if it's a remote node or a guest node. */
if (node->priv->variant != pcmk__node_variant_cluster) {
return 0;
}
// No need to request a delay if the fencing target is in our partition
if (node->details->online) {
return 0;
}
for (gIter = scheduler->nodes; gIter != NULL; gIter = gIter->next) {
pcmk_node_t *n = gIter->data;
if (n->priv->variant != pcmk__node_variant_cluster) {
continue;
}
member_count ++;
if (n->details->online) {
online_count++;
}
if (member_count == 1
|| n->priv->priority > top_priority) {
top_priority = n->priv->priority;
}
if (member_count == 1
|| n->priv->priority < lowest_priority) {
lowest_priority = n->priv->priority;
}
}
// No need to delay if we have more than half of the cluster members
if (online_count > member_count / 2) {
return 0;
}
/* All the nodes have equal priority.
* Any configured corresponding `pcmk_delay_base/max` will be applied. */
if (lowest_priority == top_priority) {
return 0;
}
if (node->priv->priority < top_priority) {
return 0;
}
return pcmk__timeout_ms2s(scheduler->priv->priority_fencing_ms);
}
pcmk_action_t *
pe_fence_op(pcmk_node_t *node, const char *op, bool optional,
const char *reason, bool priority_delay,
pcmk_scheduler_t *scheduler)
{
char *op_key = NULL;
pcmk_action_t *stonith_op = NULL;
if(op == NULL) {
op = scheduler->priv->fence_action;
}
op_key = crm_strdup_printf("%s-%s-%s",
PCMK_ACTION_STONITH, node->priv->name, op);
stonith_op = lookup_singleton(scheduler, op_key);
if(stonith_op == NULL) {
stonith_op = custom_action(NULL, op_key, PCMK_ACTION_STONITH, node,
TRUE, scheduler);
pcmk__insert_meta(stonith_op, PCMK__META_ON_NODE, node->priv->name);
pcmk__insert_meta(stonith_op, PCMK__META_ON_NODE_UUID,
node->priv->id);
pcmk__insert_meta(stonith_op, PCMK__META_STONITH_ACTION, op);
if (pcmk_is_set(scheduler->flags, pcmk__sched_enable_unfencing)) {
/* Extra work to detect device changes
*/
GString *digests_all = g_string_sized_new(1024);
GString *digests_secure = g_string_sized_new(1024);
GList *matches = find_unfencing_devices(scheduler->priv->resources,
NULL);
for (GList *gIter = matches; gIter != NULL; gIter = gIter->next) {
pcmk_resource_t *match = gIter->data;
const char *agent = g_hash_table_lookup(match->priv->meta,
PCMK_XA_TYPE);
pcmk__op_digest_t *data = NULL;
data = pe__compare_fencing_digest(match, agent, node,
scheduler);
if (data->rc == pcmk__digest_mismatch) {
optional = FALSE;
crm_notice("Unfencing node %s because the definition of "
"%s changed", pcmk__node_name(node), match->id);
if (!pcmk__is_daemon && (scheduler->priv->out != NULL)) {
pcmk__output_t *out = scheduler->priv->out;
out->info(out,
"notice: Unfencing node %s because the "
"definition of %s changed",
pcmk__node_name(node), match->id);
}
}
pcmk__g_strcat(digests_all,
match->id, ":", agent, ":",
data->digest_all_calc, ",", NULL);
pcmk__g_strcat(digests_secure,
match->id, ":", agent, ":",
data->digest_secure_calc, ",", NULL);
}
pcmk__insert_dup(stonith_op->meta, PCMK__META_DIGESTS_ALL,
digests_all->str);
g_string_free(digests_all, TRUE);
pcmk__insert_dup(stonith_op->meta, PCMK__META_DIGESTS_SECURE,
digests_secure->str);
g_string_free(digests_secure, TRUE);
g_list_free(matches);
}
} else {
free(op_key);
}
if ((scheduler->priv->priority_fencing_ms > 0U)
/* It's a suitable case where PCMK_OPT_PRIORITY_FENCING_DELAY
* applies. At least add PCMK_OPT_PRIORITY_FENCING_DELAY field as
* an indicator.
*/
&& (priority_delay
/* The priority delay needs to be recalculated if this function has
* been called by schedule_fencing_and_shutdowns() after node
* priority has already been calculated by native_add_running().
*/
|| g_hash_table_lookup(stonith_op->meta,
PCMK_OPT_PRIORITY_FENCING_DELAY) != NULL)) {
/* Add PCMK_OPT_PRIORITY_FENCING_DELAY to the fencing op even if
* it's 0 for the targeting node. So that it takes precedence over
* any possible `pcmk_delay_base/max`.
*/
char *delay_s = pcmk__itoa(node_priority_fencing_delay(node,
scheduler));
g_hash_table_insert(stonith_op->meta,
strdup(PCMK_OPT_PRIORITY_FENCING_DELAY),
delay_s);
}
if(optional == FALSE && pe_can_fence(scheduler, node)) {
pcmk__clear_action_flags(stonith_op, pcmk__action_optional);
pe_action_set_reason(stonith_op, reason, false);
} else if(reason && stonith_op->reason == NULL) {
stonith_op->reason = strdup(reason);
}
return stonith_op;
}
void
pe_free_action(pcmk_action_t *action)
{
if (action == NULL) {
return;
}
g_list_free_full(action->actions_before, free);
g_list_free_full(action->actions_after, free);
if (action->extra) {
g_hash_table_destroy(action->extra);
}
if (action->meta) {
g_hash_table_destroy(action->meta);
}
+ pcmk__free_node_copy(action->node);
free(action->cancel_task);
free(action->reason);
free(action->task);
free(action->uuid);
- free(action->node);
free(action);
}
enum pcmk__action_type
get_complex_task(const pcmk_resource_t *rsc, const char *name)
{
enum pcmk__action_type task = pcmk__parse_action(name);
if (pcmk__is_primitive(rsc)) {
switch (task) {
case pcmk__action_stopped:
case pcmk__action_started:
case pcmk__action_demoted:
case pcmk__action_promoted:
crm_trace("Folding %s back into its atomic counterpart for %s",
name, rsc->id);
--task;
break;
default:
break;
}
}
return task;
}
/*!
* \internal
* \brief Find first matching action in a list
*
* \param[in] input List of actions to search
* \param[in] uuid If not NULL, action must have this UUID
* \param[in] task If not NULL, action must have this action name
* \param[in] on_node If not NULL, action must be on this node
*
* \return First action in list that matches criteria, or NULL if none
*/
pcmk_action_t *
find_first_action(const GList *input, const char *uuid, const char *task,
const pcmk_node_t *on_node)
{
CRM_CHECK(uuid || task, return NULL);
for (const GList *gIter = input; gIter != NULL; gIter = gIter->next) {
pcmk_action_t *action = (pcmk_action_t *) gIter->data;
if (uuid != NULL && !pcmk__str_eq(uuid, action->uuid, pcmk__str_casei)) {
continue;
} else if (task != NULL && !pcmk__str_eq(task, action->task, pcmk__str_casei)) {
continue;
} else if (on_node == NULL) {
return action;
} else if (action->node == NULL) {
continue;
} else if (pcmk__same_node(on_node, action->node)) {
return action;
}
}
return NULL;
}
GList *
find_actions(GList *input, const char *key, const pcmk_node_t *on_node)
{
GList *gIter = input;
GList *result = NULL;
CRM_CHECK(key != NULL, return NULL);
for (; gIter != NULL; gIter = gIter->next) {
pcmk_action_t *action = (pcmk_action_t *) gIter->data;
if (!pcmk__str_eq(key, action->uuid, pcmk__str_casei)) {
continue;
} else if (on_node == NULL) {
crm_trace("Action %s matches (ignoring node)", key);
result = g_list_prepend(result, action);
} else if (action->node == NULL) {
crm_trace("Action %s matches (unallocated, assigning to %s)",
key, pcmk__node_name(on_node));
action->node = pe__copy_node(on_node);
result = g_list_prepend(result, action);
} else if (pcmk__same_node(on_node, action->node)) {
crm_trace("Action %s on %s matches", key, pcmk__node_name(on_node));
result = g_list_prepend(result, action);
}
}
return result;
}
GList *
find_actions_exact(GList *input, const char *key, const pcmk_node_t *on_node)
{
GList *result = NULL;
CRM_CHECK(key != NULL, return NULL);
if (on_node == NULL) {
return NULL;
}
for (GList *gIter = input; gIter != NULL; gIter = gIter->next) {
pcmk_action_t *action = (pcmk_action_t *) gIter->data;
if ((action->node != NULL)
&& pcmk__str_eq(key, action->uuid, pcmk__str_casei)
&& pcmk__same_node(on_node, action->node)) {
crm_trace("Action %s on %s matches", key, pcmk__node_name(on_node));
result = g_list_prepend(result, action);
}
}
return result;
}
/*!
* \brief Find all actions of given type for a resource
*
* \param[in] rsc Resource to search
* \param[in] node Find only actions scheduled on this node
* \param[in] task Action name to search for
* \param[in] require_node If TRUE, NULL node or action node will not match
*
* \return List of actions found (or NULL if none)
* \note If node is not NULL and require_node is FALSE, matching actions
* without a node will be assigned to node.
*/
GList *
pe__resource_actions(const pcmk_resource_t *rsc, const pcmk_node_t *node,
const char *task, bool require_node)
{
GList *result = NULL;
char *key = pcmk__op_key(rsc->id, task, 0);
if (require_node) {
result = find_actions_exact(rsc->priv->actions, key, node);
} else {
result = find_actions(rsc->priv->actions, key, node);
}
free(key);
return result;
}
/*!
* \internal
* \brief Create an action reason string based on the action itself
*
* \param[in] action Action to create reason string for
* \param[in] flag Action flag that was cleared
*
* \return Newly allocated string suitable for use as action reason
* \note It is the caller's responsibility to free() the result.
*/
char *
pe__action2reason(const pcmk_action_t *action, enum pcmk__action_flags flag)
{
const char *change = NULL;
switch (flag) {
case pcmk__action_runnable:
change = "unrunnable";
break;
case pcmk__action_migratable:
change = "unmigrateable";
break;
case pcmk__action_optional:
change = "required";
break;
default:
// Bug: caller passed unsupported flag
CRM_CHECK(change != NULL, change = "");
break;
}
return crm_strdup_printf("%s%s%s %s", change,
(action->rsc == NULL)? "" : " ",
(action->rsc == NULL)? "" : action->rsc->id,
action->task);
}
void pe_action_set_reason(pcmk_action_t *action, const char *reason,
bool overwrite)
{
if (action->reason != NULL && overwrite) {
pcmk__rsc_trace(action->rsc, "Changing %s reason from '%s' to '%s'",
action->uuid, action->reason,
pcmk__s(reason, "(none)"));
} else if (action->reason == NULL) {
pcmk__rsc_trace(action->rsc, "Set %s reason to '%s'",
action->uuid, pcmk__s(reason, "(none)"));
} else {
// crm_assert(action->reason != NULL && !overwrite);
return;
}
pcmk__str_update(&action->reason, reason);
}
/*!
* \internal
* \brief Create an action to clear a resource's history from CIB
*
* \param[in,out] rsc Resource to clear
* \param[in] node Node to clear history on
*/
void
pe__clear_resource_history(pcmk_resource_t *rsc, const pcmk_node_t *node)
{
pcmk__assert((rsc != NULL) && (node != NULL));
custom_action(rsc, pcmk__op_key(rsc->id, PCMK_ACTION_LRM_DELETE, 0),
PCMK_ACTION_LRM_DELETE, node, FALSE, rsc->priv->scheduler);
}
#define sort_return(an_int, why) do { \
free(a_uuid); \
free(b_uuid); \
crm_trace("%s (%d) %c %s (%d) : %s", \
a_xml_id, a_call_id, an_int>0?'>':an_int<0?'<':'=', \
b_xml_id, b_call_id, why); \
return an_int; \
} while(0)
int
pe__is_newer_op(const xmlNode *xml_a, const xmlNode *xml_b)
{
int a_call_id = -1;
int b_call_id = -1;
char *a_uuid = NULL;
char *b_uuid = NULL;
const char *a_xml_id = crm_element_value(xml_a, PCMK_XA_ID);
const char *b_xml_id = crm_element_value(xml_b, PCMK_XA_ID);
const char *a_node = crm_element_value(xml_a, PCMK__META_ON_NODE);
const char *b_node = crm_element_value(xml_b, PCMK__META_ON_NODE);
bool same_node = pcmk__str_eq(a_node, b_node, pcmk__str_casei);
if (same_node && pcmk__str_eq(a_xml_id, b_xml_id, pcmk__str_none)) {
/* We have duplicate PCMK__XE_LRM_RSC_OP entries in the status
* section which is unlikely to be a good thing
* - we can handle it easily enough, but we need to get
* to the bottom of why it's happening.
*/
pcmk__config_err("Duplicate " PCMK__XE_LRM_RSC_OP " entries named %s",
a_xml_id);
sort_return(0, "duplicate");
}
crm_element_value_int(xml_a, PCMK__XA_CALL_ID, &a_call_id);
crm_element_value_int(xml_b, PCMK__XA_CALL_ID, &b_call_id);
if (a_call_id == -1 && b_call_id == -1) {
/* both are pending ops so it doesn't matter since
* stops are never pending
*/
sort_return(0, "pending");
} else if (same_node && a_call_id >= 0 && a_call_id < b_call_id) {
sort_return(-1, "call id");
} else if (same_node && b_call_id >= 0 && a_call_id > b_call_id) {
sort_return(1, "call id");
} else if (a_call_id >= 0 && b_call_id >= 0
&& (!same_node || a_call_id == b_call_id)) {
/* The op and last_failed_op are the same. Order on
* PCMK_XA_LAST_RC_CHANGE.
*/
time_t last_a = -1;
time_t last_b = -1;
crm_element_value_epoch(xml_a, PCMK_XA_LAST_RC_CHANGE, &last_a);
crm_element_value_epoch(xml_b, PCMK_XA_LAST_RC_CHANGE, &last_b);
crm_trace("rc-change: %lld vs %lld",
(long long) last_a, (long long) last_b);
if (last_a >= 0 && last_a < last_b) {
sort_return(-1, "rc-change");
} else if (last_b >= 0 && last_a > last_b) {
sort_return(1, "rc-change");
}
sort_return(0, "rc-change");
} else {
/* One of the inputs is a pending operation.
* Attempt to use PCMK__XA_TRANSITION_MAGIC to determine its age relative
* to the other.
*/
int a_id = -1;
int b_id = -1;
const char *a_magic = crm_element_value(xml_a,
PCMK__XA_TRANSITION_MAGIC);
const char *b_magic = crm_element_value(xml_b,
PCMK__XA_TRANSITION_MAGIC);
CRM_CHECK(a_magic != NULL && b_magic != NULL, sort_return(0, "No magic"));
if (!decode_transition_magic(a_magic, &a_uuid, &a_id, NULL, NULL, NULL,
NULL)) {
sort_return(0, "bad magic a");
}
if (!decode_transition_magic(b_magic, &b_uuid, &b_id, NULL, NULL, NULL,
NULL)) {
sort_return(0, "bad magic b");
}
/* try to determine the relative age of the operation...
* some pending operations (e.g. a start) may have been superseded
* by a subsequent stop
*
* [a|b]_id == -1 means it's a shutdown operation and _always_ comes last
*/
if (!pcmk__str_eq(a_uuid, b_uuid, pcmk__str_casei) || a_id == b_id) {
/*
* some of the logic in here may be redundant...
*
* if the UUID from the TE doesn't match then one better
* be a pending operation.
* pending operations don't survive between elections and joins
* because we query the LRM directly
*/
if (b_call_id == -1) {
sort_return(-1, "transition + call");
} else if (a_call_id == -1) {
sort_return(1, "transition + call");
}
} else if ((a_id >= 0 && a_id < b_id) || b_id == -1) {
sort_return(-1, "transition");
} else if ((b_id >= 0 && a_id > b_id) || a_id == -1) {
sort_return(1, "transition");
}
}
/* we should never end up here */
CRM_CHECK(FALSE, sort_return(0, "default"));
}
gint
sort_op_by_callid(gconstpointer a, gconstpointer b)
{
return pe__is_newer_op((const xmlNode *) a, (const xmlNode *) b);
}
/*!
* \internal
* \brief Create a new pseudo-action for a resource
*
* \param[in,out] rsc Resource to create action for
* \param[in] task Action name
* \param[in] optional Whether action should be considered optional
* \param[in] runnable Whethe action should be considered runnable
*
* \return New action object corresponding to arguments
*/
pcmk_action_t *
pe__new_rsc_pseudo_action(pcmk_resource_t *rsc, const char *task, bool optional,
bool runnable)
{
pcmk_action_t *action = NULL;
pcmk__assert((rsc != NULL) && (task != NULL));
action = custom_action(rsc, pcmk__op_key(rsc->id, task, 0), task, NULL,
optional, rsc->priv->scheduler);
pcmk__set_action_flags(action, pcmk__action_pseudo);
if (runnable) {
pcmk__set_action_flags(action, pcmk__action_runnable);
}
return action;
}
/*!
* \internal
* \brief Add the expected result to an action
*
* \param[in,out] action Action to add expected result to
* \param[in] expected_result Expected result to add
*
* \note This is more efficient than calling pcmk__insert_meta().
*/
void
pe__add_action_expected_result(pcmk_action_t *action, int expected_result)
{
pcmk__assert((action != NULL) && (action->meta != NULL));
g_hash_table_insert(action->meta, pcmk__str_copy(PCMK__META_OP_TARGET_RC),
pcmk__itoa(expected_result));
}
diff --git a/lib/pengine/status.c b/lib/pengine/status.c
index 1fce0b32af..2e4deb07e1 100644
--- a/lib/pengine/status.c
+++ b/lib/pengine/status.c
@@ -1,546 +1,546 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <sys/param.h>
#include <crm/crm.h>
#include <crm/common/xml.h>
#include <crm/common/cib_internal.h>
#include <glib.h>
#include <crm/pengine/internal.h>
#include <pe_status_private.h>
/*!
* \brief Create a new object to hold scheduler data
*
* \return New, initialized scheduler data on success, else NULL (and set errno)
* \note Only pcmk_scheduler_t objects created with this function (as opposed
* to statically declared or directly allocated) should be used with the
* functions in this library, to allow for future extensions to the
* data type. The caller is responsible for freeing the memory with
* pe_free_working_set() when the instance is no longer needed.
*/
pcmk_scheduler_t *
pe_new_working_set(void)
{
pcmk_scheduler_t *scheduler = calloc(1, sizeof(pcmk_scheduler_t));
if (scheduler == NULL) {
return NULL;
}
scheduler->priv = calloc(1, sizeof(pcmk__scheduler_private_t));
if (scheduler->priv == NULL) {
free(scheduler);
return NULL;
}
set_working_set_defaults(scheduler);
return scheduler;
}
/*!
* \brief Free scheduler data
*
* \param[in,out] scheduler Scheduler data to free
*/
void
pe_free_working_set(pcmk_scheduler_t *scheduler)
{
if (scheduler != NULL) {
pe_reset_working_set(scheduler);
free(scheduler->priv->local_node_name);
free(scheduler->priv);
free(scheduler);
}
}
#define XPATH_DEPRECATED_RULES \
"//" PCMK_XE_OP_DEFAULTS "//" PCMK_XE_EXPRESSION \
"|//" PCMK_XE_OP "//" PCMK_XE_EXPRESSION
/*!
* \internal
* \brief Log a warning for deprecated rule syntax in operations
*
* \param[in] scheduler Scheduler data
*/
static void
check_for_deprecated_rules(pcmk_scheduler_t *scheduler)
{
// @COMPAT Drop this function when support for the syntax is dropped
xmlNode *deprecated = get_xpath_object(XPATH_DEPRECATED_RULES,
scheduler->input, LOG_NEVER);
if (deprecated != NULL) {
pcmk__warn_once(pcmk__wo_op_attr_expr,
"Support for rules with node attribute expressions in "
PCMK_XE_OP " or " PCMK_XE_OP_DEFAULTS " is deprecated "
"and will be dropped in a future release");
}
}
/*
* Unpack everything
* At the end you'll have:
* - A list of nodes
* - A list of resources (each with any dependencies on other resources)
* - A list of constraints between resources and nodes
* - A list of constraints between start/stop actions
* - A list of nodes that need to be stonith'd
* - A list of nodes that need to be shutdown
* - A list of the possible stop/start actions (without dependencies)
*/
gboolean
cluster_status(pcmk_scheduler_t * scheduler)
{
const char *new_version = NULL;
xmlNode *section = NULL;
if ((scheduler == NULL) || (scheduler->input == NULL)) {
return FALSE;
}
new_version = crm_element_value(scheduler->input, PCMK_XA_CRM_FEATURE_SET);
if (pcmk__check_feature_set(new_version) != pcmk_rc_ok) {
pcmk__config_err("Can't process CIB with feature set '%s' greater than our own '%s'",
new_version, CRM_FEATURE_SET);
return FALSE;
}
crm_trace("Beginning unpack");
if (scheduler->priv->failed != NULL) {
pcmk__xml_free(scheduler->priv->failed);
}
scheduler->priv->failed = pcmk__xe_create(NULL, "failed-ops");
if (scheduler->priv->now == NULL) {
scheduler->priv->now = crm_time_new(NULL);
}
if (pcmk__xe_attr_is_true(scheduler->input, PCMK_XA_HAVE_QUORUM)) {
pcmk__set_scheduler_flags(scheduler, pcmk__sched_quorate);
} else {
pcmk__clear_scheduler_flags(scheduler, pcmk__sched_quorate);
}
scheduler->priv->op_defaults = get_xpath_object("//" PCMK_XE_OP_DEFAULTS,
scheduler->input,
LOG_NEVER);
check_for_deprecated_rules(scheduler);
scheduler->priv->rsc_defaults = get_xpath_object("//" PCMK_XE_RSC_DEFAULTS,
scheduler->input,
LOG_NEVER);
section = get_xpath_object("//" PCMK_XE_CRM_CONFIG, scheduler->input,
LOG_TRACE);
unpack_config(section, scheduler);
if (!pcmk_any_flags_set(scheduler->flags,
pcmk__sched_location_only|pcmk__sched_quorate)
&& (scheduler->no_quorum_policy != pcmk_no_quorum_ignore)) {
pcmk__sched_warn(scheduler,
"Fencing and resource management disabled "
"due to lack of quorum");
}
section = get_xpath_object("//" PCMK_XE_NODES, scheduler->input, LOG_TRACE);
unpack_nodes(section, scheduler);
section = get_xpath_object("//" PCMK_XE_RESOURCES, scheduler->input,
LOG_TRACE);
if (!pcmk_is_set(scheduler->flags, pcmk__sched_location_only)) {
unpack_remote_nodes(section, scheduler);
}
unpack_resources(section, scheduler);
section = get_xpath_object("//" PCMK_XE_FENCING_TOPOLOGY, scheduler->input,
LOG_TRACE);
pcmk__validate_fencing_topology(section);
section = get_xpath_object("//" PCMK_XE_TAGS, scheduler->input, LOG_NEVER);
unpack_tags(section, scheduler);
if (!pcmk_is_set(scheduler->flags, pcmk__sched_location_only)) {
section = get_xpath_object("//" PCMK_XE_STATUS, scheduler->input,
LOG_TRACE);
unpack_status(section, scheduler);
}
if (!pcmk_is_set(scheduler->flags, pcmk__sched_no_counts)) {
for (GList *item = scheduler->priv->resources;
item != NULL; item = item->next) {
pcmk_resource_t *rsc = item->data;
rsc->priv->fns->count(item->data);
}
crm_trace("Cluster resource count: %d (%d disabled, %d blocked)",
scheduler->priv->ninstances,
scheduler->priv->disabled_resources,
scheduler->priv->blocked_resources);
}
if ((scheduler->priv->local_node_name != NULL)
&& (pcmk_find_node(scheduler,
scheduler->priv->local_node_name) == NULL)) {
crm_info("Creating a fake local node for %s",
scheduler->priv->local_node_name);
pe_create_node(scheduler->priv->local_node_name,
scheduler->priv->local_node_name, NULL, 0, scheduler);
}
pcmk__set_scheduler_flags(scheduler, pcmk__sched_have_status);
return TRUE;
}
/*!
* \internal
* \brief Free a list of pcmk_resource_t
*
* \param[in,out] resources List to free
*
* \note When the scheduler's resource list is freed, that includes the original
* storage for the uname and id of any Pacemaker Remote nodes in the
* scheduler's node list, so take care not to use those afterward.
* \todo Refactor pcmk_node_t to strdup() the node name.
*/
static void
pe_free_resources(GList *resources)
{
pcmk_resource_t *rsc = NULL;
GList *iterator = resources;
while (iterator != NULL) {
rsc = (pcmk_resource_t *) iterator->data;
iterator = iterator->next;
rsc->priv->fns->free(rsc);
}
if (resources != NULL) {
g_list_free(resources);
}
}
static void
pe_free_actions(GList *actions)
{
GList *iterator = actions;
while (iterator != NULL) {
pe_free_action(iterator->data);
iterator = iterator->next;
}
if (actions != NULL) {
g_list_free(actions);
}
}
static void
pe_free_nodes(GList *nodes)
{
for (GList *iterator = nodes; iterator != NULL; iterator = iterator->next) {
pcmk_node_t *node = (pcmk_node_t *) iterator->data;
// Shouldn't be possible, but to be safe ...
if (node == NULL) {
continue;
}
if (node->details == NULL) {
free(node);
continue;
}
/* This is called after pe_free_resources(), which means that we can't
* use node->private->name for Pacemaker Remote nodes.
*/
crm_trace("Freeing node %s", (pcmk__is_pacemaker_remote_node(node)?
"(guest or remote)" : pcmk__node_name(node)));
if (node->priv->attrs != NULL) {
g_hash_table_destroy(node->priv->attrs);
}
if (node->priv->utilization != NULL) {
g_hash_table_destroy(node->priv->utilization);
}
if (node->priv->digest_cache != NULL) {
g_hash_table_destroy(node->priv->digest_cache);
}
g_list_free(node->details->running_rsc);
g_list_free(node->priv->assigned_resources);
free(node->priv);
free(node->details);
free(node->assign);
free(node);
}
if (nodes != NULL) {
g_list_free(nodes);
}
}
static void
pe__free_ordering(GList *constraints)
{
GList *iterator = constraints;
while (iterator != NULL) {
pcmk__action_relation_t *order = iterator->data;
iterator = iterator->next;
free(order->task1);
free(order->task2);
free(order);
}
if (constraints != NULL) {
g_list_free(constraints);
}
}
static void
pe__free_location(GList *constraints)
{
GList *iterator = constraints;
while (iterator != NULL) {
pcmk__location_t *cons = iterator->data;
iterator = iterator->next;
- g_list_free_full(cons->nodes, free);
+ g_list_free_full(cons->nodes, pcmk__free_node_copy);
free(cons->id);
free(cons);
}
if (constraints != NULL) {
g_list_free(constraints);
}
}
/*!
* \brief Reset scheduler data to defaults without freeing it or constraints
*
* \param[in,out] scheduler Scheduler data to reset
*
* \deprecated This function is deprecated as part of the API;
* pe_reset_working_set() should be used instead.
*/
void
cleanup_calculations(pcmk_scheduler_t *scheduler)
{
if (scheduler == NULL) {
return;
}
pcmk__clear_scheduler_flags(scheduler, pcmk__sched_have_status);
if (scheduler->priv->options != NULL) {
g_hash_table_destroy(scheduler->priv->options);
}
if (scheduler->priv->singletons != NULL) {
g_hash_table_destroy(scheduler->priv->singletons);
}
if (scheduler->priv->ticket_constraints != NULL) {
g_hash_table_destroy(scheduler->priv->ticket_constraints);
}
if (scheduler->priv->templates != NULL) {
g_hash_table_destroy(scheduler->priv->templates);
}
if (scheduler->priv->tags != NULL) {
g_hash_table_destroy(scheduler->priv->tags);
}
crm_trace("deleting resources");
pe_free_resources(scheduler->priv->resources);
crm_trace("deleting actions");
pe_free_actions(scheduler->priv->actions);
crm_trace("deleting nodes");
pe_free_nodes(scheduler->nodes);
pe__free_param_checks(scheduler);
g_list_free(scheduler->priv->stop_needed);
crm_time_free(scheduler->priv->now);
pcmk__xml_free(scheduler->input);
pcmk__xml_free(scheduler->priv->failed);
pcmk__xml_free(scheduler->priv->graph);
set_working_set_defaults(scheduler);
CRM_LOG_ASSERT((scheduler->priv->location_constraints == NULL)
&& (scheduler->priv->ordering_constraints == NULL));
}
/*!
* \brief Reset scheduler data to default state without freeing it
*
* \param[in,out] scheduler Scheduler data to reset
*/
void
pe_reset_working_set(pcmk_scheduler_t *scheduler)
{
if (scheduler == NULL) {
return;
}
crm_trace("Deleting %d ordering constraints",
g_list_length(scheduler->priv->ordering_constraints));
pe__free_ordering(scheduler->priv->ordering_constraints);
scheduler->priv->ordering_constraints = NULL;
crm_trace("Deleting %d location constraints",
g_list_length(scheduler->priv->location_constraints));
pe__free_location(scheduler->priv->location_constraints);
scheduler->priv->location_constraints = NULL;
crm_trace("Deleting %d colocation constraints",
g_list_length(scheduler->priv->colocation_constraints));
g_list_free_full(scheduler->priv->colocation_constraints, free);
scheduler->priv->colocation_constraints = NULL;
cleanup_calculations(scheduler);
}
void
set_working_set_defaults(pcmk_scheduler_t *scheduler)
{
// These members must be preserved
pcmk__scheduler_private_t *priv = scheduler->priv;
pcmk__output_t *out = priv->out;
char *local_node_name = scheduler->priv->local_node_name;
// Wipe the main structs (any other members must have previously been freed)
memset(scheduler, 0, sizeof(pcmk_scheduler_t));
memset(priv, 0, sizeof(pcmk__scheduler_private_t));
// Restore the members to preserve
scheduler->priv = priv;
scheduler->priv->out = out;
scheduler->priv->local_node_name = local_node_name;
// Set defaults for everything else
scheduler->priv->next_ordering_id = 1;
scheduler->priv->next_action_id = 1;
scheduler->no_quorum_policy = pcmk_no_quorum_stop;
#if PCMK__CONCURRENT_FENCING_DEFAULT_TRUE
pcmk__set_scheduler_flags(scheduler,
pcmk__sched_symmetric_cluster
|pcmk__sched_concurrent_fencing
|pcmk__sched_stop_removed_resources
|pcmk__sched_cancel_removed_actions);
#else
pcmk__set_scheduler_flags(scheduler,
pcmk__sched_symmetric_cluster
|pcmk__sched_stop_removed_resources
|pcmk__sched_cancel_removed_actions);
#endif
}
pcmk_resource_t *
pe_find_resource(GList *rsc_list, const char *id)
{
return pe_find_resource_with_flags(rsc_list, id, pcmk_rsc_match_history);
}
pcmk_resource_t *
pe_find_resource_with_flags(GList *rsc_list, const char *id, enum pe_find flags)
{
GList *rIter = NULL;
for (rIter = rsc_list; id && rIter; rIter = rIter->next) {
pcmk_resource_t *parent = rIter->data;
pcmk_resource_t *match = parent->priv->fns->find_rsc(parent, id, NULL,
flags);
if (match != NULL) {
return match;
}
}
crm_trace("No match for %s", id);
return NULL;
}
/*!
* \brief Find a node by name or ID in a list of nodes
*
* \param[in] nodes List of nodes (as pcmk_node_t*)
* \param[in] id If not NULL, ID of node to find
* \param[in] node_name If not NULL, name of node to find
*
* \return Node from \p nodes that matches \p id if any,
* otherwise node from \p nodes that matches \p uname if any,
* otherwise NULL
*/
pcmk_node_t *
pe_find_node_any(const GList *nodes, const char *id, const char *uname)
{
pcmk_node_t *match = NULL;
if (id != NULL) {
match = pe_find_node_id(nodes, id);
}
if ((match == NULL) && (uname != NULL)) {
match = pcmk__find_node_in_list(nodes, uname);
}
return match;
}
/*!
* \brief Find a node by ID in a list of nodes
*
* \param[in] nodes List of nodes (as pcmk_node_t*)
* \param[in] id ID of node to find
*
* \return Node from \p nodes that matches \p id if any, otherwise NULL
*/
pcmk_node_t *
pe_find_node_id(const GList *nodes, const char *id)
{
for (const GList *iter = nodes; iter != NULL; iter = iter->next) {
pcmk_node_t *node = (pcmk_node_t *) iter->data;
/* @TODO Whether node IDs should be considered case-sensitive should
* probably depend on the node type, so functionizing the comparison
* would be worthwhile
*/
if (pcmk__str_eq(node->priv->id, id, pcmk__str_casei)) {
return node;
}
}
return NULL;
}
// Deprecated functions kept only for backward API compatibility
// LCOV_EXCL_START
#include <crm/pengine/status_compat.h>
/*!
* \brief Find a node by name in a list of nodes
*
* \param[in] nodes List of nodes (as pcmk_node_t*)
* \param[in] node_name Name of node to find
*
* \return Node from \p nodes that matches \p node_name if any, otherwise NULL
*/
pcmk_node_t *
pe_find_node(const GList *nodes, const char *node_name)
{
return pcmk__find_node_in_list(nodes, node_name);
}
// LCOV_EXCL_STOP
// End deprecated API
diff --git a/lib/pengine/utils.c b/lib/pengine/utils.c
index 4e7cf5e6ac..1197819c18 100644
--- a/lib/pengine/utils.c
+++ b/lib/pengine/utils.c
@@ -1,925 +1,927 @@
/*
* Copyright 2004-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <glib.h>
#include <stdbool.h>
#include <crm/crm.h>
#include <crm/common/xml.h>
#include <crm/pengine/rules.h>
#include <crm/pengine/internal.h>
#include "pe_status_private.h"
extern bool pcmk__is_daemon;
gboolean ghash_free_str_str(gpointer key, gpointer value, gpointer user_data);
/*!
* \internal
* \brief Check whether we can fence a particular node
*
* \param[in] scheduler Scheduler data
* \param[in] node Name of node to check
*
* \return true if node can be fenced, false otherwise
*/
bool
pe_can_fence(const pcmk_scheduler_t *scheduler, const pcmk_node_t *node)
{
if (pcmk__is_guest_or_bundle_node(node)) {
/* A guest or bundle node is fenced by stopping its launcher, which is
* possible if the launcher's host is either online or fenceable.
*/
pcmk_resource_t *rsc = node->priv->remote->priv->launcher;
for (GList *n = rsc->priv->active_nodes; n != NULL; n = n->next) {
pcmk_node_t *launcher_node = n->data;
if (!launcher_node->details->online
&& !pe_can_fence(scheduler, launcher_node)) {
return false;
}
}
return true;
} else if (!pcmk_is_set(scheduler->flags, pcmk__sched_fencing_enabled)) {
return false; /* Turned off */
} else if (!pcmk_is_set(scheduler->flags, pcmk__sched_have_fencing)) {
return false; /* No devices */
} else if (pcmk_is_set(scheduler->flags, pcmk__sched_quorate)) {
return true;
} else if (scheduler->no_quorum_policy == pcmk_no_quorum_ignore) {
return true;
} else if(node == NULL) {
return false;
} else if(node->details->online) {
crm_notice("We can fence %s without quorum because they're in our membership",
pcmk__node_name(node));
return true;
}
crm_trace("Cannot fence %s", pcmk__node_name(node));
return false;
}
/*!
* \internal
* \brief Copy a node object
*
* \param[in] this_node Node object to copy
*
* \return Newly allocated shallow copy of this_node
* \note This function asserts on errors and is guaranteed to return non-NULL.
+ * The caller is responsible for freeing the result using
+ * pcmk__free_node_copy().
*/
pcmk_node_t *
pe__copy_node(const pcmk_node_t *this_node)
{
pcmk_node_t *new_node = NULL;
pcmk__assert(this_node != NULL);
new_node = pcmk__assert_alloc(1, sizeof(pcmk_node_t));
new_node->assign = pcmk__assert_alloc(1,
sizeof(struct pcmk__node_assignment));
new_node->assign->probe_mode = this_node->assign->probe_mode;
new_node->assign->score = this_node->assign->score;
new_node->assign->count = this_node->assign->count;
new_node->details = this_node->details;
new_node->priv = this_node->priv;
return new_node;
}
/*!
* \internal
- * \brief Create a node hash table from a node list
+ * \brief Create a hash table of node copies from a list of nodes
*
* \param[in] list Node list
*
* \return Hash table equivalent of node list
*/
GHashTable *
pe__node_list2table(const GList *list)
{
GHashTable *result = NULL;
- result = pcmk__strkey_table(NULL, free);
+ result = pcmk__strkey_table(NULL, pcmk__free_node_copy);
for (const GList *gIter = list; gIter != NULL; gIter = gIter->next) {
pcmk_node_t *new_node = NULL;
new_node = pe__copy_node((const pcmk_node_t *) gIter->data);
g_hash_table_insert(result, (gpointer) new_node->priv->id, new_node);
}
return result;
}
/*!
* \internal
* \brief Compare two nodes by name, with numeric portions sorted numerically
*
* Sort two node names case-insensitively like strcasecmp(), but with any
* numeric portions of the name sorted numerically. For example, "node10" will
* sort higher than "node9" but lower than "remotenode9".
*
* \param[in] a First node to compare (can be \c NULL)
* \param[in] b Second node to compare (can be \c NULL)
*
* \retval -1 \c a comes before \c b (or \c a is \c NULL and \c b is not)
* \retval 0 \c a and \c b are equal (or both are \c NULL)
* \retval 1 \c a comes after \c b (or \c b is \c NULL and \c a is not)
*/
gint
pe__cmp_node_name(gconstpointer a, gconstpointer b)
{
const pcmk_node_t *node1 = (const pcmk_node_t *) a;
const pcmk_node_t *node2 = (const pcmk_node_t *) b;
if ((node1 == NULL) && (node2 == NULL)) {
return 0;
}
if (node1 == NULL) {
return -1;
}
if (node2 == NULL) {
return 1;
}
return pcmk__numeric_strcasecmp(node1->priv->name, node2->priv->name);
}
/*!
* \internal
* \brief Output node weights to stdout
*
* \param[in] rsc Use allowed nodes for this resource
* \param[in] comment Text description to prefix lines with
* \param[in] nodes If rsc is not specified, use these nodes
* \param[in,out] scheduler Scheduler data
*/
static void
pe__output_node_weights(const pcmk_resource_t *rsc, const char *comment,
GHashTable *nodes, pcmk_scheduler_t *scheduler)
{
pcmk__output_t *out = scheduler->priv->out;
// Sort the nodes so the output is consistent for regression tests
GList *list = g_list_sort(g_hash_table_get_values(nodes),
pe__cmp_node_name);
for (const GList *gIter = list; gIter != NULL; gIter = gIter->next) {
const pcmk_node_t *node = (const pcmk_node_t *) gIter->data;
out->message(out, "node-weight", rsc, comment, node->priv->name,
pcmk_readable_score(node->assign->score));
}
g_list_free(list);
}
/*!
* \internal
* \brief Log node weights at trace level
*
* \param[in] file Caller's filename
* \param[in] function Caller's function name
* \param[in] line Caller's line number
* \param[in] rsc If not NULL, include this resource's ID in logs
* \param[in] comment Text description to prefix lines with
* \param[in] nodes Nodes whose scores should be logged
*/
static void
pe__log_node_weights(const char *file, const char *function, int line,
const pcmk_resource_t *rsc, const char *comment,
GHashTable *nodes)
{
GHashTableIter iter;
pcmk_node_t *node = NULL;
// Don't waste time if we're not tracing at this point
pcmk__if_tracing({}, return);
g_hash_table_iter_init(&iter, nodes);
while (g_hash_table_iter_next(&iter, NULL, (void **) &node)) {
if (rsc) {
qb_log_from_external_source(function, file,
"%s: %s allocation score on %s: %s",
LOG_TRACE, line, 0,
comment, rsc->id,
pcmk__node_name(node),
pcmk_readable_score(node->assign->score));
} else {
qb_log_from_external_source(function, file, "%s: %s = %s",
LOG_TRACE, line, 0,
comment, pcmk__node_name(node),
pcmk_readable_score(node->assign->score));
}
}
}
/*!
* \internal
* \brief Log or output node weights
*
* \param[in] file Caller's filename
* \param[in] function Caller's function name
* \param[in] line Caller's line number
* \param[in] to_log Log if true, otherwise output
* \param[in] rsc If not NULL, use this resource's ID in logs,
* and show scores recursively for any children
* \param[in] comment Text description to prefix lines with
* \param[in] nodes Nodes whose scores should be shown
* \param[in,out] scheduler Scheduler data
*/
void
pe__show_node_scores_as(const char *file, const char *function, int line,
bool to_log, const pcmk_resource_t *rsc,
const char *comment, GHashTable *nodes,
pcmk_scheduler_t *scheduler)
{
if ((rsc != NULL) && pcmk_is_set(rsc->flags, pcmk__rsc_removed)) {
// Don't show allocation scores for orphans
return;
}
if (nodes == NULL) {
// Nothing to show
return;
}
if (to_log) {
pe__log_node_weights(file, function, line, rsc, comment, nodes);
} else {
pe__output_node_weights(rsc, comment, nodes, scheduler);
}
if (rsc == NULL) {
return;
}
// If this resource has children, repeat recursively for each
for (GList *gIter = rsc->priv->children;
gIter != NULL; gIter = gIter->next) {
pcmk_resource_t *child = (pcmk_resource_t *) gIter->data;
pe__show_node_scores_as(file, function, line, to_log, child, comment,
child->priv->allowed_nodes, scheduler);
}
}
/*!
* \internal
* \brief Compare two resources by priority
*
* \param[in] a First resource to compare (can be \c NULL)
* \param[in] b Second resource to compare (can be \c NULL)
*
* \retval -1 a's priority > b's priority (or \c b is \c NULL and \c a is not)
* \retval 0 a's priority == b's priority (or both \c a and \c b are \c NULL)
* \retval 1 a's priority < b's priority (or \c a is \c NULL and \c b is not)
*/
gint
pe__cmp_rsc_priority(gconstpointer a, gconstpointer b)
{
const pcmk_resource_t *resource1 = (const pcmk_resource_t *)a;
const pcmk_resource_t *resource2 = (const pcmk_resource_t *)b;
if (a == NULL && b == NULL) {
return 0;
}
if (a == NULL) {
return 1;
}
if (b == NULL) {
return -1;
}
if (resource1->priv->priority > resource2->priv->priority) {
return -1;
}
if (resource1->priv->priority < resource2->priv->priority) {
return 1;
}
return 0;
}
static void
resource_node_score(pcmk_resource_t *rsc, const pcmk_node_t *node, int score,
const char *tag)
{
pcmk_node_t *match = NULL;
if ((pcmk_is_set(rsc->flags, pcmk__rsc_exclusive_probes)
|| (node->assign->probe_mode == pcmk__probe_never))
&& pcmk__str_eq(tag, "symmetric_default", pcmk__str_casei)) {
/* This string comparision may be fragile, but exclusive resources and
* exclusive nodes should not have the symmetric_default constraint
* applied to them.
*/
return;
} else {
for (GList *gIter = rsc->priv->children;
gIter != NULL; gIter = gIter->next) {
pcmk_resource_t *child_rsc = (pcmk_resource_t *) gIter->data;
resource_node_score(child_rsc, node, score, tag);
}
}
match = g_hash_table_lookup(rsc->priv->allowed_nodes, node->priv->id);
if (match == NULL) {
match = pe__copy_node(node);
g_hash_table_insert(rsc->priv->allowed_nodes,
(gpointer) match->priv->id, match);
}
match->assign->score = pcmk__add_scores(match->assign->score, score);
pcmk__rsc_trace(rsc,
"Enabling %s preference (%s) for %s on %s (now %s)",
tag, pcmk_readable_score(score), rsc->id,
pcmk__node_name(node),
pcmk_readable_score(match->assign->score));
}
void
resource_location(pcmk_resource_t *rsc, const pcmk_node_t *node, int score,
const char *tag, pcmk_scheduler_t *scheduler)
{
if (node != NULL) {
resource_node_score(rsc, node, score, tag);
} else if (scheduler != NULL) {
GList *gIter = scheduler->nodes;
for (; gIter != NULL; gIter = gIter->next) {
pcmk_node_t *node_iter = (pcmk_node_t *) gIter->data;
resource_node_score(rsc, node_iter, score, tag);
}
} else {
GHashTableIter iter;
pcmk_node_t *node_iter = NULL;
g_hash_table_iter_init(&iter, rsc->priv->allowed_nodes);
while (g_hash_table_iter_next(&iter, NULL, (void **)&node_iter)) {
resource_node_score(rsc, node_iter, score, tag);
}
}
if ((node == NULL) && (score == -PCMK_SCORE_INFINITY)
&& (rsc->priv->assigned_node != NULL)) {
// @TODO Should this be more like pcmk__unassign_resource()?
crm_info("Unassigning %s from %s",
rsc->id, pcmk__node_name(rsc->priv->assigned_node));
- free(rsc->priv->assigned_node);
+ pcmk__free_node_copy(rsc->priv->assigned_node);
rsc->priv->assigned_node = NULL;
}
}
time_t
get_effective_time(pcmk_scheduler_t *scheduler)
{
if(scheduler) {
if (scheduler->priv->now == NULL) {
crm_trace("Recording a new 'now'");
scheduler->priv->now = crm_time_new(NULL);
}
return crm_time_get_seconds_since_epoch(scheduler->priv->now);
}
crm_trace("Defaulting to 'now'");
return time(NULL);
}
gboolean
get_target_role(const pcmk_resource_t *rsc, enum rsc_role_e *role)
{
enum rsc_role_e local_role = pcmk_role_unknown;
const char *value = g_hash_table_lookup(rsc->priv->meta,
PCMK_META_TARGET_ROLE);
CRM_CHECK(role != NULL, return FALSE);
if (pcmk__str_eq(value, PCMK_ROLE_STARTED,
pcmk__str_null_matches|pcmk__str_casei)) {
return FALSE;
}
if (pcmk__str_eq(PCMK_VALUE_DEFAULT, value, pcmk__str_casei)) {
// @COMPAT Deprecated since 2.1.8
pcmk__config_warn("Support for setting " PCMK_META_TARGET_ROLE
" to the explicit value '" PCMK_VALUE_DEFAULT
"' is deprecated and will be removed in a "
"future release (just leave it unset)");
return FALSE;
}
local_role = pcmk_parse_role(value);
if (local_role == pcmk_role_unknown) {
pcmk__config_err("Ignoring '" PCMK_META_TARGET_ROLE "' for %s "
"because '%s' is not valid", rsc->id, value);
return FALSE;
} else if (local_role > pcmk_role_started) {
if (pcmk_is_set(pe__const_top_resource(rsc, false)->flags,
pcmk__rsc_promotable)) {
if (local_role > pcmk_role_unpromoted) {
/* This is what we'd do anyway, just leave the default to avoid messing up the placement algorithm */
return FALSE;
}
} else {
pcmk__config_err("Ignoring '" PCMK_META_TARGET_ROLE "' for %s "
"because '%s' only makes sense for promotable "
"clones", rsc->id, value);
return FALSE;
}
}
*role = local_role;
return TRUE;
}
gboolean
order_actions(pcmk_action_t *first, pcmk_action_t *then, uint32_t flags)
{
GList *gIter = NULL;
pcmk__related_action_t *wrapper = NULL;
GList *list = NULL;
if (flags == pcmk__ar_none) {
return FALSE;
}
if ((first == NULL) || (then == NULL)) {
return FALSE;
}
crm_trace("Creating action wrappers for ordering: %s then %s",
first->uuid, then->uuid);
/* Ensure we never create a dependency on ourselves... it's happened */
pcmk__assert(first != then);
/* Filter dups, otherwise update_action_states() has too much work to do */
gIter = first->actions_after;
for (; gIter != NULL; gIter = gIter->next) {
pcmk__related_action_t *after = gIter->data;
if ((after->action == then)
&& pcmk_any_flags_set(after->flags, flags)) {
return FALSE;
}
}
wrapper = pcmk__assert_alloc(1, sizeof(pcmk__related_action_t));
wrapper->action = then;
wrapper->flags = flags;
list = first->actions_after;
list = g_list_prepend(list, wrapper);
first->actions_after = list;
wrapper = pcmk__assert_alloc(1, sizeof(pcmk__related_action_t));
wrapper->action = first;
wrapper->flags = flags;
list = then->actions_before;
list = g_list_prepend(list, wrapper);
then->actions_before = list;
return TRUE;
}
void
destroy_ticket(gpointer data)
{
pcmk__ticket_t *ticket = data;
if (ticket->state) {
g_hash_table_destroy(ticket->state);
}
free(ticket->id);
free(ticket);
}
pcmk__ticket_t *
ticket_new(const char *ticket_id, pcmk_scheduler_t *scheduler)
{
pcmk__ticket_t *ticket = NULL;
if (pcmk__str_empty(ticket_id)) {
return NULL;
}
if (scheduler->priv->ticket_constraints == NULL) {
scheduler->priv->ticket_constraints =
pcmk__strkey_table(free, destroy_ticket);
}
ticket = g_hash_table_lookup(scheduler->priv->ticket_constraints,
ticket_id);
if (ticket == NULL) {
ticket = calloc(1, sizeof(pcmk__ticket_t));
if (ticket == NULL) {
pcmk__sched_err(scheduler, "Cannot allocate ticket '%s'",
ticket_id);
return NULL;
}
crm_trace("Creating ticket entry for %s", ticket_id);
ticket->id = strdup(ticket_id);
ticket->last_granted = -1;
ticket->state = pcmk__strkey_table(free, free);
g_hash_table_insert(scheduler->priv->ticket_constraints,
pcmk__str_copy(ticket->id), ticket);
}
return ticket;
}
const char *
rsc_printable_id(const pcmk_resource_t *rsc)
{
if (pcmk_is_set(rsc->flags, pcmk__rsc_unique)) {
return rsc->id;
}
return pcmk__xe_id(rsc->priv->xml);
}
void
pe__clear_resource_flags_recursive(pcmk_resource_t *rsc, uint64_t flags)
{
pcmk__clear_rsc_flags(rsc, flags);
for (GList *gIter = rsc->priv->children;
gIter != NULL; gIter = gIter->next) {
pe__clear_resource_flags_recursive((pcmk_resource_t *) gIter->data,
flags);
}
}
void
pe__clear_resource_flags_on_all(pcmk_scheduler_t *scheduler, uint64_t flag)
{
for (GList *lpc = scheduler->priv->resources;
lpc != NULL; lpc = lpc->next) {
pcmk_resource_t *r = (pcmk_resource_t *) lpc->data;
pe__clear_resource_flags_recursive(r, flag);
}
}
void
pe__set_resource_flags_recursive(pcmk_resource_t *rsc, uint64_t flags)
{
pcmk__set_rsc_flags(rsc, flags);
for (GList *gIter = rsc->priv->children;
gIter != NULL; gIter = gIter->next) {
pe__set_resource_flags_recursive((pcmk_resource_t *) gIter->data,
flags);
}
}
void
trigger_unfencing(pcmk_resource_t *rsc, pcmk_node_t *node, const char *reason,
pcmk_action_t *dependency, pcmk_scheduler_t *scheduler)
{
if (!pcmk_is_set(scheduler->flags, pcmk__sched_enable_unfencing)) {
/* No resources require it */
return;
} else if ((rsc != NULL)
&& !pcmk_is_set(rsc->flags, pcmk__rsc_fence_device)) {
/* Wasn't a stonith device */
return;
} else if(node
&& node->details->online
&& node->details->unclean == FALSE
&& node->details->shutdown == FALSE) {
pcmk_action_t *unfence = pe_fence_op(node, PCMK_ACTION_ON, FALSE,
reason, FALSE, scheduler);
if(dependency) {
order_actions(unfence, dependency, pcmk__ar_ordered);
}
} else if(rsc) {
GHashTableIter iter;
g_hash_table_iter_init(&iter, rsc->priv->allowed_nodes);
while (g_hash_table_iter_next(&iter, NULL, (void **)&node)) {
if(node->details->online && node->details->unclean == FALSE && node->details->shutdown == FALSE) {
trigger_unfencing(rsc, node, reason, dependency, scheduler);
}
}
}
}
/*!
* \internal
* \brief Check whether shutdown has been requested for a node
*
* \param[in] node Node to check
*
* \return TRUE if node has shutdown attribute set and nonzero, FALSE otherwise
* \note This differs from simply using node->details->shutdown in that it can
* be used before that has been determined (and in fact to determine it),
* and it can also be used to distinguish requested shutdown from implicit
* shutdown of remote nodes by virtue of their connection stopping.
*/
bool
pe__shutdown_requested(const pcmk_node_t *node)
{
const char *shutdown = pcmk__node_attr(node, PCMK__NODE_ATTR_SHUTDOWN, NULL,
pcmk__rsc_node_current);
return !pcmk__str_eq(shutdown, "0", pcmk__str_null_matches);
}
/*!
* \internal
* \brief Update "recheck by" time in scheduler data
*
* \param[in] recheck Epoch time when recheck should happen
* \param[in,out] scheduler Scheduler data
* \param[in] reason What time is being updated for (for logs)
*/
void
pe__update_recheck_time(time_t recheck, pcmk_scheduler_t *scheduler,
const char *reason)
{
if ((recheck > get_effective_time(scheduler))
&& ((scheduler->priv->recheck_by == 0)
|| (scheduler->priv->recheck_by > recheck))) {
scheduler->priv->recheck_by = recheck;
crm_debug("Updated next scheduler recheck to %s for %s",
pcmk__trim(ctime(&recheck)), reason);
}
}
/*!
* \internal
* \brief Extract nvpair blocks contained by a CIB XML element into a hash table
*
* \param[in] xml_obj XML element containing blocks of nvpair elements
* \param[in] set_name If not NULL, only use blocks of this element
* \param[in] rule_data Matching parameters to use when unpacking
* (node_hash member must be NULL if \p set_name is
* PCMK_XE_META_ATTRIBUTES)
* \param[out] hash Where to store extracted name/value pairs
* \param[in] always_first If not NULL, process block with this ID first
* \param[in,out] scheduler Scheduler data containing \p xml_obj
*/
void
pe__unpack_dataset_nvpairs(const xmlNode *xml_obj, const char *set_name,
const pe_rule_eval_data_t *rule_data,
GHashTable *hash, const char *always_first,
pcmk_scheduler_t *scheduler)
{
crm_time_t *next_change = NULL;
CRM_CHECK((set_name != NULL) && (rule_data != NULL) && (hash != NULL)
&& (scheduler != NULL), return);
// Node attribute expressions are not allowed for meta-attributes
CRM_CHECK((rule_data->node_hash == NULL)
|| (strcmp(set_name, PCMK_XE_META_ATTRIBUTES) != 0), return);
if (xml_obj == NULL) {
return;
}
next_change = crm_time_new_undefined();
pe_eval_nvpairs(scheduler->input, xml_obj, set_name, rule_data, hash,
always_first, FALSE, next_change);
if (crm_time_is_defined(next_change)) {
time_t recheck = (time_t) crm_time_get_seconds_since_epoch(next_change);
pe__update_recheck_time(recheck, scheduler, "rule evaluation");
}
crm_time_free(next_change);
}
bool
pe__resource_is_disabled(const pcmk_resource_t *rsc)
{
const char *target_role = NULL;
CRM_CHECK(rsc != NULL, return false);
target_role = g_hash_table_lookup(rsc->priv->meta,
PCMK_META_TARGET_ROLE);
if (target_role) {
// If invalid, we've already logged an error when unpacking
enum rsc_role_e target_role_e = pcmk_parse_role(target_role);
if ((target_role_e == pcmk_role_stopped)
|| ((target_role_e == pcmk_role_unpromoted)
&& pcmk_is_set(pe__const_top_resource(rsc, false)->flags,
pcmk__rsc_promotable))) {
return true;
}
}
return false;
}
/*!
* \internal
* \brief Check whether a resource is running only on given node
*
* \param[in] rsc Resource to check
* \param[in] node Node to check
*
* \return true if \p rsc is running only on \p node, otherwise false
*/
bool
pe__rsc_running_on_only(const pcmk_resource_t *rsc, const pcmk_node_t *node)
{
return (rsc != NULL) && pcmk__list_of_1(rsc->priv->active_nodes)
&& pcmk__same_node((const pcmk_node_t *)
rsc->priv->active_nodes->data, node);
}
bool
pe__rsc_running_on_any(pcmk_resource_t *rsc, GList *node_list)
{
if (rsc != NULL) {
for (GList *ele = rsc->priv->active_nodes; ele; ele = ele->next) {
pcmk_node_t *node = (pcmk_node_t *) ele->data;
if (pcmk__str_in_list(node->priv->name, node_list,
pcmk__str_star_matches|pcmk__str_casei)) {
return true;
}
}
}
return false;
}
bool
pcmk__rsc_filtered_by_node(pcmk_resource_t *rsc, GList *only_node)
{
return rsc->priv->fns->active(rsc, FALSE)
&& !pe__rsc_running_on_any(rsc, only_node);
}
GList *
pe__filter_rsc_list(GList *rscs, GList *filter)
{
GList *retval = NULL;
for (GList *gIter = rscs; gIter; gIter = gIter->next) {
pcmk_resource_t *rsc = (pcmk_resource_t *) gIter->data;
/* I think the second condition is safe here for all callers of this
* function. If not, it needs to move into pe__node_text.
*/
if (pcmk__str_in_list(rsc_printable_id(rsc), filter, pcmk__str_star_matches) ||
((rsc->priv->parent != NULL)
&& pcmk__str_in_list(rsc_printable_id(rsc->priv->parent),
filter, pcmk__str_star_matches))) {
retval = g_list_prepend(retval, rsc);
}
}
return retval;
}
GList *
pe__build_node_name_list(pcmk_scheduler_t *scheduler, const char *s)
{
GList *nodes = NULL;
if (pcmk__str_eq(s, "*", pcmk__str_null_matches)) {
/* Nothing was given so return a list of all node names. Or, '*' was
* given. This would normally fall into the pe__unames_with_tag branch
* where it will return an empty list. Catch it here instead.
*/
nodes = g_list_prepend(nodes, strdup("*"));
} else {
pcmk_node_t *node = pcmk_find_node(scheduler, s);
if (node) {
/* The given string was a valid uname for a node. Return a
* singleton list containing just that uname.
*/
nodes = g_list_prepend(nodes, strdup(s));
} else {
/* The given string was not a valid uname. It's either a tag or
* it's a typo or something. In the first case, we'll return a
* list of all the unames of the nodes with the given tag. In the
* second case, we'll return a NULL pointer and nothing will
* get displayed.
*/
nodes = pe__unames_with_tag(scheduler, s);
}
}
return nodes;
}
GList *
pe__build_rsc_list(pcmk_scheduler_t *scheduler, const char *s)
{
GList *resources = NULL;
if (pcmk__str_eq(s, "*", pcmk__str_null_matches)) {
resources = g_list_prepend(resources, strdup("*"));
} else {
const uint32_t flags = pcmk_rsc_match_history|pcmk_rsc_match_basename;
pcmk_resource_t *rsc =
pe_find_resource_with_flags(scheduler->priv->resources, s, flags);
if (rsc) {
/* A colon in the name we were given means we're being asked to filter
* on a specific instance of a cloned resource. Put that exact string
* into the filter list. Otherwise, use the printable ID of whatever
* resource was found that matches what was asked for.
*/
if (strstr(s, ":") != NULL) {
resources = g_list_prepend(resources, strdup(rsc->id));
} else {
resources = g_list_prepend(resources, strdup(rsc_printable_id(rsc)));
}
} else {
/* The given string was not a valid resource name. It's a tag or a
* typo or something. See pe__build_node_name_list() for more
* detail.
*/
resources = pe__rscs_with_tag(scheduler, s);
}
}
return resources;
}
xmlNode *
pe__failed_probe_for_rsc(const pcmk_resource_t *rsc, const char *name)
{
const pcmk_resource_t *parent = pe__const_top_resource(rsc, false);
const char *rsc_id = rsc->id;
const pcmk_scheduler_t *scheduler = rsc->priv->scheduler;
if (pcmk__is_clone(parent)) {
rsc_id = pe__clone_child_id(parent);
}
for (xmlNode *xml_op = pcmk__xe_first_child(scheduler->priv->failed,
NULL, NULL, NULL);
xml_op != NULL; xml_op = pcmk__xe_next(xml_op, NULL)) {
const char *value = NULL;
char *op_id = NULL;
/* This resource operation is not a failed probe. */
if (!pcmk_xe_mask_probe_failure(xml_op)) {
continue;
}
/* This resource operation was not run on the given node. Note that if name is
* NULL, this will always succeed.
*/
value = crm_element_value(xml_op, PCMK__META_ON_NODE);
if (value == NULL || !pcmk__str_eq(value, name, pcmk__str_casei|pcmk__str_null_matches)) {
continue;
}
if (!parse_op_key(pcmk__xe_history_key(xml_op), &op_id, NULL, NULL)) {
continue; // This history entry is missing an operation key
}
/* This resource operation's ID does not match the rsc_id we are looking for. */
if (!pcmk__str_eq(op_id, rsc_id, pcmk__str_none)) {
free(op_id);
continue;
}
free(op_id);
return xml_op;
}
return NULL;
}
diff --git a/lib/services/services_linux.c b/lib/services/services_linux.c
index 0e991e4533..ff4763a20c 100644
--- a/lib/services/services_linux.c
+++ b/lib/services/services_linux.c
@@ -1,1477 +1,1478 @@
/*
* Copyright 2010-2024 the Pacemaker project contributors
*
* The version control history for this file may have further details.
*
* This source code is licensed under the GNU Lesser General Public License
* version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY.
*/
#include <crm_internal.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <errno.h>
#include <unistd.h>
#include <dirent.h>
#include <grp.h>
#include <string.h>
#include <sys/time.h>
#include <sys/resource.h>
#include "crm/crm.h"
#include "crm/common/mainloop.h"
#include "crm/services.h"
#include "crm/services_internal.h"
#include "services_private.h"
static void close_pipe(int fildes[]);
/* We have two alternative ways of handling SIGCHLD when synchronously waiting
* for spawned processes to complete. Both rely on polling a file descriptor to
* discover SIGCHLD events.
*
* If sys/signalfd.h is available (e.g. on Linux), we call signalfd() to
* generate the file descriptor. Otherwise, we use the "self-pipe trick"
* (opening a pipe and writing a byte to it when SIGCHLD is received).
*/
#ifdef HAVE_SYS_SIGNALFD_H
// signalfd() implementation
#include <sys/signalfd.h>
// Everything needed to manage SIGCHLD handling
struct sigchld_data_s {
sigset_t mask; // Signals to block now (including SIGCHLD)
sigset_t old_mask; // Previous set of blocked signals
bool ignored; // If SIGCHLD for another child has been ignored
};
// Initialize SIGCHLD data and prepare for use
static bool
sigchld_setup(struct sigchld_data_s *data)
{
sigemptyset(&(data->mask));
sigaddset(&(data->mask), SIGCHLD);
sigemptyset(&(data->old_mask));
// Block SIGCHLD (saving previous set of blocked signals to restore later)
if (sigprocmask(SIG_BLOCK, &(data->mask), &(data->old_mask)) < 0) {
crm_info("Wait for child process completion failed: %s "
QB_XS " source=sigprocmask", pcmk_rc_str(errno));
return false;
}
data->ignored = false;
return true;
}
// Get a file descriptor suitable for polling for SIGCHLD events
static int
sigchld_open(struct sigchld_data_s *data)
{
int fd;
CRM_CHECK(data != NULL, return -1);
fd = signalfd(-1, &(data->mask), SFD_NONBLOCK);
if (fd < 0) {
crm_info("Wait for child process completion failed: %s "
QB_XS " source=signalfd", pcmk_rc_str(errno));
}
return fd;
}
// Close a file descriptor returned by sigchld_open()
static void
sigchld_close(int fd)
{
if (fd > 0) {
close(fd);
}
}
// Return true if SIGCHLD was received from polled fd
static bool
sigchld_received(int fd, int pid, struct sigchld_data_s *data)
{
struct signalfd_siginfo fdsi;
ssize_t s;
if (fd < 0) {
return false;
}
s = read(fd, &fdsi, sizeof(struct signalfd_siginfo));
if (s != sizeof(struct signalfd_siginfo)) {
crm_info("Wait for child process completion failed: %s "
QB_XS " source=read", pcmk_rc_str(errno));
} else if (fdsi.ssi_signo == SIGCHLD) {
if (fdsi.ssi_pid == pid) {
return true;
} else {
/* This SIGCHLD is for another child. We have to ignore it here but
* will still need to resend it after this synchronous action has
* completed and SIGCHLD has been restored to be handled by the
* previous SIGCHLD handler, so that it will be handled.
*/
data->ignored = true;
return false;
}
}
return false;
}
// Do anything needed after done waiting for SIGCHLD
static void
sigchld_cleanup(struct sigchld_data_s *data)
{
// Restore the original set of blocked signals
if ((sigismember(&(data->old_mask), SIGCHLD) == 0)
&& (sigprocmask(SIG_UNBLOCK, &(data->mask), NULL) < 0)) {
crm_warn("Could not clean up after child process completion: %s",
pcmk_rc_str(errno));
}
// Resend any ignored SIGCHLD for other children so that they'll be handled.
if (data->ignored && kill(getpid(), SIGCHLD) != 0) {
crm_warn("Could not resend ignored SIGCHLD to ourselves: %s",
pcmk_rc_str(errno));
}
}
#else // HAVE_SYS_SIGNALFD_H not defined
// Self-pipe implementation (see above for function descriptions)
struct sigchld_data_s {
int pipe_fd[2]; // Pipe file descriptors
struct sigaction sa; // Signal handling info (with SIGCHLD)
struct sigaction old_sa; // Previous signal handling info
bool ignored; // If SIGCHLD for another child has been ignored
};
// We need a global to use in the signal handler
volatile struct sigchld_data_s *last_sigchld_data = NULL;
static void
sigchld_handler(void)
{
// We received a SIGCHLD, so trigger pipe polling
if ((last_sigchld_data != NULL)
&& (last_sigchld_data->pipe_fd[1] >= 0)
&& (write(last_sigchld_data->pipe_fd[1], "", 1) == -1)) {
crm_info("Wait for child process completion failed: %s "
QB_XS " source=write", pcmk_rc_str(errno));
}
}
static bool
sigchld_setup(struct sigchld_data_s *data)
{
int rc;
data->pipe_fd[0] = data->pipe_fd[1] = -1;
if (pipe(data->pipe_fd) == -1) {
crm_info("Wait for child process completion failed: %s "
QB_XS " source=pipe", pcmk_rc_str(errno));
return false;
}
rc = pcmk__set_nonblocking(data->pipe_fd[0]);
if (rc != pcmk_rc_ok) {
crm_info("Could not set pipe input non-blocking: %s " QB_XS " rc=%d",
pcmk_rc_str(rc), rc);
}
rc = pcmk__set_nonblocking(data->pipe_fd[1]);
if (rc != pcmk_rc_ok) {
crm_info("Could not set pipe output non-blocking: %s " QB_XS " rc=%d",
pcmk_rc_str(rc), rc);
}
// Set SIGCHLD handler
data->sa.sa_handler = (sighandler_t) sigchld_handler;
data->sa.sa_flags = 0;
sigemptyset(&(data->sa.sa_mask));
if (sigaction(SIGCHLD, &(data->sa), &(data->old_sa)) < 0) {
crm_info("Wait for child process completion failed: %s "
QB_XS " source=sigaction", pcmk_rc_str(errno));
}
data->ignored = false;
// Remember data for use in signal handler
last_sigchld_data = data;
return true;
}
static int
sigchld_open(struct sigchld_data_s *data)
{
CRM_CHECK(data != NULL, return -1);
return data->pipe_fd[0];
}
static void
sigchld_close(int fd)
{
// Pipe will be closed in sigchld_cleanup()
return;
}
static bool
sigchld_received(int fd, int pid, struct sigchld_data_s *data)
{
char ch;
if (fd < 0) {
return false;
}
// Clear out the self-pipe
while (read(fd, &ch, 1) == 1) /*omit*/;
return true;
}
static void
sigchld_cleanup(struct sigchld_data_s *data)
{
// Restore the previous SIGCHLD handler
if (sigaction(SIGCHLD, &(data->old_sa), NULL) < 0) {
crm_warn("Could not clean up after child process completion: %s",
pcmk_rc_str(errno));
}
close_pipe(data->pipe_fd);
// Resend any ignored SIGCHLD for other children so that they'll be handled.
if (data->ignored && kill(getpid(), SIGCHLD) != 0) {
crm_warn("Could not resend ignored SIGCHLD to ourselves: %s",
pcmk_rc_str(errno));
}
}
#endif
/*!
* \internal
* \brief Close the two file descriptors of a pipe
*
* \param[in,out] fildes Array of file descriptors opened by pipe()
*/
static void
close_pipe(int fildes[])
{
if (fildes[0] >= 0) {
close(fildes[0]);
fildes[0] = -1;
}
if (fildes[1] >= 0) {
close(fildes[1]);
fildes[1] = -1;
}
}
#define out_type(is_stderr) ((is_stderr)? "stderr" : "stdout")
// Maximum number of bytes of stdout or stderr we'll accept
#define MAX_OUTPUT (10 * 1024 * 1024)
static gboolean
svc_read_output(int fd, svc_action_t * op, bool is_stderr)
{
char *data = NULL;
ssize_t rc = 0;
size_t len = 0;
size_t discarded = 0;
char buf[500];
static const size_t buf_read_len = sizeof(buf) - 1;
if (fd < 0) {
crm_trace("No fd for %s", op->id);
return FALSE;
}
if (is_stderr && op->stderr_data) {
len = strlen(op->stderr_data);
data = op->stderr_data;
crm_trace("Reading %s stderr into offset %lld",
op->id, (long long) len);
} else if (is_stderr == FALSE && op->stdout_data) {
len = strlen(op->stdout_data);
data = op->stdout_data;
crm_trace("Reading %s stdout into offset %lld",
op->id, (long long) len);
} else {
crm_trace("Reading %s %s", op->id, out_type(is_stderr));
}
do {
errno = 0;
rc = read(fd, buf, buf_read_len);
if (rc > 0) {
if (len < MAX_OUTPUT) {
buf[rc] = 0;
crm_trace("Received %lld bytes of %s %s: %.80s",
(long long) rc, op->id, out_type(is_stderr), buf);
data = pcmk__realloc(data, len + rc + 1);
strcpy(data + len, buf);
len += rc;
} else {
discarded += rc;
}
} else if (errno != EINTR) { // Fatal error or EOF
rc = 0;
break;
}
} while ((rc == buf_read_len) || (rc < 0));
if (discarded > 0) {
crm_warn("Truncated %s %s to %lld bytes (discarded %lld)",
op->id, out_type(is_stderr), (long long) len,
(long long) discarded);
}
if (is_stderr) {
op->stderr_data = data;
} else {
op->stdout_data = data;
}
return rc != 0;
}
static int
dispatch_stdout(gpointer userdata)
{
svc_action_t *op = (svc_action_t *) userdata;
return svc_read_output(op->opaque->stdout_fd, op, FALSE);
}
static int
dispatch_stderr(gpointer userdata)
{
svc_action_t *op = (svc_action_t *) userdata;
return svc_read_output(op->opaque->stderr_fd, op, TRUE);
}
static void
pipe_out_done(gpointer user_data)
{
svc_action_t *op = (svc_action_t *) user_data;
crm_trace("%p", op);
op->opaque->stdout_gsource = NULL;
if (op->opaque->stdout_fd > STDOUT_FILENO) {
close(op->opaque->stdout_fd);
}
op->opaque->stdout_fd = -1;
}
static void
pipe_err_done(gpointer user_data)
{
svc_action_t *op = (svc_action_t *) user_data;
op->opaque->stderr_gsource = NULL;
if (op->opaque->stderr_fd > STDERR_FILENO) {
close(op->opaque->stderr_fd);
}
op->opaque->stderr_fd = -1;
}
static struct mainloop_fd_callbacks stdout_callbacks = {
.dispatch = dispatch_stdout,
.destroy = pipe_out_done,
};
static struct mainloop_fd_callbacks stderr_callbacks = {
.dispatch = dispatch_stderr,
.destroy = pipe_err_done,
};
static void
set_ocf_env(const char *key, const char *value, gpointer user_data)
{
if (setenv(key, value, 1) != 0) {
crm_perror(LOG_ERR, "setenv failed for key:%s and value:%s", key, value);
}
}
static void
set_ocf_env_with_prefix(gpointer key, gpointer value, gpointer user_data)
{
char buffer[500];
snprintf(buffer, sizeof(buffer), strcmp(key, "OCF_CHECK_LEVEL") != 0 ? "OCF_RESKEY_%s" : "%s", (char *)key);
set_ocf_env(buffer, value, user_data);
}
static void
set_alert_env(gpointer key, gpointer value, gpointer user_data)
{
int rc;
if (value != NULL) {
rc = setenv(key, value, 1);
} else {
rc = unsetenv(key);
}
if (rc < 0) {
crm_perror(LOG_ERR, "setenv %s=%s",
(char*)key, (value? (char*)value : ""));
} else {
crm_trace("setenv %s=%s", (char*)key, (value? (char*)value : ""));
}
}
/*!
* \internal
* \brief Add environment variables suitable for an action
*
* \param[in] op Action to use
*/
static void
add_action_env_vars(const svc_action_t *op)
{
void (*env_setter)(gpointer, gpointer, gpointer) = NULL;
if (op->agent == NULL) {
env_setter = set_alert_env; /* we deal with alert handler */
} else if (pcmk__str_eq(op->standard, PCMK_RESOURCE_CLASS_OCF, pcmk__str_casei)) {
env_setter = set_ocf_env_with_prefix;
}
if (env_setter != NULL && op->params != NULL) {
g_hash_table_foreach(op->params, env_setter, NULL);
}
if (env_setter == NULL || env_setter == set_alert_env) {
return;
}
set_ocf_env("OCF_RA_VERSION_MAJOR", PCMK_OCF_MAJOR_VERSION, NULL);
set_ocf_env("OCF_RA_VERSION_MINOR", PCMK_OCF_MINOR_VERSION, NULL);
set_ocf_env("OCF_ROOT", PCMK_OCF_ROOT, NULL);
set_ocf_env("OCF_EXIT_REASON_PREFIX", PCMK_OCF_REASON_PREFIX, NULL);
if (op->rsc) {
set_ocf_env("OCF_RESOURCE_INSTANCE", op->rsc, NULL);
}
if (op->agent != NULL) {
set_ocf_env("OCF_RESOURCE_TYPE", op->agent, NULL);
}
/* Notes: this is not added to specification yet. Sept 10,2004 */
if (op->provider != NULL) {
set_ocf_env("OCF_RESOURCE_PROVIDER", op->provider, NULL);
}
}
static void
pipe_in_single_parameter(gpointer key, gpointer value, gpointer user_data)
{
svc_action_t *op = user_data;
char *buffer = crm_strdup_printf("%s=%s\n", (char *)key, (char *) value);
size_t len = strlen(buffer);
size_t total = 0;
ssize_t ret = 0;
do {
errno = 0;
ret = write(op->opaque->stdin_fd, buffer + total, len - total);
if (ret > 0) {
total += ret;
}
} while ((errno == EINTR) && (total < len));
free(buffer);
}
/*!
* \internal
* \brief Pipe parameters in via stdin for action
*
* \param[in] op Action to use
*/
static void
pipe_in_action_stdin_parameters(const svc_action_t *op)
{
if (op->params) {
g_hash_table_foreach(op->params, pipe_in_single_parameter, (gpointer) op);
}
}
gboolean
recurring_action_timer(gpointer data)
{
svc_action_t *op = data;
crm_debug("Scheduling another invocation of %s", op->id);
/* Clean out the old result */
free(op->stdout_data);
op->stdout_data = NULL;
free(op->stderr_data);
op->stderr_data = NULL;
op->opaque->repeat_timer = 0;
services_action_async(op, NULL);
return FALSE;
}
/*!
* \internal
* \brief Finalize handling of an asynchronous operation
*
* Given a completed asynchronous operation, cancel or reschedule it as
* appropriate if recurring, call its callback if registered, stop tracking it,
* and clean it up.
*
* \param[in,out] op Operation to finalize
*
* \return Standard Pacemaker return code
* \retval EINVAL Caller supplied NULL or invalid \p op
* \retval EBUSY Uncanceled recurring action has only been cleaned up
* \retval pcmk_rc_ok Action has been freed
*
* \note If the return value is not pcmk_rc_ok, the caller is responsible for
* freeing the action.
*/
int
services__finalize_async_op(svc_action_t *op)
{
CRM_CHECK((op != NULL) && !(op->synchronous), return EINVAL);
if (op->interval_ms != 0) {
// Recurring operations must be either cancelled or rescheduled
if (op->cancel) {
services__set_cancelled(op);
cancel_recurring_action(op);
} else {
op->opaque->repeat_timer = pcmk__create_timer(op->interval_ms,
recurring_action_timer,
op);
}
}
if (op->opaque->callback != NULL) {
op->opaque->callback(op);
}
// Stop tracking the operation (as in-flight or blocked)
op->pid = 0;
services_untrack_op(op);
if ((op->interval_ms != 0) && !(op->cancel)) {
// Do not free recurring actions (they will get freed when cancelled)
services_action_cleanup(op);
return EBUSY;
}
services_action_free(op);
return pcmk_rc_ok;
}
static void
close_op_input(svc_action_t *op)
{
if (op->opaque->stdin_fd >= 0) {
close(op->opaque->stdin_fd);
}
}
static void
finish_op_output(svc_action_t *op, bool is_stderr)
{
mainloop_io_t **source;
int fd;
if (is_stderr) {
source = &(op->opaque->stderr_gsource);
fd = op->opaque->stderr_fd;
} else {
source = &(op->opaque->stdout_gsource);
fd = op->opaque->stdout_fd;
}
if (op->synchronous || *source) {
crm_trace("Finish reading %s[%d] %s",
op->id, op->pid, (is_stderr? "stderr" : "stdout"));
svc_read_output(fd, op, is_stderr);
if (op->synchronous) {
close(fd);
} else {
mainloop_del_fd(*source);
*source = NULL;
}
}
}
// Log an operation's stdout and stderr
static void
log_op_output(svc_action_t *op)
{
char *prefix = crm_strdup_printf("%s[%d] error output", op->id, op->pid);
/* The library caller has better context to know how important the output
* is, so log it at info and debug severity here. They can log it again at
* higher severity if appropriate.
*/
crm_log_output(LOG_INFO, prefix, op->stderr_data);
strcpy(prefix + strlen(prefix) - strlen("error output"), "output");
crm_log_output(LOG_DEBUG, prefix, op->stdout_data);
free(prefix);
}
// Truncate exit reasons at this many characters
#define EXIT_REASON_MAX_LEN 128
static void
parse_exit_reason_from_stderr(svc_action_t *op)
{
const char *reason_start = NULL;
const char *reason_end = NULL;
const int prefix_len = strlen(PCMK_OCF_REASON_PREFIX);
if ((op->stderr_data == NULL) ||
// Only OCF agents have exit reasons in stderr
!pcmk__str_eq(op->standard, PCMK_RESOURCE_CLASS_OCF, pcmk__str_none)) {
return;
}
// Find the last occurrence of the magic string indicating an exit reason
for (const char *cur = strstr(op->stderr_data, PCMK_OCF_REASON_PREFIX);
cur != NULL; cur = strstr(cur, PCMK_OCF_REASON_PREFIX)) {
cur += prefix_len; // Skip over magic string
reason_start = cur;
}
if ((reason_start == NULL) || (reason_start[0] == '\n')
|| (reason_start[0] == '\0')) {
return; // No or empty exit reason
}
// Exit reason goes to end of line (or end of output)
reason_end = strchr(reason_start, '\n');
if (reason_end == NULL) {
reason_end = reason_start + strlen(reason_start);
}
// Limit size of exit reason to something reasonable
if (reason_end > (reason_start + EXIT_REASON_MAX_LEN)) {
reason_end = reason_start + EXIT_REASON_MAX_LEN;
}
free(op->opaque->exit_reason);
op->opaque->exit_reason = strndup(reason_start, reason_end - reason_start);
}
/*!
* \internal
* \brief Process the completion of an asynchronous child process
*
* \param[in,out] p Child process that completed
* \param[in] pid Process ID of child
* \param[in] core (Unused)
* \param[in] signo Signal that interrupted child, if any
* \param[in] exitcode Exit status of child process
*/
static void
async_action_complete(mainloop_child_t *p, pid_t pid, int core, int signo,
int exitcode)
{
svc_action_t *op = mainloop_child_userdata(p);
mainloop_clear_child_userdata(p);
CRM_CHECK(op->pid == pid,
services__set_result(op, services__generic_error(op),
PCMK_EXEC_ERROR, "Bug in mainloop handling");
return);
/* Depending on the priority the mainloop gives the stdout and stderr
* file descriptors, this function could be called before everything has
* been read from them, so force a final read now.
*/
finish_op_output(op, true);
finish_op_output(op, false);
close_op_input(op);
if (signo == 0) {
crm_debug("%s[%d] exited with status %d", op->id, op->pid, exitcode);
services__set_result(op, exitcode, PCMK_EXEC_DONE, NULL);
log_op_output(op);
parse_exit_reason_from_stderr(op);
} else if (mainloop_child_timeout(p)) {
const char *kind = services__action_kind(op);
crm_info("%s %s[%d] timed out after %s",
kind, op->id, op->pid, pcmk__readable_interval(op->timeout));
services__format_result(op, services__generic_error(op),
PCMK_EXEC_TIMEOUT,
"%s did not complete within %s",
kind, pcmk__readable_interval(op->timeout));
} else if (op->cancel) {
/* If an in-flight recurring operation was killed because it was
* cancelled, don't treat that as a failure.
*/
crm_info("%s[%d] terminated with signal %d (%s)",
op->id, op->pid, signo, strsignal(signo));
services__set_result(op, PCMK_OCF_OK, PCMK_EXEC_CANCELLED, NULL);
} else {
crm_info("%s[%d] terminated with signal %d (%s)",
op->id, op->pid, signo, strsignal(signo));
services__format_result(op, PCMK_OCF_UNKNOWN_ERROR, PCMK_EXEC_ERROR,
"%s interrupted by %s signal",
services__action_kind(op), strsignal(signo));
}
services__finalize_async_op(op);
}
/*!
* \internal
* \brief Return agent standard's exit status for "generic error"
*
* When returning an internal error for an action, a value that is appropriate
* to the action's agent standard must be used. This function returns a value
* appropriate for errors in general.
*
* \param[in] op Action that error is for
*
* \return Exit status appropriate to agent standard
* \note Actions without a standard will get PCMK_OCF_UNKNOWN_ERROR.
*/
int
services__generic_error(const svc_action_t *op)
{
if ((op == NULL) || (op->standard == NULL)) {
return PCMK_OCF_UNKNOWN_ERROR;
}
#if PCMK__ENABLE_LSB
if (pcmk__str_eq(op->standard, PCMK_RESOURCE_CLASS_LSB, pcmk__str_casei)
&& pcmk__str_eq(op->action, PCMK_ACTION_STATUS, pcmk__str_casei)) {
return PCMK_LSB_STATUS_UNKNOWN;
}
#endif
return PCMK_OCF_UNKNOWN_ERROR;
}
/*!
* \internal
* \brief Return agent standard's exit status for "not installed"
*
* When returning an internal error for an action, a value that is appropriate
* to the action's agent standard must be used. This function returns a value
* appropriate for "not installed" errors.
*
* \param[in] op Action that error is for
*
* \return Exit status appropriate to agent standard
* \note Actions without a standard will get PCMK_OCF_UNKNOWN_ERROR.
*/
int
services__not_installed_error(const svc_action_t *op)
{
if ((op == NULL) || (op->standard == NULL)) {
return PCMK_OCF_UNKNOWN_ERROR;
}
#if PCMK__ENABLE_LSB
if (pcmk__str_eq(op->standard, PCMK_RESOURCE_CLASS_LSB, pcmk__str_casei)
&& pcmk__str_eq(op->action, PCMK_ACTION_STATUS, pcmk__str_casei)) {
return PCMK_LSB_STATUS_NOT_INSTALLED;
}
#endif
return PCMK_OCF_NOT_INSTALLED;
}
/*!
* \internal
* \brief Return agent standard's exit status for "insufficient privileges"
*
* When returning an internal error for an action, a value that is appropriate
* to the action's agent standard must be used. This function returns a value
* appropriate for "insufficient privileges" errors.
*
* \param[in] op Action that error is for
*
* \return Exit status appropriate to agent standard
* \note Actions without a standard will get PCMK_OCF_UNKNOWN_ERROR.
*/
int
services__authorization_error(const svc_action_t *op)
{
if ((op == NULL) || (op->standard == NULL)) {
return PCMK_OCF_UNKNOWN_ERROR;
}
#if PCMK__ENABLE_LSB
if (pcmk__str_eq(op->standard, PCMK_RESOURCE_CLASS_LSB, pcmk__str_casei)
&& pcmk__str_eq(op->action, PCMK_ACTION_STATUS, pcmk__str_casei)) {
return PCMK_LSB_STATUS_INSUFFICIENT_PRIV;
}
#endif
return PCMK_OCF_INSUFFICIENT_PRIV;
}
/*!
* \internal
* \brief Return agent standard's exit status for "not configured"
*
* When returning an internal error for an action, a value that is appropriate
* to the action's agent standard must be used. This function returns a value
* appropriate for "not configured" errors.
*
* \param[in] op Action that error is for
* \param[in] is_fatal Whether problem is cluster-wide instead of only local
*
* \return Exit status appropriate to agent standard
* \note Actions without a standard will get PCMK_OCF_UNKNOWN_ERROR.
*/
int
services__configuration_error(const svc_action_t *op, bool is_fatal)
{
if ((op == NULL) || (op->standard == NULL)) {
return PCMK_OCF_UNKNOWN_ERROR;
}
#if PCMK__ENABLE_LSB
if (pcmk__str_eq(op->standard, PCMK_RESOURCE_CLASS_LSB, pcmk__str_casei)
&& pcmk__str_eq(op->action, PCMK_ACTION_STATUS, pcmk__str_casei)) {
return PCMK_LSB_NOT_CONFIGURED;
}
#endif
return is_fatal? PCMK_OCF_NOT_CONFIGURED : PCMK_OCF_INVALID_PARAM;
}
/*!
* \internal
* \brief Set operation rc and status per errno from stat(), fork() or execvp()
*
* \param[in,out] op Operation to set rc and status for
* \param[in] error Value of errno after system call
*
* \return void
*/
void
services__handle_exec_error(svc_action_t * op, int error)
{
const char *name = op->opaque->exec;
if (name == NULL) {
name = op->agent;
if (name == NULL) {
name = op->id;
}
}
switch (error) { /* see execve(2), stat(2) and fork(2) */
case ENOENT: /* No such file or directory */
case EISDIR: /* Is a directory */
case ENOTDIR: /* Path component is not a directory */
case EINVAL: /* Invalid executable format */
case ENOEXEC: /* Invalid executable format */
services__format_result(op, services__not_installed_error(op),
PCMK_EXEC_NOT_INSTALLED, "%s: %s",
name, pcmk_rc_str(error));
break;
case EACCES: /* permission denied (various errors) */
case EPERM: /* permission denied (various errors) */
services__format_result(op, services__authorization_error(op),
PCMK_EXEC_ERROR, "%s: %s",
name, pcmk_rc_str(error));
break;
default:
services__set_result(op, services__generic_error(op),
PCMK_EXEC_ERROR, pcmk_rc_str(error));
}
}
/*!
* \internal
* \brief Exit a child process that failed before executing agent
*
* \param[in] op Action that failed
* \param[in] exit_status Exit status code to use
* \param[in] exit_reason Exit reason to output if for OCF agent
*/
static void
exit_child(const svc_action_t *op, int exit_status, const char *exit_reason)
{
if ((op != NULL) && (exit_reason != NULL)
&& pcmk__str_eq(op->standard, PCMK_RESOURCE_CLASS_OCF,
pcmk__str_none)) {
fprintf(stderr, PCMK_OCF_REASON_PREFIX "%s\n", exit_reason);
}
+ pcmk_common_cleanup();
_exit(exit_status);
}
static void
action_launch_child(svc_action_t *op)
{
int rc;
/* SIGPIPE is ignored (which is different from signal blocking) by the gnutls library.
* Depending on the libqb version in use, libqb may set SIGPIPE to be ignored as well.
* We do not want this to be inherited by the child process. By resetting this the signal
* to the default behavior, we avoid some potential odd problems that occur during OCF
* scripts when SIGPIPE is ignored by the environment. */
signal(SIGPIPE, SIG_DFL);
if (sched_getscheduler(0) != SCHED_OTHER) {
struct sched_param sp;
memset(&sp, 0, sizeof(sp));
sp.sched_priority = 0;
if (sched_setscheduler(0, SCHED_OTHER, &sp) == -1) {
crm_info("Could not reset scheduling policy for %s", op->id);
}
}
if (setpriority(PRIO_PROCESS, 0, 0) == -1) {
crm_info("Could not reset process priority for %s", op->id);
}
/* Man: The call setpgrp() is equivalent to setpgid(0,0)
* _and_ compiles on BSD variants too
* need to investigate if it works the same too.
*/
setpgid(0, 0);
pcmk__close_fds_in_child(false);
/* It would be nice if errors in this function could be reported as
* execution status (for example, PCMK_EXEC_NO_SECRETS for the secrets error
* below) instead of exit status. However, we've already forked, so
* exit status is all we have. At least for OCF actions, we can output an
* exit reason for the parent to parse.
*/
#if PCMK__ENABLE_CIBSECRETS
rc = pcmk__substitute_secrets(op->rsc, op->params);
if (rc != pcmk_rc_ok) {
if (pcmk__str_eq(op->action, PCMK_ACTION_STOP, pcmk__str_casei)) {
crm_info("Proceeding with stop operation for %s "
"despite being unable to load CIB secrets (%s)",
op->rsc, pcmk_rc_str(rc));
} else {
crm_err("Considering %s unconfigured "
"because unable to load CIB secrets: %s",
op->rsc, pcmk_rc_str(rc));
exit_child(op, services__configuration_error(op, false),
"Unable to load CIB secrets");
}
}
#endif
add_action_env_vars(op);
/* Become the desired user */
if (op->opaque->uid && (geteuid() == 0)) {
// If requested, set effective group
if (op->opaque->gid && (setgid(op->opaque->gid) < 0)) {
crm_err("Considering %s unauthorized because could not set "
"child group to %d: %s",
op->id, op->opaque->gid, strerror(errno));
exit_child(op, services__authorization_error(op),
"Could not set group for child process");
}
// Erase supplementary group list
// (We could do initgroups() if we kept a copy of the username)
if (setgroups(0, NULL) < 0) {
crm_err("Considering %s unauthorized because could not "
"clear supplementary groups: %s", op->id, strerror(errno));
exit_child(op, services__authorization_error(op),
"Could not clear supplementary groups for child process");
}
// Set effective user
if (setuid(op->opaque->uid) < 0) {
crm_err("Considering %s unauthorized because could not set user "
"to %d: %s", op->id, op->opaque->uid, strerror(errno));
exit_child(op, services__authorization_error(op),
"Could not set user for child process");
}
}
// Execute the agent (doesn't return if successful)
execvp(op->opaque->exec, op->opaque->args);
// An earlier stat() should have avoided most possible errors
rc = errno;
services__handle_exec_error(op, rc);
crm_err("Unable to execute %s: %s", op->id, strerror(rc));
exit_child(op, op->rc, "Child process was unable to execute file");
}
/*!
* \internal
* \brief Wait for synchronous action to complete, and set its result
*
* \param[in,out] op Action to wait for
* \param[in,out] data Child signal data
*/
static void
wait_for_sync_result(svc_action_t *op, struct sigchld_data_s *data)
{
int status = 0;
int timeout = op->timeout;
time_t start = time(NULL);
struct pollfd fds[3];
int wait_rc = 0;
const char *wait_reason = NULL;
fds[0].fd = op->opaque->stdout_fd;
fds[0].events = POLLIN;
fds[0].revents = 0;
fds[1].fd = op->opaque->stderr_fd;
fds[1].events = POLLIN;
fds[1].revents = 0;
fds[2].fd = sigchld_open(data);
fds[2].events = POLLIN;
fds[2].revents = 0;
crm_trace("Waiting for %s[%d]", op->id, op->pid);
do {
int poll_rc = poll(fds, 3, timeout);
wait_reason = NULL;
if (poll_rc > 0) {
if (fds[0].revents & POLLIN) {
svc_read_output(op->opaque->stdout_fd, op, FALSE);
}
if (fds[1].revents & POLLIN) {
svc_read_output(op->opaque->stderr_fd, op, TRUE);
}
if ((fds[2].revents & POLLIN)
&& sigchld_received(fds[2].fd, op->pid, data)) {
wait_rc = waitpid(op->pid, &status, WNOHANG);
if ((wait_rc > 0) || ((wait_rc < 0) && (errno == ECHILD))) {
// Child process exited or doesn't exist
break;
} else if (wait_rc < 0) {
wait_reason = pcmk_rc_str(errno);
crm_info("Wait for completion of %s[%d] failed: %s "
QB_XS " source=waitpid",
op->id, op->pid, wait_reason);
wait_rc = 0; // Act as if process is still running
#ifndef HAVE_SYS_SIGNALFD_H
} else {
/* The child hasn't exited, so this SIGCHLD could be for
* another child. We have to ignore it here but will still
* need to resend it after this synchronous action has
* completed and SIGCHLD has been restored to be handled by
* the previous handler, so that it will be handled.
*/
data->ignored = true;
#endif
}
}
} else if (poll_rc == 0) {
// Poll timed out with no descriptors ready
timeout = 0;
break;
} else if ((poll_rc < 0) && (errno != EINTR)) {
wait_reason = pcmk_rc_str(errno);
crm_info("Wait for completion of %s[%d] failed: %s "
QB_XS " source=poll", op->id, op->pid, wait_reason);
break;
}
timeout = op->timeout - (time(NULL) - start) * 1000;
} while ((op->timeout < 0 || timeout > 0));
crm_trace("Stopped waiting for %s[%d]", op->id, op->pid);
finish_op_output(op, true);
finish_op_output(op, false);
close_op_input(op);
sigchld_close(fds[2].fd);
if (wait_rc <= 0) {
if ((op->timeout > 0) && (timeout <= 0)) {
services__format_result(op, services__generic_error(op),
PCMK_EXEC_TIMEOUT,
"%s did not exit within specified timeout",
services__action_kind(op));
crm_info("%s[%d] timed out after %dms",
op->id, op->pid, op->timeout);
} else {
services__set_result(op, services__generic_error(op),
PCMK_EXEC_ERROR, wait_reason);
}
/* If only child hasn't been successfully waited for, yet.
This is to limit killing wrong target a bit more. */
if ((wait_rc == 0) && (waitpid(op->pid, &status, WNOHANG) == 0)) {
if (kill(op->pid, SIGKILL)) {
crm_warn("Could not kill rogue child %s[%d]: %s",
op->id, op->pid, pcmk_rc_str(errno));
}
/* Safe to skip WNOHANG here as we sent non-ignorable signal. */
while ((waitpid(op->pid, &status, 0) == (pid_t) -1)
&& (errno == EINTR)) {
/* keep waiting */;
}
}
} else if (WIFEXITED(status)) {
services__set_result(op, WEXITSTATUS(status), PCMK_EXEC_DONE, NULL);
parse_exit_reason_from_stderr(op);
crm_info("%s[%d] exited with status %d", op->id, op->pid, op->rc);
} else if (WIFSIGNALED(status)) {
int signo = WTERMSIG(status);
services__format_result(op, services__generic_error(op),
PCMK_EXEC_ERROR, "%s interrupted by %s signal",
services__action_kind(op), strsignal(signo));
crm_info("%s[%d] terminated with signal %d (%s)",
op->id, op->pid, signo, strsignal(signo));
#ifdef WCOREDUMP
if (WCOREDUMP(status)) {
crm_warn("%s[%d] dumped core", op->id, op->pid);
}
#endif
} else {
// Shouldn't be possible to get here
services__set_result(op, services__generic_error(op), PCMK_EXEC_ERROR,
"Unable to wait for child to complete");
}
}
/*!
* \internal
* \brief Execute an action whose standard uses executable files
*
* \param[in,out] op Action to execute
*
* \return Standard Pacemaker return value
* \retval EBUSY Recurring operation could not be initiated
* \retval pcmk_rc_error Synchronous action failed
* \retval pcmk_rc_ok Synchronous action succeeded, or asynchronous action
* should not be freed (because it's pending or because
* it failed to execute and was already freed)
*
* \note If the return value for an asynchronous action is not pcmk_rc_ok, the
* caller is responsible for freeing the action.
*/
int
services__execute_file(svc_action_t *op)
{
int stdout_fd[2];
int stderr_fd[2];
int stdin_fd[2] = {-1, -1};
int rc;
struct stat st;
struct sigchld_data_s data = { .ignored = false };
// Catch common failure conditions early
if (stat(op->opaque->exec, &st) != 0) {
rc = errno;
crm_info("Cannot execute '%s': %s " QB_XS " stat rc=%d",
op->opaque->exec, pcmk_rc_str(rc), rc);
services__handle_exec_error(op, rc);
goto done;
}
if (pipe(stdout_fd) < 0) {
rc = errno;
crm_info("Cannot execute '%s': %s " QB_XS " pipe(stdout) rc=%d",
op->opaque->exec, pcmk_rc_str(rc), rc);
services__handle_exec_error(op, rc);
goto done;
}
if (pipe(stderr_fd) < 0) {
rc = errno;
close_pipe(stdout_fd);
crm_info("Cannot execute '%s': %s " QB_XS " pipe(stderr) rc=%d",
op->opaque->exec, pcmk_rc_str(rc), rc);
services__handle_exec_error(op, rc);
goto done;
}
if (pcmk_is_set(pcmk_get_ra_caps(op->standard), pcmk_ra_cap_stdin)) {
if (pipe(stdin_fd) < 0) {
rc = errno;
close_pipe(stdout_fd);
close_pipe(stderr_fd);
crm_info("Cannot execute '%s': %s " QB_XS " pipe(stdin) rc=%d",
op->opaque->exec, pcmk_rc_str(rc), rc);
services__handle_exec_error(op, rc);
goto done;
}
}
if (op->synchronous && !sigchld_setup(&data)) {
close_pipe(stdin_fd);
close_pipe(stdout_fd);
close_pipe(stderr_fd);
sigchld_cleanup(&data);
services__set_result(op, services__generic_error(op), PCMK_EXEC_ERROR,
"Could not manage signals for child process");
goto done;
}
op->pid = fork();
switch (op->pid) {
case -1:
rc = errno;
close_pipe(stdin_fd);
close_pipe(stdout_fd);
close_pipe(stderr_fd);
crm_info("Cannot execute '%s': %s " QB_XS " fork rc=%d",
op->opaque->exec, pcmk_rc_str(rc), rc);
services__handle_exec_error(op, rc);
if (op->synchronous) {
sigchld_cleanup(&data);
}
goto done;
break;
case 0: /* Child */
close(stdout_fd[0]);
close(stderr_fd[0]);
if (stdin_fd[1] >= 0) {
close(stdin_fd[1]);
}
if (STDOUT_FILENO != stdout_fd[1]) {
if (dup2(stdout_fd[1], STDOUT_FILENO) != STDOUT_FILENO) {
crm_warn("Can't redirect output from '%s': %s "
QB_XS " errno=%d",
op->opaque->exec, pcmk_rc_str(errno), errno);
}
close(stdout_fd[1]);
}
if (STDERR_FILENO != stderr_fd[1]) {
if (dup2(stderr_fd[1], STDERR_FILENO) != STDERR_FILENO) {
crm_warn("Can't redirect error output from '%s': %s "
QB_XS " errno=%d",
op->opaque->exec, pcmk_rc_str(errno), errno);
}
close(stderr_fd[1]);
}
if ((stdin_fd[0] >= 0) &&
(STDIN_FILENO != stdin_fd[0])) {
if (dup2(stdin_fd[0], STDIN_FILENO) != STDIN_FILENO) {
crm_warn("Can't redirect input to '%s': %s "
QB_XS " errno=%d",
op->opaque->exec, pcmk_rc_str(errno), errno);
}
close(stdin_fd[0]);
}
if (op->synchronous) {
sigchld_cleanup(&data);
}
action_launch_child(op);
pcmk__assert(false); // action_launch_child() should not return
}
/* Only the parent reaches here */
close(stdout_fd[1]);
close(stderr_fd[1]);
if (stdin_fd[0] >= 0) {
close(stdin_fd[0]);
}
op->opaque->stdout_fd = stdout_fd[0];
rc = pcmk__set_nonblocking(op->opaque->stdout_fd);
if (rc != pcmk_rc_ok) {
crm_info("Could not set '%s' output non-blocking: %s "
QB_XS " rc=%d",
op->opaque->exec, pcmk_rc_str(rc), rc);
}
op->opaque->stderr_fd = stderr_fd[0];
rc = pcmk__set_nonblocking(op->opaque->stderr_fd);
if (rc != pcmk_rc_ok) {
crm_info("Could not set '%s' error output non-blocking: %s "
QB_XS " rc=%d",
op->opaque->exec, pcmk_rc_str(rc), rc);
}
op->opaque->stdin_fd = stdin_fd[1];
if (op->opaque->stdin_fd >= 0) {
// using buffer behind non-blocking-fd here - that could be improved
// as long as no other standard uses stdin_fd assume stonith
rc = pcmk__set_nonblocking(op->opaque->stdin_fd);
if (rc != pcmk_rc_ok) {
crm_info("Could not set '%s' input non-blocking: %s "
QB_XS " fd=%d,rc=%d", op->opaque->exec,
pcmk_rc_str(rc), op->opaque->stdin_fd, rc);
}
pipe_in_action_stdin_parameters(op);
// as long as we are handling parameters directly in here just close
close(op->opaque->stdin_fd);
op->opaque->stdin_fd = -1;
}
// after fds are setup properly and before we plug anything into mainloop
if (op->opaque->fork_callback) {
op->opaque->fork_callback(op);
}
if (op->synchronous) {
wait_for_sync_result(op, &data);
sigchld_cleanup(&data);
goto done;
}
crm_trace("Waiting async for '%s'[%d]", op->opaque->exec, op->pid);
mainloop_child_add_with_flags(op->pid, op->timeout, op->id, op,
pcmk_is_set(op->flags, SVC_ACTION_LEAVE_GROUP)? mainloop_leave_pid_group : 0,
async_action_complete);
op->opaque->stdout_gsource = mainloop_add_fd(op->id,
G_PRIORITY_LOW,
op->opaque->stdout_fd, op,
&stdout_callbacks);
op->opaque->stderr_gsource = mainloop_add_fd(op->id,
G_PRIORITY_LOW,
op->opaque->stderr_fd, op,
&stderr_callbacks);
services_add_inflight_op(op);
return pcmk_rc_ok;
done:
if (op->synchronous) {
return (op->rc == PCMK_OCF_OK)? pcmk_rc_ok : pcmk_rc_error;
} else {
return services__finalize_async_op(op);
}
}
GList *
services_os_get_single_directory_list(const char *root, gboolean files, gboolean executable)
{
GList *list = NULL;
struct dirent **namelist;
int entries = 0, lpc = 0;
char buffer[PATH_MAX];
entries = scandir(root, &namelist, NULL, alphasort);
if (entries <= 0) {
return list;
}
for (lpc = 0; lpc < entries; lpc++) {
struct stat sb;
if ('.' == namelist[lpc]->d_name[0]) {
free(namelist[lpc]);
continue;
}
snprintf(buffer, sizeof(buffer), "%s/%s", root, namelist[lpc]->d_name);
if (stat(buffer, &sb)) {
continue;
}
if (S_ISDIR(sb.st_mode)) {
if (files) {
free(namelist[lpc]);
continue;
}
} else if (S_ISREG(sb.st_mode)) {
if (files == FALSE) {
free(namelist[lpc]);
continue;
} else if (executable
&& (sb.st_mode & S_IXUSR) == 0
&& (sb.st_mode & S_IXGRP) == 0 && (sb.st_mode & S_IXOTH) == 0) {
free(namelist[lpc]);
continue;
}
}
list = g_list_append(list, strdup(namelist[lpc]->d_name));
free(namelist[lpc]);
}
free(namelist);
return list;
}
GList *
services_os_get_directory_list(const char *root, gboolean files, gboolean executable)
{
GList *result = NULL;
char *dirs = strdup(root);
char *dir = NULL;
if (pcmk__str_empty(dirs)) {
free(dirs);
return result;
}
for (dir = strtok(dirs, ":"); dir != NULL; dir = strtok(NULL, ":")) {
GList *tmp = services_os_get_single_directory_list(dir, files, executable);
if (tmp) {
result = g_list_concat(result, tmp);
}
}
free(dirs);
return result;
}
diff --git a/maint/Makefile.am b/maint/Makefile.am
index 4d3174409e..fd5373f75d 100644
--- a/maint/Makefile.am
+++ b/maint/Makefile.am
@@ -1,82 +1,82 @@
#
# Copyright 2019-2024 the Pacemaker project contributors
#
# The version control history for this file may have further details.
#
# This source code is licensed under the GNU General Public License version 2
# or later (GPLv2+) WITHOUT ANY WARRANTY.
#
# Define release-related variables
include $(top_srcdir)/mk/release.mk
include $(top_srcdir)/mk/common.mk
noinst_SCRIPTS = bumplibs
EXTRA_DIST = README
#
# Change log generation
#
# Count changes in these directories
CHANGELOG_DIRS = ../include \
../lib \
../daemons \
../tools \
../xml
.PHONY: require_last_release
require_last_release:
@if [ -z "$(CHECKOUT)" ]; then \
echo "This target must be run from a git checkout"; \
exit 1; \
elif ! "$(GIT)" rev-parse $(LAST_RELEASE) >/dev/null 2>&1; then \
echo "LAST_RELEASE must be set to a valid git tag"; \
exit 1; \
fi
.PHONY: summary
summary: require_last_release
- @printf "# Pacemaker-%s (%s)\n* %d commits with%s\n" \
+ @printf "# %s (%s)\n* %d commits with%s\n" \
"$(NEXT_RELEASE)" "$$(date +'%d %b %Y')" \
"$$("$(GIT)" log --pretty=oneline --no-merges \
$(LAST_RELEASE)..HEAD | wc -l)" \
"$$("$(GIT)" diff $(LAST_RELEASE)..HEAD --shortstat \
$(CHANGELOG_DIRS))"
.PHONY: changes
changes: summary
@printf "\n## Features added since $(LAST_RELEASE)\n\n"
@"$(GIT)" log --pretty=format:'%s' --no-merges \
--abbrev-commit $(LAST_RELEASE)..HEAD \
| sed -n -e 's/^ *Feature: */* /p' | sort -uf \
| sed -e 's/^\( *[-+*] \)\([^:]*:\)\(.*\)$$/\1**\2**\3/' \
-e 's/\([ (]\)\([A-Za-z0-9]*_[^ ,]*\)/\1`\2`/g'
@printf "\n## Fixes since $(LAST_RELEASE)\n\n"
@"$(GIT)" log --pretty=format:'%s' --no-merges \
--abbrev-commit $(LAST_RELEASE)..HEAD \
| sed -n -e 's/^ *\(Fix\|High\|Bug\): */* /p' \
| sed -e 's/\(\(pacemaker-\)?based\):/CIB:/' \
-e 's/\(\(pacemaker-\)?execd\):/executor:/' \
-e 's/\(\(pacemaker-\)?controld\):/controller:/' \
-e 's/\(\(pacemaker-\)?fenced\):/fencing:/' \
| sort -uf \
| sed -e 's/^\( *[-+*] \)\([^:]*:\)\(.*\)$$/\1**\2**\3/' \
-e 's/\([ (]\)\([A-Za-z0-9]*_[^ ,]*\)/\1`\2`/g'
@printf "\n## Public API changes since $(LAST_RELEASE)\n\n"
@"$(GIT)" log --pretty=format:'%s' --no-merges \
--abbrev-commit $(LAST_RELEASE)..HEAD \
| sed -n -e 's/^ *API: */* /p' | sort -uf \
| sed -e 's/^\( *[-+*] \)\([^:]*:\)\(.*\)$$/\1**\2**\3/' \
-e 's/\([ (]\)\([A-Za-z0-9]*_[^ ,]*\)/\1`\2`/g'
.PHONY: changelog
changelog: require_last_release
@printf "%s\n\n%s\n" \
"$$($(MAKE) $(AM_MAKEFLAGS) changes \
| grep -v 'make\(\[[0-9]*\]\)\?:')" \
"$$(cat ../ChangeLog.md)" > ../ChangeLog.md
.PHONY: authors
authors: require_last_release
"$(GIT)" log $(LAST_RELEASE)..$(COMMIT) --format='%an' | sort -u