diff --git a/cts/cli/regression.crm_attribute.exp b/cts/cli/regression.crm_attribute.exp index adad151c9f..19558586fb 100644 --- a/cts/cli/regression.crm_attribute.exp +++ b/cts/cli/regression.crm_attribute.exp @@ -1,1913 +1,1913 @@ =#=#=#= Begin test: List all available options (invalid type) =#=#=#= crm_attribute: Invalid --list-options value 'asdf'. Allowed values: cluster =#=#=#= End test: List all available options (invalid type) - Incorrect usage (64) =#=#=#= * Passed: crm_attribute - List all available options (invalid type) =#=#=#= Begin test: List all available options (invalid type) (XML) =#=#=#= crm_attribute: Invalid --list-options value 'asdf'. Allowed values: cluster =#=#=#= End test: List all available options (invalid type) (XML) - Incorrect usage (64) =#=#=#= * Passed: crm_attribute - List all available options (invalid type) (XML) =#=#=#= Begin test: List non-advanced cluster options =#=#=#= Pacemaker cluster options Also known as properties, these are options that affect behavior across the entire cluster. They are configured within cluster_property_set elements inside the crm_config subsection of the CIB configuration section. * dc-version: Pacemaker version on cluster node elected Designated Controller (DC) * Includes a hash which identifies the exact revision the code was built from. Used for diagnostic purposes. * Possible values (generated by Pacemaker): version (no default) * cluster-infrastructure: The messaging layer on which Pacemaker is currently running * Used for informational and diagnostic purposes. * Possible values (generated by Pacemaker): string (no default) * cluster-name: An arbitrary name for the cluster * This optional value is mostly for users' convenience as desired in administration, but may also be used in Pacemaker configuration rules via the #cluster-name node attribute, and by higher-level tools and resource agents. * Possible values: string (no default) * dc-deadtime: How long to wait for a response from other nodes during start-up * The optimal value will depend on the speed and load of your network and the type of switches used. * Possible values: duration (default: ) * cluster-recheck-interval: Polling interval to recheck cluster state and evaluate rules with date specifications * Pacemaker is primarily event-driven, and looks ahead to know when to recheck cluster state for failure-timeout settings and most time-based rules. However, it will also recheck the cluster after this amount of inactivity, to evaluate rules with date specifications and serve as a fail-safe for certain types of scheduler bugs. A value of 0 disables polling. A positive value sets an interval in seconds, unless other units are specified (for example, "5min"). * Possible values: duration (default: ) * fence-reaction: How a cluster node should react if notified of its own fencing * A cluster node may receive notification of a "succeeded" fencing that targeted it if fencing is misconfigured, or if fabric fencing is in use that doesn't cut cluster communication. Use "stop" to attempt to immediately stop Pacemaker and stay stopped, or "panic" to attempt to immediately reboot the local node, falling back to stop on failure. * Possible values: "stop" (default), "panic" * no-quorum-policy: What to do when the cluster does not have quorum * Possible values: "stop" (default), "freeze", "ignore", "demote", "fence", "suicide" * shutdown-lock: Whether to lock resources to a cleanly shut down node * When true, resources active on a node when it is cleanly shut down are kept "locked" to that node (not allowed to run elsewhere) until they start again on that node after it rejoins (or for at most shutdown-lock-limit, if set). Stonith resources and Pacemaker Remote connections are never locked. Clone and bundle instances and the promoted role of promotable clones are currently never locked, though support could be added in a future release. * Possible values: boolean (default: ) * shutdown-lock-limit: Do not lock resources to a cleanly shut down node longer than this * If shutdown-lock is true and this is set to a nonzero time duration, shutdown locks will expire after this much time has passed since the shutdown was initiated, even if the node has not rejoined. * Possible values: duration (default: ) * enable-acl: Enable Access Control Lists (ACLs) for the CIB * Possible values: boolean (default: ) * symmetric-cluster: Whether resources can run on any node by default * Possible values: boolean (default: ) * maintenance-mode: Whether the cluster should refrain from monitoring, starting, and stopping resources * Possible values: boolean (default: ) * start-failure-is-fatal: Whether a start failure should prevent a resource from being recovered on the same node * When true, the cluster will immediately ban a resource from a node if it fails to start there. When false, the cluster will instead check the resource's fail count against its migration-threshold. * Possible values: boolean (default: ) * enable-startup-probes: Whether the cluster should check for active resources during start-up * Possible values: boolean (default: ) * stonith-action: Action to send to fence device when a node needs to be fenced * Possible values: "reboot" (default), "off" * stonith-timeout: How long to wait for on, off, and reboot fence actions to complete by default * Possible values: duration (default: ) * have-watchdog: Whether watchdog integration is enabled * This is set automatically by the cluster according to whether SBD is detected to be in use. User-configured values are ignored. The value `true` is meaningful if diskless SBD is used and `stonith-watchdog-timeout` is nonzero. In that case, if fencing is required, watchdog-based self-fencing will be performed via SBD without requiring a fencing resource explicitly configured. * Possible values (generated by Pacemaker): boolean (default: ) * stonith-watchdog-timeout: How long before nodes can be assumed to be safely down when watchdog-based self-fencing via SBD is in use * If this is set to a positive value, lost nodes are assumed to achieve self-fencing using watchdog-based SBD within this much time. This does not require a fencing resource to be explicitly configured, though a fence_watchdog resource can be configured, to limit use to specific nodes. If this is set to 0 (the default), the cluster will never assume watchdog-based self-fencing. If this is set to a negative value, the cluster will use twice the local value of the `SBD_WATCHDOG_TIMEOUT` environment variable if that is positive, or otherwise treat this as 0. WARNING: When used, this timeout must be larger than `SBD_WATCHDOG_TIMEOUT` on all nodes that use watchdog-based SBD, and Pacemaker will refuse to start on any of those nodes where this is not true for the local value or SBD is not active. When this is set to a negative value, `SBD_WATCHDOG_TIMEOUT` must be set to the same value on all nodes that use SBD, otherwise data corruption or loss could occur. * Possible values: timeout (default: ) * stonith-max-attempts: How many times fencing can fail before it will no longer be immediately re-attempted on a target * Possible values: score (default: ) * priority-fencing-delay: Apply fencing delay targeting the lost nodes with the highest total resource priority * Apply specified delay for the fencings that are targeting the lost nodes with the highest total resource priority in case we don't have the majority of the nodes in our cluster partition, so that the more significant nodes potentially win any fencing match, which is especially meaningful under split-brain of 2-node cluster. A promoted resource instance takes the base priority + 1 on calculation if the base priority is not 0. Any static/random delays that are introduced by `pcmk_delay_base/max` configured for the corresponding fencing resources will be added to this delay. This delay should be significantly greater than, safely twice, the maximum `pcmk_delay_base/max`. By default, priority fencing delay is disabled. * Possible values: duration (default: ) * node-pending-timeout: How long to wait for a node that has joined the cluster to join the controller process group * Fence nodes that do not join the controller process group within this much time after joining the cluster, to allow the cluster to continue managing resources. A value of 0 means never fence pending nodes. Setting the value to 2h means fence nodes after 2 hours. * Possible values: duration (default: ) * cluster-delay: Maximum time for node-to-node communication * The node elected Designated Controller (DC) will consider an action failed if it does not get a response from the node executing the action within this time (after considering the action's own timeout). The "correct" value will depend on the speed and load of your network and cluster nodes. * Possible values: duration (default: ) * load-threshold: Maximum amount of system load that should be used by cluster nodes * The cluster will slow down its recovery process when the amount of system resources used (currently CPU) approaches this limit * Possible values: percentage (default: ) * node-action-limit: Maximum number of jobs that can be scheduled per node (defaults to 2x cores) * Possible values: integer (default: ) * batch-limit: Maximum number of jobs that the cluster may execute in parallel across all nodes * The "correct" value will depend on the speed and load of your network and cluster nodes. If set to 0, the cluster will impose a dynamically calculated limit when any node has a high load. * Possible values: integer (default: ) * migration-limit: The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) * Possible values: integer (default: ) - * cluster-ipc-limit: Maximum IPC message backlog before disconnecting a cluster daemon - * Raise this if log has "Evicting client" messages for cluster daemon PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). + * cluster-ipc-limit: Maximum IPC message backlog before disconnecting a command line client + * Raise this if log has "Evicting client" messages for cluster PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). * Possible values: nonnegative_integer (default: ) * stop-all-resources: Whether the cluster should stop all active resources * Possible values: boolean (default: ) * stop-removed-resources: Whether to stop resources that were removed from the configuration * Possible values: boolean (default: ) * stop-removed-actions: Whether to cancel recurring actions removed from the configuration * Possible values: boolean (default: ) * pe-error-series-max: The number of scheduler inputs resulting in errors to save * Zero to disable, -1 to store unlimited. * Possible values: integer (default: ) * pe-warn-series-max: The number of scheduler inputs resulting in warnings to save * Zero to disable, -1 to store unlimited. * Possible values: integer (default: ) * pe-input-series-max: The number of scheduler inputs without errors or warnings to save * Zero to disable, -1 to store unlimited. * Possible values: integer (default: ) * node-health-strategy: How cluster should react to node health attributes * Requires external entities to create node attributes (named with the prefix "#health") with values "red", "yellow", or "green". * Possible values: "none" (default), "migrate-on-red", "only-green", "progressive", "custom" * node-health-base: Base health score assigned to a node * Only used when "node-health-strategy" is set to "progressive". * Possible values: score (default: ) * node-health-green: The score to use for a node health attribute whose value is "green" * Only used when "node-health-strategy" is set to "custom" or "progressive". * Possible values: score (default: ) * node-health-yellow: The score to use for a node health attribute whose value is "yellow" * Only used when "node-health-strategy" is set to "custom" or "progressive". * Possible values: score (default: ) * node-health-red: The score to use for a node health attribute whose value is "red" * Only used when "node-health-strategy" is set to "custom" or "progressive". * Possible values: score (default: ) * placement-strategy: How the cluster should allocate resources to nodes * Possible values: "default" (default), "utilization", "minimal", "balanced" =#=#=#= End test: List non-advanced cluster options - OK (0) =#=#=#= * Passed: crm_attribute - List non-advanced cluster options =#=#=#= Begin test: List non-advanced cluster options (XML) =#=#=#= 1.1 Also known as properties, these are options that affect behavior across the entire cluster. They are configured within cluster_property_set elements inside the crm_config subsection of the CIB configuration section. Pacemaker cluster options Includes a hash which identifies the exact revision the code was built from. Used for diagnostic purposes. Pacemaker version on cluster node elected Designated Controller (DC) Used for informational and diagnostic purposes. The messaging layer on which Pacemaker is currently running This optional value is mostly for users' convenience as desired in administration, but may also be used in Pacemaker configuration rules via the #cluster-name node attribute, and by higher-level tools and resource agents. An arbitrary name for the cluster The optimal value will depend on the speed and load of your network and the type of switches used. How long to wait for a response from other nodes during start-up Pacemaker is primarily event-driven, and looks ahead to know when to recheck cluster state for failure-timeout settings and most time-based rules. However, it will also recheck the cluster after this amount of inactivity, to evaluate rules with date specifications and serve as a fail-safe for certain types of scheduler bugs. A value of 0 disables polling. A positive value sets an interval in seconds, unless other units are specified (for example, "5min"). Polling interval to recheck cluster state and evaluate rules with date specifications A cluster node may receive notification of a "succeeded" fencing that targeted it if fencing is misconfigured, or if fabric fencing is in use that doesn't cut cluster communication. Use "stop" to attempt to immediately stop Pacemaker and stay stopped, or "panic" to attempt to immediately reboot the local node, falling back to stop on failure. How a cluster node should react if notified of its own fencing Declare an election failed if it is not decided within this much time. If you need to adjust this value, it probably indicates the presence of a bug. Declare an election failed if it is not decided within this much time. If you need to adjust this value, it probably indicates the presence of a bug. Exit immediately if shutdown does not complete within this much time. If you need to adjust this value, it probably indicates the presence of a bug. Exit immediately if shutdown does not complete within this much time. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. Delay cluster recovery for this much time to allow for additional events to occur. Useful if your configuration is sensitive to the order in which ping updates arrive. Enabling this option will slow down cluster recovery under all conditions What to do when the cluster does not have quorum What to do when the cluster does not have quorum When true, resources active on a node when it is cleanly shut down are kept "locked" to that node (not allowed to run elsewhere) until they start again on that node after it rejoins (or for at most shutdown-lock-limit, if set). Stonith resources and Pacemaker Remote connections are never locked. Clone and bundle instances and the promoted role of promotable clones are currently never locked, though support could be added in a future release. Whether to lock resources to a cleanly shut down node If shutdown-lock is true and this is set to a nonzero time duration, shutdown locks will expire after this much time has passed since the shutdown was initiated, even if the node has not rejoined. Do not lock resources to a cleanly shut down node longer than this Enable Access Control Lists (ACLs) for the CIB Enable Access Control Lists (ACLs) for the CIB Whether resources can run on any node by default Whether resources can run on any node by default Whether the cluster should refrain from monitoring, starting, and stopping resources Whether the cluster should refrain from monitoring, starting, and stopping resources When true, the cluster will immediately ban a resource from a node if it fails to start there. When false, the cluster will instead check the resource's fail count against its migration-threshold. Whether a start failure should prevent a resource from being recovered on the same node Whether the cluster should check for active resources during start-up Whether the cluster should check for active resources during start-up By default, an inquorate node can not fence Pacemaker Remote nodes that are part of its partition as long as the cluster thinks they can be restarted. If true, inquorate nodes will be able to fence remote nodes regardless. Whether remote nodes can be fenced without quorum If false, unresponsive nodes are immediately assumed to be harmless, and resources that were active on them may be recovered elsewhere. This can result in a "split-brain" situation, potentially leading to data loss and/or service unavailability. Whether nodes may be fenced as part of recovery Action to send to fence device when a node needs to be fenced Action to send to fence device when a node needs to be fenced How long to wait for on, off, and reboot fence actions to complete by default How long to wait for on, off, and reboot fence actions to complete by default This is set automatically by the cluster according to whether SBD is detected to be in use. User-configured values are ignored. The value `true` is meaningful if diskless SBD is used and `stonith-watchdog-timeout` is nonzero. In that case, if fencing is required, watchdog-based self-fencing will be performed via SBD without requiring a fencing resource explicitly configured. Whether watchdog integration is enabled If this is set to a positive value, lost nodes are assumed to achieve self-fencing using watchdog-based SBD within this much time. This does not require a fencing resource to be explicitly configured, though a fence_watchdog resource can be configured, to limit use to specific nodes. If this is set to 0 (the default), the cluster will never assume watchdog-based self-fencing. If this is set to a negative value, the cluster will use twice the local value of the `SBD_WATCHDOG_TIMEOUT` environment variable if that is positive, or otherwise treat this as 0. WARNING: When used, this timeout must be larger than `SBD_WATCHDOG_TIMEOUT` on all nodes that use watchdog-based SBD, and Pacemaker will refuse to start on any of those nodes where this is not true for the local value or SBD is not active. When this is set to a negative value, `SBD_WATCHDOG_TIMEOUT` must be set to the same value on all nodes that use SBD, otherwise data corruption or loss could occur. How long before nodes can be assumed to be safely down when watchdog-based self-fencing via SBD is in use How many times fencing can fail before it will no longer be immediately re-attempted on a target How many times fencing can fail before it will no longer be immediately re-attempted on a target Allow performing fencing operations in parallel Allow performing fencing operations in parallel Setting this to false may lead to a "split-brain" situation, potentially leading to data loss and/or service unavailability. Whether to fence unseen nodes at start-up Apply specified delay for the fencings that are targeting the lost nodes with the highest total resource priority in case we don't have the majority of the nodes in our cluster partition, so that the more significant nodes potentially win any fencing match, which is especially meaningful under split-brain of 2-node cluster. A promoted resource instance takes the base priority + 1 on calculation if the base priority is not 0. Any static/random delays that are introduced by `pcmk_delay_base/max` configured for the corresponding fencing resources will be added to this delay. This delay should be significantly greater than, safely twice, the maximum `pcmk_delay_base/max`. By default, priority fencing delay is disabled. Apply fencing delay targeting the lost nodes with the highest total resource priority Fence nodes that do not join the controller process group within this much time after joining the cluster, to allow the cluster to continue managing resources. A value of 0 means never fence pending nodes. Setting the value to 2h means fence nodes after 2 hours. How long to wait for a node that has joined the cluster to join the controller process group The node elected Designated Controller (DC) will consider an action failed if it does not get a response from the node executing the action within this time (after considering the action's own timeout). The "correct" value will depend on the speed and load of your network and cluster nodes. Maximum time for node-to-node communication The cluster will slow down its recovery process when the amount of system resources used (currently CPU) approaches this limit Maximum amount of system load that should be used by cluster nodes Maximum number of jobs that can be scheduled per node (defaults to 2x cores) Maximum number of jobs that can be scheduled per node (defaults to 2x cores) The "correct" value will depend on the speed and load of your network and cluster nodes. If set to 0, the cluster will impose a dynamically calculated limit when any node has a high load. Maximum number of jobs that the cluster may execute in parallel across all nodes The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) - Raise this if log has "Evicting client" messages for cluster daemon PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). - Maximum IPC message backlog before disconnecting a cluster daemon + Raise this if log has "Evicting client" messages for cluster PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). + Maximum IPC message backlog before disconnecting a command line client Whether the cluster should stop all active resources Whether the cluster should stop all active resources Whether to stop resources that were removed from the configuration Whether to stop resources that were removed from the configuration Whether to cancel recurring actions removed from the configuration Whether to cancel recurring actions removed from the configuration Zero to disable, -1 to store unlimited. The number of scheduler inputs resulting in errors to save Zero to disable, -1 to store unlimited. The number of scheduler inputs resulting in warnings to save Zero to disable, -1 to store unlimited. The number of scheduler inputs without errors or warnings to save Requires external entities to create node attributes (named with the prefix "#health") with values "red", "yellow", or "green". How cluster should react to node health attributes Only used when "node-health-strategy" is set to "progressive". Base health score assigned to a node Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "green" Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "yellow" Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "red" How the cluster should allocate resources to nodes How the cluster should allocate resources to nodes =#=#=#= End test: List non-advanced cluster options (XML) - OK (0) =#=#=#= * Passed: crm_attribute - List non-advanced cluster options (XML) =#=#=#= Begin test: List all available cluster options =#=#=#= Pacemaker cluster options Also known as properties, these are options that affect behavior across the entire cluster. They are configured within cluster_property_set elements inside the crm_config subsection of the CIB configuration section. * dc-version: Pacemaker version on cluster node elected Designated Controller (DC) * Includes a hash which identifies the exact revision the code was built from. Used for diagnostic purposes. * Possible values (generated by Pacemaker): version (no default) * cluster-infrastructure: The messaging layer on which Pacemaker is currently running * Used for informational and diagnostic purposes. * Possible values (generated by Pacemaker): string (no default) * cluster-name: An arbitrary name for the cluster * This optional value is mostly for users' convenience as desired in administration, but may also be used in Pacemaker configuration rules via the #cluster-name node attribute, and by higher-level tools and resource agents. * Possible values: string (no default) * dc-deadtime: How long to wait for a response from other nodes during start-up * The optimal value will depend on the speed and load of your network and the type of switches used. * Possible values: duration (default: ) * cluster-recheck-interval: Polling interval to recheck cluster state and evaluate rules with date specifications * Pacemaker is primarily event-driven, and looks ahead to know when to recheck cluster state for failure-timeout settings and most time-based rules. However, it will also recheck the cluster after this amount of inactivity, to evaluate rules with date specifications and serve as a fail-safe for certain types of scheduler bugs. A value of 0 disables polling. A positive value sets an interval in seconds, unless other units are specified (for example, "5min"). * Possible values: duration (default: ) * fence-reaction: How a cluster node should react if notified of its own fencing * A cluster node may receive notification of a "succeeded" fencing that targeted it if fencing is misconfigured, or if fabric fencing is in use that doesn't cut cluster communication. Use "stop" to attempt to immediately stop Pacemaker and stay stopped, or "panic" to attempt to immediately reboot the local node, falling back to stop on failure. * Possible values: "stop" (default), "panic" * no-quorum-policy: What to do when the cluster does not have quorum * Possible values: "stop" (default), "freeze", "ignore", "demote", "fence", "suicide" * shutdown-lock: Whether to lock resources to a cleanly shut down node * When true, resources active on a node when it is cleanly shut down are kept "locked" to that node (not allowed to run elsewhere) until they start again on that node after it rejoins (or for at most shutdown-lock-limit, if set). Stonith resources and Pacemaker Remote connections are never locked. Clone and bundle instances and the promoted role of promotable clones are currently never locked, though support could be added in a future release. * Possible values: boolean (default: ) * shutdown-lock-limit: Do not lock resources to a cleanly shut down node longer than this * If shutdown-lock is true and this is set to a nonzero time duration, shutdown locks will expire after this much time has passed since the shutdown was initiated, even if the node has not rejoined. * Possible values: duration (default: ) * enable-acl: Enable Access Control Lists (ACLs) for the CIB * Possible values: boolean (default: ) * symmetric-cluster: Whether resources can run on any node by default * Possible values: boolean (default: ) * maintenance-mode: Whether the cluster should refrain from monitoring, starting, and stopping resources * Possible values: boolean (default: ) * start-failure-is-fatal: Whether a start failure should prevent a resource from being recovered on the same node * When true, the cluster will immediately ban a resource from a node if it fails to start there. When false, the cluster will instead check the resource's fail count against its migration-threshold. * Possible values: boolean (default: ) * enable-startup-probes: Whether the cluster should check for active resources during start-up * Possible values: boolean (default: ) * stonith-action: Action to send to fence device when a node needs to be fenced * Possible values: "reboot" (default), "off" * stonith-timeout: How long to wait for on, off, and reboot fence actions to complete by default * Possible values: duration (default: ) * have-watchdog: Whether watchdog integration is enabled * This is set automatically by the cluster according to whether SBD is detected to be in use. User-configured values are ignored. The value `true` is meaningful if diskless SBD is used and `stonith-watchdog-timeout` is nonzero. In that case, if fencing is required, watchdog-based self-fencing will be performed via SBD without requiring a fencing resource explicitly configured. * Possible values (generated by Pacemaker): boolean (default: ) * stonith-watchdog-timeout: How long before nodes can be assumed to be safely down when watchdog-based self-fencing via SBD is in use * If this is set to a positive value, lost nodes are assumed to achieve self-fencing using watchdog-based SBD within this much time. This does not require a fencing resource to be explicitly configured, though a fence_watchdog resource can be configured, to limit use to specific nodes. If this is set to 0 (the default), the cluster will never assume watchdog-based self-fencing. If this is set to a negative value, the cluster will use twice the local value of the `SBD_WATCHDOG_TIMEOUT` environment variable if that is positive, or otherwise treat this as 0. WARNING: When used, this timeout must be larger than `SBD_WATCHDOG_TIMEOUT` on all nodes that use watchdog-based SBD, and Pacemaker will refuse to start on any of those nodes where this is not true for the local value or SBD is not active. When this is set to a negative value, `SBD_WATCHDOG_TIMEOUT` must be set to the same value on all nodes that use SBD, otherwise data corruption or loss could occur. * Possible values: timeout (default: ) * stonith-max-attempts: How many times fencing can fail before it will no longer be immediately re-attempted on a target * Possible values: score (default: ) * priority-fencing-delay: Apply fencing delay targeting the lost nodes with the highest total resource priority * Apply specified delay for the fencings that are targeting the lost nodes with the highest total resource priority in case we don't have the majority of the nodes in our cluster partition, so that the more significant nodes potentially win any fencing match, which is especially meaningful under split-brain of 2-node cluster. A promoted resource instance takes the base priority + 1 on calculation if the base priority is not 0. Any static/random delays that are introduced by `pcmk_delay_base/max` configured for the corresponding fencing resources will be added to this delay. This delay should be significantly greater than, safely twice, the maximum `pcmk_delay_base/max`. By default, priority fencing delay is disabled. * Possible values: duration (default: ) * node-pending-timeout: How long to wait for a node that has joined the cluster to join the controller process group * Fence nodes that do not join the controller process group within this much time after joining the cluster, to allow the cluster to continue managing resources. A value of 0 means never fence pending nodes. Setting the value to 2h means fence nodes after 2 hours. * Possible values: duration (default: ) * cluster-delay: Maximum time for node-to-node communication * The node elected Designated Controller (DC) will consider an action failed if it does not get a response from the node executing the action within this time (after considering the action's own timeout). The "correct" value will depend on the speed and load of your network and cluster nodes. * Possible values: duration (default: ) * load-threshold: Maximum amount of system load that should be used by cluster nodes * The cluster will slow down its recovery process when the amount of system resources used (currently CPU) approaches this limit * Possible values: percentage (default: ) * node-action-limit: Maximum number of jobs that can be scheduled per node (defaults to 2x cores) * Possible values: integer (default: ) * batch-limit: Maximum number of jobs that the cluster may execute in parallel across all nodes * The "correct" value will depend on the speed and load of your network and cluster nodes. If set to 0, the cluster will impose a dynamically calculated limit when any node has a high load. * Possible values: integer (default: ) * migration-limit: The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) * Possible values: integer (default: ) - * cluster-ipc-limit: Maximum IPC message backlog before disconnecting a cluster daemon - * Raise this if log has "Evicting client" messages for cluster daemon PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). + * cluster-ipc-limit: Maximum IPC message backlog before disconnecting a command line client + * Raise this if log has "Evicting client" messages for cluster PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). * Possible values: nonnegative_integer (default: ) * stop-all-resources: Whether the cluster should stop all active resources * Possible values: boolean (default: ) * stop-removed-resources: Whether to stop resources that were removed from the configuration * Possible values: boolean (default: ) * stop-removed-actions: Whether to cancel recurring actions removed from the configuration * Possible values: boolean (default: ) * pe-error-series-max: The number of scheduler inputs resulting in errors to save * Zero to disable, -1 to store unlimited. * Possible values: integer (default: ) * pe-warn-series-max: The number of scheduler inputs resulting in warnings to save * Zero to disable, -1 to store unlimited. * Possible values: integer (default: ) * pe-input-series-max: The number of scheduler inputs without errors or warnings to save * Zero to disable, -1 to store unlimited. * Possible values: integer (default: ) * node-health-strategy: How cluster should react to node health attributes * Requires external entities to create node attributes (named with the prefix "#health") with values "red", "yellow", or "green". * Possible values: "none" (default), "migrate-on-red", "only-green", "progressive", "custom" * node-health-base: Base health score assigned to a node * Only used when "node-health-strategy" is set to "progressive". * Possible values: score (default: ) * node-health-green: The score to use for a node health attribute whose value is "green" * Only used when "node-health-strategy" is set to "custom" or "progressive". * Possible values: score (default: ) * node-health-yellow: The score to use for a node health attribute whose value is "yellow" * Only used when "node-health-strategy" is set to "custom" or "progressive". * Possible values: score (default: ) * node-health-red: The score to use for a node health attribute whose value is "red" * Only used when "node-health-strategy" is set to "custom" or "progressive". * Possible values: score (default: ) * placement-strategy: How the cluster should allocate resources to nodes * Possible values: "default" (default), "utilization", "minimal", "balanced" * ADVANCED OPTIONS: * election-timeout: Declare an election failed if it is not decided within this much time. If you need to adjust this value, it probably indicates the presence of a bug. * Possible values: duration (default: ) * shutdown-escalation: Exit immediately if shutdown does not complete within this much time. If you need to adjust this value, it probably indicates the presence of a bug. * Possible values: duration (default: ) * join-integration-timeout: If you need to adjust this value, it probably indicates the presence of a bug. * Possible values: duration (default: ) * join-finalization-timeout: If you need to adjust this value, it probably indicates the presence of a bug. * Possible values: duration (default: ) * transition-delay: Enabling this option will slow down cluster recovery under all conditions * Delay cluster recovery for this much time to allow for additional events to occur. Useful if your configuration is sensitive to the order in which ping updates arrive. * Possible values: duration (default: ) * fence-remote-without-quorum: Whether remote nodes can be fenced without quorum * By default, an inquorate node can not fence Pacemaker Remote nodes that are part of its partition as long as the cluster thinks they can be restarted. If true, inquorate nodes will be able to fence remote nodes regardless. * Possible values: boolean (default: ) * stonith-enabled: Whether nodes may be fenced as part of recovery * If false, unresponsive nodes are immediately assumed to be harmless, and resources that were active on them may be recovered elsewhere. This can result in a "split-brain" situation, potentially leading to data loss and/or service unavailability. * Possible values: boolean (default: ) * startup-fencing: Whether to fence unseen nodes at start-up * Setting this to false may lead to a "split-brain" situation, potentially leading to data loss and/or service unavailability. * Possible values: boolean (default: ) * DEPRECATED OPTIONS (will be removed in a future release): * concurrent-fencing: Allow performing fencing operations in parallel * Possible values: boolean (default: ) =#=#=#= End test: List all available cluster options - OK (0) =#=#=#= * Passed: crm_attribute - List all available cluster options =#=#=#= Begin test: List all available cluster options (XML) =#=#=#= 1.1 Also known as properties, these are options that affect behavior across the entire cluster. They are configured within cluster_property_set elements inside the crm_config subsection of the CIB configuration section. Pacemaker cluster options Includes a hash which identifies the exact revision the code was built from. Used for diagnostic purposes. Pacemaker version on cluster node elected Designated Controller (DC) Used for informational and diagnostic purposes. The messaging layer on which Pacemaker is currently running This optional value is mostly for users' convenience as desired in administration, but may also be used in Pacemaker configuration rules via the #cluster-name node attribute, and by higher-level tools and resource agents. An arbitrary name for the cluster The optimal value will depend on the speed and load of your network and the type of switches used. How long to wait for a response from other nodes during start-up Pacemaker is primarily event-driven, and looks ahead to know when to recheck cluster state for failure-timeout settings and most time-based rules. However, it will also recheck the cluster after this amount of inactivity, to evaluate rules with date specifications and serve as a fail-safe for certain types of scheduler bugs. A value of 0 disables polling. A positive value sets an interval in seconds, unless other units are specified (for example, "5min"). Polling interval to recheck cluster state and evaluate rules with date specifications A cluster node may receive notification of a "succeeded" fencing that targeted it if fencing is misconfigured, or if fabric fencing is in use that doesn't cut cluster communication. Use "stop" to attempt to immediately stop Pacemaker and stay stopped, or "panic" to attempt to immediately reboot the local node, falling back to stop on failure. How a cluster node should react if notified of its own fencing Declare an election failed if it is not decided within this much time. If you need to adjust this value, it probably indicates the presence of a bug. Declare an election failed if it is not decided within this much time. If you need to adjust this value, it probably indicates the presence of a bug. Exit immediately if shutdown does not complete within this much time. If you need to adjust this value, it probably indicates the presence of a bug. Exit immediately if shutdown does not complete within this much time. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. If you need to adjust this value, it probably indicates the presence of a bug. Delay cluster recovery for this much time to allow for additional events to occur. Useful if your configuration is sensitive to the order in which ping updates arrive. Enabling this option will slow down cluster recovery under all conditions What to do when the cluster does not have quorum What to do when the cluster does not have quorum When true, resources active on a node when it is cleanly shut down are kept "locked" to that node (not allowed to run elsewhere) until they start again on that node after it rejoins (or for at most shutdown-lock-limit, if set). Stonith resources and Pacemaker Remote connections are never locked. Clone and bundle instances and the promoted role of promotable clones are currently never locked, though support could be added in a future release. Whether to lock resources to a cleanly shut down node If shutdown-lock is true and this is set to a nonzero time duration, shutdown locks will expire after this much time has passed since the shutdown was initiated, even if the node has not rejoined. Do not lock resources to a cleanly shut down node longer than this Enable Access Control Lists (ACLs) for the CIB Enable Access Control Lists (ACLs) for the CIB Whether resources can run on any node by default Whether resources can run on any node by default Whether the cluster should refrain from monitoring, starting, and stopping resources Whether the cluster should refrain from monitoring, starting, and stopping resources When true, the cluster will immediately ban a resource from a node if it fails to start there. When false, the cluster will instead check the resource's fail count against its migration-threshold. Whether a start failure should prevent a resource from being recovered on the same node Whether the cluster should check for active resources during start-up Whether the cluster should check for active resources during start-up By default, an inquorate node can not fence Pacemaker Remote nodes that are part of its partition as long as the cluster thinks they can be restarted. If true, inquorate nodes will be able to fence remote nodes regardless. Whether remote nodes can be fenced without quorum If false, unresponsive nodes are immediately assumed to be harmless, and resources that were active on them may be recovered elsewhere. This can result in a "split-brain" situation, potentially leading to data loss and/or service unavailability. Whether nodes may be fenced as part of recovery Action to send to fence device when a node needs to be fenced Action to send to fence device when a node needs to be fenced How long to wait for on, off, and reboot fence actions to complete by default How long to wait for on, off, and reboot fence actions to complete by default This is set automatically by the cluster according to whether SBD is detected to be in use. User-configured values are ignored. The value `true` is meaningful if diskless SBD is used and `stonith-watchdog-timeout` is nonzero. In that case, if fencing is required, watchdog-based self-fencing will be performed via SBD without requiring a fencing resource explicitly configured. Whether watchdog integration is enabled If this is set to a positive value, lost nodes are assumed to achieve self-fencing using watchdog-based SBD within this much time. This does not require a fencing resource to be explicitly configured, though a fence_watchdog resource can be configured, to limit use to specific nodes. If this is set to 0 (the default), the cluster will never assume watchdog-based self-fencing. If this is set to a negative value, the cluster will use twice the local value of the `SBD_WATCHDOG_TIMEOUT` environment variable if that is positive, or otherwise treat this as 0. WARNING: When used, this timeout must be larger than `SBD_WATCHDOG_TIMEOUT` on all nodes that use watchdog-based SBD, and Pacemaker will refuse to start on any of those nodes where this is not true for the local value or SBD is not active. When this is set to a negative value, `SBD_WATCHDOG_TIMEOUT` must be set to the same value on all nodes that use SBD, otherwise data corruption or loss could occur. How long before nodes can be assumed to be safely down when watchdog-based self-fencing via SBD is in use How many times fencing can fail before it will no longer be immediately re-attempted on a target How many times fencing can fail before it will no longer be immediately re-attempted on a target Allow performing fencing operations in parallel Allow performing fencing operations in parallel Setting this to false may lead to a "split-brain" situation, potentially leading to data loss and/or service unavailability. Whether to fence unseen nodes at start-up Apply specified delay for the fencings that are targeting the lost nodes with the highest total resource priority in case we don't have the majority of the nodes in our cluster partition, so that the more significant nodes potentially win any fencing match, which is especially meaningful under split-brain of 2-node cluster. A promoted resource instance takes the base priority + 1 on calculation if the base priority is not 0. Any static/random delays that are introduced by `pcmk_delay_base/max` configured for the corresponding fencing resources will be added to this delay. This delay should be significantly greater than, safely twice, the maximum `pcmk_delay_base/max`. By default, priority fencing delay is disabled. Apply fencing delay targeting the lost nodes with the highest total resource priority Fence nodes that do not join the controller process group within this much time after joining the cluster, to allow the cluster to continue managing resources. A value of 0 means never fence pending nodes. Setting the value to 2h means fence nodes after 2 hours. How long to wait for a node that has joined the cluster to join the controller process group The node elected Designated Controller (DC) will consider an action failed if it does not get a response from the node executing the action within this time (after considering the action's own timeout). The "correct" value will depend on the speed and load of your network and cluster nodes. Maximum time for node-to-node communication The cluster will slow down its recovery process when the amount of system resources used (currently CPU) approaches this limit Maximum amount of system load that should be used by cluster nodes Maximum number of jobs that can be scheduled per node (defaults to 2x cores) Maximum number of jobs that can be scheduled per node (defaults to 2x cores) The "correct" value will depend on the speed and load of your network and cluster nodes. If set to 0, the cluster will impose a dynamically calculated limit when any node has a high load. Maximum number of jobs that the cluster may execute in parallel across all nodes The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) - Raise this if log has "Evicting client" messages for cluster daemon PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). - Maximum IPC message backlog before disconnecting a cluster daemon + Raise this if log has "Evicting client" messages for cluster PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). + Maximum IPC message backlog before disconnecting a command line client Whether the cluster should stop all active resources Whether the cluster should stop all active resources Whether to stop resources that were removed from the configuration Whether to stop resources that were removed from the configuration Whether to cancel recurring actions removed from the configuration Whether to cancel recurring actions removed from the configuration Zero to disable, -1 to store unlimited. The number of scheduler inputs resulting in errors to save Zero to disable, -1 to store unlimited. The number of scheduler inputs resulting in warnings to save Zero to disable, -1 to store unlimited. The number of scheduler inputs without errors or warnings to save Requires external entities to create node attributes (named with the prefix "#health") with values "red", "yellow", or "green". How cluster should react to node health attributes Only used when "node-health-strategy" is set to "progressive". Base health score assigned to a node Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "green" Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "yellow" Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "red" How the cluster should allocate resources to nodes How the cluster should allocate resources to nodes =#=#=#= End test: List all available cluster options (XML) - OK (0) =#=#=#= * Passed: crm_attribute - List all available cluster options (XML) =#=#=#= Begin test: Return usage error if both -p and OCF_RESOURCE_INSTANCE are empty strings =#=#=#= crm_attribute: -p/--promotion must be called from an OCF resource agent or with a resource ID specified =#=#=#= End test: Return usage error if both -p and OCF_RESOURCE_INSTANCE are empty strings - Incorrect usage (64) =#=#=#= * Passed: crm_attribute - Return usage error if both -p and OCF_RESOURCE_INSTANCE are empty strings =#=#=#= Begin test: Query the value of an attribute that does not exist =#=#=#= crm_attribute: Error performing operation: No such device or address =#=#=#= End test: Query the value of an attribute that does not exist - No such object (105) =#=#=#= * Passed: crm_attribute - Query the value of an attribute that does not exist =#=#=#= Begin test: Configure something before erasing =#=#=#= =#=#=#= Current cib after: Configure something before erasing =#=#=#= =#=#=#= End test: Configure something before erasing - OK (0) =#=#=#= * Passed: crm_attribute - Configure something before erasing =#=#=#= Begin test: Test '++' XML attribute update syntax =#=#=#= =#=#=#= Current cib after: Test '++' XML attribute update syntax =#=#=#= =#=#=#= End test: Test '++' XML attribute update syntax - OK (0) =#=#=#= * Passed: cibadmin - Test '++' XML attribute update syntax =#=#=#= Begin test: Test '+=' XML attribute update syntax =#=#=#= =#=#=#= Current cib after: Test '+=' XML attribute update syntax =#=#=#= =#=#=#= End test: Test '+=' XML attribute update syntax - OK (0) =#=#=#= * Passed: cibadmin - Test '+=' XML attribute update syntax =#=#=#= Begin test: Test '++' nvpair value update syntax =#=#=#= =#=#=#= Current cib after: Test '++' nvpair value update syntax =#=#=#= =#=#=#= End test: Test '++' nvpair value update syntax - OK (0) =#=#=#= * Passed: crm_attribute - Test '++' nvpair value update syntax =#=#=#= Begin test: Test '++' nvpair value update syntax (XML) =#=#=#= =#=#=#= Current cib after: Test '++' nvpair value update syntax (XML) =#=#=#= =#=#=#= End test: Test '++' nvpair value update syntax (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Test '++' nvpair value update syntax (XML) =#=#=#= Begin test: Test '+=' nvpair value update syntax =#=#=#= =#=#=#= Current cib after: Test '+=' nvpair value update syntax =#=#=#= =#=#=#= End test: Test '+=' nvpair value update syntax - OK (0) =#=#=#= * Passed: crm_attribute - Test '+=' nvpair value update syntax =#=#=#= Begin test: Test '+=' nvpair value update syntax (XML) =#=#=#= =#=#=#= Current cib after: Test '+=' nvpair value update syntax (XML) =#=#=#= =#=#=#= End test: Test '+=' nvpair value update syntax (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Test '+=' nvpair value update syntax (XML) =#=#=#= Begin test: Test '++' XML attribute update syntax (--score not set) =#=#=#= =#=#=#= Current cib after: Test '++' XML attribute update syntax (--score not set) =#=#=#= =#=#=#= End test: Test '++' XML attribute update syntax (--score not set) - OK (0) =#=#=#= * Passed: cibadmin - Test '++' XML attribute update syntax (--score not set) =#=#=#= Begin test: Test '+=' XML attribute update syntax (--score not set) =#=#=#= =#=#=#= Current cib after: Test '+=' XML attribute update syntax (--score not set) =#=#=#= =#=#=#= End test: Test '+=' XML attribute update syntax (--score not set) - OK (0) =#=#=#= * Passed: cibadmin - Test '+=' XML attribute update syntax (--score not set) =#=#=#= Begin test: Test '++' nvpair value update syntax (--score not set) =#=#=#= =#=#=#= Current cib after: Test '++' nvpair value update syntax (--score not set) =#=#=#= =#=#=#= End test: Test '++' nvpair value update syntax (--score not set) - OK (0) =#=#=#= * Passed: crm_attribute - Test '++' nvpair value update syntax (--score not set) =#=#=#= Begin test: Test '++' nvpair value update syntax (--score not set) (XML) =#=#=#= =#=#=#= Current cib after: Test '++' nvpair value update syntax (--score not set) (XML) =#=#=#= =#=#=#= End test: Test '++' nvpair value update syntax (--score not set) (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Test '++' nvpair value update syntax (--score not set) (XML) =#=#=#= Begin test: Test '+=' nvpair value update syntax (--score not set) =#=#=#= =#=#=#= Current cib after: Test '+=' nvpair value update syntax (--score not set) =#=#=#= =#=#=#= End test: Test '+=' nvpair value update syntax (--score not set) - OK (0) =#=#=#= * Passed: crm_attribute - Test '+=' nvpair value update syntax (--score not set) =#=#=#= Begin test: Test '+=' nvpair value update syntax (--score not set) (XML) =#=#=#= =#=#=#= Current cib after: Test '+=' nvpair value update syntax (--score not set) (XML) =#=#=#= =#=#=#= End test: Test '+=' nvpair value update syntax (--score not set) (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Test '+=' nvpair value update syntax (--score not set) (XML) =#=#=#= Begin test: Set cluster option =#=#=#= =#=#=#= Current cib after: Set cluster option =#=#=#= =#=#=#= End test: Set cluster option - OK (0) =#=#=#= * Passed: crm_attribute - Set cluster option =#=#=#= Begin test: Query new cluster option =#=#=#= =#=#=#= End test: Query new cluster option - OK (0) =#=#=#= * Passed: cibadmin - Query new cluster option =#=#=#= Begin test: Set no-quorum policy =#=#=#= =#=#=#= Current cib after: Set no-quorum policy =#=#=#= =#=#=#= End test: Set no-quorum policy - OK (0) =#=#=#= * Passed: crm_attribute - Set no-quorum policy =#=#=#= Begin test: Delete nvpair =#=#=#= =#=#=#= Current cib after: Delete nvpair =#=#=#= =#=#=#= End test: Delete nvpair - OK (0) =#=#=#= * Passed: cibadmin - Delete nvpair =#=#=#= Begin test: Create operation should fail =#=#=#= cibadmin: CIB API call failed: File exists =#=#=#= Current cib after: Create operation should fail =#=#=#= =#=#=#= End test: Create operation should fail - Requested item already exists (108) =#=#=#= * Passed: cibadmin - Create operation should fail =#=#=#= Begin test: Modify cluster options section =#=#=#= =#=#=#= Current cib after: Modify cluster options section =#=#=#= =#=#=#= End test: Modify cluster options section - OK (0) =#=#=#= * Passed: cibadmin - Modify cluster options section =#=#=#= Begin test: Query updated cluster option =#=#=#= =#=#=#= Current cib after: Query updated cluster option =#=#=#= =#=#=#= End test: Query updated cluster option - OK (0) =#=#=#= * Passed: cibadmin - Query updated cluster option =#=#=#= Begin test: Set duplicate cluster option =#=#=#= =#=#=#= Current cib after: Set duplicate cluster option =#=#=#= =#=#=#= End test: Set duplicate cluster option - OK (0) =#=#=#= * Passed: crm_attribute - Set duplicate cluster option =#=#=#= Begin test: Setting multiply defined cluster option should fail =#=#=#= crm_attribute: Please choose from one of the matches below and supply the 'id' with --attr-id Multiple attributes match name=cluster-delay Value: 60s (id=cib-bootstrap-options-cluster-delay) Value: 40s (id=duplicate-cluster-delay) =#=#=#= Current cib after: Setting multiply defined cluster option should fail =#=#=#= =#=#=#= End test: Setting multiply defined cluster option should fail - Multiple items match request (109) =#=#=#= * Passed: crm_attribute - Setting multiply defined cluster option should fail =#=#=#= Begin test: Set cluster option with -s =#=#=#= =#=#=#= Current cib after: Set cluster option with -s =#=#=#= =#=#=#= End test: Set cluster option with -s - OK (0) =#=#=#= * Passed: crm_attribute - Set cluster option with -s =#=#=#= Begin test: Delete cluster option with -i =#=#=#= Deleted crm_config option: id=(null) name=cluster-delay =#=#=#= Current cib after: Delete cluster option with -i =#=#=#= =#=#=#= End test: Delete cluster option with -i - OK (0) =#=#=#= * Passed: crm_attribute - Delete cluster option with -i =#=#=#= Begin test: Create node1 and bring it online =#=#=#= unpack_resources error: Resource start-up disabled since no STONITH resources have been defined unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity unpack_resources error: Resource start-up disabled since no STONITH resources have been defined unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity unpack_resources error: Resource start-up disabled since no STONITH resources have been defined unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity Current cluster status: * Full List of Resources: * No resources Performing Requested Modifications: * Bringing node node1 online Transition Summary: Executing Cluster Transition: Revised Cluster Status: * Node List: * Online: [ node1 ] * Full List of Resources: * No resources =#=#=#= Current cib after: Create node1 and bring it online =#=#=#= =#=#=#= End test: Create node1 and bring it online - OK (0) =#=#=#= * Passed: crm_simulate - Create node1 and bring it online =#=#=#= Begin test: Create node attribute =#=#=#= =#=#=#= Current cib after: Create node attribute =#=#=#= =#=#=#= End test: Create node attribute - OK (0) =#=#=#= * Passed: crm_attribute - Create node attribute =#=#=#= Begin test: Query new node attribute =#=#=#= =#=#=#= Current cib after: Query new node attribute =#=#=#= =#=#=#= End test: Query new node attribute - OK (0) =#=#=#= * Passed: cibadmin - Query new node attribute =#=#=#= Begin test: Create second node attribute =#=#=#= =#=#=#= Current cib after: Create second node attribute =#=#=#= =#=#=#= End test: Create second node attribute - OK (0) =#=#=#= * Passed: crm_attribute - Create second node attribute =#=#=#= Begin test: Query node attributes by pattern =#=#=#= scope=nodes name=ram value=1024M scope=nodes name=rattr value=XYZ =#=#=#= End test: Query node attributes by pattern - OK (0) =#=#=#= * Passed: crm_attribute - Query node attributes by pattern =#=#=#= Begin test: Update node attributes by pattern =#=#=#= =#=#=#= Current cib after: Update node attributes by pattern =#=#=#= =#=#=#= End test: Update node attributes by pattern - OK (0) =#=#=#= * Passed: crm_attribute - Update node attributes by pattern =#=#=#= Begin test: Delete node attributes by pattern =#=#=#= Deleted nodes attribute: id=nodes-node1-rattr name=rattr =#=#=#= Current cib after: Delete node attributes by pattern =#=#=#= =#=#=#= End test: Delete node attributes by pattern - OK (0) =#=#=#= * Passed: crm_attribute - Delete node attributes by pattern =#=#=#= Begin test: Set a transient (fail-count) node attribute =#=#=#= =#=#=#= Current cib after: Set a transient (fail-count) node attribute =#=#=#= =#=#=#= End test: Set a transient (fail-count) node attribute - OK (0) =#=#=#= * Passed: crm_attribute - Set a transient (fail-count) node attribute =#=#=#= Begin test: Query a fail count =#=#=#= scope=status name=fail-count-foo value=3 =#=#=#= Current cib after: Query a fail count =#=#=#= =#=#=#= End test: Query a fail count - OK (0) =#=#=#= * Passed: crm_failcount - Query a fail count =#=#=#= Begin test: Show node attributes with crm_simulate =#=#=#= unpack_resources error: Resource start-up disabled since no STONITH resources have been defined unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity Current cluster status: * Node List: * Online: [ node1 ] * Full List of Resources: * No resources * Node Attributes: * Node: node1: * ram : 1024M =#=#=#= End test: Show node attributes with crm_simulate - OK (0) =#=#=#= * Passed: crm_simulate - Show node attributes with crm_simulate =#=#=#= Begin test: Set a second transient node attribute =#=#=#= =#=#=#= Current cib after: Set a second transient node attribute =#=#=#= =#=#=#= End test: Set a second transient node attribute - OK (0) =#=#=#= * Passed: crm_attribute - Set a second transient node attribute =#=#=#= Begin test: Query transient node attributes by pattern =#=#=#= scope=status name=fail-count-foo value=3 scope=status name=fail-count-bar value=5 =#=#=#= End test: Query transient node attributes by pattern - OK (0) =#=#=#= * Passed: crm_attribute - Query transient node attributes by pattern =#=#=#= Begin test: Update transient node attributes by pattern =#=#=#= =#=#=#= Current cib after: Update transient node attributes by pattern =#=#=#= =#=#=#= End test: Update transient node attributes by pattern - OK (0) =#=#=#= * Passed: crm_attribute - Update transient node attributes by pattern =#=#=#= Begin test: Delete transient node attributes by pattern =#=#=#= Deleted status attribute: id=status-node1-fail-count-foo name=fail-count-foo Deleted status attribute: id=status-node1-fail-count-bar name=fail-count-bar =#=#=#= Current cib after: Delete transient node attributes by pattern =#=#=#= =#=#=#= End test: Delete transient node attributes by pattern - OK (0) =#=#=#= * Passed: crm_attribute - Delete transient node attributes by pattern =#=#=#= Begin test: crm_attribute given invalid delete usage =#=#=#= crm_attribute: Error: must specify attribute name or pattern to delete =#=#=#= End test: crm_attribute given invalid delete usage - Incorrect usage (64) =#=#=#= * Passed: crm_attribute - crm_attribute given invalid delete usage =#=#=#= Begin test: Set a utilization node attribute =#=#=#= =#=#=#= Current cib after: Set a utilization node attribute =#=#=#= =#=#=#= End test: Set a utilization node attribute - OK (0) =#=#=#= * Passed: crm_attribute - Set a utilization node attribute =#=#=#= Begin test: Query utilization node attribute =#=#=#= scope=nodes name=cpu value=1 =#=#=#= End test: Query utilization node attribute - OK (0) =#=#=#= * Passed: crm_attribute - Query utilization node attribute =#=#=#= Begin test: Replace operation should fail =#=#=#= cibadmin: CIB API call failed: Update was older than existing configuration =#=#=#= End test: Replace operation should fail - Update was older than existing configuration (103) =#=#=#= * Passed: cibadmin - Replace operation should fail =#=#=#= Begin test: Query a nonexistent promotable score attribute =#=#=#= crm_attribute: Error performing operation: No such device or address =#=#=#= End test: Query a nonexistent promotable score attribute - No such object (105) =#=#=#= * Passed: crm_attribute - Query a nonexistent promotable score attribute =#=#=#= Begin test: Query a nonexistent promotable score attribute (XML) =#=#=#= crm_attribute: Error performing operation: No such device or address =#=#=#= End test: Query a nonexistent promotable score attribute (XML) - No such object (105) =#=#=#= * Passed: crm_attribute - Query a nonexistent promotable score attribute (XML) =#=#=#= Begin test: Delete a nonexistent promotable score attribute =#=#=#= =#=#=#= End test: Delete a nonexistent promotable score attribute - OK (0) =#=#=#= * Passed: crm_attribute - Delete a nonexistent promotable score attribute =#=#=#= Begin test: Delete a nonexistent promotable score attribute (XML) =#=#=#= =#=#=#= End test: Delete a nonexistent promotable score attribute (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Delete a nonexistent promotable score attribute (XML) =#=#=#= Begin test: Query after deleting a nonexistent promotable score attribute =#=#=#= crm_attribute: Error performing operation: No such device or address =#=#=#= End test: Query after deleting a nonexistent promotable score attribute - No such object (105) =#=#=#= * Passed: crm_attribute - Query after deleting a nonexistent promotable score attribute =#=#=#= Begin test: Query after deleting a nonexistent promotable score attribute (XML) =#=#=#= crm_attribute: Error performing operation: No such device or address =#=#=#= End test: Query after deleting a nonexistent promotable score attribute (XML) - No such object (105) =#=#=#= * Passed: crm_attribute - Query after deleting a nonexistent promotable score attribute (XML) =#=#=#= Begin test: Update a nonexistent promotable score attribute =#=#=#= =#=#=#= End test: Update a nonexistent promotable score attribute - OK (0) =#=#=#= * Passed: crm_attribute - Update a nonexistent promotable score attribute =#=#=#= Begin test: Update a nonexistent promotable score attribute (XML) =#=#=#= =#=#=#= End test: Update a nonexistent promotable score attribute (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Update a nonexistent promotable score attribute (XML) =#=#=#= Begin test: Query after updating a nonexistent promotable score attribute =#=#=#= scope=status name=master-promotable-rsc value=1 =#=#=#= End test: Query after updating a nonexistent promotable score attribute - OK (0) =#=#=#= * Passed: crm_attribute - Query after updating a nonexistent promotable score attribute =#=#=#= Begin test: Query after updating a nonexistent promotable score attribute (XML) =#=#=#= =#=#=#= End test: Query after updating a nonexistent promotable score attribute (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Query after updating a nonexistent promotable score attribute (XML) =#=#=#= Begin test: Update an existing promotable score attribute =#=#=#= =#=#=#= End test: Update an existing promotable score attribute - OK (0) =#=#=#= * Passed: crm_attribute - Update an existing promotable score attribute =#=#=#= Begin test: Update an existing promotable score attribute (XML) =#=#=#= =#=#=#= End test: Update an existing promotable score attribute (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Update an existing promotable score attribute (XML) =#=#=#= Begin test: Query after updating an existing promotable score attribute =#=#=#= scope=status name=master-promotable-rsc value=5 =#=#=#= End test: Query after updating an existing promotable score attribute - OK (0) =#=#=#= * Passed: crm_attribute - Query after updating an existing promotable score attribute =#=#=#= Begin test: Query after updating an existing promotable score attribute (XML) =#=#=#= =#=#=#= End test: Query after updating an existing promotable score attribute (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Query after updating an existing promotable score attribute (XML) =#=#=#= Begin test: Delete an existing promotable score attribute =#=#=#= Deleted status attribute: id=status-1-master-promotable-rsc name=master-promotable-rsc =#=#=#= End test: Delete an existing promotable score attribute - OK (0) =#=#=#= * Passed: crm_attribute - Delete an existing promotable score attribute =#=#=#= Begin test: Delete an existing promotable score attribute (XML) =#=#=#= =#=#=#= End test: Delete an existing promotable score attribute (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Delete an existing promotable score attribute (XML) =#=#=#= Begin test: Query after deleting an existing promotable score attribute =#=#=#= crm_attribute: Error performing operation: No such device or address =#=#=#= End test: Query after deleting an existing promotable score attribute - No such object (105) =#=#=#= * Passed: crm_attribute - Query after deleting an existing promotable score attribute =#=#=#= Begin test: Query after deleting an existing promotable score attribute (XML) =#=#=#= crm_attribute: Error performing operation: No such device or address =#=#=#= End test: Query after deleting an existing promotable score attribute (XML) - No such object (105) =#=#=#= * Passed: crm_attribute - Query after deleting an existing promotable score attribute (XML) =#=#=#= Begin test: Update a promotable score attribute to -INFINITY =#=#=#= =#=#=#= End test: Update a promotable score attribute to -INFINITY - OK (0) =#=#=#= * Passed: crm_attribute - Update a promotable score attribute to -INFINITY =#=#=#= Begin test: Update a promotable score attribute to -INFINITY (XML) =#=#=#= =#=#=#= End test: Update a promotable score attribute to -INFINITY (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Update a promotable score attribute to -INFINITY (XML) =#=#=#= Begin test: Query after updating a promotable score attribute to -INFINITY =#=#=#= scope=status name=master-promotable-rsc value=-INFINITY =#=#=#= End test: Query after updating a promotable score attribute to -INFINITY - OK (0) =#=#=#= * Passed: crm_attribute - Query after updating a promotable score attribute to -INFINITY =#=#=#= Begin test: Query after updating a promotable score attribute to -INFINITY (XML) =#=#=#= =#=#=#= End test: Query after updating a promotable score attribute to -INFINITY (XML) - OK (0) =#=#=#= * Passed: crm_attribute - Query after updating a promotable score attribute to -INFINITY (XML) =#=#=#= Begin test: Try OCF_RESOURCE_INSTANCE if -p is specified with an empty string =#=#=#= scope=status name=master-promotable-rsc value=-INFINITY =#=#=#= End test: Try OCF_RESOURCE_INSTANCE if -p is specified with an empty string - OK (0) =#=#=#= * Passed: crm_attribute - Try OCF_RESOURCE_INSTANCE if -p is specified with an empty string diff --git a/cts/cli/regression.daemons.exp b/cts/cli/regression.daemons.exp index ea3e363e6a..198ca28439 100644 --- a/cts/cli/regression.daemons.exp +++ b/cts/cli/regression.daemons.exp @@ -1,750 +1,750 @@ =#=#=#= Begin test: Get CIB manager metadata =#=#=#= 1.1 Cluster options used by Pacemaker's Cluster Information Base manager Cluster Information Base manager options Enable Access Control Lists (ACLs) for the CIB Enable Access Control Lists (ACLs) for the CIB - Raise this if log has "Evicting client" messages for cluster daemon PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). + Raise this if log has "Evicting client" messages for cluster PIDs (a good value is the number of resources in the cluster multiplied by the number of nodes). - Maximum IPC message backlog before disconnecting a cluster daemon + Maximum IPC message backlog before disconnecting a command line client =#=#=#= End test: Get CIB manager metadata - OK (0) =#=#=#= * Passed: pacemaker-based - Get CIB manager metadata =#=#=#= Begin test: Get controller metadata =#=#=#= 1.1 Cluster options used by Pacemaker's controller Pacemaker controller options Includes a hash which identifies the exact revision the code was built from. Used for diagnostic purposes. Pacemaker version on cluster node elected Designated Controller (DC) Used for informational and diagnostic purposes. The messaging layer on which Pacemaker is currently running This optional value is mostly for users' convenience as desired in administration, but may also be used in Pacemaker configuration rules via the #cluster-name node attribute, and by higher-level tools and resource agents. An arbitrary name for the cluster The optimal value will depend on the speed and load of your network and the type of switches used. How long to wait for a response from other nodes during start-up Pacemaker is primarily event-driven, and looks ahead to know when to recheck cluster state for failure-timeout settings and most time-based rules. However, it will also recheck the cluster after this amount of inactivity, to evaluate rules with date specifications and serve as a fail-safe for certain types of scheduler bugs. A value of 0 disables polling. A positive value sets an interval in seconds, unless other units are specified (for example, "5min"). Polling interval to recheck cluster state and evaluate rules with date specifications A cluster node may receive notification of a "succeeded" fencing that targeted it if fencing is misconfigured, or if fabric fencing is in use that doesn't cut cluster communication. Use "stop" to attempt to immediately stop Pacemaker and stay stopped, or "panic" to attempt to immediately reboot the local node, falling back to stop on failure. Allowed values: stop, panic How a cluster node should react if notified of its own fencing Declare an election failed if it is not decided within this much time. If you need to adjust this value, it probably indicates the presence of a bug. *** Advanced Use Only *** Exit immediately if shutdown does not complete within this much time. If you need to adjust this value, it probably indicates the presence of a bug. *** Advanced Use Only *** If you need to adjust this value, it probably indicates the presence of a bug. *** Advanced Use Only *** If you need to adjust this value, it probably indicates the presence of a bug. *** Advanced Use Only *** Delay cluster recovery for this much time to allow for additional events to occur. Useful if your configuration is sensitive to the order in which ping updates arrive. *** Advanced Use Only *** Enabling this option will slow down cluster recovery under all conditions If this is set to a positive value, lost nodes are assumed to achieve self-fencing using watchdog-based SBD within this much time. This does not require a fencing resource to be explicitly configured, though a fence_watchdog resource can be configured, to limit use to specific nodes. If this is set to 0 (the default), the cluster will never assume watchdog-based self-fencing. If this is set to a negative value, the cluster will use twice the local value of the `SBD_WATCHDOG_TIMEOUT` environment variable if that is positive, or otherwise treat this as 0. WARNING: When used, this timeout must be larger than `SBD_WATCHDOG_TIMEOUT` on all nodes that use watchdog-based SBD, and Pacemaker will refuse to start on any of those nodes where this is not true for the local value or SBD is not active. When this is set to a negative value, `SBD_WATCHDOG_TIMEOUT` must be set to the same value on all nodes that use SBD, otherwise data corruption or loss could occur. How long before nodes can be assumed to be safely down when watchdog-based self-fencing via SBD is in use How many times fencing can fail before it will no longer be immediately re-attempted on a target How many times fencing can fail before it will no longer be immediately re-attempted on a target The cluster will slow down its recovery process when the amount of system resources used (currently CPU) approaches this limit Maximum amount of system load that should be used by cluster nodes Maximum number of jobs that can be scheduled per node (defaults to 2x cores) Maximum number of jobs that can be scheduled per node (defaults to 2x cores) =#=#=#= End test: Get controller metadata - OK (0) =#=#=#= * Passed: pacemaker-controld - Get controller metadata =#=#=#= Begin test: Get fencer metadata =#=#=#= 1.1 Instance attributes available for all "stonith"-class resources and used by Pacemaker's fence daemon Instance attributes available for all "stonith"-class resources If the fencing agent metadata advertises support for the "port" or "plug" parameter, that will be used as the default, otherwise "none" will be used, which tells the cluster not to supply any additional parameters. *** Advanced Use Only *** Name of agent parameter that should be set to the fencing target For example, "node1:1;node2:2,3" would tell the cluster to use port 1 for node1 and ports 2 and 3 for node2. A mapping of node names to port numbers for devices that do not support node names. Comma-separated list of nodes that can be targeted by this device (for example, "node1,node2,node3"). If pcmk_host_check is "static-list", either this or pcmk_host_map must be set. Nodes targeted by this device Use "dynamic-list" to query the device via the 'list' command; "static-list" to check the pcmk_host_list attribute; "status" to query the device via the 'status' command; or "none" to assume every device can fence every node. The default value is "static-list" if pcmk_host_map or pcmk_host_list is set; otherwise "dynamic-list" if the device supports the list operation; otherwise "status" if the device supports the status operation; otherwise "none" Allowed values: dynamic-list, static-list, status, none How to determine which nodes can be targeted by the device Enable a delay of no more than the time specified before executing fencing actions. Pacemaker derives the overall delay by taking the value of pcmk_delay_base and adding a random delay value such that the sum is kept below this maximum. Enable a delay of no more than the time specified before executing fencing actions. This enables a static delay for fencing actions, which can help avoid "death matches" where two nodes try to fence each other at the same time. If pcmk_delay_max is also used, a random delay will be added such that the total delay is kept below that value. This can be set to a single time value to apply to any node targeted by this device (useful if a separate device is configured for each target), or to a node map (for example, "node1:1s;node2:5") to set a different value for each target. Enable a base delay for fencing actions and specify base delay value. If the concurrent-fencing cluster property is "true", this specifies the maximum number of actions that can be performed in parallel on this device. A value of -1 means unlimited. The maximum number of actions can be performed in parallel on this device Some devices do not support the standard commands or may provide additional ones. Use this to specify an alternate, device-specific, command that implements the 'reboot' action. *** Advanced Use Only *** An alternate command to run instead of 'reboot' Some devices need much more/less time to complete than normal. Use this to specify an alternate, device-specific, timeout for 'reboot' actions. *** Advanced Use Only *** Specify an alternate timeout to use for 'reboot' actions instead of stonith-timeout Some devices do not support multiple connections. Operations may "fail" if the device is busy with another task. In that case, Pacemaker will automatically retry the operation if there is time remaining. Use this option to alter the number of times Pacemaker tries a 'reboot' action before giving up. *** Advanced Use Only *** The maximum number of times to try the 'reboot' command within the timeout period Some devices do not support the standard commands or may provide additional ones. Use this to specify an alternate, device-specific, command that implements the 'off' action. *** Advanced Use Only *** An alternate command to run instead of 'off' Some devices need much more/less time to complete than normal. Use this to specify an alternate, device-specific, timeout for 'off' actions. *** Advanced Use Only *** Specify an alternate timeout to use for 'off' actions instead of stonith-timeout Some devices do not support multiple connections. Operations may "fail" if the device is busy with another task. In that case, Pacemaker will automatically retry the operation if there is time remaining. Use this option to alter the number of times Pacemaker tries a 'off' action before giving up. *** Advanced Use Only *** The maximum number of times to try the 'off' command within the timeout period Some devices do not support the standard commands or may provide additional ones. Use this to specify an alternate, device-specific, command that implements the 'on' action. *** Advanced Use Only *** An alternate command to run instead of 'on' Some devices need much more/less time to complete than normal. Use this to specify an alternate, device-specific, timeout for 'on' actions. *** Advanced Use Only *** Specify an alternate timeout to use for 'on' actions instead of stonith-timeout Some devices do not support multiple connections. Operations may "fail" if the device is busy with another task. In that case, Pacemaker will automatically retry the operation if there is time remaining. Use this option to alter the number of times Pacemaker tries a 'on' action before giving up. *** Advanced Use Only *** The maximum number of times to try the 'on' command within the timeout period Some devices do not support the standard commands or may provide additional ones. Use this to specify an alternate, device-specific, command that implements the 'list' action. *** Advanced Use Only *** An alternate command to run instead of 'list' Some devices need much more/less time to complete than normal. Use this to specify an alternate, device-specific, timeout for 'list' actions. *** Advanced Use Only *** Specify an alternate timeout to use for 'list' actions instead of stonith-timeout Some devices do not support multiple connections. Operations may "fail" if the device is busy with another task. In that case, Pacemaker will automatically retry the operation if there is time remaining. Use this option to alter the number of times Pacemaker tries a 'list' action before giving up. *** Advanced Use Only *** The maximum number of times to try the 'list' command within the timeout period Some devices do not support the standard commands or may provide additional ones. Use this to specify an alternate, device-specific, command that implements the 'monitor' action. *** Advanced Use Only *** An alternate command to run instead of 'monitor' Some devices need much more/less time to complete than normal. Use this to specify an alternate, device-specific, timeout for 'monitor' actions. *** Advanced Use Only *** Specify an alternate timeout to use for 'monitor' actions instead of stonith-timeout Some devices do not support multiple connections. Operations may "fail" if the device is busy with another task. In that case, Pacemaker will automatically retry the operation if there is time remaining. Use this option to alter the number of times Pacemaker tries a 'monitor' action before giving up. *** Advanced Use Only *** The maximum number of times to try the 'monitor' command within the timeout period Some devices do not support the standard commands or may provide additional ones. Use this to specify an alternate, device-specific, command that implements the 'status' action. *** Advanced Use Only *** An alternate command to run instead of 'status' Some devices need much more/less time to complete than normal. Use this to specify an alternate, device-specific, timeout for 'status' actions. *** Advanced Use Only *** Specify an alternate timeout to use for 'status' actions instead of stonith-timeout Some devices do not support multiple connections. Operations may "fail" if the device is busy with another task. In that case, Pacemaker will automatically retry the operation if there is time remaining. Use this option to alter the number of times Pacemaker tries a 'status' action before giving up. *** Advanced Use Only *** The maximum number of times to try the 'status' command within the timeout period =#=#=#= End test: Get fencer metadata - OK (0) =#=#=#= * Passed: pacemaker-fenced - Get fencer metadata =#=#=#= Begin test: Get scheduler metadata =#=#=#= 1.1 Cluster options used by Pacemaker's scheduler Pacemaker scheduler options What to do when the cluster does not have quorum Allowed values: stop, freeze, ignore, demote, fence, suicide What to do when the cluster does not have quorum When true, resources active on a node when it is cleanly shut down are kept "locked" to that node (not allowed to run elsewhere) until they start again on that node after it rejoins (or for at most shutdown-lock-limit, if set). Stonith resources and Pacemaker Remote connections are never locked. Clone and bundle instances and the promoted role of promotable clones are currently never locked, though support could be added in a future release. Whether to lock resources to a cleanly shut down node If shutdown-lock is true and this is set to a nonzero time duration, shutdown locks will expire after this much time has passed since the shutdown was initiated, even if the node has not rejoined. Do not lock resources to a cleanly shut down node longer than this Whether resources can run on any node by default Whether resources can run on any node by default Whether the cluster should refrain from monitoring, starting, and stopping resources Whether the cluster should refrain from monitoring, starting, and stopping resources When true, the cluster will immediately ban a resource from a node if it fails to start there. When false, the cluster will instead check the resource's fail count against its migration-threshold. Whether a start failure should prevent a resource from being recovered on the same node Whether the cluster should check for active resources during start-up Whether the cluster should check for active resources during start-up By default, an inquorate node can not fence Pacemaker Remote nodes that are part of its partition as long as the cluster thinks they can be restarted. If true, inquorate nodes will be able to fence remote nodes regardless. *** Advanced Use Only *** Whether remote nodes can be fenced without quorum If false, unresponsive nodes are immediately assumed to be harmless, and resources that were active on them may be recovered elsewhere. This can result in a "split-brain" situation, potentially leading to data loss and/or service unavailability. *** Advanced Use Only *** Whether nodes may be fenced as part of recovery Action to send to fence device when a node needs to be fenced Allowed values: reboot, off Action to send to fence device when a node needs to be fenced How long to wait for on, off, and reboot fence actions to complete by default How long to wait for on, off, and reboot fence actions to complete by default This is set automatically by the cluster according to whether SBD is detected to be in use. User-configured values are ignored. The value `true` is meaningful if diskless SBD is used and `stonith-watchdog-timeout` is nonzero. In that case, if fencing is required, watchdog-based self-fencing will be performed via SBD without requiring a fencing resource explicitly configured. Whether watchdog integration is enabled Allow performing fencing operations in parallel *** Deprecated *** Setting this to false may lead to a "split-brain" situation, potentially leading to data loss and/or service unavailability. *** Advanced Use Only *** Whether to fence unseen nodes at start-up Apply specified delay for the fencings that are targeting the lost nodes with the highest total resource priority in case we don't have the majority of the nodes in our cluster partition, so that the more significant nodes potentially win any fencing match, which is especially meaningful under split-brain of 2-node cluster. A promoted resource instance takes the base priority + 1 on calculation if the base priority is not 0. Any static/random delays that are introduced by `pcmk_delay_base/max` configured for the corresponding fencing resources will be added to this delay. This delay should be significantly greater than, safely twice, the maximum `pcmk_delay_base/max`. By default, priority fencing delay is disabled. Apply fencing delay targeting the lost nodes with the highest total resource priority Fence nodes that do not join the controller process group within this much time after joining the cluster, to allow the cluster to continue managing resources. A value of 0 means never fence pending nodes. Setting the value to 2h means fence nodes after 2 hours. How long to wait for a node that has joined the cluster to join the controller process group The node elected Designated Controller (DC) will consider an action failed if it does not get a response from the node executing the action within this time (after considering the action's own timeout). The "correct" value will depend on the speed and load of your network and cluster nodes. Maximum time for node-to-node communication The "correct" value will depend on the speed and load of your network and cluster nodes. If set to 0, the cluster will impose a dynamically calculated limit when any node has a high load. Maximum number of jobs that the cluster may execute in parallel across all nodes The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) The number of live migration actions that the cluster is allowed to execute in parallel on a node (-1 means no limit) Whether the cluster should stop all active resources Whether the cluster should stop all active resources Whether to stop resources that were removed from the configuration Whether to stop resources that were removed from the configuration Whether to cancel recurring actions removed from the configuration Whether to cancel recurring actions removed from the configuration Zero to disable, -1 to store unlimited. The number of scheduler inputs resulting in errors to save Zero to disable, -1 to store unlimited. The number of scheduler inputs resulting in warnings to save Zero to disable, -1 to store unlimited. The number of scheduler inputs without errors or warnings to save Requires external entities to create node attributes (named with the prefix "#health") with values "red", "yellow", or "green". Allowed values: none, migrate-on-red, only-green, progressive, custom How cluster should react to node health attributes Only used when "node-health-strategy" is set to "progressive". Base health score assigned to a node Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "green" Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "yellow" Only used when "node-health-strategy" is set to "custom" or "progressive". The score to use for a node health attribute whose value is "red" How the cluster should allocate resources to nodes Allowed values: default, utilization, minimal, balanced How the cluster should allocate resources to nodes =#=#=#= End test: Get scheduler metadata - OK (0) =#=#=#= * Passed: pacemaker-schedulerd - Get scheduler metadata diff --git a/doc/sphinx/Pacemaker_Explained/cluster-options.rst b/doc/sphinx/Pacemaker_Explained/cluster-options.rst index 83f8a05a1c..070f486036 100644 --- a/doc/sphinx/Pacemaker_Explained/cluster-options.rst +++ b/doc/sphinx/Pacemaker_Explained/cluster-options.rst @@ -1,936 +1,938 @@ Cluster-Wide Configuration -------------------------- .. index:: pair: XML element; cib pair: XML element; configuration Configuration Layout #################### The cluster is defined by the Cluster Information Base (CIB), which uses XML notation. The simplest CIB, an empty one, looks like this: .. topic:: An empty configuration .. code-block:: xml The empty configuration above contains the major sections that make up a CIB: * ``cib``: The entire CIB is enclosed with a ``cib`` element. Certain fundamental settings are defined as attributes of this element. * ``configuration``: This section -- the primary focus of this document -- contains traditional configuration information such as what resources the cluster serves and the relationships among them. * ``crm_config``: cluster-wide configuration options * ``nodes``: the machines that host the cluster * ``resources``: the services run by the cluster * ``constraints``: indications of how resources should be placed * ``status``: This section contains the history of each resource on each node. Based on this data, the cluster can construct the complete current state of the cluster. The authoritative source for this section is the local executor (pacemaker-execd process) on each cluster node, and the cluster will occasionally repopulate the entire section. For this reason, it is never written to disk, and administrators are advised against modifying it in any way. In this document, configuration settings will be described as properties or options based on how they are defined in the CIB: * Properties are XML attributes of an XML element. * Options are name-value pairs expressed as ``nvpair`` child elements of an XML element. Normally, you will use command-line tools that abstract the XML, so the distinction will be unimportant; both properties and options are cluster settings you can tweak. Options can appear within four types of enclosing elements: * ``cluster_property_set`` * ``instance_attributes`` * ``meta_attributes`` * ``utilization`` We will refer to a set of options and its enclosing element as a *block*. .. list-table:: **Properties of an Option Block's Enclosing Element** :class: longtable :widths: 15 15 15 55 :header-rows: 1 * - Name - Type - Default - Description * - .. _option_block_id: .. index:: pair: id; cluster_property_set pair: id; instance_attributes pair: id; meta_attributes pair: id; utilization single: attribute; id (cluster_property_set) single: attribute; id (instance_attributes) single: attribute; id (meta_attributes) single: attribute; id (utilization) id - :ref:`id ` - - A unique name for the block (required) * - .. _option_block_score: .. index:: pair: score; cluster_property_set pair: score; instance_attributes pair: score; meta_attributes pair: score; utilization single: attribute; score (cluster_property_set) single: attribute; score (instance_attributes) single: attribute; score (meta_attributes) single: attribute; score (utilization) score - :ref:`score ` - 0 - Priority with which to process the block Each block may optionally contain a :ref:`rule `. .. _option_precedence: Option Precedence ################# This subsection describes the precedence of options within a set of blocks and within a single block. Options are processed as follows: * All option blocks of a given type are processed in order of their ``score`` attribute, from highest to lowest. For ``cluster_property_set``, if there is a block whose enclosing element has ``id="cib-bootstrap-options"``, then that block is always processed first regardless of score. * If a block contains a rule that evaluates to false, that block is skipped. * Within a block, options are processed in order from first to last. * The first value found for a given option is applied, and the rest are ignored. Note that this means it is pointless to configure the same option twice in a single block, because occurrences after the first one would be ignored. For example, in the following configuration snippet, the ``no-quorum-policy`` value ``demote`` is applied. ``property-set2`` has a higher score than ``property-set1``, so it's processed first. There are no rules in this snippet, so both sets are processed. Within ``property-set2``, the value ``demote`` appears first, so the later value ``freeze`` is ignored. We've already found a value for ``no-quorum-policy`` before we begin processing ``property-set1``, so its value ``stop`` is ignored. .. code-block:: xml CIB Properties ############## Certain settings are defined by CIB properties (that is, attributes of the ``cib`` tag) rather than with the rest of the cluster configuration in the ``configuration`` section. The reason is simply a matter of parsing. These options are used by the configuration database which is, by design, mostly ignorant of the content it holds. So the decision was made to place them in an easy-to-find location. .. list-table:: **CIB Properties** :class: longtable :widths: 20 15 10 55 :header-rows: 1 * - Name - Type - Default - Description * - .. _admin_epoch: .. index:: pair: admin_epoch; cib admin_epoch - :ref:`nonnegative integer ` - 0 - When a node joins the cluster, the cluster asks the node with the highest (``admin_epoch``, ``epoch``, ``num_updates``) tuple to replace the configuration on all the nodes -- which makes setting them correctly very important. ``admin_epoch`` is never modified by the cluster; you can use this to make the configurations on any inactive nodes obsolete. * - .. _epoch: .. index:: pair: epoch; cib epoch - :ref:`nonnegative integer ` - 0 - The cluster increments this every time the CIB's configuration section is updated. * - .. _num_updates: .. index:: pair: num_updates; cib num_updates - :ref:`nonnegative integer ` - 0 - The cluster increments this every time the CIB's configuration or status sections are updated, and resets it to 0 when epoch changes. * - .. _validate_with: .. index:: pair: validate-with; cib validate-with - :ref:`enumeration ` - - Determines the type of XML validation that will be done on the configuration. Allowed values are ``none`` (in which case the cluster will not require that updates conform to expected syntax) and the base names of schema files installed on the local machine (for example, "pacemaker-3.9") * - .. _remote_tls_port: .. index:: pair: remote-tls-port; cib remote-tls-port - :ref:`port ` - - If set, the CIB manager will listen for anonymously encrypted remote connections on this port, to allow CIB administration from hosts not in the cluster. No key is used, so this should be used only on a protected network where man-in-the-middle attacks can be avoided. * - .. _remote_clear_port: .. index:: pair: remote-clear-port; cib remote-clear-port - :ref:`port ` - - If set to a TCP port number, the CIB manager will listen for remote connections on this port, to allow for CIB administration from hosts not in the cluster. No encryption is used, so this should be used only on a protected network. * - .. _cib_last_written: .. index:: pair: cib-last-written; cib cib-last-written - :ref:`date/time ` - - Indicates when the configuration was last written to disk. Maintained by the cluster; for informational purposes only. * - .. _have_quorum: .. index:: pair: have-quorum; cib have-quorum - :ref:`boolean ` - - Indicates whether the cluster has quorum. If false, the cluster's response is determined by ``no-quorum-policy`` (see below). Maintained by the cluster. * - .. _dc_uuid: .. index:: pair: dc-uuid; cib dc-uuid - :ref:`text ` - - Node ID of the cluster's current designated controller (DC). Used and maintained by the cluster. * - .. _execution_date: .. index:: pair: execution-date; cib execution-date - :ref:`epoch time ` - - Time to use when evaluating rules. .. _cluster_options: Cluster Options ############### Cluster options, as you might expect, control how the cluster behaves when confronted with various situations. They are grouped into sets within the ``crm_config`` section. In advanced configurations, there may be more than one set. (This will be described later in the chapter on :ref:`rules` where we will show how to have the cluster use different sets of options during working hours than during weekends.) For now, we will describe the simple case where each option is present at most once. You can obtain an up-to-date list of cluster options, including their default values, by running the ``man pacemaker-schedulerd`` and ``man pacemaker-controld`` commands. .. list-table:: **Cluster Options** :class: longtable :widths: 25 13 12 50 :header-rows: 1 * - Name - Type - Default - Description * - .. _cluster_name: .. index:: pair: cluster option; cluster-name cluster-name - :ref:`text ` - - An (optional) name for the cluster as a whole. This is mostly for users' convenience for use as desired in administration, but can be used in the Pacemaker configuration in :ref:`rules` (as the ``#cluster-name`` :ref:`node attribute `). It may also be used by higher-level tools when displaying cluster information, and by certain resource agents (for example, the ``ocf:heartbeat:GFS2`` agent stores the cluster name in filesystem meta-data). * - .. _dc_version: .. index:: pair: cluster option; dc-version dc-version - :ref:`version ` - *detected* - Version of Pacemaker on the cluster's designated controller (DC). Maintained by the cluster, and intended for diagnostic purposes. * - .. _cluster_infrastructure: .. index:: pair: cluster option; cluster-infrastructure cluster-infrastructure - :ref:`text ` - *detected* - The messaging layer with which Pacemaker is currently running. Maintained by the cluster, and intended for informational and diagnostic purposes. * - .. _no_quorum_policy: .. index:: pair: cluster option; no-quorum-policy no-quorum-policy - :ref:`enumeration ` - stop - What to do when the cluster does not have quorum. Allowed values: * ``ignore:`` continue all resource management * ``freeze:`` continue resource management, but don't recover resources from nodes not in the affected partition * ``stop:`` stop all resources in the affected cluster partition * ``demote:`` demote promotable resources and stop all other resources in the affected cluster partition *(since 2.0.5)* * ``fence:`` fence all nodes in the affected cluster partition *(since 2.1.9)* * ``suicide:`` same as ``fence`` *(deprecated since 2.1.9)* * - .. _batch_limit: .. index:: pair: cluster option; batch-limit batch-limit - :ref:`integer ` - 0 - The maximum number of actions that the cluster may execute in parallel across all nodes. The ideal value will depend on the speed and load of your network and cluster nodes. If zero, the cluster will impose a dynamically calculated limit only when any node has high load. If -1, the cluster will not impose any limit. * - .. _migration_limit: .. index:: pair: cluster option; migration-limit migration-limit - :ref:`integer ` - -1 - The number of :ref:`live migration ` actions that the cluster is allowed to execute in parallel on a node. A value of -1 means unlimited. * - .. _load_threshold: .. index:: pair: cluster option; load-threshold load-threshold - :ref:`percentage ` - 80% - Maximum amount of system load that should be used by cluster nodes. The cluster will slow down its recovery process when the amount of system resources used (currently CPU) approaches this limit. * - .. _node_action_limit: .. index:: pair: cluster option; node-action-limit node-action-limit - :ref:`integer ` - 0 - Maximum number of jobs that can be scheduled per node. If nonpositive or invalid, double the number of cores is used as the maximum number of jobs per node. :ref:`PCMK_node_action_limit ` overrides this option on a per-node basis. * - .. _symmetric_cluster: .. index:: pair: cluster option; symmetric-cluster symmetric-cluster - :ref:`boolean ` - true - If true, resources can run on any node by default. If false, a resource is allowed to run on a node only if a :ref:`location constraint ` enables it. * - .. _stop_all_resources: .. index:: pair: cluster option; stop-all-resources stop-all-resources - :ref:`boolean ` - false - Whether all resources should be disallowed from running (can be useful during maintenance or troubleshooting) * - .. _stop_removed_resources: .. index:: pair: cluster option; stop-removed-resources stop-removed-resources - :ref:`boolean ` - true - Whether resources that have been deleted from the configuration should be stopped. This value takes precedence over :ref:`is-managed ` (that is, even unmanaged resources will be stopped when removed if this value is ``true``). * - .. _stop_removed_actions: .. index:: pair: cluster option; stop-removed-actions stop-removed-actions - :ref:`boolean ` - true - Whether recurring :ref:`operations ` that have been deleted from the configuration should be cancelled * - .. _start_failure_is_fatal: .. index:: pair: cluster option; start-failure-is-fatal start-failure-is-fatal - :ref:`boolean ` - true - Whether a failure to start a resource on a particular node prevents further start attempts on that node. If ``false``, the cluster will decide whether the node is still eligible based on the resource's current failure count and ``migration-threshold``. * - .. _enable_startup_probes: .. index:: pair: cluster option; enable-startup-probes enable-startup-probes - :ref:`boolean ` - true - Whether the cluster should check the pre-existing state of resources when the cluster starts * - .. _maintenance_mode: .. index:: pair: cluster option; maintenance-mode maintenance-mode - :ref:`boolean ` - false - If true, the cluster will not start or stop any resource in the cluster, and any recurring operations (expect those specifying ``role`` as ``Stopped``) will be paused. If true, this overrides the :ref:`maintenance ` node attribute, :ref:`is-managed ` and :ref:`maintenance ` resource meta-attributes, and :ref:`enabled ` operation meta-attribute. * - .. _stonith_enabled: .. index:: pair: cluster option; stonith-enabled stonith-enabled - :ref:`boolean ` - true - Whether the cluster is allowed to fence nodes (for example, failed nodes and nodes with resources that can't be stopped). If true, at least one fence device must be configured before resources are allowed to run. If false, unresponsive nodes are immediately assumed to be running no resources, and resource recovery on online nodes starts without any further protection (which can mean *data loss* if the unresponsive node still accesses shared storage, for example). See also the :ref:`requires ` resource meta-attribute. This option applies only to fencing scheduled by the cluster, not to requests initiated externally (such as with the ``stonith_admin`` command-line tool). * - .. _stonith_action: .. index:: pair: cluster option; stonith-action stonith-action - :ref:`enumeration ` - reboot - Action the cluster should send to the fence agent when a node must be fenced. Allowed values are ``reboot`` and ``off``. * - .. _stonith_timeout: .. index:: pair: cluster option; stonith-timeout stonith-timeout - :ref:`duration ` - 60s - How long to wait for ``on``, ``off``, and ``reboot`` fence actions to complete by default. * - .. _stonith_max_attempts: .. index:: pair: cluster option; stonith-max-attempts stonith-max-attempts - :ref:`score ` - 10 - How many times fencing can fail for a target before the cluster will no longer immediately re-attempt it. Any value below 1 will be ignored, and the default will be used instead. * - .. _have_watchdog: .. index:: pair: cluster option; have-watchdog have-watchdog - :ref:`boolean ` - *detected* - Whether watchdog integration is enabled. This is set automatically by the cluster according to whether SBD is detected to be in use. User-configured values are ignored. The value `true` is meaningful if diskless SBD is used and :ref:`stonith-watchdog-timeout ` is nonzero. In that case, if fencing is required, watchdog-based self-fencing will be performed via SBD without requiring a fencing resource explicitly configured. * - .. _stonith_watchdog_timeout: .. index:: pair: cluster option; stonith-watchdog-timeout stonith-watchdog-timeout - :ref:`timeout ` - 0 - If nonzero, and the cluster detects ``have-watchdog`` as ``true``, then watchdog-based self-fencing will be performed via SBD when fencing is required. If this is set to a positive value, lost nodes are assumed to achieve self-fencing within this much time. This does not require a fencing resource to be explicitly configured, though a fence_watchdog resource can be configured, to limit use to specific nodes. If this is set to 0 (the default), the cluster will never assume watchdog-based self-fencing. If this is set to a negative value, the cluster will use twice the local value of the ``SBD_WATCHDOG_TIMEOUT`` environment variable if that is positive, or otherwise treat this as 0. **Warning:** When used, this timeout must be larger than ``SBD_WATCHDOG_TIMEOUT`` on all nodes that use watchdog-based SBD, and Pacemaker will refuse to start on any of those nodes where this is not true for the local value or SBD is not active. When this is set to a negative value, ``SBD_WATCHDOG_TIMEOUT`` must be set to the same value on all nodes that use SBD, otherwise data corruption or loss could occur. * - .. _concurrent-fencing: .. index:: pair: cluster option; concurrent-fencing concurrent-fencing - :ref:`boolean ` - false - Whether the cluster is allowed to initiate multiple fence actions concurrently. Fence actions initiated externally, such as via the ``stonith_admin`` tool or an application such as DLM, or by the fencer itself such as recurring device monitors and ``status`` and ``list`` commands, are not limited by this option. * - .. _fence_reaction: .. index:: pair: cluster option; fence-reaction fence-reaction - :ref:`enumeration ` - stop - How should a cluster node react if notified of its own fencing? A cluster node may receive notification of a "succeeded" fencing that targeted it if fencing is misconfigured, or if fabric fencing is in use that doesn't cut cluster communication. Allowed values are ``stop`` to attempt to immediately stop Pacemaker and stay stopped, or ``panic`` to attempt to immediately reboot the local node, falling back to stop on failure. The default is likely to be changed to ``panic`` in a future release. *(since 2.0.3)* * - .. _priority_fencing_delay: .. index:: pair: cluster option; priority-fencing-delay priority-fencing-delay - :ref:`duration ` - 0 - Apply this delay to any fencing targeting the lost nodes with the highest total resource priority in case we don't have the majority of the nodes in our cluster partition, so that the more significant nodes potentially win any fencing match (especially meaningful in a split-brain of a 2-node cluster). A promoted resource instance takes the resource's priority plus 1 if the resource's priority is not 0. Any static or random delays introduced by ``pcmk_delay_base`` and ``pcmk_delay_max`` configured for the corresponding fencing resources will be added to this delay. This delay should be significantly greater than (safely twice) the maximum delay from those parameters. *(since 2.0.4)* * - .. _node_pending_timeout: .. index:: pair: cluster option; node-pending-timeout node-pending-timeout - :ref:`duration ` - 0 - Fence nodes that do not join the controller process group within this much time after joining the cluster, to allow the cluster to continue managing resources. A value of 0 means never fence pending nodes. Setting the value to 2h means fence nodes after 2 hours. *(since 2.1.7)* * - .. _cluster_delay: .. index:: pair: cluster option; cluster-delay cluster-delay - :ref:`duration ` - 60s - If the DC requires an action to be executed on another node, it will consider the action failed if it does not get a response from the other node within this time (beyond the action's own timeout). The ideal value will depend on the speed and load of your network and cluster nodes. * - .. _dc_deadtime: .. index:: pair: cluster option; dc-deadtime dc-deadtime - :ref:`duration ` - 20s - How long to wait for a response from other nodes when electing a DC. The ideal value will depend on the speed and load of your network and cluster nodes. * - .. _cluster_ipc_limit: .. index:: pair: cluster option; cluster-ipc-limit cluster-ipc-limit - :ref:`nonnegative integer ` - 500 - - The maximum IPC message backlog before one cluster daemon will - disconnect another. This is of use in large clusters, for which a good - value is the number of resources in the cluster multiplied by the number - of nodes. The default of 500 is also the minimum. Raise this if you see - "Evicting client" log messages for cluster daemon process IDs. + - The maximum IPC message backlog before a cluster daemon will disconnect + a command line client. Other cluster daemons are not subject to this + limit as long as they are still processing messages. This is of use in + large clusters, for which a good value is the number of resources in the + cluster multiplied by the number of nodes. The default of 500 is also + the minimum. Raise this if you see "Evicting client" log messages for + cluster process IDs. * - .. _pe_error_series_max: .. index:: pair: cluster option; pe-error-series-max pe-error-series-max - :ref:`integer ` - -1 - The number of scheduler inputs resulting in errors to save. These inputs can be helpful during troubleshooting and when reporting issues. A negative value means save all inputs, and 0 means save none. * - .. _pe_warn_series_max: .. index:: pair: cluster option; pe-warn-series-max pe-warn-series-max - :ref:`integer ` - 5000 - The number of scheduler inputs resulting in warnings to save. These inputs can be helpful during troubleshooting and when reporting issues. A negative value means save all inputs, and 0 means save none. * - .. _pe_input_series_max: .. index:: pair: cluster option; pe-input-series-max pe-input-series-max - :ref:`integer ` - 4000 - The number of "normal" scheduler inputs to save. These inputs can be helpful during troubleshooting and when reporting issues. A negative value means save all inputs, and 0 means save none. * - .. _enable_acl: .. index:: pair: cluster option; enable-acl enable-acl - :ref:`boolean ` - false - Whether :ref:`access control lists ` should be used to authorize CIB modifications * - .. _placement_strategy: .. index:: pair: cluster option; placement-strategy placement-strategy - :ref:`enumeration ` - default - How the cluster should assign resources to nodes (see :ref:`utilization`). Allowed values are ``default``, ``utilization``, ``balanced``, and ``minimal``. * - .. _node_health_strategy: .. index:: pair: cluster option; node-health-strategy node-health-strategy - :ref:`enumeration ` - none - How the cluster should react to :ref:`node health ` attributes. Allowed values are ``none``, ``migrate-on-red``, ``only-green``, ``progressive``, and ``custom``. * - .. _node_health_base: .. index:: pair: cluster option; node-health-base node-health-base - :ref:`score ` - 0 - The base health score assigned to a node. Only used when ``node-health-strategy`` is ``progressive``. * - .. _node_health_green: .. index:: pair: cluster option; node-health-green node-health-green - :ref:`score ` - 0 - The score to use for a node health attribute whose value is ``green``. Only used when ``node-health-strategy`` is ``progressive`` or ``custom``. * - .. _node_health_yellow: .. index:: pair: cluster option; node-health-yellow node-health-yellow - :ref:`score ` - 0 - The score to use for a node health attribute whose value is ``yellow``. Only used when ``node-health-strategy`` is ``progressive`` or ``custom``. * - .. _node_health_red: .. index:: pair: cluster option; node-health-red node-health-red - :ref:`score ` - -INFINITY - The score to use for a node health attribute whose value is ``red``. Only used when ``node-health-strategy`` is ``progressive`` or ``custom``. * - .. _cluster_recheck_interval: .. index:: pair: cluster option; cluster-recheck-interval cluster-recheck-interval - :ref:`duration ` - 15min - Pacemaker is primarily event-driven, and looks ahead to know when to recheck the cluster for failure-timeout settings and most time-based rules *(since 2.0.3)*. However, it will also recheck the cluster after this amount of inactivity. This has three main effects: * :ref:`Rules ` using ``date_spec`` are guaranteed to be checked only this often. * If :ref:`fencing ` fails enough to reach :ref:`stonith-max-attempts `, attempts will begin again after at most this time. * It serves as a fail-safe in case of certain scheduler bugs. If the scheduler incorrectly determines only some of the actions needed to react to a particular event, it will often correctly determine the rest after at most this time. A value of 0 disables this polling. * - .. _shutdown_lock: .. index:: pair: cluster option; shutdown-lock shutdown-lock - :ref:`boolean ` - false - The default of false allows active resources to be recovered elsewhere when their node is cleanly shut down, which is what the vast majority of users will want. However, some users prefer to make resources highly available only for failures, with no recovery for clean shutdowns. If this option is true, resources active on a node when it is cleanly shut down are kept "locked" to that node (not allowed to run elsewhere) until they start again on that node after it rejoins (or for at most ``shutdown-lock-limit``, if set). Stonith resources and Pacemaker Remote connections are never locked. Clone and bundle instances and the promoted role of promotable clones are currently never locked, though support could be added in a future release. Locks may be manually cleared using the ``--refresh`` option of ``crm_resource`` (both the resource and node must be specified; this works with remote nodes if their connection resource's ``target-role`` is set to ``Stopped``, but not if Pacemaker Remote is stopped on the remote node without disabling the connection resource). *(since 2.0.4)* * - .. _shutdown_lock_limit: .. index:: pair: cluster option; shutdown-lock-limit shutdown-lock-limit - :ref:`duration ` - 0 - If ``shutdown-lock`` is true, and this is set to a nonzero time duration, locked resources will be allowed to start after this much time has passed since the node shutdown was initiated, even if the node has not rejoined. (This works with remote nodes only if their connection resource's ``target-role`` is set to ``Stopped``.) *(since 2.0.4)* * - .. _startup_fencing: .. index:: pair: cluster option; startup-fencing startup-fencing - :ref:`boolean ` - true - *Advanced Use Only:* Whether the cluster should fence unseen nodes at start-up. Setting this to false is unsafe, because the unseen nodes could be active and running resources but unreachable. ``dc-deadtime`` acts as a grace period before this fencing, since a DC must be elected to schedule fencing. * - .. _election_timeout: .. index:: pair: cluster option; election-timeout election-timeout - :ref:`duration ` - 2min - *Advanced Use Only:* If a winner is not declared within this much time of starting an election, the node that initiated the election will declare itself the winner. * - .. _shutdown_escalation: .. index:: pair: cluster option; shutdown-escalation shutdown-escalation - :ref:`duration ` - 20min - *Advanced Use Only:* The controller will exit immediately if a shutdown does not complete within this much time. * - .. _join_integration_timeout: .. index:: pair: cluster option; join-integration-timeout join-integration-timeout - :ref:`duration ` - 3min - *Advanced Use Only:* If you need to adjust this value, it probably indicates the presence of a bug. * - .. _join_finalization_timeout: .. index:: pair: cluster option; join-finalization-timeout join-finalization-timeout - :ref:`duration ` - 30min - *Advanced Use Only:* If you need to adjust this value, it probably indicates the presence of a bug. * - .. _transition_delay: .. index:: pair: cluster option; transition-delay transition-delay - :ref:`duration ` - 0s - *Advanced Use Only:* Delay cluster recovery for the configured interval to allow for additional or related events to occur. This can be useful if your configuration is sensitive to the order in which ping updates arrive. Enabling this option will slow down cluster recovery under all conditions. diff --git a/lib/common/ipc_server.c b/lib/common/ipc_server.c index 8169bb41fd..8474852b84 100644 --- a/lib/common/ipc_server.c +++ b/lib/common/ipc_server.c @@ -1,1158 +1,1182 @@ /* * Copyright 2004-2025 the Pacemaker project contributors * * The version control history for this file may have further details. * * This source code is licensed under the GNU Lesser General Public License * version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY. */ #include #include #include #include #include #include #include #include #include #include #include "crmcommon_private.h" /* Evict clients whose event queue grows this large (by default) */ #define PCMK_IPC_DEFAULT_QUEUE_MAX 500 static GHashTable *client_connections = NULL; /*! * \internal * \brief Count IPC clients * * \return Number of active IPC client connections */ guint pcmk__ipc_client_count(void) { return client_connections? g_hash_table_size(client_connections) : 0; } /*! * \internal * \brief Execute a function for each active IPC client connection * * \param[in] func Function to call * \param[in,out] user_data Pointer to pass to function * * \note The parameters are the same as for g_hash_table_foreach(). */ void pcmk__foreach_ipc_client(GHFunc func, gpointer user_data) { if ((func != NULL) && (client_connections != NULL)) { g_hash_table_foreach(client_connections, func, user_data); } } pcmk__client_t * pcmk__find_client(const qb_ipcs_connection_t *c) { if (client_connections) { return g_hash_table_lookup(client_connections, c); } crm_trace("No client found for %p", c); return NULL; } pcmk__client_t * pcmk__find_client_by_id(const char *id) { if ((client_connections != NULL) && (id != NULL)) { gpointer key; pcmk__client_t *client = NULL; GHashTableIter iter; g_hash_table_iter_init(&iter, client_connections); while (g_hash_table_iter_next(&iter, &key, (gpointer *) & client)) { if (strcmp(client->id, id) == 0) { return client; } } } crm_trace("No client found with id='%s'", pcmk__s(id, "")); return NULL; } /*! * \internal * \brief Get a client identifier for use in log messages * * \param[in] c Client * * \return Client's name, client's ID, or a string literal, as available * \note This is intended to be used in format strings like "client %s". */ const char * pcmk__client_name(const pcmk__client_t *c) { if (c == NULL) { return "(unspecified)"; } else if (c->name != NULL) { return c->name; } else if (c->id != NULL) { return c->id; } else { return "(unidentified)"; } } void pcmk__client_cleanup(void) { if (client_connections != NULL) { int active = g_hash_table_size(client_connections); if (active > 0) { crm_warn("Exiting with %d active IPC client%s", active, pcmk__plural_s(active)); } g_hash_table_destroy(client_connections); client_connections = NULL; } } void pcmk__drop_all_clients(qb_ipcs_service_t *service) { qb_ipcs_connection_t *c = NULL; if (service == NULL) { return; } c = qb_ipcs_connection_first_get(service); while (c != NULL) { qb_ipcs_connection_t *last = c; c = qb_ipcs_connection_next_get(service, last); /* There really shouldn't be anyone connected at this point */ crm_notice("Disconnecting client %p, pid=%d...", last, pcmk__client_pid(last)); qb_ipcs_disconnect(last); qb_ipcs_connection_unref(last); } } /*! * \internal * \brief Allocate a new pcmk__client_t object based on an IPC connection * * \param[in] c IPC connection (NULL to allocate generic client) * \param[in] key Connection table key (NULL to use sane default) * \param[in] uid_client UID corresponding to c (ignored if c is NULL) * * \return Pointer to new pcmk__client_t (guaranteed not to be \c NULL) */ static pcmk__client_t * client_from_connection(qb_ipcs_connection_t *c, void *key, uid_t uid_client) { pcmk__client_t *client = pcmk__assert_alloc(1, sizeof(pcmk__client_t)); if (c) { client->user = pcmk__uid2username(uid_client); if (client->user == NULL) { client->user = pcmk__str_copy("#unprivileged"); crm_err("Unable to enforce ACLs for user ID %d, assuming unprivileged", uid_client); } client->ipcs = c; pcmk__set_client_flags(client, pcmk__client_ipc); client->pid = pcmk__client_pid(c); if (key == NULL) { key = c; } } client->id = crm_generate_uuid(); if (key == NULL) { key = client->id; } if (client_connections == NULL) { crm_trace("Creating IPC client table"); client_connections = g_hash_table_new(g_direct_hash, g_direct_equal); } g_hash_table_insert(client_connections, key, client); return client; } /*! * \brief Allocate a new pcmk__client_t object and generate its ID * * \param[in] key What to use as connections hash table key (NULL to use ID) * * \return Pointer to new pcmk__client_t (asserts on failure) */ pcmk__client_t * pcmk__new_unauth_client(void *key) { return client_from_connection(NULL, key, 0); } pcmk__client_t * pcmk__new_client(qb_ipcs_connection_t *c, uid_t uid_client, gid_t gid_client) { gid_t uid_cluster = 0; gid_t gid_cluster = 0; pcmk__client_t *client = NULL; CRM_CHECK(c != NULL, return NULL); if (pcmk_daemon_user(&uid_cluster, &gid_cluster) < 0) { static bool need_log = TRUE; if (need_log) { crm_warn("Could not find user and group IDs for user %s", CRM_DAEMON_USER); need_log = FALSE; } } if (uid_client != 0) { crm_trace("Giving group %u access to new IPC connection", gid_cluster); /* Passing -1 to chown(2) means don't change */ qb_ipcs_connection_auth_set(c, -1, gid_cluster, S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP); } /* TODO: Do our own auth checking, return NULL if unauthorized */ client = client_from_connection(c, NULL, uid_client); if ((uid_client == 0) || (uid_client == uid_cluster)) { /* Remember when a connection came from root or hacluster */ pcmk__set_client_flags(client, pcmk__client_privileged); } crm_debug("New IPC client %s for PID %u with uid %d and gid %d", client->id, client->pid, uid_client, gid_client); return client; } static struct iovec * pcmk__new_ipc_event(void) { return (struct iovec *) pcmk__assert_alloc(2, sizeof(struct iovec)); } /*! * \brief Free an I/O vector created by pcmk__ipc_prepare_iov() * * \param[in,out] event I/O vector to free */ void pcmk_free_ipc_event(struct iovec *event) { if (event != NULL) { free(event[0].iov_base); free(event[1].iov_base); free(event); } } static void free_event(gpointer data) { pcmk_free_ipc_event((struct iovec *) data); } static void add_event(pcmk__client_t *c, struct iovec *iov) { if (c->event_queue == NULL) { c->event_queue = g_queue_new(); } g_queue_push_tail(c->event_queue, iov); } void pcmk__free_client(pcmk__client_t *c) { if (c == NULL) { return; } if (client_connections) { if (c->ipcs) { crm_trace("Destroying %p/%p (%d remaining)", c, c->ipcs, g_hash_table_size(client_connections) - 1); g_hash_table_remove(client_connections, c->ipcs); } else { crm_trace("Destroying remote connection %p (%d remaining)", c, g_hash_table_size(client_connections) - 1); g_hash_table_remove(client_connections, c->id); } } if (c->event_timer) { g_source_remove(c->event_timer); } if (c->event_queue) { crm_debug("Destroying %d events", g_queue_get_length(c->event_queue)); g_queue_free_full(c->event_queue, free_event); } free(c->id); free(c->name); free(c->user); if (c->buffer != NULL) { g_byte_array_free(c->buffer, TRUE); c->buffer = NULL; } if (c->remote) { if (c->remote->auth_timeout) { g_source_remove(c->remote->auth_timeout); } if (c->remote->tls_session != NULL) { /* @TODO Reduce duplication at callers. Put here everything * necessary to tear down and free tls_session. */ gnutls_deinit(c->remote->tls_session); } free(c->remote->buffer); free(c->remote); } free(c); } /*! * \internal * \brief Raise IPC eviction threshold for a client, if allowed * * \param[in,out] client Client to modify * \param[in] qmax New threshold */ void pcmk__set_client_queue_max(pcmk__client_t *client, const char *qmax) { int rc = pcmk_rc_ok; long long qmax_ll = 0LL; unsigned int orig_value = 0U; CRM_CHECK(client != NULL, return); orig_value = client->queue_max; if (pcmk_is_set(client->flags, pcmk__client_privileged)) { rc = pcmk__scan_ll(qmax, &qmax_ll, 0LL); if (rc == pcmk_rc_ok) { if ((qmax_ll <= 0LL) || (qmax_ll > UINT_MAX)) { rc = ERANGE; } else { client->queue_max = (unsigned int) qmax_ll; } } } else { rc = EACCES; } if (rc != pcmk_rc_ok) { crm_info("Could not set IPC threshold for client %s[%u] to %s: %s", pcmk__client_name(client), client->pid, pcmk__s(qmax, "default"), pcmk_rc_str(rc)); } else if (client->queue_max != orig_value) { crm_debug("IPC threshold for client %s[%u] is now %u (was %u)", pcmk__client_name(client), client->pid, client->queue_max, orig_value); } } int pcmk__client_pid(qb_ipcs_connection_t *c) { struct qb_ipcs_connection_stats stats; stats.client_pid = 0; qb_ipcs_connection_stats_get(c, &stats, 0); return stats.client_pid; } /*! * \internal * \brief Retrieve message XML from data read from client IPC * * \param[in,out] c IPC client connection * \param[out] id Where to store message ID from libqb header * \param[out] flags Where to store flags from libqb header * * \return Message XML on success, NULL otherwise */ xmlNode * pcmk__client_data2xml(pcmk__client_t *c, uint32_t *id, uint32_t *flags) { xmlNode *xml = NULL; pcmk__ipc_header_t *header = (void *) c->buffer->data; char *text = (char *) header + sizeof(pcmk__ipc_header_t); if (!pcmk__valid_ipc_header(header)) { return NULL; } if (id) { *id = header->qb.id; } if (flags) { *flags = header->flags; } if (pcmk_is_set(header->flags, crm_ipc_proxied)) { /* Mark this client as being the endpoint of a proxy connection. * Proxy connections responses are sent on the event channel, to avoid * blocking the controller serving as proxy. */ pcmk__set_client_flags(c, pcmk__client_proxied); } pcmk__assert(text[header->size - 1] == 0); xml = pcmk__xml_parse(text); crm_log_xml_trace(xml, "[IPC received]"); return xml; } static int crm_ipcs_flush_events(pcmk__client_t *c); static gboolean crm_ipcs_flush_events_cb(gpointer data) { pcmk__client_t *c = data; c->event_timer = 0; crm_ipcs_flush_events(c); return FALSE; } /*! * \internal * \brief Add progressive delay before next event queue flush * * \param[in,out] c Client connection to add delay to * \param[in] queue_len Current event queue length */ static inline void delay_next_flush(pcmk__client_t *c, unsigned int queue_len) { /* Delay a maximum of 1.5 seconds */ guint delay = (queue_len < 5)? (1000 + 100 * queue_len) : 1500; c->event_timer = pcmk__create_timer(delay, crm_ipcs_flush_events_cb, c); } /*! * \internal * \brief Send client any messages in its queue * * \param[in,out] c Client to flush * * \return Standard Pacemaker return value */ static int crm_ipcs_flush_events(pcmk__client_t *c) { int rc = pcmk_rc_ok; ssize_t qb_rc = 0; unsigned int sent = 0; unsigned int queue_len = 0; if (c == NULL) { return rc; + } - } else if (c->event_timer) { + if (c->event_timer != 0) { /* There is already a timer, wait until it goes off */ crm_trace("Timer active for %p - %d", c->ipcs, c->event_timer); return rc; } - if (c->event_queue) { + if ((c->event_queue != NULL) && !g_queue_is_empty(c->event_queue)) { queue_len = g_queue_get_length(c->event_queue); } + while (sent < 100) { - pcmk__ipc_header_t *header = NULL; struct iovec *event = NULL; - if (c->event_queue) { - // We don't pop unless send is successful - event = g_queue_peek_head(c->event_queue); - } - if (event == NULL) { // Queue is empty + if ((c->event_queue == NULL) || g_queue_is_empty(c->event_queue)) { break; } + // We don't pop unless send is successful + event = g_queue_peek_head(c->event_queue); + /* Retry sending the event up to five times. If we get -EAGAIN, sleep * a very short amount of time (too long here is bad) and try again. * If we simply exit the while loop on -EAGAIN, we'll have to wait until * the timer fires off again (up to 1.5 seconds - see delay_next_flush) * to retry sending the message. * * In that case, the queue may just continue to grow faster than we are * processing it, eventually leading to daemons timing out waiting for * replies, which will cause wider failures. */ for (unsigned int retries = 5; retries > 0; retries--) { qb_rc = qb_ipcs_event_sendv(c->ipcs, event, 2); - if (qb_rc < 0) { - if (retries == 1 || qb_rc != -EAGAIN) { - rc = (int) -qb_rc; - goto no_more_retries; - } else { - pcmk__sleep_ms(5); - } - } else { + if (qb_rc >= 0) { break; } - } - event = g_queue_pop_head(c->event_queue); + if (retries == 1 || qb_rc != -EAGAIN) { + rc = (int) -qb_rc; + goto no_more_retries; + } + + pcmk__sleep_ms(5); + } + /* No need to assign the return value here because event is still the + * same since we peeked it earlier. + */ + g_queue_pop_head(c->event_queue); sent++; - header = event[0].iov_base; - crm_trace("Event %" PRId32 " to %p[%u] (%zd bytes) sent: %.120s", - header->qb.id, c->ipcs, c->pid, qb_rc, - (char *) (event[1].iov_base)); + + pcmk__if_tracing( + { + pcmk__ipc_header_t *header = event[0].iov_base; + crm_trace("Event %" PRId32 " to %p[%u] (%zd bytes) sent: %.120s", + header->qb.id, c->ipcs, c->pid, qb_rc, + (char *) (event[1].iov_base)); + }, + {} + ); + pcmk_free_ipc_event(event); } no_more_retries: queue_len -= sent; if (sent > 0 || queue_len) { crm_trace("Sent %u events (%u remaining) for %p[%d]: %s (%zd)", sent, queue_len, c->ipcs, c->pid, pcmk_rc_str(rc), qb_rc); } - if (queue_len) { + if (queue_len == 0) { + /* Event queue is empty, there is no backlog */ + c->queue_backlog = 0; + return rc; + } - /* Allow clients to briefly fall behind on processing incoming messages, - * but drop completely unresponsive clients so the connection doesn't - * consume resources indefinitely. + /* Allow clients to briefly fall behind on processing incoming messages, + * but drop completely unresponsive clients so the connection doesn't + * consume resources indefinitely. + */ + if (queue_len > QB_MAX(c->queue_max, PCMK_IPC_DEFAULT_QUEUE_MAX)) { + /* Don't evict clients with: + * - A new backlog. + * - A shrinking backlog (the client is processing messages faster than + * the server is sending them). + * - Any messages sent in this flush call (the server is sending + * messages faster than the client is processing them, but the client + * is not dead). This could allow the queue to grow without bound, + * so only trusted clients (daemons for the moment) are allowed to do + * so. + * + * @TODO We could extend this to other command line tools that we + * trust, or perhaps to any client that has pcmk__client_privileged + * set. */ - if (queue_len > QB_MAX(c->queue_max, PCMK_IPC_DEFAULT_QUEUE_MAX)) { - if ((c->queue_backlog <= 1) || (queue_len < c->queue_backlog)) { - /* Don't evict for a new or shrinking backlog */ - crm_warn("Client with process ID %u has a backlog of %u messages " - QB_XS " %p", c->pid, queue_len, c->ipcs); - } else { - crm_err("Evicting client with process ID %u due to backlog of %u messages " - QB_XS " %p", c->pid, queue_len, c->ipcs); - c->queue_backlog = 0; - qb_ipcs_disconnect(c->ipcs); - return rc; - } - } - - c->queue_backlog = queue_len; - delay_next_flush(c, queue_len); + if ((c->queue_backlog <= 1) + || (queue_len < c->queue_backlog) + || ((sent > 0) && (pcmk__parse_server(c->name) != pcmk_ipc_unknown))) { + crm_warn("Client with process ID %u has a backlog of %u messages " + QB_XS " %p", c->pid, queue_len, c->ipcs); - } else { - /* Event queue is empty, there is no backlog */ - c->queue_backlog = 0; + } else { + crm_err("Evicting client with process ID %u due to backlog of %u messages " + QB_XS " %p", c->pid, queue_len, c->ipcs); + c->queue_backlog = 0; + qb_ipcs_disconnect(c->ipcs); + return rc; + } } + c->queue_backlog = queue_len; + delay_next_flush(c, queue_len); + return rc; } /*! * \internal * \brief Create an I/O vector for sending an IPC XML message * * If the message is too large to fit into a single buffer, this function will * prepare an I/O vector that only holds as much as fits. The remainder can be * prepared in a separate call by keeping a running count of the number of times * this function has been called and passing that in for \p index. * * \param[in] request Identifier for libqb response header * \param[in] message Message to send * \param[in] index How many times this function has been called - basically, * a count of how many chunks of \p message have already * been sent * \param[out] result Where to store prepared I/O vector - NULL on error * \param[out] bytes Size of prepared data in bytes (includes header) * * \return Standard Pacemaker return code */ int pcmk__ipc_prepare_iov(uint32_t request, const GString *message, uint16_t index, struct iovec **result, ssize_t *bytes) { struct iovec *iov = NULL; unsigned int payload_size = 0; unsigned int total = 0; unsigned int max_send_size = crm_ipc_default_buffer_size(); unsigned int max_chunk_size = 0; size_t offset = 0; pcmk__ipc_header_t *header = NULL; int rc = pcmk_rc_ok; if ((message == NULL) || (result == NULL)) { rc = EINVAL; goto done; } header = calloc(1, sizeof(pcmk__ipc_header_t)); if (header == NULL) { rc = ENOMEM; goto done; } *result = NULL; iov = pcmk__new_ipc_event(); iov[0].iov_len = sizeof(pcmk__ipc_header_t); iov[0].iov_base = header; header->version = PCMK__IPC_VERSION; /* We are passed an index, which is basically how many times this function * has been called. This is how we support multi-part IPC messages. We * need to convert that into an offset into the buffer that we want to start * reading from. * * Each call to this function can send max_send_size, but this also includes * the header and a null terminator character for the end of the payload. * We need to subtract those out here. */ max_chunk_size = max_send_size - iov[0].iov_len - 1; offset = index * max_chunk_size; /* How much of message is left to send? This does not include the null * terminator character. */ payload_size = message->len - offset; /* How much would be transmitted, including the header size and null * terminator character for the buffer? */ total = iov[0].iov_len + payload_size + 1; if (total >= max_send_size) { /* The entire packet is too big to fit in a single buffer. Calculate * how much of it we can send - buffer size, minus header size, minus * one for the null terminator. */ payload_size = max_chunk_size; header->size = payload_size + 1; iov[1].iov_base = strndup(message->str + offset, payload_size); if (iov[1].iov_base == NULL) { rc = ENOMEM; goto done; } iov[1].iov_len = header->size; rc = pcmk_rc_ipc_more; } else { /* The entire packet fits in a single buffer. We can copy the entirety * of it into the payload. */ header->size = payload_size + 1; iov[1].iov_base = pcmk__str_copy(message->str + offset); iov[1].iov_len = header->size; } header->part_id = index; header->qb.size = iov[0].iov_len + iov[1].iov_len; header->qb.id = (int32_t)request; /* Replying to a specific request */ if ((rc == pcmk_rc_ok) && (index != 0)) { pcmk__set_ipc_flags(header->flags, "multipart ipc", crm_ipc_multipart | crm_ipc_multipart_end); } else if (rc == pcmk_rc_ipc_more) { pcmk__set_ipc_flags(header->flags, "multipart ipc", crm_ipc_multipart); } *result = iov; pcmk__assert(header->qb.size > 0); if (bytes != NULL) { *bytes = header->qb.size; } done: if ((rc != pcmk_rc_ok) && (rc != pcmk_rc_ipc_more)) { pcmk_free_ipc_event(iov); } return rc; } /* Return the next available ID for a server event. * * For the parts of a multipart event, all parts should have the same ID as * the first part. */ static uint32_t id_for_server_event(pcmk__ipc_header_t *header) { static uint32_t id = 1; if (pcmk_is_set(header->flags, crm_ipc_multipart) && (header->part_id != 0)) { return id; } else { id++; return id; } } int pcmk__ipc_send_iov(pcmk__client_t *c, struct iovec *iov, uint32_t flags) { int rc = pcmk_rc_ok; pcmk__ipc_header_t *header = iov[0].iov_base; /* _ALL_ replies to proxied connections need to be sent as events */ if (pcmk_is_set(c->flags, pcmk__client_proxied) && !pcmk_is_set(flags, crm_ipc_server_event)) { /* The proxied flag lets us know this was originally meant to be a * response, even though we're sending it over the event channel. */ pcmk__set_ipc_flags(flags, "server event", crm_ipc_server_event|crm_ipc_proxied_relay_response); } pcmk__set_ipc_flags(header->flags, "server event", flags); if (pcmk_is_set(flags, crm_ipc_server_event)) { /* Server events don't use an ID, though we do set one in * pcmk__ipc_prepare_iov if the event is in response to a particular * request. In that case, we don't want to set a new ID here that * overwrites that one. * * @TODO: Since server event IDs aren't used anywhere, do we really * need to set this for any reason other than ease of logging? */ if (header->qb.id == 0) { header->qb.id = id_for_server_event(header); } if (pcmk_is_set(flags, crm_ipc_server_free)) { crm_trace("Sending the original to %p[%d]", c->ipcs, c->pid); add_event(c, iov); } else { struct iovec *iov_copy = pcmk__new_ipc_event(); crm_trace("Sending a copy to %p[%d]", c->ipcs, c->pid); iov_copy[0].iov_len = iov[0].iov_len; iov_copy[0].iov_base = malloc(iov[0].iov_len); memcpy(iov_copy[0].iov_base, iov[0].iov_base, iov[0].iov_len); iov_copy[1].iov_len = iov[1].iov_len; iov_copy[1].iov_base = malloc(iov[1].iov_len); memcpy(iov_copy[1].iov_base, iov[1].iov_base, iov[1].iov_len); add_event(c, iov_copy); } rc = crm_ipcs_flush_events(c); } else { ssize_t qb_rc; char *part_text = NULL; CRM_LOG_ASSERT(header->qb.id != 0); /* Replying to a specific request */ if (pcmk_is_set(header->flags, crm_ipc_multipart_end)) { part_text = crm_strdup_printf(" (final part %d) ", header->part_id); } else if (pcmk_is_set(header->flags, crm_ipc_multipart)) { if (header->part_id == 0) { part_text = crm_strdup_printf(" (initial part %d) ", header->part_id); } else { part_text = crm_strdup_printf(" (part %d) ", header->part_id); } } else { part_text = crm_strdup_printf(" "); } qb_rc = qb_ipcs_response_sendv(c->ipcs, iov, 2); if (qb_rc < header->qb.size) { if (qb_rc < 0) { rc = (int) -qb_rc; } crm_notice("Response %" PRId32 "%sto pid %u failed: %s " QB_XS " bytes=%" PRId32 " rc=%zd ipcs=%p", header->qb.id, part_text, c->pid, pcmk_rc_str(rc), header->qb.size, qb_rc, c->ipcs); } else { crm_trace("Response %" PRId32 "%ssent, %zd bytes to %p[%u]", header->qb.id, part_text, qb_rc, c->ipcs, c->pid); crm_trace("Text = %s", (char *) iov[1].iov_base); } free(part_text); if (pcmk_is_set(flags, crm_ipc_server_free)) { pcmk_free_ipc_event(iov); } crm_ipcs_flush_events(c); } if ((rc == EPIPE) || (rc == ENOTCONN)) { crm_trace("Client %p disconnected", c->ipcs); } return rc; } int pcmk__ipc_send_xml(pcmk__client_t *c, uint32_t request, const xmlNode *message, uint32_t flags) { struct iovec *iov = NULL; int rc = pcmk_rc_ok; GString *iov_buffer = NULL; uint16_t index = 0; bool event_or_proxied = false; if (c == NULL) { return EINVAL; } iov_buffer = g_string_sized_new(1024); pcmk__xml_string(message, 0, iov_buffer, 0); /* Testing crm_ipc_server_event is obvious. pcmk__client_proxied is less * obvious. According to pcmk__ipc_send_iov, replies to proxied connections * need to be sent as events. However, do_local_notify (which calls this * function) will clear all flags so we can't go just by crm_ipc_server_event. * * Changing do_local_notify to check for a proxied connection first results * in processes on the Pacemaker Remote node (like cibadmin or crm_mon) * timing out when waiting for a reply. */ event_or_proxied = pcmk_is_set(flags, crm_ipc_server_event) || pcmk_is_set(c->flags, pcmk__client_proxied); do { rc = pcmk__ipc_prepare_iov(request, iov_buffer, index, &iov, NULL); switch (rc) { case pcmk_rc_ok: /* No more chunks to send after this one */ pcmk__set_ipc_flags(flags, "send data", crm_ipc_server_free); rc = pcmk__ipc_send_iov(c, iov, flags); if (event_or_proxied) { if (rc == EAGAIN) { /* Return pcmk_rc_ok instead so callers don't have to know * whether they passed an event or not when interpreting * the return code. */ rc = pcmk_rc_ok; } } else { /* EAGAIN is an error for IPC messages. We don't have a * send queue for these, so we need to try again. If there * was some other error, we need to break out of this loop * and report it. * * FIXME: Retry limit for EAGAIN? */ if (rc == EAGAIN) { break; } } goto done; case pcmk_rc_ipc_more: /* There are more chunks to send after this one */ pcmk__set_ipc_flags(flags, "send data", crm_ipc_server_free); rc = pcmk__ipc_send_iov(c, iov, flags); /* Did an error occur during transmission? */ if (event_or_proxied) { /* EAGAIN is not an error for server events. The event * will be queued for transmission and we will attempt * sending it again the next time pcmk__ipc_send_iov is * called, or when the crm_ipcs_flush_events_cb happens. */ if ((rc != pcmk_rc_ok) && (rc != EAGAIN)) { goto done; } index++; break; } else { /* EAGAIN is an error for IPC messages. We don't have a * send queue for these, so we need to try again. If there * was some other error, we need to break out of this loop * and report it. * * FIXME: Retry limit for EAGAIN? */ if (rc == pcmk_rc_ok) { index++; break; } else if (rc == EAGAIN) { break; } else { goto done; } } default: /* An error occurred during preparation */ goto done; } } while (true); done: if ((rc != pcmk_rc_ok) && (rc != EAGAIN)) { crm_notice("IPC message to pid %u failed: %s " QB_XS " rc=%d", c->pid, pcmk_rc_str(rc), rc); } g_string_free(iov_buffer, TRUE); return rc; } /*! * \internal * \brief Create an acknowledgement with a status code to send to a client * * \param[in] function Calling function * \param[in] line Source file line within calling function * \param[in] flags IPC flags to use when sending * \param[in] tag Element name to use for acknowledgement * \param[in] ver IPC protocol version (can be NULL) * \param[in] status Exit status code to add to ack * * \return Newly created XML for ack * * \note The caller is responsible for freeing the return value with * \c pcmk__xml_free(). */ xmlNode * pcmk__ipc_create_ack_as(const char *function, int line, uint32_t flags, const char *tag, const char *ver, crm_exit_t status) { xmlNode *ack = NULL; if (pcmk_is_set(flags, crm_ipc_client_response)) { ack = pcmk__xe_create(NULL, tag); pcmk__xe_set(ack, PCMK_XA_FUNCTION, function); pcmk__xe_set_int(ack, PCMK__XA_LINE, line); pcmk__xe_set_int(ack, PCMK_XA_STATUS, (int) status); pcmk__xe_set(ack, PCMK__XA_IPC_PROTO_VERSION, ver); } return ack; } /*! * \internal * \brief Send an acknowledgement with a status code to a client * * \param[in] function Calling function * \param[in] line Source file line within calling function * \param[in] c Client to send ack to * \param[in] request Request ID being replied to * \param[in] flags IPC flags to use when sending * \param[in] tag Element name to use for acknowledgement * \param[in] ver IPC protocol version (can be NULL) * \param[in] status Status code to send with acknowledgement * * \return Standard Pacemaker return code */ int pcmk__ipc_send_ack_as(const char *function, int line, pcmk__client_t *c, uint32_t request, uint32_t flags, const char *tag, const char *ver, crm_exit_t status) { int rc = pcmk_rc_ok; xmlNode *ack = pcmk__ipc_create_ack_as(function, line, flags, tag, ver, status); if (ack != NULL) { crm_trace("Ack'ing IPC message from client %s as <%s status=%d>", pcmk__client_name(c), tag, status); crm_log_xml_trace(ack, "sent-ack"); c->request_id = 0; rc = pcmk__ipc_send_xml(c, request, ack, flags); pcmk__xml_free(ack); } return rc; } /*! * \internal * \brief Add an IPC server to the main loop for the CIB manager API * * \param[out] ipcs_ro New IPC server for read-only CIB manager API * \param[out] ipcs_rw New IPC server for read/write CIB manager API * \param[out] ipcs_shm New IPC server for shared-memory CIB manager API * \param[in] ro_cb IPC callbacks for read-only API * \param[in] rw_cb IPC callbacks for read/write and shared-memory APIs * * \note This function exits fatally if unable to create the servers. * \note There is no actual difference between the three IPC endpoints other * than their names. */ void pcmk__serve_based_ipc(qb_ipcs_service_t **ipcs_ro, qb_ipcs_service_t **ipcs_rw, qb_ipcs_service_t **ipcs_shm, struct qb_ipcs_service_handlers *ro_cb, struct qb_ipcs_service_handlers *rw_cb) { *ipcs_ro = mainloop_add_ipc_server(PCMK__SERVER_BASED_RO, QB_IPC_NATIVE, ro_cb); *ipcs_rw = mainloop_add_ipc_server(PCMK__SERVER_BASED_RW, QB_IPC_NATIVE, rw_cb); *ipcs_shm = mainloop_add_ipc_server(PCMK__SERVER_BASED_SHM, QB_IPC_SHM, rw_cb); if (*ipcs_ro == NULL || *ipcs_rw == NULL || *ipcs_shm == NULL) { crm_err("Failed to create the CIB manager: exiting and inhibiting respawn"); crm_warn("Verify pacemaker and pacemaker_remote are not both enabled"); crm_exit(CRM_EX_FATAL); } } /*! * \internal * \brief Destroy IPC servers for the CIB manager API * * \param[out] ipcs_ro IPC server for read-only the CIB manager API * \param[out] ipcs_rw IPC server for read/write the CIB manager API * \param[out] ipcs_shm IPC server for shared-memory the CIB manager API * * \note This is a convenience function for calling qb_ipcs_destroy() for each * argument. */ void pcmk__stop_based_ipc(qb_ipcs_service_t *ipcs_ro, qb_ipcs_service_t *ipcs_rw, qb_ipcs_service_t *ipcs_shm) { qb_ipcs_destroy(ipcs_ro); qb_ipcs_destroy(ipcs_rw); qb_ipcs_destroy(ipcs_shm); } /*! * \internal * \brief Add an IPC server to the main loop for the controller API * * \param[in] cb IPC callbacks * * \return Newly created IPC server */ qb_ipcs_service_t * pcmk__serve_controld_ipc(struct qb_ipcs_service_handlers *cb) { return mainloop_add_ipc_server(CRM_SYSTEM_CRMD, QB_IPC_NATIVE, cb); } /*! * \internal * \brief Add an IPC server to the main loop for the attribute manager API * * \param[out] ipcs Where to store newly created IPC server * \param[in] cb IPC callbacks * * \note This function exits fatally if unable to create the servers. */ void pcmk__serve_attrd_ipc(qb_ipcs_service_t **ipcs, struct qb_ipcs_service_handlers *cb) { *ipcs = mainloop_add_ipc_server(PCMK__VALUE_ATTRD, QB_IPC_NATIVE, cb); if (*ipcs == NULL) { crm_crit("Exiting fatally because unable to serve " PCMK__SERVER_ATTRD " IPC (verify pacemaker and pacemaker_remote are not both " "enabled)"); crm_exit(CRM_EX_FATAL); } } /*! * \internal * \brief Add an IPC server to the main loop for the fencer API * * \param[out] ipcs Where to store newly created IPC server * \param[in] cb IPC callbacks * * \note This function exits fatally if unable to create the servers. */ void pcmk__serve_fenced_ipc(qb_ipcs_service_t **ipcs, struct qb_ipcs_service_handlers *cb) { *ipcs = mainloop_add_ipc_server_with_prio("stonith-ng", QB_IPC_NATIVE, cb, QB_LOOP_HIGH); if (*ipcs == NULL) { crm_err("Failed to create fencer: exiting and inhibiting respawn."); crm_warn("Verify pacemaker and pacemaker_remote are not both enabled."); crm_exit(CRM_EX_FATAL); } } /*! * \internal * \brief Add an IPC server to the main loop for the pacemakerd API * * \param[out] ipcs Where to store newly created IPC server * \param[in] cb IPC callbacks * * \note This function exits with CRM_EX_OSERR if unable to create the servers. */ void pcmk__serve_pacemakerd_ipc(qb_ipcs_service_t **ipcs, struct qb_ipcs_service_handlers *cb) { *ipcs = mainloop_add_ipc_server(CRM_SYSTEM_MCP, QB_IPC_NATIVE, cb); if (*ipcs == NULL) { crm_err("Couldn't start pacemakerd IPC server"); crm_warn("Verify pacemaker and pacemaker_remote are not both enabled."); /* sub-daemons are observed by pacemakerd. Thus we exit CRM_EX_FATAL * if we want to prevent pacemakerd from restarting them. * With pacemakerd we leave the exit-code shown to e.g. systemd * to what it was prior to moving the code here from pacemakerd.c */ crm_exit(CRM_EX_OSERR); } } /*! * \internal * \brief Add an IPC server to the main loop for the scheduler API * * \param[in] cb IPC callbacks * * \return Newly created IPC server * \note This function exits fatally if unable to create the servers. */ qb_ipcs_service_t * pcmk__serve_schedulerd_ipc(struct qb_ipcs_service_handlers *cb) { return mainloop_add_ipc_server(CRM_SYSTEM_PENGINE, QB_IPC_NATIVE, cb); } diff --git a/lib/common/options.c b/lib/common/options.c index 361d2bb5d9..fe714c8dfe 100644 --- a/lib/common/options.c +++ b/lib/common/options.c @@ -1,1530 +1,1530 @@ /* * Copyright 2004-2025 the Pacemaker project contributors * * The version control history for this file may have further details. * * This source code is licensed under the GNU Lesser General Public License * version 2.1 or later (LGPLv2.1+) WITHOUT ANY WARRANTY. */ #include #include #include #include #include #include #include #include /* * Option metadata */ static const pcmk__cluster_option_t cluster_options[] = { /* name, old name, type, allowed values, * default value, validator, * flags, * short description, * long description */ { PCMK_OPT_DC_VERSION, NULL, PCMK_VALUE_VERSION, NULL, NULL, NULL, pcmk__opt_controld|pcmk__opt_generated, N_("Pacemaker version on cluster node elected Designated Controller " "(DC)"), N_("Includes a hash which identifies the exact revision the code was " "built from. Used for diagnostic purposes."), }, { PCMK_OPT_CLUSTER_INFRASTRUCTURE, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_controld|pcmk__opt_generated, N_("The messaging layer on which Pacemaker is currently running"), N_("Used for informational and diagnostic purposes."), }, { PCMK_OPT_CLUSTER_NAME, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_controld, N_("An arbitrary name for the cluster"), N_("This optional value is mostly for users' convenience as desired " "in administration, but may also be used in Pacemaker " "configuration rules via the #cluster-name node attribute, and " "by higher-level tools and resource agents."), }, { PCMK_OPT_DC_DEADTIME, NULL, PCMK_VALUE_DURATION, NULL, "20s", pcmk__valid_interval_spec, pcmk__opt_controld, N_("How long to wait for a response from other nodes during start-up"), N_("The optimal value will depend on the speed and load of your " "network and the type of switches used."), }, { PCMK_OPT_CLUSTER_RECHECK_INTERVAL, NULL, PCMK_VALUE_DURATION, NULL, "15min", pcmk__valid_interval_spec, pcmk__opt_controld, N_("Polling interval to recheck cluster state and evaluate rules " "with date specifications"), N_("Pacemaker is primarily event-driven, and looks ahead to know when " "to recheck cluster state for failure-timeout settings and most " "time-based rules. However, it will also recheck the cluster after " "this amount of inactivity, to evaluate rules with date " "specifications and serve as a fail-safe for certain types of " "scheduler bugs. A value of 0 disables polling. A positive value " "sets an interval in seconds, unless other units are specified " "(for example, \"5min\")."), }, { PCMK_OPT_FENCE_REACTION, NULL, PCMK_VALUE_SELECT, PCMK_VALUE_STOP ", " PCMK_VALUE_PANIC, PCMK_VALUE_STOP, NULL, pcmk__opt_controld, N_("How a cluster node should react if notified of its own fencing"), N_("A cluster node may receive notification of a \"succeeded\" " "fencing that targeted it if fencing is misconfigured, or if " "fabric fencing is in use that doesn't cut cluster communication. " "Use \"stop\" to attempt to immediately stop Pacemaker and stay " "stopped, or \"panic\" to attempt to immediately reboot the local " "node, falling back to stop on failure."), }, { PCMK_OPT_ELECTION_TIMEOUT, NULL, PCMK_VALUE_DURATION, NULL, "2min", pcmk__valid_interval_spec, pcmk__opt_controld|pcmk__opt_advanced, N_("Declare an election failed if it is not decided within this much " "time. If you need to adjust this value, it probably indicates " "the presence of a bug."), NULL, }, { PCMK_OPT_SHUTDOWN_ESCALATION, NULL, PCMK_VALUE_DURATION, NULL, "20min", pcmk__valid_interval_spec, pcmk__opt_controld|pcmk__opt_advanced, N_("Exit immediately if shutdown does not complete within this much " "time. If you need to adjust this value, it probably indicates " "the presence of a bug."), NULL, }, { PCMK_OPT_JOIN_INTEGRATION_TIMEOUT, NULL, PCMK_VALUE_DURATION, NULL, "3min", pcmk__valid_interval_spec, pcmk__opt_controld|pcmk__opt_advanced, N_("If you need to adjust this value, it probably indicates " "the presence of a bug."), NULL, }, { PCMK_OPT_JOIN_FINALIZATION_TIMEOUT, NULL, PCMK_VALUE_DURATION, NULL, "30min", pcmk__valid_interval_spec, pcmk__opt_controld|pcmk__opt_advanced, N_("If you need to adjust this value, it probably indicates " "the presence of a bug."), NULL, }, { PCMK_OPT_TRANSITION_DELAY, NULL, PCMK_VALUE_DURATION, NULL, "0s", pcmk__valid_interval_spec, pcmk__opt_controld|pcmk__opt_advanced, N_("Enabling this option will slow down cluster recovery under all " "conditions"), N_("Delay cluster recovery for this much time to allow for additional " "events to occur. Useful if your configuration is sensitive to " "the order in which ping updates arrive."), }, { PCMK_OPT_NO_QUORUM_POLICY, NULL, PCMK_VALUE_SELECT, PCMK_VALUE_STOP ", " PCMK_VALUE_FREEZE ", " PCMK_VALUE_IGNORE ", " PCMK_VALUE_DEMOTE ", " PCMK_VALUE_FENCE ", " PCMK_VALUE_FENCE_LEGACY, PCMK_VALUE_STOP, pcmk__valid_no_quorum_policy, pcmk__opt_schedulerd, N_("What to do when the cluster does not have quorum"), NULL, }, { PCMK_OPT_SHUTDOWN_LOCK, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether to lock resources to a cleanly shut down node"), N_("When true, resources active on a node when it is cleanly shut down " "are kept \"locked\" to that node (not allowed to run elsewhere) " "until they start again on that node after it rejoins (or for at " "most shutdown-lock-limit, if set). Stonith resources and " "Pacemaker Remote connections are never locked. Clone and bundle " "instances and the promoted role of promotable clones are " "currently never locked, though support could be added in a future " "release."), }, { PCMK_OPT_SHUTDOWN_LOCK_LIMIT, NULL, PCMK_VALUE_DURATION, NULL, "0", pcmk__valid_interval_spec, pcmk__opt_schedulerd, N_("Do not lock resources to a cleanly shut down node longer than " "this"), N_("If shutdown-lock is true and this is set to a nonzero time " "duration, shutdown locks will expire after this much time has " "passed since the shutdown was initiated, even if the node has not " "rejoined."), }, { PCMK_OPT_ENABLE_ACL, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, pcmk__valid_boolean, pcmk__opt_based, N_("Enable Access Control Lists (ACLs) for the CIB"), NULL, }, { PCMK_OPT_SYMMETRIC_CLUSTER, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether resources can run on any node by default"), NULL, }, { PCMK_OPT_MAINTENANCE_MODE, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether the cluster should refrain from monitoring, starting, and " "stopping resources"), NULL, }, { PCMK_OPT_START_FAILURE_IS_FATAL, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether a start failure should prevent a resource from being " "recovered on the same node"), N_("When true, the cluster will immediately ban a resource from a node " "if it fails to start there. When false, the cluster will instead " "check the resource's fail count against its migration-threshold.") }, { PCMK_OPT_ENABLE_STARTUP_PROBES, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether the cluster should check for active resources during " "start-up"), NULL, }, // Fencing-related options { PCMK_OPT_FENCE_REMOTE_WITHOUT_QUORUM, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, pcmk__valid_boolean, pcmk__opt_schedulerd|pcmk__opt_advanced, N_("Whether remote nodes can be fenced without quorum"), N_("By default, an inquorate node can not fence Pacemaker Remote nodes " "that are part of its partition as long as the cluster thinks they " "can be restarted. If true, inquorate nodes will be able to fence " "remote nodes regardless."), }, { PCMK_OPT_STONITH_ENABLED, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, pcmk__valid_boolean, pcmk__opt_schedulerd|pcmk__opt_advanced, N_("Whether nodes may be fenced as part of recovery"), N_("If false, unresponsive nodes are immediately assumed to be " "harmless, and resources that were active on them may be recovered " "elsewhere. This can result in a \"split-brain\" situation, " "potentially leading to data loss and/or service unavailability."), }, { PCMK_OPT_STONITH_ACTION, NULL, PCMK_VALUE_SELECT, PCMK_ACTION_REBOOT ", " PCMK_ACTION_OFF, PCMK_ACTION_REBOOT, pcmk__is_fencing_action, pcmk__opt_schedulerd, N_("Action to send to fence device when a node needs to be fenced"), NULL, }, { PCMK_OPT_STONITH_TIMEOUT, NULL, PCMK_VALUE_DURATION, NULL, "60s", pcmk__valid_interval_spec, pcmk__opt_schedulerd, N_("How long to wait for on, off, and reboot fence actions to complete " "by default"), NULL, }, { PCMK_OPT_HAVE_WATCHDOG, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, pcmk__valid_boolean, pcmk__opt_schedulerd|pcmk__opt_generated, N_("Whether watchdog integration is enabled"), N_("This is set automatically by the cluster according to whether SBD " "is detected to be in use. User-configured values are ignored. " "The value `true` is meaningful if diskless SBD is used and " "`stonith-watchdog-timeout` is nonzero. In that case, if fencing " "is required, watchdog-based self-fencing will be performed via " "SBD without requiring a fencing resource explicitly configured."), }, { /* @COMPAT Currently, unparsable values default to -1 (auto-calculate), * while missing values default to 0 (disable). All values are accepted * (unless the controller finds that the value conflicts with the * SBD_WATCHDOG_TIMEOUT). * * At a compatibility break: properly validate as a timeout, let * either negative values or a particular string like "auto" mean auto- * calculate, and use 0 as the single default for when the option either * is unset or fails to validate. */ PCMK_OPT_STONITH_WATCHDOG_TIMEOUT, NULL, PCMK_VALUE_TIMEOUT, NULL, "0", NULL, pcmk__opt_controld, N_("How long before nodes can be assumed to be safely down when " "watchdog-based self-fencing via SBD is in use"), N_("If this is set to a positive value, lost nodes are assumed to " "achieve self-fencing using watchdog-based SBD within this much " "time. This does not require a fencing resource to be explicitly " "configured, though a fence_watchdog resource can be configured, to " "limit use to specific nodes. If this is set to 0 (the default), " "the cluster will never assume watchdog-based self-fencing. If this " "is set to a negative value, the cluster will use twice the local " "value of the `SBD_WATCHDOG_TIMEOUT` environment variable if that " "is positive, or otherwise treat this as 0. WARNING: When used, " "this timeout must be larger than `SBD_WATCHDOG_TIMEOUT` on all " "nodes that use watchdog-based SBD, and Pacemaker will refuse to " "start on any of those nodes where this is not true for the local " "value or SBD is not active. When this is set to a negative value, " "`SBD_WATCHDOG_TIMEOUT` must be set to the same value on all nodes " "that use SBD, otherwise data corruption or loss could occur."), }, { PCMK_OPT_STONITH_MAX_ATTEMPTS, NULL, PCMK_VALUE_SCORE, NULL, "10", pcmk__valid_positive_int, pcmk__opt_controld, N_("How many times fencing can fail before it will no longer be " "immediately re-attempted on a target"), NULL, }, { PCMK_OPT_CONCURRENT_FENCING, NULL, PCMK_VALUE_BOOLEAN, NULL, #if PCMK__CONCURRENT_FENCING_DEFAULT_TRUE PCMK_VALUE_TRUE, #else PCMK_VALUE_FALSE, #endif pcmk__valid_boolean, pcmk__opt_schedulerd|pcmk__opt_deprecated, N_("Allow performing fencing operations in parallel"), NULL, }, { PCMK_OPT_STARTUP_FENCING, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, pcmk__valid_boolean, pcmk__opt_schedulerd|pcmk__opt_advanced, N_("Whether to fence unseen nodes at start-up"), N_("Setting this to false may lead to a \"split-brain\" situation, " "potentially leading to data loss and/or service unavailability."), }, { PCMK_OPT_PRIORITY_FENCING_DELAY, NULL, PCMK_VALUE_DURATION, NULL, "0", pcmk__valid_interval_spec, pcmk__opt_schedulerd, N_("Apply fencing delay targeting the lost nodes with the highest " "total resource priority"), N_("Apply specified delay for the fencings that are targeting the lost " "nodes with the highest total resource priority in case we don't " "have the majority of the nodes in our cluster partition, so that " "the more significant nodes potentially win any fencing match, " "which is especially meaningful under split-brain of 2-node " "cluster. A promoted resource instance takes the base priority + 1 " "on calculation if the base priority is not 0. Any static/random " "delays that are introduced by `pcmk_delay_base/max` configured " "for the corresponding fencing resources will be added to this " "delay. This delay should be significantly greater than, safely " "twice, the maximum `pcmk_delay_base/max`. By default, priority " "fencing delay is disabled."), }, { PCMK_OPT_NODE_PENDING_TIMEOUT, NULL, PCMK_VALUE_DURATION, NULL, "0", pcmk__valid_interval_spec, pcmk__opt_schedulerd, N_("How long to wait for a node that has joined the cluster to join " "the controller process group"), N_("Fence nodes that do not join the controller process group within " "this much time after joining the cluster, to allow the cluster " "to continue managing resources. A value of 0 means never fence " "pending nodes. Setting the value to 2h means fence nodes after " "2 hours."), }, { PCMK_OPT_CLUSTER_DELAY, NULL, PCMK_VALUE_DURATION, NULL, "60s", pcmk__valid_interval_spec, pcmk__opt_schedulerd, N_("Maximum time for node-to-node communication"), N_("The node elected Designated Controller (DC) will consider an action " "failed if it does not get a response from the node executing the " "action within this time (after considering the action's own " "timeout). The \"correct\" value will depend on the speed and " "load of your network and cluster nodes.") }, // Limits { PCMK_OPT_LOAD_THRESHOLD, NULL, PCMK_VALUE_PERCENTAGE, NULL, "80%", pcmk__valid_percentage, pcmk__opt_controld, N_("Maximum amount of system load that should be used by cluster " "nodes"), N_("The cluster will slow down its recovery process when the amount of " "system resources used (currently CPU) approaches this limit"), }, { PCMK_OPT_NODE_ACTION_LIMIT, NULL, PCMK_VALUE_INTEGER, NULL, "0", pcmk__valid_int, pcmk__opt_controld, N_("Maximum number of jobs that can be scheduled per node (defaults to " "2x cores)"), NULL, }, { PCMK_OPT_BATCH_LIMIT, NULL, PCMK_VALUE_INTEGER, NULL, "0", pcmk__valid_int, pcmk__opt_schedulerd, N_("Maximum number of jobs that the cluster may execute in parallel " "across all nodes"), N_("The \"correct\" value will depend on the speed and load of your " "network and cluster nodes. If set to 0, the cluster will " "impose a dynamically calculated limit when any node has a " "high load."), }, { PCMK_OPT_MIGRATION_LIMIT, NULL, PCMK_VALUE_INTEGER, NULL, "-1", pcmk__valid_int, pcmk__opt_schedulerd, N_("The number of live migration actions that the cluster is allowed " "to execute in parallel on a node (-1 means no limit)"), NULL, }, { /* @TODO This is actually ignored if not strictly positive. We should * overhaul value types in Pacemaker Explained. There are lots of * inaccurate ranges (assumptions of 32-bit width, "nonnegative" when * positive is required, etc.). * * Maybe a single integer type with the allowed range specified would be * better. * * Drop the PCMK_VALUE_NONNEGATIVE_INTEGER constant if we do this before * a release. */ PCMK_OPT_CLUSTER_IPC_LIMIT, NULL, PCMK_VALUE_NONNEGATIVE_INTEGER, NULL, "500", pcmk__valid_positive_int, pcmk__opt_based, - N_("Maximum IPC message backlog before disconnecting a cluster daemon"), + N_("Maximum IPC message backlog before disconnecting a command line client"), N_("Raise this if log has \"Evicting client\" messages for cluster " - "daemon PIDs (a good value is the number of resources in the " - "cluster multiplied by the number of nodes)."), + "PIDs (a good value is the number of resources in the cluster " + "multiplied by the number of nodes)."), }, // Stopping resources and removed resources { PCMK_OPT_STOP_ALL_RESOURCES, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether the cluster should stop all active resources"), NULL, }, { PCMK_OPT_STOP_REMOVED_RESOURCES, PCMK_OPT_STOP_ORPHAN_RESOURCES, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether to stop resources that were removed from the " "configuration"), NULL, }, { PCMK_OPT_STOP_REMOVED_ACTIONS, PCMK_OPT_STOP_ORPHAN_ACTIONS, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, pcmk__valid_boolean, pcmk__opt_schedulerd, N_("Whether to cancel recurring actions removed from the " "configuration"), NULL, }, // Storing inputs { PCMK_OPT_PE_ERROR_SERIES_MAX, NULL, PCMK_VALUE_INTEGER, NULL, "-1", pcmk__valid_int, pcmk__opt_schedulerd, N_("The number of scheduler inputs resulting in errors to save"), N_("Zero to disable, -1 to store unlimited."), }, { PCMK_OPT_PE_WARN_SERIES_MAX, NULL, PCMK_VALUE_INTEGER, NULL, "5000", pcmk__valid_int, pcmk__opt_schedulerd, N_("The number of scheduler inputs resulting in warnings to save"), N_("Zero to disable, -1 to store unlimited."), }, { PCMK_OPT_PE_INPUT_SERIES_MAX, NULL, PCMK_VALUE_INTEGER, NULL, "4000", pcmk__valid_int, pcmk__opt_schedulerd, N_("The number of scheduler inputs without errors or warnings to save"), N_("Zero to disable, -1 to store unlimited."), }, // Node health { PCMK_OPT_NODE_HEALTH_STRATEGY, NULL, PCMK_VALUE_SELECT, PCMK_VALUE_NONE ", " PCMK_VALUE_MIGRATE_ON_RED ", " PCMK_VALUE_ONLY_GREEN ", " PCMK_VALUE_PROGRESSIVE ", " PCMK_VALUE_CUSTOM, PCMK_VALUE_NONE, pcmk__validate_health_strategy, pcmk__opt_schedulerd, N_("How cluster should react to node health attributes"), N_("Requires external entities to create node attributes (named with " "the prefix \"#health\") with values \"red\", \"yellow\", or " "\"green\".") }, { PCMK_OPT_NODE_HEALTH_BASE, NULL, PCMK_VALUE_SCORE, NULL, "0", pcmk__valid_int, pcmk__opt_schedulerd, N_("Base health score assigned to a node"), N_("Only used when \"node-health-strategy\" is set to " "\"progressive\"."), }, { PCMK_OPT_NODE_HEALTH_GREEN, NULL, PCMK_VALUE_SCORE, NULL, "0", pcmk__valid_int, pcmk__opt_schedulerd, N_("The score to use for a node health attribute whose value is " "\"green\""), N_("Only used when \"node-health-strategy\" is set to \"custom\" or " "\"progressive\"."), }, { PCMK_OPT_NODE_HEALTH_YELLOW, NULL, PCMK_VALUE_SCORE, NULL, "0", pcmk__valid_int, pcmk__opt_schedulerd, N_("The score to use for a node health attribute whose value is " "\"yellow\""), N_("Only used when \"node-health-strategy\" is set to \"custom\" or " "\"progressive\"."), }, { PCMK_OPT_NODE_HEALTH_RED, NULL, PCMK_VALUE_SCORE, NULL, "-INFINITY", pcmk__valid_int, pcmk__opt_schedulerd, N_("The score to use for a node health attribute whose value is " "\"red\""), N_("Only used when \"node-health-strategy\" is set to \"custom\" or " "\"progressive\".") }, // Placement strategy { PCMK_OPT_PLACEMENT_STRATEGY, NULL, PCMK_VALUE_SELECT, PCMK_VALUE_DEFAULT ", " PCMK_VALUE_UTILIZATION ", " PCMK_VALUE_MINIMAL ", " PCMK_VALUE_BALANCED, PCMK_VALUE_DEFAULT, pcmk__valid_placement_strategy, pcmk__opt_schedulerd, N_("How the cluster should allocate resources to nodes"), NULL, }, { NULL, }, }; static const pcmk__cluster_option_t fencing_params[] = { /* name, old name, type, allowed values, * default value, validator, * flags, * short description, * long description */ { PCMK_STONITH_HOST_ARGUMENT, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_advanced, N_("Name of agent parameter that should be set to the fencing target"), N_("If the fencing agent metadata advertises support for the \"port\" " "or \"plug\" parameter, that will be used as the default, " "otherwise \"none\" will be used, which tells the cluster not to " "supply any additional parameters."), }, { PCMK_STONITH_HOST_MAP, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_none, N_("A mapping of node names to port numbers for devices that do not " "support node names."), N_("For example, \"node1:1;node2:2,3\" would tell the cluster to use " "port 1 for node1 and ports 2 and 3 for node2."), }, { PCMK_STONITH_HOST_LIST, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_none, N_("Nodes targeted by this device"), N_("Comma-separated list of nodes that can be targeted by this device " "(for example, \"node1,node2,node3\"). If pcmk_host_check is " "\"static-list\", either this or pcmk_host_map must be set."), }, { PCMK_STONITH_HOST_CHECK, NULL, PCMK_VALUE_SELECT, PCMK_VALUE_DYNAMIC_LIST ", " PCMK_VALUE_STATIC_LIST ", " PCMK_VALUE_STATUS ", " PCMK_VALUE_NONE, NULL, NULL, pcmk__opt_none, N_("How to determine which nodes can be targeted by the device"), N_("Use \"dynamic-list\" to query the device via the 'list' command; " "\"static-list\" to check the pcmk_host_list attribute; " "\"status\" to query the device via the 'status' command; or " "\"none\" to assume every device can fence every node. " "The default value is \"static-list\" if pcmk_host_map or " "pcmk_host_list is set; otherwise \"dynamic-list\" if the device " "supports the list operation; otherwise \"status\" if the device " "supports the status operation; otherwise \"none\""), }, { PCMK_STONITH_DELAY_MAX, NULL, PCMK_VALUE_DURATION, NULL, "0s", NULL, pcmk__opt_none, N_("Enable a delay of no more than the time specified before executing " "fencing actions."), N_("Enable a delay of no more than the time specified before executing " "fencing actions. Pacemaker derives the overall delay by taking " "the value of pcmk_delay_base and adding a random delay value such " "that the sum is kept below this maximum."), }, { PCMK_STONITH_DELAY_BASE, NULL, PCMK_VALUE_STRING, NULL, "0s", NULL, pcmk__opt_none, N_("Enable a base delay for fencing actions and specify base delay " "value."), N_("This enables a static delay for fencing actions, which can help " "avoid \"death matches\" where two nodes try to fence each other " "at the same time. If pcmk_delay_max is also used, a random delay " "will be added such that the total delay is kept below that value. " "This can be set to a single time value to apply to any node " "targeted by this device (useful if a separate device is " "configured for each target), or to a node map (for example, " "\"node1:1s;node2:5\") to set a different value for each target."), }, { PCMK_STONITH_ACTION_LIMIT, NULL, PCMK_VALUE_INTEGER, NULL, "1", NULL, pcmk__opt_none, N_("The maximum number of actions can be performed in parallel on this " "device"), N_("If the concurrent-fencing cluster property is \"true\", this " "specifies the maximum number of actions that can be performed in " "parallel on this device. A value of -1 means unlimited."), }, { "pcmk_reboot_action", NULL, PCMK_VALUE_STRING, NULL, PCMK_ACTION_REBOOT, NULL, pcmk__opt_advanced, N_("An alternate command to run instead of 'reboot'"), N_("Some devices do not support the standard commands or may provide " "additional ones. Use this to specify an alternate, device-" "specific, command that implements the 'reboot' action."), }, { "pcmk_reboot_timeout", NULL, PCMK_VALUE_TIMEOUT, NULL, "60s", NULL, pcmk__opt_advanced, N_("Specify an alternate timeout to use for 'reboot' actions instead " "of stonith-timeout"), N_("Some devices need much more/less time to complete than normal. " "Use this to specify an alternate, device-specific, timeout for " "'reboot' actions."), }, { "pcmk_reboot_retries", NULL, PCMK_VALUE_INTEGER, NULL, "2", NULL, pcmk__opt_advanced, N_("The maximum number of times to try the 'reboot' command within the " "timeout period"), N_("Some devices do not support multiple connections. Operations may " "\"fail\" if the device is busy with another task. In that case, " "Pacemaker will automatically retry the operation if there is time " "remaining. Use this option to alter the number of times Pacemaker " "tries a 'reboot' action before giving up."), }, { "pcmk_off_action", NULL, PCMK_VALUE_STRING, NULL, PCMK_ACTION_OFF, NULL, pcmk__opt_advanced, N_("An alternate command to run instead of 'off'"), N_("Some devices do not support the standard commands or may provide " "additional ones. Use this to specify an alternate, device-" "specific, command that implements the 'off' action."), }, { "pcmk_off_timeout", NULL, PCMK_VALUE_TIMEOUT, NULL, "60s", NULL, pcmk__opt_advanced, N_("Specify an alternate timeout to use for 'off' actions instead of " "stonith-timeout"), N_("Some devices need much more/less time to complete than normal. " "Use this to specify an alternate, device-specific, timeout for " "'off' actions."), }, { "pcmk_off_retries", NULL, PCMK_VALUE_INTEGER, NULL, "2", NULL, pcmk__opt_advanced, N_("The maximum number of times to try the 'off' command within the " "timeout period"), N_("Some devices do not support multiple connections. Operations may " "\"fail\" if the device is busy with another task. In that case, " "Pacemaker will automatically retry the operation if there is time " "remaining. Use this option to alter the number of times Pacemaker " "tries a 'off' action before giving up."), }, { "pcmk_on_action", NULL, PCMK_VALUE_STRING, NULL, PCMK_ACTION_ON, NULL, pcmk__opt_advanced, N_("An alternate command to run instead of 'on'"), N_("Some devices do not support the standard commands or may provide " "additional ones. Use this to specify an alternate, device-" "specific, command that implements the 'on' action."), }, { "pcmk_on_timeout", NULL, PCMK_VALUE_TIMEOUT, NULL, "60s", NULL, pcmk__opt_advanced, N_("Specify an alternate timeout to use for 'on' actions instead of " "stonith-timeout"), N_("Some devices need much more/less time to complete than normal. " "Use this to specify an alternate, device-specific, timeout for " "'on' actions."), }, { "pcmk_on_retries", NULL, PCMK_VALUE_INTEGER, NULL, "2", NULL, pcmk__opt_advanced, N_("The maximum number of times to try the 'on' command within the " "timeout period"), N_("Some devices do not support multiple connections. Operations may " "\"fail\" if the device is busy with another task. In that case, " "Pacemaker will automatically retry the operation if there is time " "remaining. Use this option to alter the number of times Pacemaker " "tries a 'on' action before giving up."), }, { "pcmk_list_action", NULL, PCMK_VALUE_STRING, NULL, PCMK_ACTION_LIST, NULL, pcmk__opt_advanced, N_("An alternate command to run instead of 'list'"), N_("Some devices do not support the standard commands or may provide " "additional ones. Use this to specify an alternate, device-" "specific, command that implements the 'list' action."), }, { "pcmk_list_timeout", NULL, PCMK_VALUE_TIMEOUT, NULL, "60s", NULL, pcmk__opt_advanced, N_("Specify an alternate timeout to use for 'list' actions instead of " "stonith-timeout"), N_("Some devices need much more/less time to complete than normal. " "Use this to specify an alternate, device-specific, timeout for " "'list' actions."), }, { "pcmk_list_retries", NULL, PCMK_VALUE_INTEGER, NULL, "2", NULL, pcmk__opt_advanced, N_("The maximum number of times to try the 'list' command within the " "timeout period"), N_("Some devices do not support multiple connections. Operations may " "\"fail\" if the device is busy with another task. In that case, " "Pacemaker will automatically retry the operation if there is time " "remaining. Use this option to alter the number of times Pacemaker " "tries a 'list' action before giving up."), }, { "pcmk_monitor_action", NULL, PCMK_VALUE_STRING, NULL, PCMK_ACTION_MONITOR, NULL, pcmk__opt_advanced, N_("An alternate command to run instead of 'monitor'"), N_("Some devices do not support the standard commands or may provide " "additional ones. Use this to specify an alternate, device-" "specific, command that implements the 'monitor' action."), }, { "pcmk_monitor_timeout", NULL, PCMK_VALUE_TIMEOUT, NULL, "60s", NULL, pcmk__opt_advanced, N_("Specify an alternate timeout to use for 'monitor' actions instead " "of stonith-timeout"), N_("Some devices need much more/less time to complete than normal. " "Use this to specify an alternate, device-specific, timeout for " "'monitor' actions."), }, { "pcmk_monitor_retries", NULL, PCMK_VALUE_INTEGER, NULL, "2", NULL, pcmk__opt_advanced, N_("The maximum number of times to try the 'monitor' command within " "the timeout period"), N_("Some devices do not support multiple connections. Operations may " "\"fail\" if the device is busy with another task. In that case, " "Pacemaker will automatically retry the operation if there is time " "remaining. Use this option to alter the number of times Pacemaker " "tries a 'monitor' action before giving up."), }, { "pcmk_status_action", NULL, PCMK_VALUE_STRING, NULL, PCMK_ACTION_STATUS, NULL, pcmk__opt_advanced, N_("An alternate command to run instead of 'status'"), N_("Some devices do not support the standard commands or may provide " "additional ones. Use this to specify an alternate, device-" "specific, command that implements the 'status' action."), }, { "pcmk_status_timeout", NULL, PCMK_VALUE_TIMEOUT, NULL, "60s", NULL, pcmk__opt_advanced, N_("Specify an alternate timeout to use for 'status' actions instead " "of stonith-timeout"), N_("Some devices need much more/less time to complete than normal. " "Use this to specify an alternate, device-specific, timeout for " "'status' actions."), }, { "pcmk_status_retries", NULL, PCMK_VALUE_INTEGER, NULL, "2", NULL, pcmk__opt_advanced, N_("The maximum number of times to try the 'status' command within " "the timeout period"), N_("Some devices do not support multiple connections. Operations may " "\"fail\" if the device is busy with another task. In that case, " "Pacemaker will automatically retry the operation if there is time " "remaining. Use this option to alter the number of times Pacemaker " "tries a 'status' action before giving up."), }, { NULL, }, }; static const pcmk__cluster_option_t primitive_meta[] = { /* name, old name, type, allowed values, * default value, validator, * flags, * short description, * long description */ { PCMK_META_PRIORITY, NULL, PCMK_VALUE_SCORE, NULL, "0", NULL, pcmk__opt_none, N_("Resource assignment priority"), N_("If not all resources can be active, the cluster will stop " "lower-priority resources in order to keep higher-priority ones " "active."), }, { PCMK_META_CRITICAL, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, NULL, pcmk__opt_none, N_("Default value for influence in colocation constraints"), N_("Use this value as the default for influence in all colocation " "constraints involving this resource, as well as in the implicit " "colocation constraints created if this resource is in a group."), }, { PCMK_META_TARGET_ROLE, NULL, PCMK_VALUE_SELECT, PCMK_ROLE_STOPPED ", " PCMK_ROLE_STARTED ", " PCMK_ROLE_UNPROMOTED ", " PCMK_ROLE_PROMOTED, PCMK_ROLE_STARTED, NULL, pcmk__opt_none, N_("State the cluster should attempt to keep this resource in"), N_("\"Stopped\" forces the resource to be stopped. " "\"Started\" allows the resource to be started (and in the case of " "promotable clone resources, promoted if appropriate). " "\"Unpromoted\" allows the resource to be started, but only in the " "unpromoted role if the resource is promotable. " "\"Promoted\" is equivalent to \"Started\"."), }, { PCMK_META_IS_MANAGED, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, NULL, pcmk__opt_none, N_("Whether the cluster is allowed to actively change the resource's " "state"), N_("If false, the cluster will not start, stop, promote, or demote the " "resource on any node. Recurring actions for the resource are " "unaffected. If true, a true value for the maintenance-mode " "cluster option, the maintenance node attribute, or the " "maintenance resource meta-attribute overrides this."), }, { PCMK_META_MAINTENANCE, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, NULL, pcmk__opt_none, N_("If true, the cluster will not schedule any actions involving the " "resource"), N_("If true, the cluster will not start, stop, promote, or demote the " "resource on any node, and will pause any recurring monitors " "(except those specifying role as \"Stopped\"). If false, a true " "value for the maintenance-mode cluster option or maintenance node " "attribute overrides this."), }, { PCMK_META_RESOURCE_STICKINESS, NULL, PCMK_VALUE_SCORE, NULL, NULL, NULL, pcmk__opt_none, N_("Score to add to the current node when a resource is already " "active"), N_("Score to add to the current node when a resource is already " "active. This allows running resources to stay where they are, " "even if they would be placed elsewhere if they were being started " "from a stopped state. " "The default is 1 for individual clone instances, and 0 for all " "other resources."), }, { PCMK_META_REQUIRES, NULL, PCMK_VALUE_SELECT, PCMK_VALUE_NOTHING ", " PCMK_VALUE_QUORUM ", " PCMK_VALUE_FENCING ", " PCMK_VALUE_UNFENCING, NULL, NULL, pcmk__opt_none, N_("Conditions under which the resource can be started"), N_("Conditions under which the resource can be started. " "\"nothing\" means the cluster can always start this resource. " "\"quorum\" means the cluster can start this resource only if a " "majority of the configured nodes are active. " "\"fencing\" means the cluster can start this resource only if a " "majority of the configured nodes are active and any failed or " "unknown nodes have been fenced. " "\"unfencing\" means the cluster can start this resource only if " "a majority of the configured nodes are active and any failed or " "unknown nodes have been fenced, and only on nodes that have been " "unfenced. " "The default is \"quorum\" for resources with a class of stonith; " "otherwise, \"unfencing\" if unfencing is active in the cluster; " "otherwise, \"fencing\" if the stonith-enabled cluster option is " "true; " "otherwise, \"quorum\"."), }, { PCMK_META_MIGRATION_THRESHOLD, NULL, PCMK_VALUE_SCORE, NULL, PCMK_VALUE_INFINITY, NULL, pcmk__opt_none, N_("Number of failures on a node before the resource becomes " "ineligible to run there."), N_("Number of failures that may occur for this resource on a node, " "before that node is marked ineligible to host this resource. A " "value of 0 indicates that this feature is disabled (the node will " "never be marked ineligible). By contrast, the cluster treats " "\"INFINITY\" (the default) as a very large but finite number. " "This option has an effect only if the failed operation specifies " "its on-fail attribute as \"restart\" (the default), and " "additionally for failed start operations, if the " "start-failure-is-fatal cluster property is set to false."), }, { PCMK_META_FAILURE_TIMEOUT, NULL, PCMK_VALUE_DURATION, NULL, "0", NULL, pcmk__opt_none, N_("Number of seconds before acting as if a failure had not occurred"), N_("Number of seconds after a failed action for this resource before " "acting as if the failure had not occurred, and potentially " "allowing the resource back to the node on which it failed. " "A value of 0 indicates that this feature is disabled."), }, { PCMK_META_MULTIPLE_ACTIVE, NULL, PCMK_VALUE_SELECT, PCMK_VALUE_BLOCK ", " PCMK_VALUE_STOP_ONLY ", " PCMK_VALUE_STOP_START ", " PCMK_VALUE_STOP_UNEXPECTED, PCMK_VALUE_STOP_START, NULL, pcmk__opt_none, N_("What to do if the cluster finds the resource active on more than " "one node"), N_("What to do if the cluster finds the resource active on more than " "one node. " "\"block\" means to mark the resource as unmanaged. " "\"stop_only\" means to stop all active instances of this resource " "and leave them stopped. " "\"stop_start\" means to stop all active instances of this " "resource and start the resource in one location only. " "\"stop_unexpected\" means to stop all active instances of this " "resource except where the resource should be active. (This should " "be used only when extra instances are not expected to disrupt " "existing instances, and the resource agent's monitor of an " "existing instance is capable of detecting any problems that could " "be caused. Note that any resources ordered after this one will " "still need to be restarted.)"), }, { PCMK_META_ALLOW_MIGRATE, NULL, PCMK_VALUE_BOOLEAN, NULL, NULL, NULL, pcmk__opt_none, N_("Whether the cluster should try to \"live migrate\" this resource " "when it needs to be moved"), N_("Whether the cluster should try to \"live migrate\" this resource " "when it needs to be moved. " "The default is true for ocf:pacemaker:remote resources, and false " "otherwise."), }, { PCMK_META_ALLOW_UNHEALTHY_NODES, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_FALSE, NULL, pcmk__opt_none, N_("Whether the resource should be allowed to run on a node even if " "the node's health score would otherwise prevent it"), NULL, }, { PCMK_META_CONTAINER_ATTRIBUTE_TARGET, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_none, N_("Where to check user-defined node attributes"), N_("Whether to check user-defined node attributes on the physical host " "where a container is running or on the local node. This is " "usually set for a bundle resource and inherited by the bundle's " "primitive resource. " "A value of \"host\" means to check user-defined node attributes " "on the underlying physical host. Any other value means to check " "user-defined node attributes on the local node (for a bundled " "primitive resource, this is the bundle node)."), }, { PCMK_META_REMOTE_NODE, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_none, N_("Name of the Pacemaker Remote guest node this resource is " "associated with, if any"), N_("Name of the Pacemaker Remote guest node this resource is " "associated with, if any. If specified, this both enables the " "resource as a guest node and defines the unique name used to " "identify the guest node. The guest must be configured to run the " "Pacemaker Remote daemon when it is started. " "WARNING: This value cannot overlap with any resource or node " "IDs."), }, { PCMK_META_REMOTE_ADDR, NULL, PCMK_VALUE_STRING, NULL, NULL, NULL, pcmk__opt_none, N_("If remote-node is specified, the IP address or hostname used to " "connect to the guest via Pacemaker Remote"), N_("If remote-node is specified, the IP address or hostname used to " "connect to the guest via Pacemaker Remote. The Pacemaker Remote " "daemon on the guest must be configured to accept connections on " "this address. " "The default is the value of the remote-node meta-attribute."), }, { PCMK_META_REMOTE_PORT, NULL, PCMK_VALUE_PORT, NULL, "3121", NULL, pcmk__opt_none, N_("If remote-node is specified, port on the guest used for its " "Pacemaker Remote connection"), N_("If remote-node is specified, the port on the guest used for its " "Pacemaker Remote connection. The Pacemaker Remote daemon on the " "guest must be configured to listen on this port."), }, { PCMK_META_REMOTE_CONNECT_TIMEOUT, NULL, PCMK_VALUE_TIMEOUT, NULL, "60s", NULL, pcmk__opt_none, N_("If remote-node is specified, how long before a pending Pacemaker " "Remote guest connection times out."), NULL, }, { PCMK_META_REMOTE_ALLOW_MIGRATE, NULL, PCMK_VALUE_BOOLEAN, NULL, PCMK_VALUE_TRUE, NULL, pcmk__opt_none, N_("If remote-node is specified, this acts as the allow-migrate " "meta-attribute for the implicit remote connection resource " "(ocf:pacemaker:remote)."), NULL, }, { NULL, }, }; /* * Environment variable option handling */ /*! * \internal * \brief Get the value of a Pacemaker environment variable option * * If an environment variable option is set, with either a \c "PCMK_" or (for * backward compatibility) \c "HA_" prefix, log and return the value. * * \param[in] option Environment variable name (without prefix) * * \return Value of environment variable, or \c NULL if not set */ const char * pcmk__env_option(const char *option) { // @COMPAT Drop support for "HA_" options eventually static const char *const prefixes[] = { "PCMK", "HA" }; CRM_CHECK(!pcmk__str_empty(option), return NULL); for (int i = 0; i < PCMK__NELEM(prefixes); i++) { char *env_name = crm_strdup_printf("%s_%s", prefixes[i], option); const char *value = getenv(env_name); if (value != NULL) { crm_trace("Found %s = %s", env_name, value); free(env_name); return value; } free(env_name); } crm_trace("Nothing found for %s", option); return NULL; } /*! * \internal * \brief Set or unset a Pacemaker environment variable option * * Set an environment variable option with a \c "PCMK_" prefix and optionally * an \c "HA_" prefix for backward compatibility. * * \param[in] option Environment variable name (without prefix) * \param[in] value New value (or NULL to unset) * \param[in] compat If false and \p value is not \c NULL, set only * \c "PCMK_