diff --git a/doc/sphinx/Pacemaker_Explained/constraints.rst b/doc/sphinx/Pacemaker_Explained/constraints.rst index 902f1d069e..a78d6c2bb1 100644 --- a/doc/sphinx/Pacemaker_Explained/constraints.rst +++ b/doc/sphinx/Pacemaker_Explained/constraints.rst @@ -1,1120 +1,1077 @@ .. index:: single: constraint single: resource; constraint .. _constraints: Resource Constraints -------------------- -.. index:: - single: resource; score - single: node; score - -Scores -###### - -Scores of all kinds are integral to how the cluster works. -Practically everything from moving a resource to deciding which -resource to stop in a degraded cluster is achieved by manipulating -scores in some way. - -Scores are calculated per resource and node. Any node with a -negative score for a resource can't run that resource. The cluster -places a resource on the node with the highest score for it. - -Infinity Math -_____________ - -Pacemaker implements **INFINITY** (or equivalently, **+INFINITY**) internally as a -score of 1,000,000. Addition and subtraction with it follow these three basic -rules: - -* Any value + **INFINITY** = **INFINITY** - -* Any value - **INFINITY** = -**INFINITY** - -* **INFINITY** - **INFINITY** = **-INFINITY** - -.. note:: - - What if you want to use a score higher than 1,000,000? Typically this possibility - arises when someone wants to base the score on some external metric that might - go above 1,000,000. - - The short answer is you can't. - - The long answer is it is sometimes possible work around this limitation - creatively. You may be able to set the score to some computed value based on - the external metric rather than use the metric directly. For nodes, you can - store the metric as a node attribute, and query the attribute when computing - the score (possibly as part of a custom resource agent). - .. _location-constraint: .. index:: single: location constraint single: constraint; location Deciding Which Nodes a Resource Can Run On ########################################## *Location constraints* tell the cluster which nodes a resource can run on. There are two alternative strategies. One way is to say that, by default, resources can run anywhere, and then the location constraints specify nodes that are not allowed (an *opt-out* cluster). The other way is to start with nothing able to run anywhere, and use location constraints to selectively enable allowed nodes (an *opt-in* cluster). Whether you should choose opt-in or opt-out depends on your personal preference and the make-up of your cluster. If most of your resources can run on most of the nodes, then an opt-out arrangement is likely to result in a simpler configuration. On the other-hand, if most resources can only run on a small subset of nodes, an opt-in configuration might be simpler. .. index:: pair: XML element; rsc_location single: constraint; rsc_location Location Properties ___________________ .. table:: **Attributes of a rsc_location Element** :class: longtable :widths: 1 1 4 +--------------------+---------+----------------------------------------------------------------------------------------------+ | Attribute | Default | Description | +====================+=========+==============================================================================================+ | id | | .. index:: | | | | single: rsc_location; attribute, id | | | | single: attribute; id (rsc_location) | | | | single: id; rsc_location attribute | | | | | | | | A unique name for the constraint (required) | +--------------------+---------+----------------------------------------------------------------------------------------------+ | rsc | | .. index:: | | | | single: rsc_location; attribute, rsc | | | | single: attribute; rsc (rsc_location) | | | | single: rsc; rsc_location attribute | | | | | | | | The name of the resource to which this constraint | | | | applies. A location constraint must either have a | | | | ``rsc``, have a ``rsc-pattern``, or contain at | | | | least one resource set. | +--------------------+---------+----------------------------------------------------------------------------------------------+ | rsc-pattern | | .. index:: | | | | single: rsc_location; attribute, rsc-pattern | | | | single: attribute; rsc-pattern (rsc_location) | | | | single: rsc-pattern; rsc_location attribute | | | | | | | | A pattern matching the names of resources to which | | | | this constraint applies. The syntax is the same as | | | | `POSIX `_ | | | | extended regular expressions, with the addition of an | | | | initial ``!`` indicating that resources *not* matching | | | | the pattern are selected. If the regular expression | | | | contains submatches, and the constraint is governed by | | | | a :ref:`rule `, the submatches can be | | | | referenced as ``%1`` through ``%9`` in the rule's | | | | ``score-attribute`` or a rule expression's ``attribute`` | | | | (see :ref:`s-rsc-pattern-rules`). A location constraint | | | | must either have a ``rsc``, have a ``rsc-pattern``, or | | | | contain at least one resource set. | +--------------------+---------+----------------------------------------------------------------------------------------------+ | node | | .. index:: | | | | single: rsc_location; attribute, node | | | | single: attribute; node (rsc_location) | | | | single: node; rsc_location attribute | | | | | | | | The name of the node to which this constraint applies. | | | | A location constraint must either have a ``node`` and | | | | ``score``, or contain at least one rule. | +--------------------+---------+----------------------------------------------------------------------------------------------+ | score | | .. index:: | | | | single: rsc_location; attribute, score | | | | single: attribute; score (rsc_location) | | | | single: score; rsc_location attribute | | | | | | | | Positive values indicate a preference for running the | | | | affected resource(s) on ``node`` -- the higher the value, | | | | the stronger the preference. Negative values indicate | | | | the resource(s) should avoid this node (a value of | | | | **-INFINITY** changes "should" to "must"). A location | | | | constraint must either have a ``node`` and ``score``, | | | | or contain at least one rule. | +--------------------+---------+----------------------------------------------------------------------------------------------+ | resource-discovery | always | .. index:: | | | | single: rsc_location; attribute, resource-discovery | | | | single: attribute; resource-discovery (rsc_location) | | | | single: resource-discovery; rsc_location attribute | | | | | | | | Whether Pacemaker should perform resource discovery | | | | (that is, check whether the resource is already running) | | | | for this resource on this node. This should normally be | | | | left as the default, so that rogue instances of a | | | | service can be stopped when they are running where they | | | | are not supposed to be. However, there are two | | | | situations where disabling resource discovery is a good | | | | idea: when a service is not installed on a node, | | | | discovery might return an error (properly written OCF | | | | agents will not, so this is usually only seen with other | | | | agent types); and when Pacemaker Remote is used to scale | | | | a cluster to hundreds of nodes, limiting resource | | | | discovery to allowed nodes can significantly boost | | | | performance. | | | | | | | | * ``always:`` Always perform resource discovery for | | | | the specified resource on this node. | | | | | | | | * ``never:`` Never perform resource discovery for the | | | | specified resource on this node. This option should | | | | generally be used with a -INFINITY score, although | | | | that is not strictly required. | | | | | | | | * ``exclusive:`` Perform resource discovery for the | | | | specified resource only on this node (and other nodes | | | | similarly marked as ``exclusive``). Multiple location | | | | constraints using ``exclusive`` discovery for the | | | | same resource across different nodes creates a subset | | | | of nodes resource-discovery is exclusive to. If a | | | | resource is marked for ``exclusive`` discovery on one | | | | or more nodes, that resource is only allowed to be | | | | placed within that subset of nodes. | +--------------------+---------+----------------------------------------------------------------------------------------------+ .. warning:: Setting ``resource-discovery`` to ``never`` or ``exclusive`` removes Pacemaker's ability to detect and stop unwanted instances of a service running where it's not supposed to be. It is up to the system administrator (you!) to make sure that the service can *never* be active on nodes without ``resource-discovery`` (such as by leaving the relevant software uninstalled). .. index:: single: Asymmetrical Clusters single: Opt-In Clusters Asymmetrical "Opt-In" Clusters ______________________________ To create an opt-in cluster, start by preventing resources from running anywhere by default: .. code-block:: none # crm_attribute --name symmetric-cluster --update false Then start enabling nodes. The following fragment says that the web server prefers **sles-1**, the database prefers **sles-2** and both can fail over to **sles-3** if their most preferred node fails. .. topic:: Opt-in location constraints for two resources .. code-block:: xml .. index:: single: Symmetrical Clusters single: Opt-Out Clusters Symmetrical "Opt-Out" Clusters ______________________________ To create an opt-out cluster, start by allowing resources to run anywhere by default: .. code-block:: none # crm_attribute --name symmetric-cluster --update true Then start disabling nodes. The following fragment is the equivalent of the above opt-in configuration. .. topic:: Opt-out location constraints for two resources .. code-block:: xml .. _node-score-equal: What if Two Nodes Have the Same Score _____________________________________ If two nodes have the same score, then the cluster will choose one. This choice may seem random and may not be what was intended, however the cluster was not given enough information to know any better. .. topic:: Constraints where a resource prefers two nodes equally .. code-block:: xml In the example above, assuming no other constraints and an inactive cluster, **Webserver** would probably be placed on **sles-1** and **Database** on **sles-2**. It would likely have placed **Webserver** based on the node's uname and **Database** based on the desire to spread the resource load evenly across the cluster. However other factors can also be involved in more complex configurations. .. _s-rsc-pattern: Specifying locations using pattern matching ___________________________________________ A location constraint can affect all resources whose IDs match a given pattern. The following example bans resources named **ip-httpd**, **ip-asterisk**, **ip-gateway**, etc., from **node1**. .. topic:: Location constraint banning all resources matching a pattern from one node .. code-block:: xml .. index:: single: constraint; ordering single: resource; start order .. _s-resource-ordering: Specifying the Order in which Resources Should Start/Stop ######################################################### *Ordering constraints* tell the cluster the order in which certain resource actions should occur. .. important:: Ordering constraints affect *only* the ordering of resource actions; they do *not* require that the resources be placed on the same node. If you want resources to be started on the same node *and* in a specific order, you need both an ordering constraint *and* a colocation constraint (see :ref:`s-resource-colocation`), or alternatively, a group (see :ref:`group-resources`). .. index:: pair: XML element; rsc_order pair: constraint; rsc_order Ordering Properties ___________________ .. table:: **Attributes of a rsc_order Element** :class: longtable :widths: 1 2 4 +--------------+----------------------------+-------------------------------------------------------------------+ | Field | Default | Description | +==============+============================+===================================================================+ | id | | .. index:: | | | | single: rsc_order; attribute, id | | | | single: attribute; id (rsc_order) | | | | single: id; rsc_order attribute | | | | | | | | A unique name for the constraint | +--------------+----------------------------+-------------------------------------------------------------------+ | first | | .. index:: | | | | single: rsc_order; attribute, first | | | | single: attribute; first (rsc_order) | | | | single: first; rsc_order attribute | | | | | | | | Name of the resource that the ``then`` resource | | | | depends on | +--------------+----------------------------+-------------------------------------------------------------------+ | then | | .. index:: | | | | single: rsc_order; attribute, then | | | | single: attribute; then (rsc_order) | | | | single: then; rsc_order attribute | | | | | | | | Name of the dependent resource | +--------------+----------------------------+-------------------------------------------------------------------+ | first-action | start | .. index:: | | | | single: rsc_order; attribute, first-action | | | | single: attribute; first-action (rsc_order) | | | | single: first-action; rsc_order attribute | | | | | | | | The action that the ``first`` resource must complete | | | | before ``then-action`` can be initiated for the ``then`` | | | | resource. Allowed values: ``start``, ``stop``, | | | | ``promote``, ``demote``. | +--------------+----------------------------+-------------------------------------------------------------------+ | then-action | value of ``first-action`` | .. index:: | | | | single: rsc_order; attribute, then-action | | | | single: attribute; then-action (rsc_order) | | | | single: first-action; rsc_order attribute | | | | | | | | The action that the ``then`` resource can execute only | | | | after the ``first-action`` on the ``first`` resource has | | | | completed. Allowed values: ``start``, ``stop``, | | | | ``promote``, ``demote``. | +--------------+----------------------------+-------------------------------------------------------------------+ | kind | Mandatory | .. index:: | | | | single: rsc_order; attribute, kind | | | | single: attribute; kind (rsc_order) | | | | single: kind; rsc_order attribute | | | | | | | | How to enforce the constraint. Allowed values: | | | | | | | | * ``Mandatory:`` ``then-action`` will never be initiated | | | | for the ``then`` resource unless and until ``first-action`` | | | | successfully completes for the ``first`` resource. | | | | | | | | * ``Optional:`` The constraint applies only if both specified | | | | resource actions are scheduled in the same transition | | | | (that is, in response to the same cluster state). This | | | | means that ``then-action`` is allowed on the ``then`` | | | | resource regardless of the state of the ``first`` resource, | | | | but if both actions happen to be scheduled at the same time, | | | | they will be ordered. | | | | | | | | * ``Serialize:`` Ensure that the specified actions are never | | | | performed concurrently for the specified resources. | | | | ``First-action`` and ``then-action`` can be executed in either | | | | order, but one must complete before the other can be initiated. | | | | An example use case is when resource start-up puts a high load | | | | on the host. | +--------------+----------------------------+-------------------------------------------------------------------+ | symmetrical | TRUE for ``Mandatory`` and | .. index:: | | | ``Optional`` kinds. FALSE | single: rsc_order; attribute, symmetrical | | | for ``Serialize`` kind. | single: attribute; symmetrical (rsc)order) | | | | single: symmetrical; rsc_order attribute | | | | | | | | If true, the reverse of the constraint applies for the | | | | opposite action (for example, if B starts after A starts, | | | | then B stops before A stops). ``Serialize`` orders cannot | | | | be symmetrical. | +--------------+----------------------------+-------------------------------------------------------------------+ ``Promote`` and ``demote`` apply to :ref:`promotable ` clone resources. Optional and mandatory ordering _______________________________ Here is an example of ordering constraints where **Database** *must* start before **Webserver**, and **IP** *should* start before **Webserver** if they both need to be started: .. topic:: Optional and mandatory ordering constraints .. code-block:: xml Because the above example lets ``symmetrical`` default to TRUE, **Webserver** must be stopped before **Database** can be stopped, and **Webserver** should be stopped before **IP** if they both need to be stopped. Symmetric and asymmetric ordering _________________________________ A mandatory symmetric ordering of "start A then start B" implies not only that the start actions must be ordered, but that B is not allowed to be active unless A is active. For example, if the ordering is added to the configuration when A is stopped (due to target-role, failure, etc.) and B is already active, then B will be stopped. By contrast, asymmetric ordering of "start A then start B" means the stops can occur in either order, which implies that B *can* remain active in the same situation. .. index:: single: colocation single: constraint; colocation single: resource; location relative to other resources .. _s-resource-colocation: Placing Resources Relative to other Resources ############################################# *Colocation constraints* tell the cluster that the location of one resource depends on the location of another one. Colocation has an important side-effect: it affects the order in which resources are assigned to a node. Think about it: You can't place A relative to B unless you know where B is [#]_. So when you are creating colocation constraints, it is important to consider whether you should colocate A with B, or B with A. .. important:: Colocation constraints affect *only* the placement of resources; they do *not* require that the resources be started in a particular order. If you want resources to be started on the same node *and* in a specific order, you need both an ordering constraint (see :ref:`s-resource-ordering`) *and* a colocation constraint, or alternatively, a group (see :ref:`group-resources`). .. index:: pair: XML element; rsc_colocation single: constraint; rsc_colocation Colocation Properties _____________________ .. table:: **Attributes of a rsc_colocation Constraint** :class: longtable :widths: 2 2 5 +----------------+----------------+--------------------------------------------------------+ | Field | Default | Description | +================+================+========================================================+ | id | | .. index:: | | | | single: rsc_colocation; attribute, id | | | | single: attribute; id (rsc_colocation) | | | | single: id; rsc_colocation attribute | | | | | | | | A unique name for the constraint (required). | +----------------+----------------+--------------------------------------------------------+ | rsc | | .. index:: | | | | single: rsc_colocation; attribute, rsc | | | | single: attribute; rsc (rsc_colocation) | | | | single: rsc; rsc_colocation attribute | | | | | | | | The name of a resource that should be located | | | | relative to ``with-rsc``. A colocation constraint must | | | | either contain at least one | | | | :ref:`resource set `, or specify both | | | | ``rsc`` and ``with-rsc``. | +----------------+----------------+--------------------------------------------------------+ | with-rsc | | .. index:: | | | | single: rsc_colocation; attribute, with-rsc | | | | single: attribute; with-rsc (rsc_colocation) | | | | single: with-rsc; rsc_colocation attribute | | | | | | | | The name of the resource used as the colocation | | | | target. The cluster will decide where to put this | | | | resource first and then decide where to put ``rsc``. | | | | A colocation constraint must either contain at least | | | | one :ref:`resource set `, or specify | | | | both ``rsc`` and ``with-rsc``. | +----------------+----------------+--------------------------------------------------------+ | node-attribute | #uname | .. index:: | | | | single: rsc_colocation; attribute, node-attribute | | | | single: attribute; node-attribute (rsc_colocation) | | | | single: node-attribute; rsc_colocation attribute | | | | | | | | If ``rsc`` and ``with-rsc`` are specified, this node | | | | attribute must be the same on the node running ``rsc`` | | | | and the node running ``with-rsc`` for the constraint | | | | to be satisfied. (For details, see | | | | :ref:`s-coloc-attribute`.) | +----------------+----------------+--------------------------------------------------------+ | score | 0 | .. index:: | | | | single: rsc_colocation; attribute, score | | | | single: attribute; score (rsc_colocation) | | | | single: score; rsc_colocation attribute | | | | | | | | Positive values indicate the resources should run on | | | | the same node. Negative values indicate the resources | | | | should run on different nodes. Values of | | | | +/- ``INFINITY`` change "should" to "must". | +----------------+----------------+--------------------------------------------------------+ | rsc-role | Started | .. index:: | | | | single: clone; ordering constraint, rsc-role | | | | single: ordering constraint; rsc-role (clone) | | | | single: rsc-role; clone ordering constraint | | | | | | | | If ``rsc`` and ``with-rsc`` are specified, and ``rsc`` | | | | is a :ref:`promotable clone `, | | | | the constraint applies only to ``rsc`` instances in | | | | this role. Allowed values: ``Started``, ``Stopped``, | | | | ``Promoted``, ``Unpromoted``. For details, see | | | | :ref:`promotable-clone-constraints`. | +----------------+----------------+--------------------------------------------------------+ | with-rsc-role | Started | .. index:: | | | | single: clone; ordering constraint, with-rsc-role | | | | single: ordering constraint; with-rsc-role (clone) | | | | single: with-rsc-role; clone ordering constraint | | | | | | | | If ``rsc`` and ``with-rsc`` are specified, and | | | | ``with-rsc`` is a | | | | :ref:`promotable clone `, the | | | | constraint applies only to ``with-rsc`` instances in | | | | this role. Allowed values: ``Started``, ``Stopped``, | | | | ``Promoted``, ``Unpromoted``. For details, see | | | | :ref:`promotable-clone-constraints`. | +----------------+----------------+--------------------------------------------------------+ | influence | value of | .. index:: | | | ``critical`` | single: rsc_colocation; attribute, influence | | | meta-attribute | single: attribute; influence (rsc_colocation) | | | for ``rsc`` | single: influence; rsc_colocation attribute | | | | | | | | Whether to consider the location preferences of | | | | ``rsc`` when ``with-rsc`` is already active. Allowed | | | | values: ``true``, ``false``. For details, see | | | | :ref:`s-coloc-influence`. *(since 2.1.0)* | +----------------+----------------+--------------------------------------------------------+ Mandatory Placement ___________________ Mandatory placement occurs when the constraint's score is **+INFINITY** or **-INFINITY**. In such cases, if the constraint can't be satisfied, then the **rsc** resource is not permitted to run. For ``score=INFINITY``, this includes cases where the ``with-rsc`` resource is not active. If you need resource **A** to always run on the same machine as resource **B**, you would add the following constraint: .. topic:: Mandatory colocation constraint for two resources .. code-block:: xml Remember, because **INFINITY** was used, if **B** can't run on any of the cluster nodes (for whatever reason) then **A** will not be allowed to run. Whether **A** is running or not has no effect on **B**. Alternatively, you may want the opposite -- that **A** *cannot* run on the same machine as **B**. In this case, use ``score="-INFINITY"``. .. topic:: Mandatory anti-colocation constraint for two resources .. code-block:: xml Again, by specifying **-INFINITY**, the constraint is binding. So if the only place left to run is where **B** already is, then **A** may not run anywhere. As with **INFINITY**, **B** can run even if **A** is stopped. However, in this case **A** also can run if **B** is stopped, because it still meets the constraint of **A** and **B** not running on the same node. Advisory Placement __________________ If mandatory placement is about "must" and "must not", then advisory placement is the "I'd prefer if" alternative. For colocation constraints with scores greater than **-INFINITY** and less than **INFINITY**, the cluster will try to accommodate your wishes, but may ignore them if other factors outweigh the colocation score. Those factors might include other constraints, resource stickiness, failure thresholds, whether other resources would be prevented from being active, etc. .. topic:: Advisory colocation constraint for two resources .. code-block:: xml .. _s-coloc-attribute: Colocation by Node Attribute ____________________________ The ``node-attribute`` property of a colocation constraints allows you to express the requirement, "these resources must be on similar nodes". As an example, imagine that you have two Storage Area Networks (SANs) that are not controlled by the cluster, and each node is connected to one or the other. You may have two resources **r1** and **r2** such that **r2** needs to use the same SAN as **r1**, but doesn't necessarily have to be on the same exact node. In such a case, you could define a :ref:`node attribute ` named **san**, with the value **san1** or **san2** on each node as appropriate. Then, you could colocate **r2** with **r1** using ``node-attribute`` set to **san**. .. _s-coloc-influence: Colocation Influence ____________________ By default, if A is colocated with B, the cluster will take into account A's preferences when deciding where to place B, to maximize the chance that both resources can run. For a detailed look at exactly how this occurs, see `Colocation Explained `_. However, if ``influence`` is set to ``false`` in the colocation constraint, this will happen only if B is inactive and needing to be started. If B is already active, A's preferences will have no effect on placing B. An example of what effect this would have and when it would be desirable would be a nonessential reporting tool colocated with a resource-intensive service that takes a long time to start. If the reporting tool fails enough times to reach its migration threshold, by default the cluster will want to move both resources to another node if possible. Setting ``influence`` to ``false`` on the colocation constraint would mean that the reporting tool would be stopped in this situation instead, to avoid forcing the service to move. The ``critical`` resource meta-attribute is a convenient way to specify the default for all colocation constraints and groups involving a particular resource. .. note:: If a noncritical resource is a member of a group, all later members of the group will be treated as noncritical, even if they are marked as (or left to default to) critical. .. _s-resource-sets: Resource Sets ############# .. index:: single: constraint; resource set single: resource; resource set *Resource sets* allow multiple resources to be affected by a single constraint. .. topic:: A set of 3 resources .. code-block:: xml Resource sets are valid inside ``rsc_location``, ``rsc_order`` (see :ref:`s-resource-sets-ordering`), ``rsc_colocation`` (see :ref:`s-resource-sets-colocation`), and ``rsc_ticket`` (see :ref:`ticket-constraints`) constraints. A resource set has a number of properties that can be set, though not all have an effect in all contexts. .. index:: pair: XML element; resource_set .. table:: **Attributes of a resource_set Element** :class: longtable :widths: 2 2 5 +-------------+------------------+--------------------------------------------------------+ | Field | Default | Description | +=============+==================+========================================================+ | id | | .. index:: | | | | single: resource_set; attribute, id | | | | single: attribute; id (resource_set) | | | | single: id; resource_set attribute | | | | | | | | A unique name for the set (required) | +-------------+------------------+--------------------------------------------------------+ | sequential | true | .. index:: | | | | single: resource_set; attribute, sequential | | | | single: attribute; sequential (resource_set) | | | | single: sequential; resource_set attribute | | | | | | | | Whether the members of the set must be acted on in | | | | order. Meaningful within ``rsc_order`` and | | | | ``rsc_colocation``. | +-------------+------------------+--------------------------------------------------------+ | require-all | true | .. index:: | | | | single: resource_set; attribute, require-all | | | | single: attribute; require-all (resource_set) | | | | single: require-all; resource_set attribute | | | | | | | | Whether all members of the set must be active before | | | | continuing. With the current implementation, the | | | | cluster may continue even if only one member of the | | | | set is started, but if more than one member of the set | | | | is starting at the same time, the cluster will still | | | | wait until all of those have started before continuing | | | | (this may change in future versions). Meaningful | | | | within ``rsc_order``. | +-------------+------------------+--------------------------------------------------------+ | role | | .. index:: | | | | single: resource_set; attribute, role | | | | single: attribute; role (resource_set) | | | | single: role; resource_set attribute | | | | | | | | The constraint applies only to resource set members | | | | that are :ref:`s-resource-promotable` in this | | | | role. Meaningful within ``rsc_location``, | | | | ``rsc_colocation`` and ``rsc_ticket``. | | | | Allowed values: ``Started``, ``Promoted``, | | | | ``Unpromoted``. For details, see | | | | :ref:`promotable-clone-constraints`. | +-------------+------------------+--------------------------------------------------------+ | action | value of | .. index:: | | | ``first-action`` | single: resource_set; attribute, action | | | in the enclosing | single: attribute; action (resource_set) | | | ordering | single: action; resource_set attribute | | | constraint | | | | | The action that applies to *all members* of the set. | | | | Meaningful within ``rsc_order``. Allowed values: | | | | ``start``, ``stop``, ``promote``, ``demote``. | +-------------+------------------+--------------------------------------------------------+ | score | | .. index:: | | | | single: resource_set; attribute, score | | | | single: attribute; score (resource_set) | | | | single: score; resource_set attribute | | | | | | | | *Advanced use only.* Use a specific score for this | | | | set within the constraint. | +-------------+------------------+--------------------------------------------------------+ .. _s-resource-sets-ordering: Ordering Sets of Resources ########################## A common situation is for an administrator to create a chain of ordered resources, such as: .. topic:: A chain of ordered resources .. code-block:: xml .. topic:: Visual representation of the four resources' start order for the above constraints .. image:: images/resource-set.png :alt: Ordered set Ordered Set ___________ To simplify this situation, :ref:`s-resource-sets` can be used within ordering constraints: .. topic:: A chain of ordered resources expressed as a set .. code-block:: xml While the set-based format is not less verbose, it is significantly easier to get right and maintain. .. important:: If you use a higher-level tool, pay attention to how it exposes this functionality. Depending on the tool, creating a set **A B** may be equivalent to **A then B**, or **B then A**. Ordering Multiple Sets ______________________ The syntax can be expanded to allow sets of resources to be ordered relative to each other, where the members of each individual set may be ordered or unordered (controlled by the ``sequential`` property). In the example below, **A** and **B** can both start in parallel, as can **C** and **D**, however **C** and **D** can only start once *both* **A** *and* **B** are active. .. topic:: Ordered sets of unordered resources .. code-block:: xml .. topic:: Visual representation of the start order for two ordered sets of unordered resources .. image:: images/two-sets.png :alt: Two ordered sets Of course either set -- or both sets -- of resources can also be internally ordered (by setting ``sequential="true"``) and there is no limit to the number of sets that can be specified. .. topic:: Advanced use of set ordering - Three ordered sets, two of which are internally unordered .. code-block:: xml .. topic:: Visual representation of the start order for the three sets defined above .. image:: images/three-sets.png :alt: Three ordered sets .. important:: An ordered set with ``sequential=false`` makes sense only if there is another set in the constraint. Otherwise, the constraint has no effect. Resource Set OR Logic _____________________ The unordered set logic discussed so far has all been "AND" logic. To illustrate this take the 3 resource set figure in the previous section. Those sets can be expressed, **(A and B) then (C) then (D) then (E and F)**. Say for example we want to change the first set, **(A and B)**, to use "OR" logic so the sets look like this: **(A or B) then (C) then (D) then (E and F)**. This functionality can be achieved through the use of the ``require-all`` option. This option defaults to TRUE which is why the "AND" logic is used by default. Setting ``require-all=false`` means only one resource in the set needs to be started before continuing on to the next set. .. topic:: Resource Set "OR" logic: Three ordered sets, where the first set is internally unordered with "OR" logic .. code-block:: xml .. important:: An ordered set with ``require-all=false`` makes sense only in conjunction with ``sequential=false``. Think of it like this: ``sequential=false`` modifies the set to be an unordered set using "AND" logic by default, and adding ``require-all=false`` flips the unordered set's "AND" logic to "OR" logic. .. _s-resource-sets-colocation: Colocating Sets of Resources ############################ Another common situation is for an administrator to create a set of colocated resources. The simplest way to do this is to define a resource group (see :ref:`group-resources`), but that cannot always accurately express the desired relationships. For example, maybe the resources do not need to be ordered. Another way would be to define each relationship as an individual constraint, but that causes a difficult-to-follow constraint explosion as the number of resources and combinations grow. .. topic:: Colocation chain as individual constraints, where A is placed first, then B, then C, then D .. code-block:: xml To express complicated relationships with a simplified syntax [#]_, :ref:`resource sets ` can be used within colocation constraints. .. topic:: Equivalent colocation chain expressed using **resource_set** .. code-block:: xml .. note:: Within a ``resource_set``, the resources are listed in the order they are *placed*, which is the reverse of the order in which they are *colocated*. In the above example, resource **A** is placed before resource **B**, which is the same as saying resource **B** is colocated with resource **A**. As with individual constraints, a resource that can't be active prevents any resource that must be colocated with it from being active. In both of the two previous examples, if **B** is unable to run, then both **C** and by inference **D** must remain stopped. .. important:: If you use a higher-level tool, pay attention to how it exposes this functionality. Depending on the tool, creating a set **A B** may be equivalent to **A with B**, or **B with A**. Resource sets can also be used to tell the cluster that entire *sets* of resources must be colocated relative to each other, while the individual members within any one set may or may not be colocated relative to each other (determined by the set's ``sequential`` property). In the following example, resources **B**, **C**, and **D** will each be colocated with **A** (which will be placed first). **A** must be able to run in order for any of the resources to run, but any of **B**, **C**, or **D** may be stopped without affecting any of the others. .. topic:: Using colocated sets to specify a shared dependency .. code-block:: xml .. note:: Pay close attention to the order in which resources and sets are listed. While the members of any one sequential set are placed first to last (i.e., the colocation dependency is last with first), multiple sets are placed last to first (i.e. the colocation dependency is first with last). .. important:: A colocated set with ``sequential="false"`` makes sense only if there is another set in the constraint. Otherwise, the constraint has no effect. There is no inherent limit to the number and size of the sets used. The only thing that matters is that in order for any member of one set in the constraint to be active, all members of sets listed after it must also be active (and naturally on the same node); and if a set has ``sequential="true"``, then in order for one member of that set to be active, all members listed before it must also be active. If desired, you can restrict the dependency to instances of promotable clone resources that are in a specific role, using the set's ``role`` property. .. topic:: Colocation in which the members of the middle set have no interdependencies, and the last set listed applies only to promoted instances .. code-block:: xml .. topic:: Visual representation of the above example (resources are placed from left to right) .. image:: ../shared/images/pcmk-colocated-sets.png :alt: Colocation chain .. note:: Unlike ordered sets, colocated sets do not use the ``require-all`` option. External Resource Dependencies ############################## Sometimes, a resource will depend on services that are not managed by the cluster. An example might be a resource that requires a file system that is not managed by the cluster but mounted by systemd at boot time. To accommodate this, the pacemaker systemd service depends on a normally empty target called ``resource-agents-deps.target``. The system administrator may create a unit drop-in for that target specifying the dependencies, to ensure that the services are started before Pacemaker starts and stopped after Pacemaker stops. Typically, this is accomplished by placing a unit file in the ``/etc/systemd/system/resource-agents-deps.target.d`` directory, with directives such as ``Requires`` and ``After`` specifying the dependencies as needed. .. [#] While the human brain is sophisticated enough to read the constraint in any order and choose the correct one depending on the situation, the cluster is not quite so smart. Yet. .. [#] which is not the same as saying easy to follow diff --git a/doc/sphinx/Pacemaker_Explained/options.rst b/doc/sphinx/Pacemaker_Explained/options.rst index 1e21400663..6436c17443 100644 --- a/doc/sphinx/Pacemaker_Explained/options.rst +++ b/doc/sphinx/Pacemaker_Explained/options.rst @@ -1,760 +1,802 @@ Cluster-Wide Configuration -------------------------- .. index:: pair: XML element; cib pair: XML element; configuration Configuration Layout #################### The cluster is defined by the Cluster Information Base (CIB), which uses XML notation. The simplest CIB, an empty one, looks like this: .. topic:: An empty configuration .. code-block:: xml The empty configuration above contains the major sections that make up a CIB: * ``cib``: The entire CIB is enclosed with a ``cib`` element. Certain fundamental settings are defined as attributes of this element. * ``configuration``: This section -- the primary focus of this document -- contains traditional configuration information such as what resources the cluster serves and the relationships among them. * ``crm_config``: cluster-wide configuration options * ``nodes``: the machines that host the cluster * ``resources``: the services run by the cluster * ``constraints``: indications of how resources should be placed * ``status``: This section contains the history of each resource on each node. Based on this data, the cluster can construct the complete current state of the cluster. The authoritative source for this section is the local executor (pacemaker-execd process) on each cluster node, and the cluster will occasionally repopulate the entire section. For this reason, it is never written to disk, and administrators are advised against modifying it in any way. In this document, configuration settings will be described as properties or options based on how they are defined in the CIB: * Properties are XML attributes of an XML element. * Options are name-value pairs expressed as ``nvpair`` child elements of an XML element. Normally, you will use command-line tools that abstract the XML, so the distinction will be unimportant; both properties and options are cluster settings you can tweak. Configuration Value Types ######################### Throughout this document, configuration values will be designated as having one of the following types: .. table:: **Configuration Value Types** :class: longtable :widths: 1 3 +-------------------+-------------------------------------------------------+ | Type | Description | +===================+=======================================================+ | boolean | .. _boolean: | | | | | | .. index:: | | | pair: type; boolean | | | | | | Case-insensitive true/false value where "1", "yes", | | | "y", "on", and "true" evaluate as true and "0", "no", | | | "n", "off", "false", and unset evaluate as false | +-------------------+-------------------------------------------------------+ | date/time | .. _date_time: | | | | | | .. index:: | | | pair: type; date/time | | | | | | Textual timestamp like "Sat Dec 21 11:47:45 2013" | +-------------------+-------------------------------------------------------+ | enumeration | .. _enumeration: | | | | | | .. index:: | | | pair: type; enumeration | | | | | | Text that must be one of a set of defined values | | | (which will be listed in the description) | +-------------------+-------------------------------------------------------+ | integer | .. _integer: | | | | | | .. index:: | | | pair: type; integer | | | | | | 32-bit signed integer value (-2,147,483,648 to | | | 2,147,483,647) | +-------------------+-------------------------------------------------------+ | nonnegative | .. _nonnegative_integer: | | integer | | | | .. index:: | | | pair: type; nonnegative integer | | | | | | 32-bit nonnegative integer value (0 to 2,147,483,647) | +-------------------+-------------------------------------------------------+ | port | .. _port: | | | | | | .. index:: | | | pair: type; port | | | | | | Integer TCP port number (0 to 65535) | +-------------------+-------------------------------------------------------+ + | score | .. _score: | + | | | + | | .. index:: | + | | pair: type; score | + | | | + | | A Pacemaker score can be an integer between | + | | -1,000,000 and 1,000,000, or a string alias: | + | | ``INFINITY`` or ``+INFINITY`` is equivalent to | + | | 1,000,000, ``-INFINITY`` is equivalent to -1,000,000, | + | | and ``red``, ``yellow``, and ``green`` are equivalent | + | | to integers as described in :ref:`node-health`. | + +-------------------+-------------------------------------------------------+ | text | .. _text: | | | | | | .. index:: | | | pair: type; text | | | | | | A text string | +-------------------+-------------------------------------------------------+ | version | .. _version: | | | | | | .. index:: | | | pair: type; version | | | | | | Version number (three integers separated by dots) | +-------------------+-------------------------------------------------------+ +Scores +______ + +Scores are integral to how Pacemaker works. Practically everything from moving +a resource to deciding which resource to stop in a degraded cluster is achieved +by manipulating scores in some way. + +Scores are calculated per resource and node. Any node with a negative score for +a resource can't run that resource. The cluster places a resource on the node +with the highest score for it. + +Score addition and subtraction follow these rules: + +* Any value (including ``INFINITY``) - ``INFINITY`` = ``-INFINITY`` +* ``INFINITY`` + any value other than ``-INFINITY`` = ``INFINITY`` + +.. note:: + + What if you want to use a score higher than 1,000,000? Typically this possibility + arises when someone wants to base the score on some external metric that might + go above 1,000,000. + + The short answer is you can't. + + The long answer is it is sometimes possible work around this limitation + creatively. You may be able to set the score to some computed value based on + the external metric rather than use the metric directly. For nodes, you can + store the metric as a node attribute, and query the attribute when computing + the score (possibly as part of a custom resource agent). + CIB Properties ############## Certain settings are defined by CIB properties (that is, attributes of the ``cib`` tag) rather than with the rest of the cluster configuration in the ``configuration`` section. The reason is simply a matter of parsing. These options are used by the configuration database which is, by design, mostly ignorant of the content it holds. So the decision was made to place them in an easy-to-find location. .. list-table:: **CIB Properties** :class: longtable :widths: 2 2 2 5 :header-rows: 1 * - Attribute - Type - Default - Description * - .. _admin_epoch: .. index:: pair: admin_epoch; cib admin_epoch - :ref:`nonnegative integer ` - 0 - When a node joins the cluster, the cluster asks the node with the highest (``admin_epoch``, ``epoch``, ``num_updates``) tuple to replace the configuration on all the nodes -- which makes setting them correctly very important. ``admin_epoch`` is never modified by the cluster; you can use this to make the configurations on any inactive nodes obsolete. * - .. _epoch: .. index:: pair: epoch; cib epoch - :ref:`nonnegative integer ` - 0 - The cluster increments this every time the CIB's configuration section is updated. * - .. _num_updates: .. index:: pair: num_updates; cib num_updates - :ref:`nonnegative integer ` - 0 - The cluster increments this every time the CIB's configuration or status sections are updated, and resets it to 0 when epoch changes. * - .. _validate_with: .. index:: pair: validate-with; cib validate-with - :ref:`enumeration ` - - Determines the type of XML validation that will be done on the configuration. Allowed values are ``none`` (in which case the cluster will not require that updates conform to expected syntax) and the base names of schema files installed on the local machine (for example, "pacemaker-3.9") * - .. _remote_tls_port: .. index:: pair: remote-tls-port; cib remote-tls-port - :ref:`port ` - - If set, the CIB manager will listen for anonymously encrypted remote connections on this port, to allow CIB administration from hosts not in the cluster. No key is used, so this should be used only on a protected network where man-in-the-middle attacks can be avoided. * - .. _remote_clear_port: .. index:: pair: remote-clear-port; cib remote-clear-port - :ref:`port ` - - If set to a TCP port number, the CIB manager will listen for remote connections on this port, to allow for CIB administration from hosts not in the cluster. No encryption is used, so this should be used only on a protected network. * - .. _cib_last_written: .. index:: pair: cib-last-written; cib cib-last-written - :ref:`date/time ` - - Indicates when the configuration was last written to disk. Maintained by the cluster; for informational purposes only. * - .. _have_quorum: .. index:: pair: have-quorum; cib have-quorum - :ref:`boolean ` - - Indicates whether the cluster has quorum. If false, the cluster's response is determined by ``no-quorum-policy`` (see below). Maintained by the cluster. * - .. _dc_uuid: .. index:: pair: dc-uuid; cib dc-uuid - :ref:`text ` - - Node ID of the cluster's current designated controller (DC). Used and maintained by the cluster. .. _cluster_options: Cluster Options ############### Cluster options, as you might expect, control how the cluster behaves when confronted with various situations. They are grouped into sets within the ``crm_config`` section. In advanced configurations, there may be more than one set. (This will be described later in the chapter on :ref:`rules` where we will show how to have the cluster use different sets of options during working hours than during weekends.) For now, we will describe the simple case where each option is present at most once. You can obtain an up-to-date list of cluster options, including their default values, by running the ``man pacemaker-schedulerd`` and ``man pacemaker-controld`` commands. .. table:: **Cluster Options** :class: longtable :widths: 2 1 4 +---------------------------+---------+----------------------------------------------------+ | Option | Default | Description | +===========================+=========+====================================================+ | cluster-name | | .. index:: | | | | pair: cluster option; cluster-name | | | | | | | | An (optional) name for the cluster as a whole. | | | | This is mostly for users' convenience for use | | | | as desired in administration, but this can be | | | | used in the Pacemaker configuration in | | | | :ref:`rules` (as the ``#cluster-name`` | | | | :ref:`node attribute | | | | `. It may | | | | also be used by higher-level tools when | | | | displaying cluster information, and by | | | | certain resource agents (for example, the | | | | ``ocf:heartbeat:GFS2`` agent stores the | | | | cluster name in filesystem meta-data). | +---------------------------+---------+----------------------------------------------------+ | dc-version | | .. index:: | | | | pair: cluster option; dc-version | | | | | | | | Version of Pacemaker on the cluster's DC. | | | | Determined automatically by the cluster. Often | | | | includes the hash which identifies the exact | | | | Git changeset it was built from. Used for | | | | diagnostic purposes. | +---------------------------+---------+----------------------------------------------------+ | cluster-infrastructure | | .. index:: | | | | pair: cluster option; cluster-infrastructure | | | | | | | | The messaging stack on which Pacemaker is | | | | currently running. Determined automatically by | | | | the cluster. Used for informational and | | | | diagnostic purposes. | +---------------------------+---------+----------------------------------------------------+ | no-quorum-policy | stop | .. index:: | | | | pair: cluster option; no-quorum-policy | | | | | | | | What to do when the cluster does not have | | | | quorum. Allowed values: | | | | | | | | * ``ignore:`` continue all resource management | | | | * ``freeze:`` continue resource management, but | | | | don't recover resources from nodes not in the | | | | affected partition | | | | * ``stop:`` stop all resources in the affected | | | | cluster partition | | | | * ``demote:`` demote promotable resources and | | | | stop all other resources in the affected | | | | cluster partition *(since 2.0.5)* | | | | * ``suicide:`` fence all nodes in the affected | | | | cluster partition | +---------------------------+---------+----------------------------------------------------+ | batch-limit | 0 | .. index:: | | | | pair: cluster option; batch-limit | | | | | | | | The maximum number of actions that the cluster | | | | may execute in parallel across all nodes. The | | | | "correct" value will depend on the speed and | | | | load of your network and cluster nodes. If zero, | | | | the cluster will impose a dynamically calculated | | | | limit only when any node has high load. If -1, the | | | | cluster will not impose any limit. | +---------------------------+---------+----------------------------------------------------+ | migration-limit | -1 | .. index:: | | | | pair: cluster option; migration-limit | | | | | | | | The number of | | | | :ref:`live migration ` actions | | | | that the cluster is allowed to execute in | | | | parallel on a node. A value of -1 means | | | | unlimited. | +---------------------------+---------+----------------------------------------------------+ | symmetric-cluster | true | .. index:: | | | | pair: cluster option; symmetric-cluster | | | | | | | | Whether resources can run on any node by default | | | | (if false, a resource is allowed to run on a | | | | node only if a | | | | :ref:`location constraint ` | | | | enables it) | +---------------------------+---------+----------------------------------------------------+ | stop-all-resources | false | .. index:: | | | | pair: cluster option; stop-all-resources | | | | | | | | Whether all resources should be disallowed from | | | | running (can be useful during maintenance) | +---------------------------+---------+----------------------------------------------------+ | stop-orphan-resources | true | .. index:: | | | | pair: cluster option; stop-orphan-resources | | | | | | | | Whether resources that have been deleted from | | | | the configuration should be stopped. This value | | | | takes precedence over | | | | :ref:`is-managed ` (that is, even | | | | unmanaged resources will be stopped when orphaned | | | | if this value is ``true``). | +---------------------------+---------+----------------------------------------------------+ | stop-orphan-actions | true | .. index:: | | | | pair: cluster option; stop-orphan-actions | | | | | | | | Whether recurring :ref:`operations ` | | | | that have been deleted from the configuration | | | | should be cancelled | +---------------------------+---------+----------------------------------------------------+ | start-failure-is-fatal | true | .. index:: | | | | pair: cluster option; start-failure-is-fatal | | | | | | | | Whether a failure to start a resource on a | | | | particular node prevents further start attempts | | | | on that node? If ``false``, the cluster will | | | | decide whether the node is still eligible based | | | | on the resource's current failure count and | | | | :ref:`migration-threshold `. | +---------------------------+---------+----------------------------------------------------+ | enable-startup-probes | true | .. index:: | | | | pair: cluster option; enable-startup-probes | | | | | | | | Whether the cluster should check the | | | | pre-existing state of resources when the cluster | | | | starts | +---------------------------+---------+----------------------------------------------------+ | maintenance-mode | false | .. _maintenance_mode: | | | | | | | | .. index:: | | | | pair: cluster option; maintenance-mode | | | | | | | | If true, the cluster will not start or stop any | | | | resource in the cluster, and any recurring | | | | operations (expect those specifying ``role`` as | | | | ``Stopped``) will be paused. If true, this | | | | overrides the | | | | :ref:`maintenance ` node | | | | attribute, :ref:`is-managed ` and | | | | :ref:`maintenance ` resource | | | | meta-attributes, and :ref:`enabled ` | | | | operation meta-attribute. | +---------------------------+---------+----------------------------------------------------+ | stonith-enabled | true | .. index:: | | | | pair: cluster option; stonith-enabled | | | | | | | | Whether the cluster is allowed to fence nodes | | | | (for example, failed nodes and nodes with | | | | resources that can't be stopped). | | | | | | | | If true, at least one fence device must be | | | | configured before resources are allowed to run. | | | | | | | | If false, unresponsive nodes are immediately | | | | assumed to be running no resources, and resource | | | | recovery on online nodes starts without any | | | | further protection (which can mean *data loss* | | | | if the unresponsive node still accesses shared | | | | storage, for example). See also the | | | | :ref:`requires ` resource | | | | meta-attribute. | +---------------------------+---------+----------------------------------------------------+ | stonith-action | reboot | .. index:: | | | | pair: cluster option; stonith-action | | | | | | | | Action the cluster should send to the fence agent | | | | when a node must be fenced. Allowed values are | | | | ``reboot``, ``off``, and (for legacy agents only) | | | | ``poweroff``. | +---------------------------+---------+----------------------------------------------------+ | stonith-timeout | 60s | .. index:: | | | | pair: cluster option; stonith-timeout | | | | | | | | How long to wait for ``on``, ``off``, and | | | | ``reboot`` fence actions to complete by default. | +---------------------------+---------+----------------------------------------------------+ | stonith-max-attempts | 10 | .. index:: | | | | pair: cluster option; stonith-max-attempts | | | | | | | | How many times fencing can fail for a target | | | | before the cluster will no longer immediately | | | | re-attempt it. | +---------------------------+---------+----------------------------------------------------+ | stonith-watchdog-timeout | 0 | .. index:: | | | | pair: cluster option; stonith-watchdog-timeout | | | | | | | | If nonzero, and the cluster detects | | | | ``have-watchdog`` as ``true``, then watchdog-based | | | | self-fencing will be performed via SBD when | | | | fencing is required, without requiring a fencing | | | | resource explicitly configured. | | | | | | | | If this is set to a positive value, unseen nodes | | | | are assumed to self-fence within this much time. | | | | | | | | **Warning:** It must be ensured that this value is | | | | larger than the ``SBD_WATCHDOG_TIMEOUT`` | | | | environment variable on all nodes. Pacemaker | | | | verifies the settings individually on all nodes | | | | and prevents startup or shuts down if configured | | | | wrongly on the fly. It is strongly recommended | | | | that ``SBD_WATCHDOG_TIMEOUT`` be set to the same | | | | value on all nodes. | | | | | | | | If this is set to a negative value, and | | | | ``SBD_WATCHDOG_TIMEOUT`` is set, twice that value | | | | will be used. | | | | | | | | **Warning:** In this case, it is essential (and | | | | currently not verified by pacemaker) that | | | | ``SBD_WATCHDOG_TIMEOUT`` is set to the same | | | | value on all nodes. | +---------------------------+---------+----------------------------------------------------+ | concurrent-fencing | false | .. index:: | | | | pair: cluster option; concurrent-fencing | | | | | | | | Whether the cluster is allowed to initiate | | | | multiple fence actions concurrently. Fence actions | | | | initiated externally, such as via the | | | | ``stonith_admin`` tool or an application such as | | | | DLM, or by the fencer itself such as recurring | | | | device monitors and ``status`` and ``list`` | | | | commands, are not limited by this option. | +---------------------------+---------+----------------------------------------------------+ | fence-reaction | stop | .. index:: | | | | pair: cluster option; fence-reaction | | | | | | | | How should a cluster node react if notified of its | | | | own fencing? A cluster node may receive | | | | notification of its own fencing if fencing is | | | | misconfigured, or if fabric fencing is in use that | | | | doesn't cut cluster communication. Allowed values | | | | are ``stop`` to attempt to immediately stop | | | | pacemaker and stay stopped, or ``panic`` to | | | | attempt to immediately reboot the local node, | | | | falling back to stop on failure. The default is | | | | likely to be changed to ``panic`` in a future | | | | release. *(since 2.0.3)* | +---------------------------+---------+----------------------------------------------------+ | priority-fencing-delay | 0 | .. index:: | | | | pair: cluster option; priority-fencing-delay | | | | | | | | Apply this delay to any fencing targeting the lost | | | | nodes with the highest total resource priority in | | | | case we don't have the majority of the nodes in | | | | our cluster partition, so that the more | | | | significant nodes potentially win any fencing | | | | match (especially meaningful in a split-brain of a | | | | 2-node cluster). A promoted resource instance | | | | takes the resource's priority plus 1 if the | | | | resource's priority is not 0. Any static or random | | | | delays introduced by ``pcmk_delay_base`` and | | | | ``pcmk_delay_max`` configured for the | | | | corresponding fencing resources will be added to | | | | this delay. This delay should be significantly | | | | greater than (safely twice) the maximum delay from | | | | those parameters. *(since 2.0.4)* | +---------------------------+---------+----------------------------------------------------+ | node-pending-timeout | 10min | .. index:: | | | | pair: cluster option; node-pending-timeout | | | | | | | | A node that has joined the cluster can be pending | | | | on joining the process group. We wait up to this | | | | much time for it. If it times out, fencing | | | | targeting the node will be issued if enabled. | | | | *(since 2.1.7)* | +---------------------------+---------+----------------------------------------------------+ | cluster-delay | 60s | .. index:: | | | | pair: cluster option; cluster-delay | | | | | | | | Estimated maximum round-trip delay over the | | | | network (excluding action execution). If the DC | | | | requires an action to be executed on another node, | | | | it will consider the action failed if it does not | | | | get a response from the other node in this time | | | | (after considering the action's own timeout). The | | | | "correct" value will depend on the speed and load | | | | of your network and cluster nodes. | +---------------------------+---------+----------------------------------------------------+ | dc-deadtime | 20s | .. index:: | | | | pair: cluster option; dc-deadtime | | | | | | | | How long to wait for a response from other nodes | | | | during startup. The "correct" value will depend on | | | | the speed/load of your network and the type of | | | | switches used. | +---------------------------+---------+----------------------------------------------------+ | cluster-ipc-limit | 500 | .. index:: | | | | pair: cluster option; cluster-ipc-limit | | | | | | | | The maximum IPC message backlog before one cluster | | | | daemon will disconnect another. This is of use in | | | | large clusters, for which a good value is the | | | | number of resources in the cluster multiplied by | | | | the number of nodes. The default of 500 is also | | | | the minimum. Raise this if you see | | | | "Evicting client" messages for cluster daemon PIDs | | | | in the logs. | +---------------------------+---------+----------------------------------------------------+ | pe-error-series-max | -1 | .. index:: | | | | pair: cluster option; pe-error-series-max | | | | | | | | The number of scheduler inputs resulting in errors | | | | to save. Used when reporting problems. A value of | | | | -1 means unlimited (report all), and 0 means none. | +---------------------------+---------+----------------------------------------------------+ | pe-warn-series-max | 5000 | .. index:: | | | | pair: cluster option; pe-warn-series-max | | | | | | | | The number of scheduler inputs resulting in | | | | warnings to save. Used when reporting problems. A | | | | value of -1 means unlimited (report all), and 0 | | | | means none. | +---------------------------+---------+----------------------------------------------------+ | pe-input-series-max | 4000 | .. index:: | | | | pair: cluster option; pe-input-series-max | | | | | | | | The number of "normal" scheduler inputs to save. | | | | Used when reporting problems. A value of -1 means | | | | unlimited (report all), and 0 means none. | +---------------------------+---------+----------------------------------------------------+ | enable-acl | false | .. index:: | | | | pair: cluster option; enable-acl | | | | | | | | Whether :ref:`acl` should be used to authorize | | | | modifications to the CIB | +---------------------------+---------+----------------------------------------------------+ | placement-strategy | default | .. index:: | | | | pair: cluster option; placement-strategy | | | | | | | | How the cluster should assign resources to nodes | | | | (see :ref:`utilization`). Allowed values are | | | | ``default``, ``utilization``, ``balanced``, and | | | | ``minimal``. | +---------------------------+---------+----------------------------------------------------+ | node-health-strategy | none | .. index:: | | | | pair: cluster option; node-health-strategy | | | | | | | | How the cluster should react to node health | | | | attributes (see :ref:`node-health`). Allowed values| | | | are ``none``, ``migrate-on-red``, ``only-green``, | | | | ``progressive``, and ``custom``. | +---------------------------+---------+----------------------------------------------------+ | node-health-base | 0 | .. index:: | | | | pair: cluster option; node-health-base | | | | | | | | The base health score assigned to a node. Only | | | | used when ``node-health-strategy`` is | | | | ``progressive``. | +---------------------------+---------+----------------------------------------------------+ | node-health-green | 0 | .. index:: | | | | pair: cluster option; node-health-green | | | | | | | | The score to use for a node health attribute whose | | | | value is ``green``. Only used when | | | | ``node-health-strategy`` is ``progressive`` or | | | | ``custom``. | +---------------------------+---------+----------------------------------------------------+ | node-health-yellow | 0 | .. index:: | | | | pair: cluster option; node-health-yellow | | | | | | | | The score to use for a node health attribute whose | | | | value is ``yellow``. Only used when | | | | ``node-health-strategy`` is ``progressive`` or | | | | ``custom``. | +---------------------------+---------+----------------------------------------------------+ | node-health-red | 0 | .. index:: | | | | pair: cluster option; node-health-red | | | | | | | | The score to use for a node health attribute whose | | | | value is ``red``. Only used when | | | | ``node-health-strategy`` is ``progressive`` or | | | | ``custom``. | +---------------------------+---------+----------------------------------------------------+ | cluster-recheck-interval | 15min | .. index:: | | | | pair: cluster option; cluster-recheck-interval | | | | | | | | Pacemaker is primarily event-driven, and looks | | | | ahead to know when to recheck the cluster for | | | | failure timeouts and most time-based rules | | | | *(since 2.0.3)*. However, it will also recheck the | | | | cluster after this amount of inactivity. This has | | | | two goals: rules with ``date_spec`` are only | | | | guaranteed to be checked this often, and it also | | | | serves as a fail-safe for some kinds of scheduler | | | | bugs. A value of 0 disables this polling; positive | | | | values are a time interval. | +---------------------------+---------+----------------------------------------------------+ | shutdown-lock | false | .. index:: | | | | pair: cluster option; shutdown-lock | | | | | | | | The default of false allows active resources to be | | | | recovered elsewhere when their node is cleanly | | | | shut down, which is what the vast majority of | | | | users will want. However, some users prefer to | | | | make resources highly available only for failures, | | | | with no recovery for clean shutdowns. If this | | | | option is true, resources active on a node when it | | | | is cleanly shut down are kept "locked" to that | | | | node (not allowed to run elsewhere) until they | | | | start again on that node after it rejoins (or for | | | | at most ``shutdown-lock-limit``, if set). Stonith | | | | resources and Pacemaker Remote connections are | | | | never locked. Clone and bundle instances and the | | | | promoted role of promotable clones are currently | | | | never locked, though support could be added in a | | | | future release. Locks may be manually cleared | | | | using the ``--refresh`` option of ``crm_resource`` | | | | (both the resource and node must be specified; | | | | this works with remote nodes if their connection | | | | resource's ``target-role`` is set to ``Stopped``, | | | | but not if Pacemaker Remote is stopped on the | | | | remote node without disabling the connection | | | | resource). *(since 2.0.4)* | +---------------------------+---------+----------------------------------------------------+ | shutdown-lock-limit | 0 | .. index:: | | | | pair: cluster option; shutdown-lock-limit | | | | | | | | If ``shutdown-lock`` is true, and this is set to a | | | | nonzero time duration, locked resources will be | | | | allowed to start after this much time has passed | | | | since the node shutdown was initiated, even if the | | | | node has not rejoined. (This works with remote | | | | nodes only if their connection resource's | | | | ``target-role`` is set to ``Stopped``.) | | | | *(since 2.0.4)* | +---------------------------+---------+----------------------------------------------------+ | remove-after-stop | false | .. index:: | | | | pair: cluster option; remove-after-stop | | | | | | | | *Deprecated* Should the cluster remove | | | | resources from Pacemaker's executor after they are | | | | stopped? Values other than the default are, at | | | | best, poorly tested and potentially dangerous. | | | | This option is deprecated and will be removed in a | | | | future release. | +---------------------------+---------+----------------------------------------------------+ | startup-fencing | true | .. index:: | | | | pair: cluster option; startup-fencing | | | | | | | | *Advanced Use Only:* Should the cluster fence | | | | unseen nodes at start-up? Setting this to false is | | | | unsafe, because the unseen nodes could be active | | | | and running resources but unreachable. | +---------------------------+---------+----------------------------------------------------+ | election-timeout | 2min | .. index:: | | | | pair: cluster option; election-timeout | | | | | | | | *Advanced Use Only:* If you need to adjust this | | | | value, it probably indicates the presence of a bug.| +---------------------------+---------+----------------------------------------------------+ | shutdown-escalation | 20min | .. index:: | | | | pair: cluster option; shutdown-escalation | | | | | | | | *Advanced Use Only:* If you need to adjust this | | | | value, it probably indicates the presence of a bug.| +---------------------------+---------+----------------------------------------------------+ | join-integration-timeout | 3min | .. index:: | | | | pair: cluster option; join-integration-timeout | | | | | | | | *Advanced Use Only:* If you need to adjust this | | | | value, it probably indicates the presence of a bug.| +---------------------------+---------+----------------------------------------------------+ | join-finalization-timeout | 30min | .. index:: | | | | pair: cluster option; join-finalization-timeout | | | | | | | | *Advanced Use Only:* If you need to adjust this | | | | value, it probably indicates the presence of a bug.| +---------------------------+---------+----------------------------------------------------+ | transition-delay | 0s | .. index:: | | | | pair: cluster option; transition-delay | | | | | | | | *Advanced Use Only:* Delay cluster recovery for | | | | the configured interval to allow for additional or | | | | related events to occur. This can be useful if | | | | your configuration is sensitive to the order in | | | | which ping updates arrive. Enabling this option | | | | will slow down cluster recovery under all | | | | conditions. | +---------------------------+---------+----------------------------------------------------+