Page MenuHomeClusterLabs Projects
Files F3687341

No OneTemporary
Actions

View Options

diff --git a/doc/sphinx/Pacemaker_Explained/local-options.rst b/doc/sphinx/Pacemaker_Explained/local-options.rst
index 5e4b8e67ba..102f232198 100644
--- a/doc/sphinx/Pacemaker_Explained/local-options.rst
+++ b/doc/sphinx/Pacemaker_Explained/local-options.rst
@@ -1,721 +1,723 @@
Host-Local Configuration
------------------------
.. index::
pair: XML element; configuration
.. note:: Directory and file paths below may differ on your system depending on
your Pacemaker build settings. Check your Pacemaker configuration
file to find the correct paths.
Configuration Value Types
#########################
Throughout this document, configuration values will be designated as having one
of the following types:
.. list-table:: **Configuration Value Types**
:class: longtable
:widths: 1 3
:header-rows: 1
* - Type
- Description
* - .. _boolean:
.. index::
pair: type; boolean
boolean
- Case-insensitive text value where ``1``, ``yes``, ``y``, ``on``,
and ``true`` evaluate as true and ``0``, ``no``, ``n``, ``off``,
``false``, and unset evaluate as false
* - .. _date_time:
.. index::
pair: type; date/time
date/time
- Textual timestamp like ``Sat Dec 21 11:47:45 2013``
* - .. _duration:
.. index::
pair: type; duration
duration
- A time duration, specified either like a :ref:`timeout <timeout>` or an
`ISO 8601 duration <https://en.wikipedia.org/wiki/ISO_8601#Durations>`_.
A duration may be up to approximately 49 days but is intended for much
smaller time periods.
* - .. _enumeration:
.. index::
pair: type; enumeration
enumeration
- Text that must be one of a set of defined values (which will be listed
in the description)
* - .. _epoch_time:
.. index::
pair: type; epoch_time
epoch_time
- Time as the integer number of seconds since the Unix epoch,
``1970-01-01 00:00:00 +0000 (UTC)``.
* - .. _id:
.. index::
pair: type; id
id
- A text string starting with a letter or underbar, followed by any
combination of letters, numbers, dashes, dots, and/or underbars; when
used for a property named ``id``, the string must be unique across all
``id`` properties in the CIB
* - .. _integer:
.. index::
pair: type; integer
integer
- 32-bit signed integer value (-2,147,483,648 to 2,147,483,647)
* - .. _iso8601:
.. index::
pair: type; iso8601
ISO 8601
- An `ISO 8601 <https://en.wikipedia.org/wiki/ISO_8601>`_ date/time.
* - .. _nonnegative_integer:
.. index::
pair: type; nonnegative integer
nonnegative integer
- 32-bit nonnegative integer value (0 to 2,147,483,647)
* - .. _percentage:
.. index::
pair: type; percentage
percentage
- Floating-point number followed by an optional percent sign ('%')
* - .. _port:
.. index::
pair: type; port
port
- Integer TCP port number (0 to 65535)
* - .. _range:
.. index::
pair: type; range
range
- A range may be a single nonnegative integer or a dash-separated range of
nonnegative integers. Either the first or last value may be omitted to
leave the range open-ended. Examples: ``0``, ``3-``, ``-5``, ``4-6``.
* - .. _score:
.. index::
pair: type; score
score
- A Pacemaker score can be an integer between -1,000,000 and 1,000,000, or
a string alias: ``INFINITY`` or ``+INFINITY`` is equivalent to
1,000,000, ``-INFINITY`` is equivalent to -1,000,000, and ``red``,
``yellow``, and ``green`` are equivalent to integers as described in
:ref:`node-health`.
* - .. _text:
.. index::
pair: type; text
text
- A text string
* - .. _timeout:
.. index::
pair: type; timeout
timeout
- A time duration, specified as a bare number (in which case it is
considered to be in seconds) or a number with a unit (``ms`` or ``msec``
for milliseconds, ``us`` or ``usec`` for microseconds, ``s`` or ``sec``
for seconds, ``m`` or ``min`` for minutes, ``h`` or ``hr`` for hours)
optionally with whitespace before and/or after the number.
* - .. _version:
.. index::
pair: type; version
version
- Version number (any combination of alphanumeric characters, dots, and
dashes, starting with a number).
Scores
______
Scores are integral to how Pacemaker works. Practically everything from moving
a resource to deciding which resource to stop in a degraded cluster is achieved
by manipulating scores in some way.
Scores are calculated per resource and node. Any node with a negative score for
a resource can't run that resource. The cluster places a resource on the node
with the highest score for it.
Score addition and subtraction follow these rules:
* Any value (including ``INFINITY``) - ``INFINITY`` = ``-INFINITY``
* ``INFINITY`` + any value other than ``-INFINITY`` = ``INFINITY``
.. note::
What if you want to use a score higher than 1,000,000? Typically this possibility
arises when someone wants to base the score on some external metric that might
go above 1,000,000.
The short answer is you can't.
The long answer is it is sometimes possible work around this limitation
creatively. You may be able to set the score to some computed value based on
the external metric rather than use the metric directly. For nodes, you can
store the metric as a node attribute, and query the attribute when computing
the score (possibly as part of a custom resource agent).
Local Options
#############
Most Pacemaker configuration is in the cluster-wide CIB, but some host-local
configuration options either are needed at startup, before the CIB is read, or
provide per-host overrides of cluster-wide options.
These options are configured as environment variables set when Pacemaker is
started, in the format ``<NAME>="<VALUE>"``. These are typically set in a file
whose location varies by OS (most commonly ``/etc/sysconfig/pacemaker`` or
``/etc/default/pacemaker``; this documentation was generated on a system using
|PCMK_CONFIG_FILE|).
.. list-table:: **Local Options**
:class: longtable
:widths: 2 2 2 5
:header-rows: 1
* - Name
- Type
- Default
- Description
* - .. _cib_pam_service:
.. index::
pair: node option; CIB_pam_service
CIB_pam_service
- :ref:`text <text>`
- login
- PAM service to use for remote CIB client authentication (passed to
``pam_start``).
* - .. _pcmk_logfacility:
.. index::
pair: node option; PCMK_logfacility
PCMK_logfacility
- :ref:`enumeration <enumeration>`
- daemon
- Enable logging via the system log or journal, using the specified log
facility. Messages sent here are of value to all Pacemaker
administrators. This can be disabled using ``none``, but that is not
recommended. Allowed values:
* ``none``
* ``daemon``
* ``user``
* ``local0``
* ``local1``
* ``local2``
* ``local3``
* ``local4``
* ``local5``
* ``local6``
* ``local7``
* - .. _pcmk_logpriority:
.. index::
pair: node option; PCMK_logpriority
PCMK_logpriority
- :ref:`enumeration <enumeration>`
- notice
- Unless system logging is disabled using ``PCMK_logfacility=none``,
messages of the specified log severity and higher will be sent to the
system log. The default is appropriate for most installations. Allowed
values:
* ``emerg``
* ``alert``
* ``crit``
* ``error``
* ``warning``
* ``notice``
* ``info``
* ``debug``
* - .. _pcmk_logfile:
.. index::
pair: node option; PCMK_logfile
PCMK_logfile
- :ref:`text <text>`
- |PCMK_LOG_FILE|
- Unless set to ``none``, more detailed log messages will be sent to the
specified file (in addition to the system log, if enabled). These
messages may have extended information, and will include messages of info
severity. This log is of more use to developers and advanced system
administrators, and when reporting problems. Note: The default is
|PCMK_CONTAINER_LOG_FILE| (inside the container) for bundled container
nodes; this would typically be mapped to a different path on the host
running the container.
* - .. _pcmk_logfile_mode:
.. index::
pair: node option; PCMK_logfile_mode
PCMK_logfile_mode
- :ref:`text <text>`
- 0660
- Pacemaker will set the permissions on the detail log to this value (see
``chmod(1)``).
* - .. _pcmk_debug:
.. index::
pair: node option; PCMK_debug
PCMK_debug
- :ref:`enumeration <enumeration>`
- no
- Whether to send debug severity messages to the detail log. This may be
set for all subsystems (``yes`` or ``no``) or for specific (comma-
separated) subsystems. Allowed subsystems are:
* ``pacemakerd``
* ``pacemaker-attrd``
* ``pacemaker-based``
* ``pacemaker-controld``
* ``pacemaker-execd``
* ``pacemaker-fenced``
* ``pacemaker-schedulerd``
Example: ``PCMK_debug="pacemakerd,pacemaker-execd"``
* - .. _pcmk_stderr:
.. index::
pair: node option; PCMK_stderr
PCMK_stderr
- :ref:`boolean <boolean>`
- no
- *Advanced Use Only:* Whether to send daemon log messages to stderr. This
would be useful only during troubleshooting, when starting Pacemaker
manually on the command line.
Setting this option in the configuration file is pointless, since the
file is not read when starting Pacemaker manually. However, it can be set
directly as an environment variable on the command line.
* - .. _pcmk_trace_functions:
.. index::
pair: node option; PCMK_trace_functions
PCMK_trace_functions
- :ref:`text <text>`
-
- *Advanced Use Only:* Send debug and trace severity messages from these
(comma-separated) source code functions to the detail log.
Example:
``PCMK_trace_functions="func1,func2"``
* - .. _pcmk_trace_files:
.. index::
pair: node option; PCMK_trace_files
PCMK_trace_files
- :ref:`text <text>`
-
- *Advanced Use Only:* Send debug and trace severity messages from all
functions in these (comma-separated) source file names to the detail log.
Example: ``PCMK_trace_files="file1.c,file2.c"``
* - .. _pcmk_trace_formats:
.. index::
pair: node option; PCMK_trace_formats
PCMK_trace_formats
- :ref:`text <text>`
-
- *Advanced Use Only:* Send trace severity messages that are generated by
these (comma-separated) format strings in the source code to the detail
log.
Example: ``PCMK_trace_formats="Error: %s (%d)"``
* - .. _pcmk_trace_tags:
.. index::
pair: node option; PCMK_trace_tags
PCMK_trace_tags
- :ref:`text <text>`
-
- *Advanced Use Only:* Send debug and trace severity messages related to
these (comma-separated) resource IDs to the detail log.
Example: ``PCMK_trace_tags="client-ip,dbfs"``
* - .. _pcmk_blackbox:
.. index::
pair: node option; PCMK_blackbox
PCMK_blackbox
- :ref:`enumeration <enumeration>`
- no
- *Advanced Use Only:* Enable blackbox logging globally (``yes`` or ``no``)
or by subsystem. A blackbox contains a rolling buffer of all logs (of all
severities). Blackboxes are stored under |CRM_BLACKBOX_DIR| by default,
by default, and their contents can be viewed using the ``qb-blackbox(8)``
command.
The blackbox recorder can be enabled at start using this variable, or at
runtime by sending a Pacemaker subsystem daemon process a ``SIGUSR1`` or
``SIGTRAP`` signal, and disabled by sending ``SIGUSR2`` (see
``kill(1)``). The blackbox will be written after a crash, assertion
failure, or ``SIGTRAP`` signal.
See :ref:`PCMK_debug <pcmk_debug>` for allowed subsystems.
Example:
``PCMK_blackbox="pacemakerd,pacemaker-execd"``
* - .. _pcmk_trace_blackbox:
.. index::
pair: node option; PCMK_trace_blackbox
PCMK_trace_blackbox
- :ref:`enumeration <enumeration>`
-
- *Advanced Use Only:* Write a blackbox whenever the message at the
specified function and line is logged. Multiple entries may be comma-
separated.
Example: ``PCMK_trace_blackbox="remote.c:144,remote.c:149"``
* - .. _pcmk_node_start_state:
.. index::
pair: node option; PCMK_node_start_state
PCMK_node_start_state
- :ref:`enumeration <enumeration>`
- default
- By default, the local host will join the cluster in an online or standby
state when Pacemaker first starts depending on whether it was previously
put into standby mode. If this variable is set to ``standby`` or
``online``, it will force the local host to join in the specified state.
* - .. _pcmk_node_action_limit:
.. index::
pair: node option; PCMK_node_action_limit
PCMK_node_action_limit
- :ref:`nonnegative integer <nonnegative_integer>`
-
- If set, this overrides the :ref:`node-action-limit <node_action_limit>`
cluster option on this node to specify the maximum number of jobs that
can be scheduled on this node (or 0 to use twice the number of CPU
cores).
* - .. _pcmk_fail_fast:
.. index::
pair: node option; PCMK_fail_fast
PCMK_fail_fast
- :ref:`boolean <boolean>`
- no
- By default, if a Pacemaker subsystem crashes, the main ``pacemakerd``
process will attempt to restart it. If this variable is set to ``yes``,
``pacemakerd`` will panic the local host instead.
* - .. _pcmk_panic_action:
.. index::
pair: node option; PCMK_panic_action
PCMK_panic_action
- :ref:`enumeration <enumeration>`
- reboot
- Pacemaker will panic the local host under certain conditions. By default,
this means rebooting the host. This variable can change that behavior: if
``crash``, trigger a kernel crash (useful if you want a kernel dump to
investigate); if ``sync-reboot`` or ``sync-crash``, synchronize
filesystems before rebooting the host or triggering a kernel crash. The
sync values are more likely to preserve log messages, but with the risk
that the host may be left active if the synchronization hangs.
* - .. _pcmk_authkey_location:
.. index::
pair: node option; PCMK_authkey_location
PCMK_authkey_location
- :ref:`text <text>`
- |PCMK_AUTHKEY_FILE|
- Use the contents of this file as the authorization key to use with
- Pacemaker Remote connections. This file must be readable by Pacemaker
- daemons (that is, it must allow read permissions to either the
- |CRM_DAEMON_USER| user or the |CRM_DAEMON_GROUP| group), and its contents
- must be identical on all nodes.
+ :ref:`Pacemaker Remote <pacemaker_remote>` connections. This file must
+ be readable by Pacemaker daemons (that is, it must allow read
+ permissions to either the |CRM_DAEMON_USER| user or the
+ |CRM_DAEMON_GROUP| group), and its contents must be identical on all
+ nodes.
* - .. _pcmk_remote_address:
.. index::
pair: node option; PCMK_remote_address
PCMK_remote_address
- :ref:`text <text>`
-
- - By default, if the Pacemaker Remote service is run on the local node, it
- will listen for connections on all IP addresses. This may be set to one
- address to listen on instead, as a resolvable hostname or as a numeric
- IPv4 or IPv6 address. When resolving names or listening on all addresses,
- IPv6 will be preferred if available. When listening on an IPv6 address,
- IPv4 clients will be supported via IPv4-mapped IPv6 addresses.
+ - By default, if the :ref:`Pacemaker Remote <pacemaker_remote>` service is
+ run on the local node, it will listen for connections on all IP
+ addresses. This may be set to one address to listen on instead, as a
+ resolvable hostname or as a numeric IPv4 or IPv6 address. When resolving
+ names or listening on all addresses, IPv6 will be preferred if
+ available. When listening on an IPv6 address, IPv4 clients will be
+ supported via IPv4-mapped IPv6 addresses.
Example: ``PCMK_remote_address="192.0.2.1"``
* - .. _pcmk_remote_port:
.. index::
pair: node option; PCMK_remote_port
PCMK_remote_port
- :ref:`port <port>`
- 3121
- - Use this TCP port number for Pacemaker Remote node connections. This
- value must be the same on all nodes.
+ - Use this TCP port number for :ref:`Pacemaker Remote <pacemaker_remote>`
+ node connections. This value must be the same on all nodes.
* - .. _pcmk_remote_pid1:
.. index::
pair: node option; PCMK_remote_pid1
PCMK_remote_pid1
- :ref:`enumeration <enumeration>`
- default
- *Advanced Use Only:* When a bundle resource's ``run-command`` option is
- left to default, Pacemaker Remote runs as PID 1 in the bundle's
- containers. When it does so, it loads environment variables from the
- container's |PCMK_INIT_ENV_FILE| and performs the PID 1 responsibility of
- reaping dead subprocesses.
+ left to default, :ref:`Pacemaker Remote <pacemaker_remote>` runs as PID
+ 1 in the bundle's containers. When it does so, it loads environment
+ variables from the container's |PCMK_INIT_ENV_FILE| and performs the PID
+ 1 responsibility of reaping dead subprocesses.
This option controls whether those actions are performed when Pacemaker
Remote is not running as PID 1. It is intended primarily for developer
testing but can be useful when ``run-command`` is set to a separate,
custom PID 1 process that launches Pacemaker Remote.
* ``full``: Pacemaker Remote loads environment variables from
|PCMK_INIT_ENV_FILE| and reaps dead subprocesses.
* ``vars``: Pacemaker Remote loads environment variables from
|PCMK_INIT_ENV_FILE| but does not reap dead subprocesses.
* ``default``: Pacemaker Remote performs neither action.
If Pacemaker Remote is running as PID 1, this option is ignored, and the
behavior is the same as for ``full``.
* - .. _pcmk_tls_priorities:
.. index::
pair: node option; PCMK_tls_priorities
PCMK_tls_priorities
- :ref:`text <text>`
- |PCMK__GNUTLS_PRIORITIES|
- *Advanced Use Only:* These GnuTLS cipher priorities will be used for TLS
- connections (whether for Pacemaker Remote connections or remote CIB
- access, when enabled). See:
+ connections (whether for :ref:`Pacemaker Remote <pacemaker_remote>`
+ connections or remote CIB access, when enabled). See:
https://gnutls.org/manual/html_node/Priority-Strings.html
Pacemaker will append ``":+ANON-DH"`` for remote CIB access and
``":+DHE-PSK:+PSK"`` for Pacemaker Remote connections, as they are
required for the respective functionality.
Example:
``PCMK_tls_priorities="SECURE128:+SECURE192"``
* - .. _pcmk_dh_max_bits:
.. index::
pair: node option; PCMK_dh_max_bits
PCMK_dh_max_bits
- :ref:`nonnegative integer <nonnegative_integer>`
- 0 (no maximum)
- *Advanced Use Only:* Set an upper bound on the bit length of the prime
number generated for Diffie-Hellman parameters needed by TLS connections.
The default is no maximum.
- The server (Pacemaker Remote daemon, or CIB manager configured to accept
- remote clients) will use this value to provide a ceiling for the value
- recommended by the GnuTLS library. The library will only accept a limited
- number of specific values, which vary by library version, so setting
- these is recommended only when required for compatibility with specific
- client versions.
+ The server (:ref:`Pacemaker Remote <pacemaker_remote>` daemon, or CIB
+ manager configured to accept remote clients) will use this value to
+ provide a ceiling for the value recommended by the GnuTLS library. The
+ library will only accept a limited number of specific values, which vary
+ by library version, so setting these is recommended only when required
+ for compatibility with specific client versions.
Clients do not use ``PCMK_dh_max_bits``.
* - .. _pcmk_ipc_type:
.. index::
pair: node option; PCMK_ipc_type
PCMK_ipc_type
- :ref:`enumeration <enumeration>`
- shared-mem
- *Advanced Use Only:* Force use of a particular IPC method. Allowed values:
* ``shared-mem``
* ``socket``
* ``posix``
* ``sysv``
* - .. _pcmk_ipc_buffer:
.. index::
pair: node option; PCMK_ipc_buffer
PCMK_ipc_buffer
- :ref:`nonnegative integer <nonnegative_integer>`
- 131072
- *Advanced Use Only:* Specify an IPC buffer size in bytes. This can be
useful when connecting to large clusters that result in messages
exceeding the default size (which will also result in log messages
referencing this variable).
* - .. _pcmk_cluster_type:
.. index::
pair: node option; PCMK_cluster_type
PCMK_cluster_type
- :ref:`enumeration <enumeration>`
- corosync
- *Advanced Use Only:* Specify the cluster layer to be used. If unset,
Pacemaker will detect and use a supported cluster layer, if available.
Currently, ``"corosync"`` is the only supported cluster layer. If
multiple layers are supported in the future, this will allow overriding
Pacemaker's automatic detection to select a specific one.
* - .. _pcmk_schema_directory:
.. index::
pair: node option; PCMK_schema_directory
PCMK_schema_directory
- :ref:`text <text>`
- |PCMK_SCHEMA_DIR|
- *Advanced Use Only:* Specify an alternate location for RNG schemas and
XSL transforms.
* - .. _pcmk_remote_schema_directory:
.. index::
pair: node option; PCMK_remote_schema_directory
PCMK_remote_schema_directory
- :ref:`text <text>`
- |PCMK__REMOTE_SCHEMA_DIR|
- - *Advanced Use Only:* Specify an alternate location on Pacemaker Remote
- nodes for storing newer RNG schemas and XSL transforms fetched from
- the cluster.
+ - *Advanced Use Only:* Specify an alternate location on
+ :ref:`Pacemaker Remote <pacemaker_remote>` nodes for storing newer RNG
+ schemas and XSL transforms fetched from the cluster.
* - .. _pcmk_valgrind_enabled:
.. index::
pair: node option; PCMK_valgrind_enabled
PCMK_valgrind_enabled
- :ref:`enumeration <enumeration>`
- no
- *Advanced Use Only:* Whether subsystem daemons should be run under
``valgrind``. Allowed values are the same as for ``PCMK_debug``.
* - .. _pcmk_callgrind_enabled:
.. index::
pair: node option; PCMK_callgrind_enabled
PCMK_callgrind_enabled
- :ref:`enumeration <enumeration>`
- no
- *Advanced Use Only:* Whether subsystem daemons should be run under
``valgrind`` with the ``callgrind`` tool enabled. Allowed values are the
same as for ``PCMK_debug``.
* - .. _sbd_sync_resource_startup:
.. index::
pair: node option; SBD_SYNC_RESOURCE_STARTUP
SBD_SYNC_RESOURCE_STARTUP
- :ref:`boolean <boolean>`
-
- If true, ``pacemakerd`` waits for a ping from ``sbd`` during startup
before starting other Pacemaker daemons, and during shutdown after
stopping other Pacemaker daemons but before exiting. Default value is set
based on the ``--with-sbd-sync-default`` configure script option.
* - .. _sbd_watchdog_timeout:
.. index::
pair: node option; SBD_WATCHDOG_TIMEOUT
SBD_WATCHDOG_TIMEOUT
- :ref:`duration <duration>`
-
- If the ``stonith-watchdog-timeout`` cluster property is set to a negative
or invalid value, use double this value as the default if positive, or
use 0 as the default otherwise. This value must be greater than the value
of ``stonith-watchdog-timeout`` if both are set.
* - .. _valgrind_opts:
.. index::
pair: node option; VALGRIND_OPTS
VALGRIND_OPTS
- :ref:`text <text>`
-
- *Advanced Use Only:* Pass these options to valgrind, when enabled (see
``valgrind(1)``). ``"--vgdb=no"`` should usually be specified because
``pacemaker-execd`` can lower privileges when executing commands, which
would otherwise leave a bunch of unremovable files in ``/tmp``.
diff --git a/doc/sphinx/Pacemaker_Explained/nodes.rst b/doc/sphinx/Pacemaker_Explained/nodes.rst
index b700010cf3..e4d2591047 100644
--- a/doc/sphinx/Pacemaker_Explained/nodes.rst
+++ b/doc/sphinx/Pacemaker_Explained/nodes.rst
@@ -1,473 +1,555 @@
-Cluster Nodes
--------------
+.. index::
+ single: node
+
+Nodes
+-----
+
+Pacemaker supports two basic types of nodes: *cluster nodes* and *Pacemaker
+Remote nodes*.
+
+.. index::
+ single: node; cluster node
+
+Cluster nodes
+_____________
+
+Cluster nodes run Corosync and all Pacemaker components. They may run cluster
+resources, run all Pacemaker command-line tools, execute fencing actions, count
+toward cluster quorum, and serve as the cluster's Designated Controller (DC).
+
+Every cluster must have at least one cluster node. Scalability is limited by
+the cluster layer to around 32 cluster nodes.
+
+.. _pacemaker_remote:
+
+.. index::
+ pair: node; Pacemaker Remote
+
+Pacemaker Remote nodes
+______________________
+
+Pacemaker Remote nodes do not run Corosync or the usual Pacemaker components.
+Instead, they run only the *remote executor* (``pacemaker-remoted``), which
+waits for Pacemaker on a cluster node to give it instructions.
+
+They may run cluster resources and most command-line tools, but cannot perform
+other functions of full cluster nodes such as fencing execution, quorum voting,
+or DC eligibility.
-Defining a Cluster Node
-_______________________
+There is no hard limit on the number of Pacemaker Remote nodes.
+
+.. NOTE::
+
+ *Remote* in this document has nothing to do with physical proximity and
+ instead refers to the node not being a member of the underlying Corosync
+ cluster. Pacemaker Remote nodes are subject to the same latency
+ requirements as cluster nodes, which means they are typically in the same
+ data center.
+
+There are three types of Pacemaker Remote nodes:
+
+* A *remote node* boots outside Pacemaker control, and is typically a physical
+ host. The connection to the remote node is managed as a :ref:`special type of
+ resource <remote_nodes>` configured by the user.
+
+* A *guest node* is a virtual machine or container configured to run
+ Pacemaker's remote executor when launched, and is launched and managed by the
+ cluster as a standard resource configured by the user with :ref:`special
+ options <guest_nodes>`.
+
+* A *bundle node* is a guest node created for a container that is launched and
+ managed by the cluster as part of a :ref:`bundle <s-resource-bundle>`
+ resource configured by the user.
+
+.. NOTE::
+
+ It is important to distinguish the various roles a virtual machine can serve
+ in Pacemaker clusters:
+
+ * A virtual machine can run the full cluster stack, in which case it is a
+ cluster node and is not itself managed by the cluster.
+ * A virtual machine can be managed by the cluster as a simple resource,
+ without the cluster having any awareness of the services running within
+ it. The virtual machine is *opaque* to the cluster.
+ * A virtual machine can be a guest node, allowing the cluster to manage
+ both the virtual machine and resources running within it. The virtual
+ machine is *transparent* to the cluster.
+
+Defining a Node
+_______________
Each cluster node will have an entry in the ``nodes`` section containing at
least an ID and a name. A cluster node's ID is defined by the cluster layer
(Corosync).
.. topic:: **Example Corosync cluster node entry**
.. code-block:: xml
<node id="101" uname="pcmk-1"/>
-In normal circumstances, the admin should let the cluster populate this
-information automatically from the cluster layer.
+Pacemaker Remote nodes are defined by a resource in the ``resources`` section.
+Remote nodes and guest nodes may optionally have an entry in the ``nodes``
+section, primarily for permanent :ref:`node attributes <node-attributes>`.
+Normally, the user should let the cluster populate the ``nodes`` section
+automatically.
+
+.. index::
+ single: node; name
.. _node_name:
Where Pacemaker Gets the Node Name
##################################
The name that Pacemaker uses for a node in the configuration does not have to
-be the same as its local hostname. Pacemaker uses the following for a Corosync
+be the same as its local hostname. Pacemaker uses the following for a cluster
node's name, in order of most preferred first:
* The value of ``name`` in the ``nodelist`` section of ``corosync.conf``
* The value of ``ring0_addr`` in the ``nodelist`` section of ``corosync.conf``
* The local hostname (value of ``uname -n``)
+A Pacemaker Remote node's name is defined in its resource configuration.
+
If the cluster is running, the ``crm_node -n`` command will display the local
node's name as used by the cluster.
If a Corosync ``nodelist`` is used, ``crm_node --name-for-id`` with a Corosync
node ID will display the name used by the node with the given Corosync
``nodeid``, for example:
.. code-block:: none
crm_node --name-for-id 2
.. index::
single: node; attribute
single: node attribute
.. _node_attributes:
Node Attributes
_______________
Pacemaker allows node-specific values to be specified using *node attributes*.
A node attribute has a name, and may have a distinct value for each node.
Node attributes come in two types, *permanent* and *transient*. Permanent node
attributes are kept within the ``node`` entry, and keep their values even if
the cluster restarts on a node. Transient node attributes are kept in the CIB's
``status`` section, and go away when the cluster stops on the node.
While certain node attributes have specific meanings to the cluster, they are
mainly intended to allow administrators and resource agents to track any
information desired.
For example, an administrator might choose to define node attributes for how
much RAM and disk space each node has, which OS each uses, or which server room
rack each node is in.
Users can configure :ref:`rules` that use node attributes to affect where
resources are placed.
Setting and querying node attributes
####################################
Node attributes can be set and queried using the ``crm_attribute`` and
``attrd_updater`` commands, so that the user does not have to deal with XML
configuration directly.
Here is an example command to set a permanent node attribute, and the XML
configuration that would be generated:
.. topic:: **Result of using crm_attribute to specify which kernel pcmk-1 is running**
.. code-block:: none
# crm_attribute --type nodes --node pcmk-1 --name kernel --update $(uname -r)
.. code-block:: xml
<node id="1" uname="pcmk-1">
<instance_attributes id="nodes-1-attributes">
<nvpair id="nodes-1-kernel" name="kernel" value="3.10.0-862.14.4.el7.x86_64"/>
</instance_attributes>
</node>
To read back the value that was just set:
.. code-block:: none
# crm_attribute --type nodes --node pcmk-1 --name kernel --query
scope=nodes name=kernel value=3.10.0-862.14.4.el7.x86_64
The ``--type nodes`` indicates that this is a permanent node attribute;
``--type status`` would indicate a transient node attribute.
.. warning::
Attribute values with newline or tab characters are currently displayed with
newlines as ``"\n"`` and tabs as ``"\t"``, when ``crm_attribute`` or
``attrd_updater`` query commands use ``--output-as=text`` or leave
``--output-as`` unspecified:
.. code-block:: none
# crm_attribute -N node1 -n test_attr -v "$(echo -e "a\nb\tc")" -t status
# crm_attribute -N node1 -n test_attr --query -t status
scope=status name=test_attr value=a\nb\tc
This format is deprecated. In a future release, the values will be displayed
with literal whitespace characters:
.. code-block:: none
# crm_attribute -N node1 -n test_attr --query -t status
scope=status name=test_attr value=a
b c
Users should either avoid attribute values with newlines and tabs, or ensure
that they can handle both formats.
However, it's best to use ``--output-as=xml`` when parsing attribute values
from output. Newlines, tabs, and special characters are replaced with XML
character references that a conforming XML processor can recognize and
convert to literals *(since 2.1.8)*:
.. code-block:: none
# crm_attribute -N node1 -n test_attr --query -t status --output-as=xml
<pacemaker-result api-version="2.35" request="crm_attribute -N laptop -n test_attr --query -t status --output-as=xml">
<attribute name="test_attr" value="a&#10;b&#9;c" scope="status"/>
<status code="0" message="OK"/>
</pacemaker-result>
.. _special_node_attributes:
Special node attributes
#######################
Certain node attributes have special meaning to the cluster.
Node attribute names beginning with ``#`` are considered reserved for these
special attributes. Some special attributes do not start with ``#``, for
historical reasons.
Certain special attributes are set automatically by the cluster, should never
be modified directly, and can be used only within :ref:`rules`; these are
listed under
:ref:`built-in node attributes <node-attribute-expressions-special>`.
For true/false values, the cluster considers a value of "1", "y", "yes", "on",
or "true" (case-insensitively) to be true, "0", "n", "no", "off", "false", or
unset to be false, and anything else to be an error.
.. table:: **Node attributes with special significance**
:class: longtable
:widths: 1 2
+----------------------------+-----------------------------------------------------+
| Name | Description |
+============================+=====================================================+
| fail-count-* | .. index:: |
| | pair: node attribute; fail-count |
| | |
| | Attributes whose names start with |
| | ``fail-count-`` are managed by the cluster |
| | to track how many times particular resource |
| | operations have failed on this node. These |
| | should be queried and cleared via the |
| | ``crm_failcount`` or |
| | ``crm_resource --cleanup`` commands rather |
| | than directly. |
+----------------------------+-----------------------------------------------------+
| last-failure-* | .. index:: |
| | pair: node attribute; last-failure |
| | |
| | Attributes whose names start with |
| | ``last-failure-`` are managed by the cluster |
| | to track when particular resource operations |
| | have most recently failed on this node. |
| | These should be cleared via the |
| | ``crm_failcount`` or |
| | ``crm_resource --cleanup`` commands rather |
| | than directly. |
+----------------------------+-----------------------------------------------------+
| maintenance | .. _node_maintenance: |
| | |
| | .. index:: |
| | pair: node attribute; maintenance |
| | |
| | If true, the cluster will not start or stop any |
| | resources on this node. Any resources active on the |
| | node become unmanaged, and any recurring operations |
| | for those resources (except those specifying |
| | ``role`` as ``Stopped``) will be paused. The |
| | :ref:`maintenance-mode <maintenance_mode>` cluster |
| | option, if true, overrides this. If this attribute |
| | is true, it overrides the |
| | :ref:`is-managed <is_managed>` and |
| | :ref:`maintenance <rsc_maintenance>` |
| | meta-attributes of affected resources and |
| | :ref:`enabled <op_enabled>` meta-attribute for |
| | affected recurring actions. Pacemaker should not be |
| | restarted on a node that is in single-node |
| | maintenance mode. |
+----------------------------+-----------------------------------------------------+
| probe_complete | .. index:: |
| | pair: node attribute; probe_complete |
| | |
| | This is managed by the cluster to detect |
| | when nodes need to be reprobed, and should |
| | never be used directly. |
+----------------------------+-----------------------------------------------------+
| resource-discovery-enabled | .. index:: |
| | pair: node attribute; resource-discovery-enabled |
| | |
| | If the node is a remote node, fencing is enabled, |
| | and this attribute is explicitly set to false |
| | (unset means true in this case), resource discovery |
| | (probes) will not be done on this node. This is |
| | highly discouraged; the ``resource-discovery`` |
| | location constraint property is preferred for this |
| | purpose. |
+----------------------------+-----------------------------------------------------+
| shutdown | .. index:: |
| | pair: node attribute; shutdown |
| | |
| | This is managed by the cluster to orchestrate the |
| | shutdown of a node, and should never be used |
| | directly. |
+----------------------------+-----------------------------------------------------+
| site-name | .. index:: |
| | pair: node attribute; site-name |
| | |
| | If set, this will be used as the value of the |
| | ``#site-name`` node attribute used in rules. (If |
| | not set, the value of the ``cluster-name`` cluster |
| | option will be used as ``#site-name`` instead.) |
+----------------------------+-----------------------------------------------------+
| standby | .. index:: |
| | pair: node attribute; standby |
| | |
| | If true, the node is in standby mode. This is |
| | typically set and queried via the ``crm_standby`` |
| | command rather than directly. |
+----------------------------+-----------------------------------------------------+
| terminate | .. index:: |
| | pair: node attribute; terminate |
| | |
| | If the value is true or begins with any nonzero |
| | number, the node will be fenced. This is typically |
| | set by tools rather than directly. |
+----------------------------+-----------------------------------------------------+
| #digests-* | .. index:: |
| | pair: node attribute; #digests |
| | |
| | Attributes whose names start with ``#digests-`` are |
| | managed by the cluster to detect when |
| | :ref:`unfencing` needs to be redone, and should |
| | never be used directly. |
+----------------------------+-----------------------------------------------------+
| #node-unfenced | .. index:: |
| | pair: node attribute; #node-unfenced |
| | |
| | When the node was last unfenced (as seconds since |
| | the epoch). This is managed by the cluster and |
| | should never be used directly. |
+----------------------------+-----------------------------------------------------+
.. index::
single: node; health
.. _node-health:
Tracking Node Health
____________________
A node may be functioning adequately as far as cluster membership is concerned,
and yet be "unhealthy" in some respect that makes it an undesirable location
for resources. For example, a disk drive may be reporting SMART errors, or the
CPU may be highly loaded.
Pacemaker offers a way to automatically move resources off unhealthy nodes.
.. index::
single: node attribute; health
Node Health Attributes
######################
Pacemaker will treat any node attribute whose name starts with ``#health`` as
an indicator of node health. Node health attributes may have one of the
following values:
.. table:: **Allowed Values for Node Health Attributes**
:widths: 1 4
+------------+--------------------------------------------------------------+
| Value | Intended significance |
+============+==============================================================+
| ``red`` | .. index:: |
| | single: red; node health attribute value |
| | single: node attribute; health (red) |
| | |
| | This indicator is unhealthy |
+------------+--------------------------------------------------------------+
| ``yellow`` | .. index:: |
| | single: yellow; node health attribute value |
| | single: node attribute; health (yellow) |
| | |
| | This indicator is becoming unhealthy |
+------------+--------------------------------------------------------------+
| ``green`` | .. index:: |
| | single: green; node health attribute value |
| | single: node attribute; health (green) |
| | |
| | This indicator is healthy |
+------------+--------------------------------------------------------------+
| *integer* | .. index:: |
| | single: score; node health attribute value |
| | single: node attribute; health (score) |
| | |
| | A numeric score to apply to all resources on this node (0 or |
| | positive is healthy, negative is unhealthy) |
+------------+--------------------------------------------------------------+
.. index::
pair: cluster option; node-health-strategy
Node Health Strategy
####################
Pacemaker assigns a node health score to each node, as the sum of the values of
all its node health attributes. This score will be used as a location
constraint applied to this node for all resources.
The ``node-health-strategy`` cluster option controls how Pacemaker responds to
changes in node health attributes, and how it translates ``red``, ``yellow``,
and ``green`` to scores.
Allowed values are:
.. table:: **Node Health Strategies**
:widths: 1 4
+----------------+----------------------------------------------------------+
| Value | Effect |
+================+==========================================================+
| none | .. index:: |
| | single: node-health-strategy; none |
| | single: none; node-health-strategy value |
| | |
| | Do not track node health attributes at all. |
+----------------+----------------------------------------------------------+
| migrate-on-red | .. index:: |
| | single: node-health-strategy; migrate-on-red |
| | single: migrate-on-red; node-health-strategy value |
| | |
| | Assign the value of ``-INFINITY`` to ``red``, and 0 to |
| | ``yellow`` and ``green``. This will cause all resources |
| | to move off the node if any attribute is ``red``. |
+----------------+----------------------------------------------------------+
| only-green | .. index:: |
| | single: node-health-strategy; only-green |
| | single: only-green; node-health-strategy value |
| | |
| | Assign the value of ``-INFINITY`` to ``red`` and |
| | ``yellow``, and 0 to ``green``. This will cause all |
| | resources to move off the node if any attribute is |
| | ``red`` or ``yellow``. |
+----------------+----------------------------------------------------------+
| progressive | .. index:: |
| | single: node-health-strategy; progressive |
| | single: progressive; node-health-strategy value |
| | |
| | Assign the value of the ``node-health-red`` cluster |
| | option to ``red``, the value of ``node-health-yellow`` |
| | to ``yellow``, and the value of ``node-health-green`` to |
| | ``green``. Each node is additionally assigned a score of |
| | ``node-health-base`` (this allows resources to start |
| | even if some attributes are ``yellow``). This strategy |
| | gives the administrator finer control over how important |
| | each value is. |
+----------------+----------------------------------------------------------+
| custom | .. index:: |
| | single: node-health-strategy; custom |
| | single: custom; node-health-strategy value |
| | |
| | Track node health attributes using the same values as |
| | ``progressive`` for ``red``, ``yellow``, and ``green``, |
| | but do not take them into account. The administrator is |
| | expected to implement a policy by defining :ref:`rules` |
| | referencing node health attributes. |
+----------------+----------------------------------------------------------+
Exempting a Resource from Health Restrictions
#############################################
If you want a resource to be able to run on a node even if its health score
would otherwise prevent it, set the resource's ``allow-unhealthy-nodes``
meta-attribute to ``true`` *(available since 2.1.3)*.
This is particularly useful for node health agents, to allow them to detect
when the node becomes healthy again. If you configure a health agent without
this setting, then the health agent will be banned from an unhealthy node,
and you will have to investigate and clear the health attribute manually once
it is healthy to allow resources on the node again.
If you want the meta-attribute to apply to a clone, it must be set on the clone
itself, not on the resource being cloned.
Configuring Node Health Agents
##############################
Since Pacemaker calculates node health based on node attributes, any method
that sets node attributes may be used to measure node health. The most common
are resource agents and custom daemons.
Pacemaker provides examples that can be used directly or as a basis for custom
code. The ``ocf:pacemaker:HealthCPU``, ``ocf:pacemaker:HealthIOWait``, and
``ocf:pacemaker:HealthSMART`` resource agents set node health attributes based
on CPU and disk status.
To take advantage of this feature, add the resource to your cluster (generally
as a cloned resource with a recurring monitor action, to continually check the
health of all nodes). For example:
.. topic:: Example HealthIOWait resource configuration
.. code-block:: xml
<clone id="resHealthIOWait-clone">
<primitive class="ocf" id="HealthIOWait" provider="pacemaker" type="HealthIOWait">
<instance_attributes id="resHealthIOWait-instance_attributes">
<nvpair id="resHealthIOWait-instance_attributes-red_limit" name="red_limit" value="30"/>
<nvpair id="resHealthIOWait-instance_attributes-yellow_limit" name="yellow_limit" value="10"/>
</instance_attributes>
<operations>
<op id="resHealthIOWait-monitor-interval-5" interval="5" name="monitor" timeout="5"/>
<op id="resHealthIOWait-start-interval-0s" interval="0s" name="start" timeout="10s"/>
<op id="resHealthIOWait-stop-interval-0s" interval="0s" name="stop" timeout="10s"/>
</operations>
</primitive>
</clone>
The resource agents use ``attrd_updater`` to set proper status for each node
running this resource, as a node attribute whose name starts with ``#health``
(for ``HealthIOWait``, the node attribute is named ``#health-iowait``).
When a node is no longer faulty, you can force the cluster to make it available
to take resources without waiting for the next monitor, by setting the node
health attribute to green. For example:
.. topic:: **Force node1 to be marked as healthy**
.. code-block:: none
# attrd_updater --name "#health-iowait" --update "green" --node "node1"
diff --git a/doc/sphinx/Pacemaker_Explained/resources.rst b/doc/sphinx/Pacemaker_Explained/resources.rst
index 26449d3f85..b0dca49670 100644
--- a/doc/sphinx/Pacemaker_Explained/resources.rst
+++ b/doc/sphinx/Pacemaker_Explained/resources.rst
@@ -1,800 +1,912 @@
.. _resource:
-Cluster Resources
------------------
+Resources
+---------
.. _s-resource-primitive:
-What is a Cluster Resource?
-###########################
-
.. index::
single: resource
A *resource* is a service managed by Pacemaker. The simplest type of resource,
a *primitive*, is described in this chapter. More complex forms, such as groups
and clones, are described in later chapters.
Every primitive has a *resource agent* that provides Pacemaker a standardized
interface for managing the service. This allows Pacemaker to be agnostic about
the services it manages. Pacemaker doesn't need to understand how the service
works because it relies on the resource agent to do the right thing when asked.
Every resource has a *standard* (also called *class*) specifying the interface
that its resource agent follows, and a *type* identifying the specific service
being managed.
.. _s-resource-supported:
.. index::
single: resource; standard
Resource Standards
##################
Pacemaker can use resource agents complying with these standards, described in
more detail below:
* ocf
* lsb
* systemd
* service
* stonithd
* nagios *(deprecated since 2.1.6)*
* upstart *(deprecated since 2.1.0)*
Support for some standards is controlled by build options and so might not be
available in any particular build of Pacemaker. The command ``crm_resource
--list-standards`` will show which standards are supported by the local build.
.. index::
single: resource; OCF
single: OCF; resources
single: Open Cluster Framework; resources
Open Cluster Framework
______________________
The Open Cluster Framework (OCF) Resource Agent API is a ClusterLabs
standard for managing services. It is the most preferred since it is
specifically designed for use in a Pacemaker cluster.
OCF agents are scripts that support a variety of actions including ``start``,
``stop``, and ``monitor``. They may accept parameters, making them more
flexible than other standards. The number and purpose of parameters is left to
the agent, which advertises them via the ``meta-data`` action.
Unlike other standards, OCF agents have a *provider* as well as a standard and
type.
For more information, see the "Resource Agents" chapter of *Pacemaker
Administration* and the `OCF standard
<https://github.com/ClusterLabs/OCF-spec/tree/main/ra>`_.
.. _s-resource-supported-systemd:
.. index::
single: Resource; Systemd
single: Systemd; resources
Systemd
_______
Most Linux distributions use `Systemd
<http://www.freedesktop.org/wiki/Software/systemd>`_ for system initialization
and service management. *Unit files* specify how to manage services and are
usually provided by the distribution.
Pacemaker can manage systemd services. Simply create a resource with
``systemd`` as the resource standard and the unit file name as the resource
type. Do *not* run ``systemctl enable`` on the unit.
.. important::
Make sure that any systemd services to be controlled by the cluster are
*not* enabled to start at boot.
.. index::
single: resource; LSB
single: LSB; resources
single: Linux Standard Base; resources
Linux Standard Base
___________________
*LSB* resource agents, also known as `SysV-style
<https://en.wikipedia.org/wiki/Init#SysV-style init scripts>`_, are scripts that
provide start, stop, and status actions for a service.
They are provided by some operating system distributions. If a full path is not
given, they are assumed to be located in a directory specified when your
Pacemaker software was built (usually ``/etc/init.d``).
In order to be used with Pacemaker, they must conform to the `LSB specification
<http://refspecs.linux-foundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html>`_
as it relates to init scripts.
.. warning::
Some LSB scripts do not fully comply with the standard. For details on how
to check whether your script is LSB-compatible, see the "Resource Agents"
chapter of `Pacemaker Administration`. Common problems include:
* Not implementing the ``status`` action
* Not observing the correct exit status codes
* Starting a started resource returns an error
* Stopping a stopped resource returns an error
.. important::
Make sure the host is *not* configured to start any LSB services at boot
that will be controlled by the cluster.
.. index::
single: Resource; System Services
single: System Service; resources
System Services
_______________
Since there are various types of system services (``systemd``,
``upstart``, and ``lsb``), Pacemaker supports a special ``service`` alias which
intelligently figures out which one applies to a given cluster node.
This is particularly useful when the cluster contains a mix of
``systemd``, ``upstart``, and ``lsb``.
In order, Pacemaker will try to find the named service as:
* an LSB init script
* a Systemd unit file
* an Upstart job
.. index::
single: Resource; STONITH
single: STONITH; resources
STONITH
_______
The ``stonith`` standard is used for managing fencing devices, discussed later
in :ref:`fencing`.
.. index::
single: Resource; Nagios Plugins
single: Nagios Plugins; resources
Nagios Plugins
______________
Nagios Plugins are a way to monitor services. Pacemaker can use these as
resources, to react to a change in the service's status.
To use plugins as resources, Pacemaker must have been built with support, and
OCF-style meta-data for the plugins must be installed on nodes that can run
them. Meta-data for several common plugins is provided by the
`nagios-agents-metadata <https://github.com/ClusterLabs/nagios-agents-metadata>`_
project.
The supported parameters for such a resource are same as the long options of
the plugin.
Start and monitor actions for plugin resources are implemented as invoking the
plugin. A plugin result of "OK" (0) is treated as success, a result of "WARN"
(1) is treated as a successful but degraded service, and any other result is
considered a failure.
A plugin resource is not going to change its status after recovery by
restarting the plugin, so using them alone does not make sense with ``on-fail``
set (or left to default) to ``restart``. Another value could make sense, for
example, if you want to fence or standby nodes that cannot reach some external
service.
A more common use case for plugin resources is to configure them with a
``container`` meta-attribute set to the name of another resource that actually
makes the service available, such as a virtual machine or container.
With ``container`` set, the plugin resource will automatically be colocated
with the containing resource and ordered after it, and the containing resource
will be considered failed if the plugin resource fails. This allows monitoring
of a service inside a virtual machine or container, with recovery of the
virtual machine or container if the service fails.
.. warning::
Nagios support is deprecated in Pacemaker. Support will be dropped entirely
at the next major release of Pacemaker.
For monitoring a service inside a virtual machine or container, the
recommended alternative is to configure the virtual machine as a guest node
or the container as a :ref:`bundle <s-resource-bundle>`. For other use
cases, or when the virtual machine or container image cannot be modified,
the recommended alternative is to write a custom OCF agent for the service
(which may even call the Nagios plugin as part of its status action).
.. index::
single: Resource; Upstart
single: Upstart; resources
Upstart
_______
Some Linux distributions previously used `Upstart
<https://upstart.ubuntu.com/>`_ for system initialization and service
management. Pacemaker is able to manage services using Upstart if the local
system supports them and support was enabled when your Pacemaker software was
built.
The *jobs* that specify how services are managed are usually provided by the
operating system distribution.
.. important::
Make sure the host is *not* configured to start any Upstart services at boot
that will be controlled by the cluster.
.. warning::
Upstart support is deprecated in Pacemaker. Upstart is no longer actively
maintained, and test platforms for it are no longer readily usable. Support
will be dropped entirely at the next major release of Pacemaker.
.. _primitive-resource:
Resource Properties
###################
These values tell the cluster which resource agent to use for the resource,
where to find that resource agent and what standards it conforms to.
.. table:: **Properties of a Primitive Resource**
:widths: 1 4
+-------------+------------------------------------------------------------------+
| Field | Description |
+=============+==================================================================+
| id | .. index:: |
| | single: id; resource |
| | single: resource; property, id |
| | |
| | Your name for the resource |
+-------------+------------------------------------------------------------------+
| class | .. index:: |
| | single: class; resource |
| | single: resource; property, class |
| | |
| | The standard the resource agent conforms to. Allowed values: |
| | ``lsb``, ``ocf``, ``service``, ``stonith``, ``systemd``, |
| | ``nagios`` *(deprecated since 2.1.6)*, and ``upstart`` |
| | *(deprecated since 2.1.0)* |
+-------------+------------------------------------------------------------------+
| description | .. index:: |
| | single: description; resource |
| | single: resource; property, description |
| | |
| | A description of the Resource Agent, intended for local use. |
| | E.g. ``IP address for website`` |
+-------------+------------------------------------------------------------------+
| type | .. index:: |
| | single: type; resource |
| | single: resource; property, type |
| | |
| | The name of the Resource Agent you wish to use. E.g. |
| | ``IPaddr`` or ``Filesystem`` |
+-------------+------------------------------------------------------------------+
| provider | .. index:: |
| | single: provider; resource |
| | single: resource; property, provider |
| | |
| | The OCF spec allows multiple vendors to supply the same resource |
| | agent. To use the OCF resource agents supplied by the Heartbeat |
| | project, you would specify ``heartbeat`` here. |
+-------------+------------------------------------------------------------------+
The XML definition of a resource can be queried with the **crm_resource** tool.
For example:
.. code-block:: none
# crm_resource --resource Email --query-xml
might produce:
.. topic:: A system resource definition
.. code-block:: xml
<primitive id="Email" class="service" type="exim"/>
.. note::
One of the main drawbacks to system services (LSB, systemd or
Upstart) resources is that they do not allow any parameters!
.. topic:: An OCF resource definition
.. code-block:: xml
<primitive id="Public-IP" class="ocf" type="IPaddr" provider="heartbeat">
<instance_attributes id="Public-IP-params">
<nvpair id="Public-IP-ip" name="ip" value="192.0.2.2"/>
</instance_attributes>
</primitive>
.. _resource_options:
Resource Options
################
Resources have two types of options: *meta-attributes* and *instance attributes*.
Meta-attributes apply to any type of resource, while instance attributes
are specific to each resource agent.
Resource Meta-Attributes
________________________
Meta-attributes are used by the cluster to decide how a resource should
behave and can be easily set using the ``--meta`` option of the
**crm_resource** command.
.. list-table:: **Meta-attributes of a Primitive Resource**
:class: longtable
:widths: 2 2 3 5
:header-rows: 1
* - Name
- Type
- Default
- Description
* - .. _meta_priority:
.. index::
single: priority; resource option
single: resource; option, priority
priority
- :ref:`score <score>`
- 0
- If not all resources can be active, the cluster will stop lower-priority
resources in order to keep higher-priority ones active.
* - .. _meta_critical:
.. index::
single: critical; resource option
single: resource; option, critical
critical
- :ref:`boolean <boolean>`
- true
- Use this value as the default for ``influence`` in all
:ref:`colocation constraints <s-resource-colocation>` involving this
resource, as well as in the implicit colocation constraints created if
this resource is in a :ref:`group <group-resources>`. For details, see
:ref:`s-coloc-influence`. *(since 2.1.0)*
* - .. _meta_target_role:
.. index::
single: target-role; resource option
single: resource; option, target-role
target-role
- :ref:`enumeration <enumeration>`
- Started
- What state should the cluster attempt to keep this resource in? Allowed
values:
* ``Stopped:`` Force the resource to be stopped
* ``Started:`` Allow the resource to be started (and in the case of
:ref:`promotable <s-resource-promotable>` clone resources, promoted if
appropriate)
* ``Unpromoted:`` Allow the resource to be started, but only in the
unpromoted role if the resource is
:ref:`promotable <s-resource-promotable>`
* ``Promoted:`` Equivalent to ``Started``
* - .. _meta_is_managed:
.. _is_managed:
.. index::
single: is-managed; resource option
single: resource; option, is-managed
is-managed
- :ref:`boolean <boolean>`
- true
- If false, the cluster will not start, stop, promote, or demote the
resource on any node. Recurring actions for the resource are
unaffected. Maintenance mode overrides this setting.
* - .. _meta_maintenance:
.. _rsc_maintenance:
.. index::
single: maintenance; resource option
single: resource; option, maintenance
maintenance
- :ref:`boolean <boolean>`
- false
- If true, the cluster will not start, stop, promote, or demote the
resource on any node, and will pause any recurring monitors (except those
specifying ``role`` as ``Stopped``). If true, the
:ref:`maintenance-mode <maintenance_mode>` cluster option or
:ref:`maintenance <node_maintenance>` node attribute overrides this.
* - .. _meta_resource_stickiness:
.. _resource-stickiness:
.. index::
single: resource-stickiness; resource option
single: resource; option, resource-stickiness
resource-stickiness
- :ref:`score <score>`
- 1 for individual clone instances, 0 for all other resources
- A score that will be added to the current node when a resource is already
active. This allows running resources to stay where they are, even if
they would be placed elsewhere if they were being started from a stopped
state.
* - .. _meta_requires:
.. _requires:
.. index::
single: requires; resource option
single: resource; option, requires
requires
- :ref:`enumeration <enumeration>`
- ``quorum`` for resources with a ``class`` of ``stonith``, otherwise
``unfencing`` if unfencing is active in the cluster, otherwise
``fencing`` if ``stonith-enabled`` is true, otherwise ``quorum``
- Conditions under which the resource can be started. Allowed values:
* ``nothing:`` The cluster can always start this resource.
* ``quorum:`` The cluster can start this resource only if a majority of
the configured nodes are active.
* ``fencing:`` The cluster can start this resource only if a majority of
the configured nodes are active *and* any failed or unknown nodes have
been :ref:`fenced <fencing>`.
* ``unfencing:`` The cluster can only start this resource if a majority
of the configured nodes are active *and* any failed or unknown nodes
have been fenced *and* only on nodes that have been
:ref:`unfenced <unfencing>`.
* - .. _meta_migration_threshold:
.. index::
single: migration-threshold; resource option
single: resource; option, migration-threshold
migration-threshold
- :ref:`score <score>`
- INFINITY
- How many failures may occur for this resource on a node, before this node
is marked ineligible to host this resource. A value of 0 indicates that
this feature is disabled (the node will never be marked ineligible); by
contrast, the cluster treats ``INFINITY`` (the default) as a very large
but finite number. This option has an effect only if the failed operation
specifies ``on-fail`` as ``restart`` (the default), and additionally for
failed ``start`` operations, if the cluster property
``start-failure-is-fatal`` is ``false``.
* - .. _meta_failure_timeout:
.. index::
single: failure-timeout; resource option
single: resource; option, failure-timeout
failure-timeout
- :ref:`duration <duration>`
- 0
- How many seconds to wait before acting as if the failure had not
occurred, and potentially allowing the resource back to the node on which
it failed. A value of 0 indicates that this feature is disabled.
* - .. _meta_multiple_active:
.. index::
single: multiple-active; resource option
single: resource; option, multiple-active
multiple-active
- :ref:`enumeration <enumeration>`
- stop_start
- What should the cluster do if it ever finds the resource active on more
than one node? Allowed values:
* ``block``: mark the resource as unmanaged
* ``stop_only``: stop all active instances and leave them that way
* ``stop_start``: stop all active instances and start the resource in one
location only
* ``stop_unexpected``: stop all active instances except where the
resource should be active (this should be used only when extra
instances are not expected to disrupt existing instances, and the
resource agent's monitor of an existing instance is capable of
detecting any problems that could be caused; note that any resources
ordered after this will still need to be restarted) *(since 2.1.3)*
* - .. _meta_allow_migrate:
.. index::
single: allow-migrate; resource option
single: resource; option, allow-migrate
allow-migrate
- :ref:`boolean <boolean>`
- true for ``ocf:pacemaker:remote`` resources, false otherwise
- Whether the cluster should try to "live migrate" this resource when it
needs to be moved (see :ref:`live-migration`)
* - .. _meta_allow_unhealthy_nodes:
.. index::
single: allow-unhealthy-nodes; resource option
single: resource; option, allow-unhealthy-nodes
allow-unhealthy-nodes
- :ref:`boolean <boolean>`
- false
- Whether the resource should be able to run on a node even if the node's
health score would otherwise prevent it (see :ref:`node-health`) *(since
2.1.3)*
* - .. _meta_container_attribute_target:
.. index::
single: container-attribute-target; resource option
single: resource; option, container-attribute-target
container-attribute-target
- :ref:`enumeration <enumeration>`
-
- Specific to bundle resources; see :ref:`s-bundle-attributes`
- * - .. _meta_remote_node:
-
- .. index::
- single: remote-node; resource option
- single: resource; option, remote-node
-
- remote-node
- - :ref:`text <text>`
- -
- - The name of the Pacemaker Remote guest node this resource is associated
- with, if any. If specified, this both enables the resource as a guest
- node and defines the unique name used to identify the guest node. The
- guest must be configured to run the Pacemaker Remote daemon when it is
- started. **WARNING:** This value cannot overlap with any resource or node
- IDs.
-
- * - .. _meta_remote_addr:
-
- .. index::
- single: remote-addr; resource option
- single: resource; option, remote-addr
-
- remote-addr
- - :ref:`text <text>`
- - value of ``remote-node``
- - If ``remote-node`` is specified, the IP address or hostname used to
- connect to the guest via Pacemaker Remote. The Pacemaker Remote daemon on
- the guest must be configured to accept connections on this address.
-
- * - .. _meta_remote_port:
-
- .. index::
- single: remote-port; resource option
- single: resource; option, remote-port
-
- remote-port
- - :ref:`port <port>`
- - 3121
- - If ``remote-node`` is specified, the port on the guest used for its
- Pacemaker Remote connection. The Pacemaker Remote daemon on the guest
- must be configured to listen on this port.
-
- * - .. _meta_remote_connect_timeout:
-
- .. index::
- single: remote-connect-timeout; resource option
- single: resource; option, remote-connect-timeout
-
- remote-connect-timeout
- - :ref:`timeout <timeout>`
- - 60s
- - If ``remote-node`` is specified, how long before a pending guest
- connection will time out.
-
- * - .. _meta_remote_allow_migrate:
-
- .. index::
- single: remote-allow-migrate; resource option
- single: resource; option, remote-allow-migrate
-
- remote-allow-migrate
- - :ref:`boolean <boolean>`
- - true
- - If ``remote-node`` is specified, this acts as the ``allow-migrate``
- meta-attribute for the implicit remote connection resource
- (``ocf:pacemaker:remote``).
-
-
As an example of setting resource options, if you performed the following
commands on an LSB Email resource:
.. code-block:: none
# crm_resource --meta --resource Email --set-parameter priority --parameter-value 100
# crm_resource -m -r Email -p multiple-active -v block
the resulting resource definition might be:
.. topic:: An LSB resource with cluster options
.. code-block:: xml
<primitive id="Email" class="lsb" type="exim">
<meta_attributes id="Email-meta_attributes">
<nvpair id="Email-meta_attributes-priority" name="priority" value="100"/>
<nvpair id="Email-meta_attributes-multiple-active" name="multiple-active" value="block"/>
</meta_attributes>
</primitive>
In addition to the cluster-defined meta-attributes described above, you may
also configure arbitrary meta-attributes of your own choosing. Most commonly,
this would be done for use in :ref:`rules <rules>`. For example, an IT department
might define a custom meta-attribute to indicate which company department each
resource is intended for. To reduce the chance of name collisions with
cluster-defined meta-attributes added in the future, it is recommended to use
a unique, organization-specific prefix for such attributes.
.. _s-resource-defaults:
Setting Global Defaults for Resource Meta-Attributes
____________________________________________________
To set a default value for a resource option, add it to the
``rsc_defaults`` section with ``crm_attribute``. For example,
.. code-block:: none
# crm_attribute --type rsc_defaults --name is-managed --update false
would prevent the cluster from starting or stopping any of the
resources in the configuration (unless of course the individual
resources were specifically enabled by having their ``is-managed`` set to
``true``).
Resource Instance Attributes
____________________________
The resource agents of some resource standards (lsb, systemd and upstart *not*
among them) can be given parameters which determine how they behave and which
instance of a service they control.
If your resource agent supports parameters, you can add them with the
``crm_resource`` command. For example,
.. code-block:: none
# crm_resource --resource Public-IP --set-parameter ip --parameter-value 192.0.2.2
would create an entry in the resource like this:
.. topic:: An example OCF resource with instance attributes
.. code-block:: xml
<primitive id="Public-IP" class="ocf" type="IPaddr" provider="heartbeat">
<instance_attributes id="params-public-ip">
<nvpair id="public-ip-addr" name="ip" value="192.0.2.2"/>
</instance_attributes>
</primitive>
For an OCF resource, the result would be an environment variable
called ``OCF_RESKEY_ip`` with a value of ``192.0.2.2``.
The list of instance attributes supported by an OCF resource agent can be
found by calling the resource agent with the ``meta-data`` command.
The output contains an XML description of all the supported
attributes, their purpose and default values.
.. topic:: Displaying the metadata for the Dummy resource agent template
.. code-block:: none
# export OCF_ROOT=/usr/lib/ocf
# $OCF_ROOT/resource.d/pacemaker/Dummy meta-data
.. code-block:: xml
<?xml version="1.0"?>
<!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
<resource-agent name="Dummy" version="2.0">
<version>1.1</version>
<longdesc lang="en">
This is a dummy OCF resource agent. It does absolutely nothing except keep track
of whether it is running or not, and can be configured so that actions fail or
take a long time. Its purpose is primarily for testing, and to serve as a
template for resource agent writers.
</longdesc>
<shortdesc lang="en">Example stateless resource agent</shortdesc>
<parameters>
<parameter name="state" unique-group="state">
<longdesc lang="en">
Location to store the resource state in.
</longdesc>
<shortdesc lang="en">State file</shortdesc>
<content type="string" default="/var/run/Dummy-RESOURCE_ID.state" />
</parameter>
<parameter name="passwd" reloadable="1">
<longdesc lang="en">
Fake password field
</longdesc>
<shortdesc lang="en">Password</shortdesc>
<content type="string" default="" />
</parameter>
<parameter name="fake" reloadable="1">
<longdesc lang="en">
Fake attribute that can be changed to cause a reload
</longdesc>
<shortdesc lang="en">Fake attribute that can be changed to cause a reload</shortdesc>
<content type="string" default="dummy" />
</parameter>
<parameter name="op_sleep" reloadable="1">
<longdesc lang="en">
Number of seconds to sleep during operations. This can be used to test how
the cluster reacts to operation timeouts.
</longdesc>
<shortdesc lang="en">Operation sleep duration in seconds.</shortdesc>
<content type="string" default="0" />
</parameter>
<parameter name="fail_start_on" reloadable="1">
<longdesc lang="en">
Start, migrate_from, and reload-agent actions will return failure if running on
the host specified here, but the resource will run successfully anyway (future
monitor calls will find it running). This can be used to test on-fail=ignore.
</longdesc>
<shortdesc lang="en">Report bogus start failure on specified host</shortdesc>
<content type="string" default="" />
</parameter>
<parameter name="envfile" reloadable="1">
<longdesc lang="en">
If this is set, the environment will be dumped to this file for every call.
</longdesc>
<shortdesc lang="en">Environment dump file</shortdesc>
<content type="string" default="" />
</parameter>
</parameters>
<actions>
<action name="start" timeout="20s" />
<action name="stop" timeout="20s" />
<action name="monitor" timeout="20s" interval="10s" depth="0"/>
<action name="reload" timeout="20s" />
<action name="reload-agent" timeout="20s" />
<action name="migrate_to" timeout="20s" />
<action name="migrate_from" timeout="20s" />
<action name="validate-all" timeout="20s" />
<action name="meta-data" timeout="5s" />
</actions>
</resource-agent>
+
+
+Pacemaker Remote Resources
+##########################
+
+:ref:`Pacemaker Remote <pacemaker_remote>` nodes are defined by resources.
+
+.. _remote_nodes:
+
+.. index::
+ single: node; remote
+ single: Pacemaker Remote; remote node
+ single: remote node
+
+Remote nodes
+____________
+
+A remote node is defined by a connection resource using the special,
+built-in **ocf:pacemaker:remote** resource agent.
+
+.. list-table:: **ocf:pacemaker:remote Instance Attributes**
+ :class: longtable
+ :widths: 2 2 3 5
+ :header-rows: 1
+
+ * - Name
+ - Type
+ - Default
+ - Description
+
+ * - .. _remote_server:
+
+ .. index::
+ pair: remote node; server
+
+ server
+ - :ref:`text <text>`
+ - resource ID
+ - Hostname or IP address used to connect to the remote node. The remote
+ executor on the remote node must be configured to accept connections on
+ this address.
+
+ * - .. _remote_port:
+
+ .. index::
+ pair: remote node; port
+
+ port
+ - :ref:`port <port>`
+ - 3121
+ - TCP port on the remote node used for its Pacemaker Remote connection.
+ The remote executor on the remote node must be configured to listen on
+ this port.
+
+ * - .. _remote_reconnect_interval:
+
+ .. index::
+ pair: remote node; reconnect_interval
+
+ reconnect_interval
+ - :ref:`duration <duration>`
+ - 0
+ - If positive, the cluster will attempt to reconnect to a remote node
+ at this interval after an active connection has been lost. Otherwise,
+ the cluster will attempt to reconnect immediately (after any fencing, if
+ needed).
+
+.. _guest_nodes:
+
+.. index::
+ single: node; guest
+ single: Pacemaker Remote; guest node
+ single: guest node
+
+Guest Nodes
+___________
+
+When configuring a virtual machine as a guest node, the virtual machine is
+created using one of the usual resource agents for that purpose (for example,
+**ocf:heartbeat:VirtualDomain** or **ocf:heartbeat:Xen**), with additional
+meta-attributes.
+
+No restrictions are enforced on what agents may be used to create a guest node,
+but obviously the agent must create a distinct environment capable of running
+the remote executor and cluster resources. An additional requirement is that
+fencing the node hosting the guest node resource must be sufficient for
+ensuring the guest node is stopped. This means that not all hypervisors
+supported by **VirtualDomain** may be used to create guest nodes; if the guest
+can survive the hypervisor being fenced, it is unsuitable for use as a guest
+node.
+
+.. list-table:: **Guest node meta-attributes**
+ :class: longtable
+ :widths: 2 2 3 5
+ :header-rows: 1
+
+ * - Name
+ - Type
+ - Default
+ - Description
+
+ * - .. _meta_remote_node:
+
+ .. index::
+ single: remote-node; resource option
+ single: resource; option, remote-node
+
+ remote-node
+ - :ref:`text <text>`
+ -
+ - If specified, this resource defines a guest node using this node name.
+ The guest must be configured to run the remote executor when it is
+ started. This value *must not* be the same as any resource or node ID.
+
+ * - .. _meta_remote_addr:
+
+ .. index::
+ single: remote-addr; resource option
+ single: resource; option, remote-addr
+
+ remote-addr
+ - :ref:`text <text>`
+ - value of ``remote-node``
+ - If ``remote-node`` is specified, the hostname or IP address used to
+ connect to the guest. The remote executor on the guest must be
+ configured to accept connections on this address.
+
+ * - .. _meta_remote_port:
+
+ .. index::
+ single: remote-port; resource option
+ single: resource; option, remote-port
+
+ remote-port
+ - :ref:`port <port>`
+ - 3121
+ - If ``remote-node`` is specified, the port on the guest used for its
+ Pacemaker Remote connection. The remote executor on the guest must be
+ configured to listen on this port.
+
+ * - .. _meta_remote_connect_timeout:
+
+ .. index::
+ single: remote-connect-timeout; resource option
+ single: resource; option, remote-connect-timeout
+
+ remote-connect-timeout
+ - :ref:`timeout <timeout>`
+ - 60s
+ - If ``remote-node`` is specified, how long before a pending guest
+ connection will time out.
+
+ * - .. _meta_remote_allow_migrate:
+
+ .. index::
+ single: remote-allow-migrate; resource option
+ single: resource; option, remote-allow-migrate
+
+ remote-allow-migrate
+ - :ref:`boolean <boolean>`
+ - true
+ - If ``remote-node`` is specified, this acts as the ``allow-migrate``
+ meta-attribute for its implicitly created remote connection resource
+ (``ocf:pacemaker:remote``).
+
+Removing Pacemaker Remote Nodes
+_______________________________
+
+If the resource creating a remote node connection or guest node is removed from
+the configuration, status output may continue to show the affected node (as
+offline).
+
+If you want to get rid of that output, run the following command, replacing
+``$NODE_NAME`` appropriately:
+
+.. code-block:: none
+
+ # crm_node --force --remove $NODE_NAME
+
+.. WARNING::
+
+ Be absolutely sure that there are no references to the node's resource in the
+ configuration before running the above command.

File Metadata

Mime Type
text/x-diff
Expires
Mon, Apr 21, 7:15 PM (12 h, 28 m)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
1665455
Default Alt Text
(91 KB)

Event Timeline