diff --git a/doc/sphinx/Pacemaker_Explained/local-options.rst b/doc/sphinx/Pacemaker_Explained/local-options.rst index 915a65b9ec..74785f5b66 100644 --- a/doc/sphinx/Pacemaker_Explained/local-options.rst +++ b/doc/sphinx/Pacemaker_Explained/local-options.rst @@ -1,750 +1,750 @@ Host-Local Configuration ------------------------ .. index:: pair: XML element; configuration .. note:: Directory and file paths below may differ on your system depending on your Pacemaker build settings. Check your Pacemaker configuration file to find the correct paths. Configuration Value Types ######################### Throughout this document, configuration values will be designated as having one of the following types: .. list-table:: **Configuration Value Types** :class: longtable :widths: 1 3 :header-rows: 1 * - Type - Description * - .. _boolean: .. index:: pair: type; boolean boolean - Case-insensitive text value where ``1``, ``yes``, ``y``, ``on``, and ``true`` evaluate as true and ``0``, ``no``, ``n``, ``off``, ``false``, and unset evaluate as false * - .. _date_time: .. index:: pair: type; date/time date/time - Textual timestamp like ``Sat Dec 21 11:47:45 2013`` * - .. _duration: .. index:: pair: type; duration duration - A time duration, specified either like a :ref:`timeout ` or an `ISO 8601 duration `_. A duration may be up to approximately 49 days but is intended for much smaller time periods. * - .. _enumeration: .. index:: pair: type; enumeration enumeration - Text that must be one of a set of defined values (which will be listed in the description) * - .. _epoch_time: .. index:: pair: type; epoch_time epoch_time - Time as the integer number of seconds since the Unix epoch, ``1970-01-01 00:00:00 +0000 (UTC)``. * - .. _id: .. index:: pair: type; id id - A text string starting with a letter or underbar, followed by any combination of letters, numbers, dashes, dots, and/or underbars; when used for a property named ``id``, the string must be unique across all ``id`` properties in the CIB * - .. _integer: .. index:: pair: type; integer integer - 32-bit signed integer value (-2,147,483,648 to 2,147,483,647) * - .. _iso8601: .. index:: pair: type; iso8601 ISO 8601 - An `ISO 8601 `_ date/time. * - .. _nonnegative_integer: .. index:: pair: type; nonnegative integer nonnegative integer - 32-bit nonnegative integer value (0 to 2,147,483,647) * - .. _percentage: .. index:: pair: type; percentage percentage - Floating-point number followed by an optional percent sign ('%') * - .. _port: .. index:: pair: type; port port - Integer TCP port number (0 to 65535) * - .. _range: .. index:: pair: type; range range - A range may be a single nonnegative integer or a dash-separated range of nonnegative integers. Either the first or last value may be omitted to leave the range open-ended. Examples: ``0``, ``3-``, ``-5``, ``4-6``. * - .. _score: .. index:: pair: type; score score - A Pacemaker score can be an integer between -1,000,000 and 1,000,000, or a string alias: ``INFINITY`` or ``+INFINITY`` is equivalent to 1,000,000, ``-INFINITY`` is equivalent to -1,000,000, and ``red``, ``yellow``, and ``green`` are equivalent to integers as described in :ref:`node-health`. * - .. _text: .. index:: pair: type; text text - A text string * - .. _timeout: .. index:: pair: type; timeout timeout - A time duration, specified as a bare number (in which case it is considered to be in seconds) or a number with a unit (``ms`` or ``msec`` for milliseconds, ``us`` or ``usec`` for microseconds, ``s`` or ``sec`` for seconds, ``m`` or ``min`` for minutes, ``h`` or ``hr`` for hours) optionally with whitespace before and/or after the number. * - .. _version: .. index:: pair: type; version version - Version number (any combination of alphanumeric characters, dots, and dashes, starting with a number). Scores ______ Scores are integral to how Pacemaker works. Practically everything from moving a resource to deciding which resource to stop in a degraded cluster is achieved by manipulating scores in some way. Scores are calculated per resource and node. Any node with a negative score for a resource can't run that resource. The cluster places a resource on the node with the highest score for it. Score addition and subtraction follow these rules: * Any value (including ``INFINITY``) - ``INFINITY`` = ``-INFINITY`` * ``INFINITY`` + any value other than ``-INFINITY`` = ``INFINITY`` .. note:: What if you want to use a score higher than 1,000,000? Typically this possibility arises when someone wants to base the score on some external metric that might go above 1,000,000. The short answer is you can't. The long answer is it is sometimes possible work around this limitation creatively. You may be able to set the score to some computed value based on the external metric rather than use the metric directly. For nodes, you can store the metric as a node attribute, and query the attribute when computing the score (possibly as part of a custom resource agent). Local Options ############# Pacemaker supports several host-local configuration options. These options can be configured on each node in the main Pacemaker configuration file (|PCMK_CONFIG_FILE|) in the format ``=""``. They work by setting environment variables when Pacemaker daemons start up. .. list-table:: **Local Options** :class: longtable :widths: 2 2 2 5 :header-rows: 1 * - Name - Type - Default - Description * - .. _cib_pam_service: .. index:: pair: node option; CIB_pam_service CIB_pam_service - :ref:`text ` - login - PAM service to use for remote CIB client authentication (passed to ``pam_start``). * - .. _pcmk_logfacility: .. index:: pair: node option; PCMK_logfacility PCMK_logfacility - :ref:`enumeration ` - daemon - Enable logging via the system log or journal, using the specified log facility. Messages sent here are of value to all Pacemaker administrators. This can be disabled using ``none``, but that is not recommended. Allowed values: * ``none`` * ``daemon`` * ``user`` * ``local0`` * ``local1`` * ``local2`` * ``local3`` * ``local4`` * ``local5`` * ``local6`` * ``local7`` * - .. _pcmk_logpriority: .. index:: pair: node option; PCMK_logpriority PCMK_logpriority - :ref:`enumeration ` - notice - Unless system logging is disabled using ``PCMK_logfacility=none``, messages of the specified log severity and higher will be sent to the system log. The default is appropriate for most installations. Allowed values: * ``emerg`` * ``alert`` * ``crit`` * ``error`` * ``warning`` * ``notice`` * ``info`` * ``debug`` * - .. _pcmk_logfile: .. index:: pair: node option; PCMK_logfile PCMK_logfile - :ref:`text ` - |PCMK_LOG_FILE| - Unless set to ``none``, more detailed log messages will be sent to the specified file (in addition to the system log, if enabled). These messages may have extended information, and will include messages of info severity. This log is of more use to developers and advanced system administrators, and when reporting problems. Note: The default is |PCMK_CONTAINER_LOG_FILE| (inside the container) for bundled container nodes; this would typically be mapped to a different path on the host running the container. * - .. _pcmk_logfile_mode: .. index:: pair: node option; PCMK_logfile_mode PCMK_logfile_mode - :ref:`text ` - 0660 - Pacemaker will set the permissions on the detail log to this value (see ``chmod(1)``). * - .. _pcmk_debug: .. index:: pair: node option; PCMK_debug PCMK_debug - :ref:`enumeration ` - no - Whether to send debug severity messages to the detail log. This may be set for all subsystems (``yes`` or ``no``) or for specific (comma- separated) subsystems. Allowed subsystems are: * ``pacemakerd`` * ``pacemaker-attrd`` * ``pacemaker-based`` * ``pacemaker-controld`` * ``pacemaker-execd`` * ``pacemaker-fenced`` * ``pacemaker-schedulerd`` Example: ``PCMK_debug="pacemakerd,pacemaker-execd"`` * - .. _pcmk_stderr: .. index:: pair: node option; PCMK_stderr PCMK_stderr - :ref:`boolean ` - no - *Advanced Use Only:* Whether to send daemon log messages to stderr. This would be useful only during troubleshooting, when starting Pacemaker manually on the command line. Setting this option in the configuration file is pointless, since the file is not read when starting Pacemaker manually. However, it can be set directly as an environment variable on the command line. * - .. _pcmk_trace_functions: .. index:: pair: node option; PCMK_trace_functions PCMK_trace_functions - :ref:`text ` - - *Advanced Use Only:* Send debug and trace severity messages from these (comma-separated) source code functions to the detail log. Example: ``PCMK_trace_functions="func1,func2"`` * - .. _pcmk_trace_files: .. index:: pair: node option; PCMK_trace_files PCMK_trace_files - :ref:`text ` - - *Advanced Use Only:* Send debug and trace severity messages from all functions in these (comma-separated) source file names to the detail log. Example: ``PCMK_trace_files="file1.c,file2.c"`` * - .. _pcmk_trace_formats: .. index:: pair: node option; PCMK_trace_formats PCMK_trace_formats - :ref:`text ` - - *Advanced Use Only:* Send trace severity messages that are generated by these (comma-separated) format strings in the source code to the detail log. Example: ``PCMK_trace_formats="Error: %s (%d)"`` * - .. _pcmk_trace_tags: .. index:: pair: node option; PCMK_trace_tags PCMK_trace_tags - :ref:`text ` - - *Advanced Use Only:* Send debug and trace severity messages related to these (comma-separated) resource IDs to the detail log. Example: ``PCMK_trace_tags="client-ip,dbfs"`` * - .. _pcmk_blackbox: .. index:: pair: node option; PCMK_blackbox PCMK_blackbox - :ref:`enumeration ` - no - *Advanced Use Only:* Enable blackbox logging globally (``yes`` or ``no``) or by subsystem. A blackbox contains a rolling buffer of all logs (of all severities). Blackboxes are stored under |CRM_BLACKBOX_DIR| by default, by default, and their contents can be viewed using the ``qb-blackbox(8)`` command. The blackbox recorder can be enabled at start using this variable, or at runtime by sending a Pacemaker subsystem daemon process a ``SIGUSR1`` or ``SIGTRAP`` signal, and disabled by sending ``SIGUSR2`` (see ``kill(1)``). The blackbox will be written after a crash, assertion failure, or ``SIGTRAP`` signal. See :ref:`PCMK_debug ` for allowed subsystems. Example: ``PCMK_blackbox="pacemakerd,pacemaker-execd"`` * - .. _pcmk_trace_blackbox: .. index:: pair: node option; PCMK_trace_blackbox PCMK_trace_blackbox - :ref:`enumeration ` - - *Advanced Use Only:* Write a blackbox whenever the message at the specified function and line is logged. Multiple entries may be comma- separated. Example: ``PCMK_trace_blackbox="remote.c:144,remote.c:149"`` * - .. _pcmk_node_start_state: .. index:: pair: node option; PCMK_node_start_state PCMK_node_start_state - :ref:`enumeration ` - default - By default, the local host will join the cluster in an online or standby state when Pacemaker first starts depending on whether it was previously put into standby mode. If this variable is set to ``standby`` or ``online``, it will force the local host to join in the specified state. * - .. _pcmk_node_action_limit: .. index:: pair: node option; PCMK_node_action_limit PCMK_node_action_limit - :ref:`nonnegative integer ` - - Specify the maximum number of jobs that can be scheduled on this node. If set, this overrides the :ref:`node-action-limit ` cluster option on this node. * - .. _pcmk_shutdown_delay: .. index:: pair: node option; PCMK_shutdown_delay PCMK_shutdown_delay - :ref:`timeout ` - - Specify a delay before shutting down ``pacemakerd`` after shutting down all other Pacemaker daemons. * - .. _pcmk_fail_fast: .. index:: pair: node option; PCMK_fail_fast PCMK_fail_fast - :ref:`boolean ` - no - By default, if a Pacemaker subsystem crashes, the main ``pacemakerd`` process will attempt to restart it. If this variable is set to ``yes``, ``pacemakerd`` will panic the local host instead. * - .. _pcmk_panic_action: .. index:: pair: node option; PCMK_panic_action PCMK_panic_action - :ref:`enumeration ` - reboot - Pacemaker will panic the local host under certain conditions. By default, this means rebooting the host. This variable can change that behavior: if ``crash``, trigger a kernel crash (useful if you want a kernel dump to investigate); if ``sync-reboot`` or ``sync-crash``, synchronize filesystems before rebooting the host or triggering a kernel crash. The sync values are more likely to preserve log messages, but with the risk that the host may be left active if the synchronization hangs. * - .. _pcmk_authkey_location: .. index:: pair: node option; PCMK_authkey_location PCMK_authkey_location - :ref:`text ` - |PCMK_AUTHKEY_FILE| - Use the contents of this file as the authorization key to use with Pacemaker Remote connections. This file must be readable by Pacemaker daemons (that is, it must allow read permissions to either the |CRM_DAEMON_USER| user or the |CRM_DAEMON_GROUP| group), and its contents must be identical on all nodes. * - .. _pcmk_remote_address: .. index:: pair: node option; PCMK_remote_address PCMK_remote_address - :ref:`text ` - - By default, if the Pacemaker Remote service is run on the local node, it will listen for connections on all IP addresses. This may be set to one address to listen on instead, as a resolvable hostname or as a numeric IPv4 or IPv6 address. When resolving names or listening on all addresses, IPv6 will be preferred if available. When listening on an IPv6 address, IPv4 clients will be supported via IPv4-mapped IPv6 addresses. Example: ``PCMK_remote_address="192.0.2.1"`` * - .. _pcmk_remote_port: .. index:: pair: node option; PCMK_remote_port PCMK_remote_port - :ref:`port ` - 3121 - Use this TCP port number for Pacemaker Remote node connections. This value must be the same on all nodes. * - .. _pcmk_remote_pid1: .. index:: pair: node option; PCMK_remote_pid1 PCMK_remote_pid1 - :ref:`enumeration ` - default - *Advanced Use Only:* When a bundle resource's ``run-command`` option is left to default, Pacemaker Remote runs as PID 1 in the bundle's containers. When it does so, it loads environment variables from the container's |PCMK_INIT_ENV_FILE| and performs the PID 1 responsibility of reaping dead subprocesses. This option controls whether those actions are performed when Pacemaker Remote is not running as PID 1. It is intended primarily for developer testing but can be useful when ``run-command`` is set to a separate, custom PID 1 process that launches Pacemaker Remote. * ``full``: Pacemaker Remote loads environment variables from |PCMK_INIT_ENV_FILE| and reaps dead subprocesses. * ``vars``: Pacemaker Remote loads environment variables from |PCMK_INIT_ENV_FILE| but does not reap dead subprocesses. * ``default``: Pacemaker Remote performs neither action. If Pacemaker Remote is running as PID 1, this option is ignored, and the behavior is the same as for ``full``. * - .. _pcmk_tls_priorities: .. index:: pair: node option; PCMK_tls_priorities PCMK_tls_priorities - :ref:`text ` - |PCMK_GNUTLS_PRIORITIES| - *Advanced Use Only:* These GnuTLS cipher priorities will be used for TLS connections (whether for Pacemaker Remote connections or remote CIB access, when enabled). See: https://gnutls.org/manual/html_node/Priority-Strings.html Pacemaker will append ``":+ANON-DH"`` for remote CIB access and ``":+DHE-PSK:+PSK"`` for Pacemaker Remote connections, as they are required for the respective functionality. Example: ``PCMK_tls_priorities="SECURE128:+SECURE192"`` * - .. _pcmk_dh_min_bits: .. index:: pair: node option; PCMK_dh_min_bits PCMK_dh_min_bits - :ref:`nonnegative integer ` - 0 (no minimum) - *Advanced Use Only:* Set a lower bound on the bit length of the prime number generated for Diffie-Hellman parameters needed by TLS connections. The default is no minimum. The server (Pacemaker Remote daemon, or CIB manager configured to accept remote clients) will use this value to provide a floor for the value recommended by the GnuTLS library. The library will only accept a limited number of specific values, which vary by library version, so setting these is recommended only when required for compatibility with specific client versions. Clients (connecting cluster nodes or remote CIB commands) will require that the server use a prime of at least this size. This is recommended only when the value must be lowered in order for the client's GnuTLS library to accept a connection to an older server. * - .. _pcmk_dh_max_bits: .. index:: pair: node option; PCMK_dh_max_bits PCMK_dh_max_bits - :ref:`nonnegative integer ` - 0 (no maximum) - *Advanced Use Only:* Set an upper bound on the bit length of the prime number generated for Diffie-Hellman parameters needed by TLS connections. The default is no maximum. The server (Pacemaker Remote daemon, or CIB manager configured to accept remote clients) will use this value to provide a ceiling for the value recommended by the GnuTLS library. The library will only accept a limited number of specific values, which vary by library version, so setting these is recommended only when required for compatibility with specific client versions. Clients do not use ``PCMK_dh_max_bits``. * - .. _pcmk_ipc_type: .. index:: pair: node option; PCMK_ipc_type PCMK_ipc_type - :ref:`enumeration ` - shared-mem - *Advanced Use Only:* Force use of a particular IPC method. Allowed values: * ``shared-mem`` * ``socket`` * ``posix`` * ``sysv`` * - .. _pcmk_ipc_buffer: .. index:: pair: node option; PCMK_ipc_buffer PCMK_ipc_buffer - :ref:`nonnegative integer ` - 131072 - *Advanced Use Only:* Specify an IPC buffer size in bytes. This can be useful when connecting to large clusters that result in messages exceeding the default size (which will also result in log messages referencing this variable). * - .. _pcmk_cluster_type: .. index:: pair: node option; PCMK_cluster_type PCMK_cluster_type - :ref:`enumeration ` - corosync - *Advanced Use Only:* Specify the cluster layer to be used. If unset, Pacemaker will detect and use a supported cluster layer, if available. Currently, ``"corosync"`` is the only supported cluster layer. If multiple layers are supported in the future, this will allow overriding Pacemaker's automatic detection to select a specific one. * - .. _pcmk_schema_directory: .. index:: pair: node option; PCMK_schema_directory PCMK_schema_directory - :ref:`text ` - |CRM_SCHEMA_DIRECTORY| - *Advanced Use Only:* Specify an alternate location for RNG schemas and XSL transforms. * - .. _pcmk_remote_schema_directory: .. index:: pair: node option; PCMK_remote_schema_directory PCMK_remote_schema_directory - :ref:`text ` - |PCMK__REMOTE_SCHEMA_DIR| - *Advanced Use Only:* Specify an alternate location on Pacemaker Remote nodes for storing newer RNG schemas and XSL transforms fetched from the cluster. * - .. _pcmk_valgrind_enabled: .. index:: pair: node option; PCMK_valgrind_enabled PCMK_valgrind_enabled - :ref:`enumeration ` - no - *Advanced Use Only:* Whether subsystem daemons should be run under ``valgrind``. Allowed values are the same as for ``PCMK_debug``. * - .. _pcmk_callgrind_enabled: .. index:: pair: node option; PCMK_callgrind_enabled PCMK_callgrind_enabled - :ref:`enumeration ` - no - *Advanced Use Only:* Whether subsystem daemons should be run under ``valgrind`` with the ``callgrind`` tool enabled. Allowed values are the same as for ``PCMK_debug``. * - .. _sbd_sync_resource_startup: .. index:: - pair:: node option; SBD_SYNC_RESOURCE_STARTUP + pair: node option; SBD_SYNC_RESOURCE_STARTUP SBD_SYNC_RESOURCE_STARTUP - :ref:`boolean ` - - If true, ``pacemakerd`` waits for a ping from ``sbd`` during startup before starting other Pacemaker daemons, and during shutdown after stopping other Pacemaker daemons but before exiting. Default value is set based on the ``--with-sbd-sync-default`` configure script option. * - .. _sbd_watchdog_timeout: .. index:: - pair:: node option; SBD_WATCHDOG_TIMEOUT + pair: node option; SBD_WATCHDOG_TIMEOUT SBD_WATCHDOG_TIMEOUT - :ref:`duration ` - - If the ``stonith-watchdog-timeout`` cluster property is set to a negative or invalid value, use double this value as the default if positive, or use 0 as the default otherwise. This value must be greater than the value of ``stonith-watchdog-timeout`` if both are set. * - .. _valgrind_opts: .. index:: pair: node option; VALGRIND_OPTS VALGRIND_OPTS - :ref:`text ` - - *Advanced Use Only:* Pass these options to valgrind, when enabled (see ``valgrind(1)``). ``"--vgdb=no"`` should usually be specified because ``pacemaker-execd`` can lower privileges when executing commands, which would otherwise leave a bunch of unremovable files in ``/tmp``. diff --git a/doc/sphinx/Pacemaker_Explained/status.rst b/doc/sphinx/Pacemaker_Explained/status.rst index 6384edaef6..2cdf20c7e8 100644 --- a/doc/sphinx/Pacemaker_Explained/status.rst +++ b/doc/sphinx/Pacemaker_Explained/status.rst @@ -1,368 +1,459 @@ .. index:: - single: status - single: XML element, status + pair: XML element; status -Status -- Here be dragons -------------------------- +Status +------ +Pacemaker automatically generates a ``status`` section in the CIB (inside the +``cib`` element, at the same level as ``configuration``). The status is +transient, and is not stored to disk with the rest of the CIB. -Most users never need to understand the contents of the status section -and can be happy with the output from ``crm_mon``. +The section's structure and contents are internal to Pacemaker and subject to +change from release to release. Its often obscure element and attribute names +are kept for historical reasons, to maintain compatibility with older versions +during rolling upgrades. + +Users should not modify the section directly, though various command-line tool +options affect it indirectly. -However for those with a curious inclination, this section attempts to -provide an overview of its contents. .. index:: - single: node; status + pair: XML element; node_state + single: node; state -Node Status -########### +Node State +########## -In addition to the cluster's configuration, the CIB holds an -up-to-date representation of each cluster node in the ``status`` section. +The ``status`` element contains ``node_state`` elements for each node in the +cluster (and potentially nodes that have been removed from the configuration +since the cluster started). The ``node_state`` element has attributes that +allow the cluster to determine whether the node is healthy. -.. topic:: A bare-bones status entry for a healthy node **cl-virt-1** +.. topic:: Example minimal node state entry .. code-block:: xml - + -Users are highly recommended *not* to modify any part of a node's -state *directly*. The cluster will periodically regenerate the entire -section from authoritative sources, so any changes should be done -with the tools appropriate to those sources. - -.. table:: **Authoritative Sources for State Information** - :widths: 1 1 - - +----------------------+----------------------+ - | CIB Object | Authoritative Source | - +======================+======================+ - | node_state | pacemaker-controld | - +----------------------+----------------------+ - | transient_attributes | pacemaker-attrd | - +----------------------+----------------------+ - | lrm | pacemaker-execd | - +----------------------+----------------------+ - -The fields used in the ``node_state`` objects are named as they are -largely for historical reasons, to maintain compatibility with older versions. - -.. table:: **Node Status Fields** - :widths: 1 3 - - +------------------+----------------------------------------------------------+ - | Field | Description | - +==================+==========================================================+ - | id | .. index: | - | | single: id; node status | - | | single: node; status, id | - | | | - | | Unique identifier for the node. Corosync-based clusters | - | | use a numeric counter. | - +------------------+----------------------------------------------------------+ - | uname | .. index:: | - | | single: uname; node status | - | | single: node; status, uname | - | | | - | | The node's name as known by the cluster | - +------------------+----------------------------------------------------------+ - | in_ccm | .. index:: | - | | single: in_ccm; node status | - | | single: node; status, in_ccm | - | | | - | | Is the node a member at the cluster communication later? | - | | Allowed values: ``true``, ``false``. | - +------------------+----------------------------------------------------------+ - | crmd | .. index:: | - | | single: crmd; node status | - | | single: node; status, crmd | - | | | - | | Is the node a member at the pacemaker layer? Allowed | - | | values: ``online``, ``offline``. | - +------------------+----------------------------------------------------------+ - | crm-debug-origin | .. index:: | - | | single: crm-debug-origin; node status | - | | single: node; status, crm-debug-origin | - | | | - | | The name of the source function that made the most | - | | recent change (for debugging purposes). | - +------------------+----------------------------------------------------------+ - | join | .. index:: | - | | single: join; node status | - | | single: node; status, join | - | | | - | | Does the node participate in hosting resources? | - | | Allowed values: ``down``, ``pending``, ``member``. | - | | ``banned``. | - +------------------+----------------------------------------------------------+ - | expected | .. index:: | - | | single: expected; node status | - | | single: node; status, expected | - | | | - | | Expected value for ``join``. | - +------------------+----------------------------------------------------------+ - -The cluster uses these fields to determine whether, at the node level, the -node is healthy or is in a failed state and needs to be fenced. - +.. list-table:: **Attributes of a node_state Element** + :class: longtable + :widths: 1 1 3 + :header-rows: 1 + + * - Name + - Type + - Description + * - .. _node_state_id: + + .. index:: + pair: node_state; id + + id + - :ref:`text ` + - Node ID (identical to ``id`` of corresponding ``node`` element in the + ``configuration`` section) + * - .. node_state_uname: + + .. index:: + pair: node_state; uname + + uname + - :ref:`text ` + - Node name (identical to ``uname`` of corresponding ``node`` element in the + ``configuration`` section) + * - .. node_state_in_ccm: + + .. index:: + pair: node_state; in_ccm + + in_ccm + - :ref:`epoch time ` *(since 2.1.7; previously boolean)* + - If the node's controller is currently in the cluster layer's membership, + this is the epoch time at which it joined (or 1 if the node is in the + process of leaving the cluster), otherwise 0 *(since 2.1.7; previously, + it was "true" or "false")* + * - .. node_state_crmd: + + .. index:: + pair: node_state; crmd + + crmd + - :ref:`epoch time ` *(since 2.1.7; previously an enumeration)* + - If the node's controller is currently in the cluster layer's controller + messaging group, this is the epoch time at which it joined, otherwise 0 + *(since 2.1.7; previously, the value was either "online" or "offline")* + * - .. node_state_crm_debug_origin: + + .. index:: + pair: node_state; crm-debug-origin + + crm-debug-origin + - :ref:`text ` + - Name of the source code function that recorded this ``node_state`` + element (for debugging) + * - .. node_state_join: + + .. index:: + pair: node_state; join + + join + - :ref:`enumeration ` + - Current status of node's controller join sequence (and thus whether it + is eligible to run resources). Allowed values: + + * ``down``: Not yet joined + * ``pending``: In the process of joining or leaving + * ``member``: Fully joined + * ``banned``: Rejected by DC + * - .. node_state_expected: + + .. index:: + pair: node_state; expected + + expected + - :ref:`enumeration ` + - What cluster expects ``join`` to be in the immediate future. Allowed + values are same as for ``join``. + + +.. _transient_attributes: + +.. index:: + pair: XML element; transient_attributes + single: node; transient attribute + single: node attribute; transient + Transient Node Attributes ######################### -Like regular :ref:`node_attributes`, the name/value -pairs listed in the ``transient_attributes`` section help to describe the -node. However they are forgotten by the cluster when the node goes offline. -This can be useful, for instance, when you want a node to be in standby mode -(not able to run resources) just until the next reboot. - -In addition to any values the administrator sets, the cluster will -also store information about failed resources here. +The ``transient_attributes`` section specifies transient +:ref:`node_attributes`. In addition to any values set by the administrator or +resource agents using the ``attrd_updater`` or ``crm_attribute`` tools, the +cluster stores various state information here. -.. topic:: A set of transient node attributes for node **cl-virt-1** +.. topic:: Example transient node attributes for a node .. code-block:: xml - -In the above example, we can see that a monitor on the ``pingd:0`` resource has -failed once, at 09:22:22 UTC 6 April 2009. [#]_. -We also see that the node is connected to three **pingd** peers and that -all known resources have been checked for on this machine (``probe_complete``). - .. index:: - single: Operation History + pair: XML element; lrm + pair: XML element; lrm_resources + pair: node; history -Operation History -################# - -A node's resource history is held in the ``lrm_resources`` element (a child -of the ``lrm`` element). The information stored here includes enough -information for the cluster to stop the resource safely if it is -removed from the ``configuration`` section. Specifically, the resource's -``id``, ``class``, ``type`` and ``provider`` are stored. +Node History +############ -.. topic:: A record of the ``apcstonith`` resource +Each ``node_state`` element contains an ``lrm`` element with a history of +certain resource actions performed on the node. The ``lrm`` element contains an +``lrm_resources`` element. - .. code-block:: xml +.. index:: + pair: XML element; lrm_resource + pair: resource; history - - -Additionally, we store history entries for certain actions. +Resource History +________________ + +The ``lrm_resources`` element contains an ``lrm_resource`` element for each +resource that has had an action performed on the node. + +An ``lrm_resource`` entry has attributes allowing the cluster to stop the +resource safely even if it is removed from the configuration. Specifically, the +resource's ``id``, ``class``, ``type`` and ``provider`` are recorded. + +.. index:: + pair: XML element; lrm_rsc_op + pair: action; history + +Action History +______________ -.. table:: **Attributes of an lrm_rsc_op element** +Each ``lrm_resource`` element contains an ``lrm_rsc_op`` element for each +recorded action performed for that resource on that node. (Not all actions are +recorded, just enough to determine the resource's state.) + +.. list-table:: **Attributes of an lrm_rsc_op element** :class: longtable - :widths: 1 3 - - +------------------+----------------------------------------------------------+ - | Field | Description | - +==================+==========================================================+ - | id | .. index:: | - | | single: id; action status | - | | single: action; status, id | - | | | - | | Identifier for the history entry constructed from the | - | | resource ID, action name, and operation interval. | - +------------------+----------------------------------------------------------+ - | call-id | .. index:: | - | | single: call-id; action status | - | | single: action; status, call-id | - | | | - | | A node-specific counter used to determine the order in | - | | which actions were executed. | - +------------------+----------------------------------------------------------+ - | operation | .. index:: | - | | single: operation; action status | - | | single: action; status, operation | - | | | - | | The action name the resource agent was invoked with. | - +------------------+----------------------------------------------------------+ - | interval | .. index:: | - | | single: interval; action status | - | | single: action; status, interval | - | | | - | | The frequency, in milliseconds, at which the operation | - | | will be repeated. One-time execution is indicated by 0. | - +------------------+----------------------------------------------------------+ - | op-status | .. index:: | - | | single: op-status; action status | - | | single: action; status, op-status | - | | | - | | The execution status of this action. The meanings of | - | | these codes are internal to Pacemaker. | - +------------------+----------------------------------------------------------+ - | rc-code | .. index:: | - | | single: rc-code; action status | - | | single: action; status, rc-code | - | | | - | | The resource agent's exit status for this action. Refer | - | | to the *Resource Agents* chapter of | - | | *Pacemaker Administration* for how these values are | - | | interpreted. | - +------------------+----------------------------------------------------------+ - | last-rc-change | .. index:: | - | | single: last-rc-change; action status | - | | single: action; status, last-rc-change | - | | | - | | Machine-local date/time, in seconds since epoch, at | - | | which the action first returned the current value of | - | | ``rc-code``. For diagnostic purposes. | - +------------------+----------------------------------------------------------+ - | exec-time | .. index:: | - | | single: exec-time; action status | - | | single: action; status, exec-time | - | | | - | | Time, in milliseconds, that the action was running for. | - | | For diagnostic purposes. | - +------------------+----------------------------------------------------------+ - | queue-time | .. index:: | - | | single: queue-time; action status | - | | single: action; status, queue-time | - | | | - | | Time, in seconds, that the action was queued for in the | - | | local executor. For diagnostic purposes. | - +------------------+----------------------------------------------------------+ - | crm_feature_set | .. index:: | - | | single: crm_feature_set; action status | - | | single: action; status, crm_feature_set | - | | | - | | The Pacemaker feature set used to record this entry. | - +------------------+----------------------------------------------------------+ - | transition-key | .. index:: | - | | single: transition-key; action status | - | | single: action; status, transition-key | - | | | - | | A concatenation of the action's graph action number, the | - | | graph number, the expected result and the UUID of the | - | | controller instance that scheduled it. This is used to | - | | construct ``transition-magic`` (below). | - +------------------+----------------------------------------------------------+ - | transition-magic | .. index:: | - | | single: transition-magic; action status | - | | single: action; status, transition-magic | - | | | - | | A concatenation of ``op-status``, ``rc-code`` | - | | and ``transition-key``. Guaranteed to be unique for the | - | | life of the cluster (which ensures it is part of CIB | - | | update notifications) and contains all the information | - | | needed for the controller to correctly analyze and | - | | process the completed action. Most importantly, the | - | | decomposed elements tell the controller if the history | - | | entry was expected and whether it failed. | - +------------------+----------------------------------------------------------+ - | op-digest | .. index:: | - | | single: op-digest; action status | - | | single: action; status, op-digest | - | | | - | | An MD5 sum representing the parameters passed to the | - | | action. Used to detect changes to the configuration, to | - | | restart resources if necessary. | - +------------------+----------------------------------------------------------+ - | crm-debug-origin | .. index:: | - | | single: crm-debug-origin; action status | - | | single: action; status, crm-debug-origin | - | | | - | | The origin of the current values. For diagnostic | - | | purposes. | - +------------------+----------------------------------------------------------+ - + :widths: 1 1 3 + :header-rows: 1 + + * - Name + - Type + - Description + * - .. _lrm_rsc_op_id: + + .. index:: + pair: lrm_rsc_op; id + + id + - :ref:`text ` + - Identifier for the history entry constructed from the resource ID, + action name or history entry type, and action interval. + * - .. _lrm_rsc_op_operation_key: + + .. index:: + pair: lrm_rsc_op; operation_key + + operation_key + - :ref:`text ` + - Identifier for the action that was executed, constructed from the + resource ID, action name, and action interval. + * - .. _lrm_rsc_op_operation: + + .. index:: + pair: lrm_rsc_op; operation + + operation + - :ref:`text ` + - The name of the action the history entry is for + * - .. _lrm_rsc_op_crm_debug_origin: + + .. index:: + pair: lrm_rsc_op; crm-debug-origin + + crm-debug-origin + - :ref:`text ` + - Name of the source code function that recorded this entry (for + debugging) + * - .. _lrm_rsc_op_crm_feature_set: + + .. index:: + pair: lrm_rsc_op; crm_feature_set + + crm_feature_set + - :ref:`version ` + - The Pacemaker feature set used to record this entry. + * - .. _lrm_rsc_op_transition_key: + + .. index:: + pair: lrm_rsc_op; transition-key + + transition-key + - :ref:`text ` + - A concatenation of the action's transition graph action number, the + transition graph number, the action's expected result, and the UUID of + the controller instance that scheduled it. + * - .. _lrm_rsc_op_transition_magic: + + .. index:: + pair: lrm_rsc_op; transition-magic + + transition-magic + - :ref:`text ` + - A concatenation of ``op-status``, ``rc-code``, and ``transition-key``. + * - .. _lrm_rsc_op_exit_reason: + + .. index:: + pair: lrm_rsc_op; exit-reason + + exit-reason + - :ref:`text ` + - An error message (if available) from the resource agent or Pacemaker if + the action did not return success. + * - .. _lrm_rsc_op_on_node: + + .. index:: + pair: lrm_rsc_op; on_node + + on_node + - :ref:`text ` + - The name of the node that executed the action (identical to the + ``uname`` of the enclosing ``node_state`` element) + * - .. _lrm_rsc_op_call_id: + + .. index:: + pair: lrm_rsc_op; call-id + + call-id + - :ref:`integer ` + - A node-specific counter used to determine the order in which actions + were executed. + * - .. _lrm_rsc_op_rc_code: + + .. index:: + pair: lrm_rsc_op; rc-code + + rc-code + - :ref:`integer ` + - The resource agent's exit status for this action. Refer to the *Resource + Agents* chapter of *Pacemaker Administration* for how these values are + interpreted. + * - .. _lrm_rsc_op_op_status: + + .. index:: + pair: lrm_rsc_op; op-status + + op-status + - :ref:`integer ` + - The execution status of this action. The meanings of these codes are + internal to Pacemaker. + * - .. _lrm_rsc_op_interval: + + .. index:: + pair: lrm_rsc_op; interval + + interval + - :ref:`nonnegative integer ` + - If the action is recurring, its frequency (in milliseconds), otherwise + 0. + * - .. _lrm_rsc_op_last_rc_change: + + .. index:: + pair: lrm_rsc_op; last-rc-change + + last-rc-change + - :ref:`epoch time ` + - Node-local time at which the action first returned the current value of + ``rc-code``. + * - .. _lrm_rsc_op_exec_time: + + .. index:: + pair: lrm_rsc_op; exec-time + + exec-time + - :ref:`integer ` + - Time (in seconds) that action execution took (if known) + * - .. _lrm_rsc_op_queue_time: + + .. index:: + pair: lrm_rsc_op; queue-time + + queue-time + - :ref:`integer ` + - Time (in seconds) that action was queued in the local executor (if known) + * - .. _lrm_rsc_op_op_digest: + + .. index:: + pair: lrm_rsc_op; op-digest + + op-digest + - :ref:`text ` + - If present, this is a hash of the parameters passed to the action. If a + hash of the currently configured parameters does not match this, that + means the resource configuration changed since the action was performed, + and the resource must be reloaded or restarted. + * - .. _lrm_rsc_op_op_restart_digest: + + .. index:: + pair: lrm_rsc_op; op-restart-digest + + op-restart-digest + - :ref:`text ` + - If present, the resource agent supports reloadable parameters, and this + is a hash of the non-reloadable parameters passed to the action. This + allows the cluster to choose between reload and restart when one is + needed. + * - .. _lrm_rsc_op_op_secure_digest: + + .. index:: + pair: lrm_rsc_op; op-secure-digest + + op-secure-digest + - :ref:`text ` + - If present, the resource agent marks some parameters as sensitive, and + this is a hash of the non-sensitive parameters passed to the action. + This allows the value of sensitive parameters to be removed from a saved + copy of the CIB while still allowing scheduler simulations to be + performed on that copy. + + Simple Operation History Example ________________________________ .. topic:: A monitor operation (determines current state of the ``apcstonith`` resource) .. code-block:: xml -In the above example, the action is a non-recurring monitor operation -often referred to as a "probe" for the ``apcstonith`` resource. +The above example shows the history entry for a probe (non-recurring monitor +operation) for the ``apcstonith`` resource. The cluster schedules probes for every configured resource on a node when the node first starts, in order to determine the resource's current state before it takes any further action. From the ``transition-key``, we can see that this was the 22nd action of the 2nd graph produced by this instance of the controller (2668bbeb-06d5-40f9-936d-24cb7f87006a). The third field of the ``transition-key`` contains a 7, which indicates that the cluster expects to find the resource inactive. By looking at the ``rc-code`` property, we see that this was the case. As that is the only action recorded for this node, we can conclude that the cluster started the resource elsewhere. Complex Operation History Example _________________________________ .. topic:: Resource history of a ``pingd`` clone with multiple entries .. code-block:: xml When more than one history entry exists, it is important to first sort them by ``call-id`` before interpreting them. Once sorted, the above example can be summarized as: -#. A non-recurring monitor operation returning 7 (not running), with a ``call-id`` of 3 +#. A non-recurring monitor operation returning 7 (not running), with a + ``call-id`` of 3 #. A stop operation returning 0 (success), with a ``call-id`` of 32 #. A start operation returning 0 (success), with a ``call-id`` of 33 #. A recurring monitor returning 0 (success), with a ``call-id`` of 34 The cluster processes each history entry to build up a picture of the resource's state. After the first and second entries, it is considered stopped, and after the third it considered active. Based on the last operation, we can tell that the resource is currently active. Additionally, from the presence of a ``stop`` operation with a lower ``call-id`` than that of the ``start`` operation, we can conclude that the resource has been restarted. Specifically this occurred as part of actions 11 and 31 of transition 11 from the controller instance with the key ``2668bbeb...``. This information can be helpful for locating the relevant section of the logs when looking for the source of a failure. - -.. [#] You can use the standard ``date`` command to print a human-readable version - of any seconds-since-epoch value, for example ``date -d @1239009742``.