**This describes proposed configuration changes for a possible Pacemaker 3.0.0 release. Discussion should be directed to the [[ /w/clusterlabs/clusterlabs_mailing_lists/ | users mailing list]].** These changes are only proposed and may not make it into the final release. == The build process == === Dependencies === In general, Pacemaker 3 will support dependency versions that are available in supported releases of major Linux distributions (usually but not always including long-term support releases). The following are specifically targeted for support: * Debian 10 "buster" and later * RHEL 8.0 and later * SUSE 15.0 and later * Ubuntu 18 "bionic" and later The only changes are these new minimum dependency versions: * libdbus 1.5.12 * libgnutls 3.1.7 (also now required rather than optional) * libqb 1.0.1 * libxml2 2.9.2 * pkg-config 0.28 (or pkgconf) * Python 3.6 * rpm 4.14.0 (if building RPMs) === configure script === The following configure script options are no longer supported: * `--enable-legacy-links` //(deprecated in Pacemaker 2.1.0)// * `--enable-compat-2.0` //(introduced in 2.1.0 explicitly for the 2.1 series only)// * `--enable-upstart` //(deprecated in Pacemaker 2.1.0)// * `--with-gnutls` //(GnuTLS is now required and cannot be disabled)// * `--with-nagios` //(deprecated in Pacemaker 2.1.6)// === RPMs === * The `RPMDEST` Makefile variable may no longer be set to `toplevel`. The default of `subtree`, and explicit paths, are still supported. * The spec file now longer supports the `--with-doc` and `--without-doc` options. Documentation will always be built. * Pacemaker's logrotate script is now part of the pacemaker package instead of the pacemaker-cli package. * The `--with stonithd`/`--without stonithd` spec file option, which controls support for Linux-HA fencing agents, has been renamed to `--with linuxha/--without linuxha` --- == Resource agents == * The obsolete and deprecated **ocf:pacemaker:o2cb** resource agent (which manages the no-longer-used ocf_controld.pcmk daemon for OCFS2 file systems) has been dropped. --- == Start-up environment variables == These are typically set under /etc/sysconfig or /etc/default. * Pacemaker no longer uses the `HA_cib_timeout`, `HA_shutdown_delay`, `PCMK_cib_timeout`, `PCMK_dh_min_bits`, or `PCMK_shutdown_delay` variables. * Pacemaker now interprets the value of `PCMK_panic_action` case-sensitively, and invalid values will be logged as a warning. --- == Configuration Information Base (CIB) XML changes == * **XML syntax in the CIB must now be correctly formed.** Previously, some simple XML syntax errors would be ignored. Upgrades will not succeed if the XML syntax is invalid, so ensure it is valid //before// upgrading any node in the cluster. * **Deprecated schemas have been dropped.** The `validate-with` attribute of the `cib` XML element may no longer be omitted or set to the deprecated values `pacemaker-0.6`, `transitional-0.6`, `pacemaker-0.7`, `pacemaker-1.1`, `pacemaker-next`, or any unknown value. Upgrades will not succeed unless the CIB has a supported value for `validate-with` //before// the upgrade, so ensure one is set before upgrading any node in the cluster. (Disabling validation by setting `validate-with` to `none` remains allowed and deprecated.) * **Schema names are now case-sensitive.** The `validate-with` attribute of the `cib` XML element must now match the schema file names by case (all lowercase). Upgrades will not succeed unless the CIB has a supported value for `validate-with` //before// the upgrade, so ensure one is set before upgrading any node in the cluster. * Support for the following deprecated (and mostly undocumented) features has been dropped. Any uses in existing configurations will be automatically and silently converted to current syntax when possible. They will not be allowed in new configurations, or in existing ones after running `cibadmin --upgrade`. ** cluster options: `remove-after-stop`, and the `stonith-action` cluster option set to `poweroff` ** nodes: `type` set to `ping` ** `primitive` resources: `nagios` or `upstart` class ** `group` resources: `collocated` and `ordered` group options; groups with no members will be ignored (previously, they could have some effect if used in constraints) ** `master` resources ** `bundle` resources: using `rkt` to launch containers ** resource meta-attributes: `can-fail`, `can_fail`, `restart-type`, and `role-after-failure` ** rules: `moon` in `date_spec` elements ** constraints: `lifetime` elements in any constraint, `ordering` attribute of resource sets in colocation constraints, and `score` in ordering constraints * Error checking is stricter (many of these errors are possible only with schema validation disabled) ** Clones using non-OCF resource agents and the `globally-unique` clone option set to `true` will be considered an error and ignored (currently, a warning is logged but the resource is allowed) ** Invalid fencing topology levels are now disallowed by the schema (rather than waiting until they are rejected by the fencer; valid levels are 1 through 9) ** The value of an `acl_target` element's `id` attribute must be of the XML ID type (noncompliant names will be given an auto-generated ID and moved to the `name` attribute) ** Resource history entries with an invalid `rc-code` (which should be possible only with manual mis-editing or a bug) will now be ignored rather than treated as action failures ** Location constraints with an invalid `boolean-op` or `role`, colocation constraints with an invalid `rsc-role` or `with-rsc-role`, and ticket constraints with an invalid `rsc-role` are now ignored (previously, the default value would be used). ** An `nvpair` elements with an invalid `id-ref` attribute is now ignored even if it has a `name` and/or `value` attribute as well ** An `nvpair` element with a valid `id-ref` attribute and also a `value` attribute now ignores the `value` attribute ** If a `cluster_property_set`, `instance_attributes`, `meta_attributes`, or `utilization` element contains multiple top-level rules, only the first will now be interpreted ** A `node` with an invalid `type` is now ignored (previously, they were treated as `ping` nodes, contributing to quorum but unable to run resources) * Rule syntax is now be more strictly interpreted ** A date expression is now treated as not passing if it is missing an `id`; contains an invalid `operation`; contains a `date_spec` or `duration` with a missing `id` or an attribute with an invalid range; has an `operation` of `in_range`, `gt`, or `lt` with an invalid or missing `start` or `end`; or uses `date_spec` without a `date_spec` element ** An attribute expression is now treated as not passing if it is missing an `id`; has an invalid `value-source` or `type`; or has `value` missing when it is required or present when it is unused ** A resource expression is now treated as not passing if it is missing an `id` ** An operation expression is now treated as not passing if it is missing an `id` or `interval` or has an invalid `interval` ** A rule is now treated as not passing if it is missing an `id` or has an invalid `boolean-op` ** An rule with a `boolean-op` of `or` with no conditions is now properly treated as passing. ** The schema will no longer allow rule syntax where it is unused, including: `op_expression` in a rule anywhere other than `op_defaults`; `rsc_expression` in a rule anywhere other than `rsc_defaults` or `op_defaults`; `score` or `score-attribute` in a rule anywhere other than a location constraint; `value-source` in a rule `expression` anywhere other than a location constraint; `monthdays`, `weekdays`, `weekyears`, or `moon` in a `duration`; `yearsdays` anywhere (the correct spelling `yeardays` is allowed only in a `time_spec`); an invalid `role` name in any `rule` * Changes in defaults ** The documentation states that the default for `action` in a `resource_set` is the enclosing constraint's action. However until now `start` has actually been used as the default. The behavior has changed to match the documentation. ** `action` configured as a fence device parameter is now ignored rather than treated as a default fencing action. ** The `concurrent-fencing` cluster option now defaults to true and is deprecated ** The `globally-unique` clone option now defaults to `true` when `clone-node-max` is greater than 1 * A negative `migration-threshold`, which is invalid and was previously treated as 1, is now treated as 0 (no limit). * Previously, a rule `expression` with the empty string as `value` would never be treated as passing. Now, it passes when the corresponding node attribute is the empty string. * Pacemaker has always allowed custom settings to be configured in cluster options and resource options, which will be ignored by Pacemaker but can be used by users' own tooling. The user has to take care not to conflict with any setting names that Pacemaker uses. As of Pacemaker 2.0.0, when support for certain deprecated options was dropped, the schema enforced that users could not use those names as custom settings, to prevent misunderstandings. Those restrictions have now been dropped, and if those names are configured, they will be ignored as with any other custom setting.

== Start-up environment variables == These are typically set under /etc/sysconfig or /etc/default. * Pacemaker no longer uses the `HA_cib_timeout`, `HA_shutdown_delay`, `PCMK_cib_timeout`, `PCMK_dh_min_bits`, or `PCMK_shutdown_delay` variables. * Pacemaker now interprets the value of `PCMK_panic_action` case-sensitively, and invalid values will be logged as a warning. --- == Configuration Information Base (CIB) XML changes == * **XML syntax in the CIB must now be correctly formed.** Previously, some simple XML syntax errors would be ignored. Upgrades will not succeed if the XML syntax is invalid, so ensure it is valid //before// upgrading any node in the cluster (if you are running Pacemaker 2.1.8 or later, you can check the logs for deprecation warnings). * **Deprecated schemas have been dropped.** The `validate-with` attribute of the `cib` XML element may no longer be omitted or set to the deprecated values `pacemaker-0.6`, `transitional-0.6`, `pacemaker-0.7`, `pacemaker-1.1`, `pacemaker-next`, or any unknown value. Upgrades will not succeed unless the CIB has a supported value for `validate-with` //before// the upgrade, so ensure one is set before upgrading any node in the cluster. (Disabling validation by setting `validate-with` to `none` remains allowed and deprecated.) * **Schema names are now case-sensitive.** The `validate-with` attribute of the `cib` XML element must now match the schema file names by case (all lowercase). Upgrades will not succeed unless the CIB has a supported value for `validate-with` //before// the upgrade, so ensure one is set before upgrading any node in the cluster. * Support for the following deprecated (and mostly undocumented) features has been dropped. Any uses in existing configurations will be automatically and silently converted to current syntax when possible. They will not be allowed in new configurations, or in existing ones after running `cibadmin --upgrade`. ** cluster options: `crmd-finalization-timeout`, `crmd-integration-timeout`, `crmd-transition-delay`, `remove-after-stop`, and the `stonith-action` cluster option set to `poweroff` ** nodes: `type` set to `ping` ** `primitive` resources: `nagios` or `upstart` class ** `master` resources (use `clone` resource with `promotable` meta-attribute set to `true` instead) ** `bundle` resources: using `rkt` to launch containers; `masters` meta-attribute ** operation options: `can-fail`, `can_fail`, and `role_after_failure` ** resource options: `restart-type` ** rules: `moon` in `date_spec` elements ** constraints: `lifetime` elements in any constraint * Error checking is stricter (most of these errors are possible only with schema validation disabled, and so will not affect the vast majority of users) ** Invalid fencing topology levels are now disallowed by the schema (rather than waiting until they are rejected by the fencer; valid levels are 1 through 9) ** Resource history entries with an invalid `rc-code` (which should be possible only with manual mis-editing or a bug) will now be ignored rather than treated as action failures ** Location constraints with an invalid `score`, `boolean-op`, or `role`, colocation constraints with an invalid `score`, `rsc-role`, or `with-rsc-role`, and ticket constraints with an invalid `rsc-role` are now ignored (previously, the default value would be used). ** An `nvpair` element with an invalid `id-ref` attribute is now ignored even if it has a `name` and/or `value` attribute as well ** An `nvpair` element with a valid `id-ref` attribute and also `name` and/or `value` attributes now ignores the `name` and `value` attributes ** If a `cluster_property_set`, `instance_attributes`, `meta_attributes`, or `utilization` element contains multiple top-level rules, only the first will now be interpreted ** A `node` with an invalid `type` is now ignored (previously, they were treated as `ping` nodes, contributing to quorum but unable to run resources) ** Any `instance_attributes` in `rsc_defaults` is now ignored * Rule syntax is more strictly interpreted ** A date expression is now treated as not passing if it is missing an `id`; contains an invalid `operation`; contains a `date_spec` or `duration` with a missing `id` or an attribute with an invalid range; has an `operation` of `in_range`, `gt`, or `lt` with an invalid or missing `start` or `end`; or uses `date_spec` without a `date_spec` element ** An attribute expression is now treated as not passing if it is missing an `id`; has an invalid `value-source` or `type`; or has `value` missing when it is required or present when it is unused ** A resource expression is now treated as not passing if it is missing an `id` ** An operation expression is now treated as not passing if it is missing an `id` or `interval` or has an invalid `interval` ** A rule is now treated as not passing if it is missing an `id` or has an invalid `boolean-op` ** An rule with a `boolean-op` of `or` with no conditions is now properly treated as passing. ** The schema will no longer allow rule syntax where it is unused, including: `op_expression` in a rule anywhere other than `op_defaults`; `rsc_expression` in a rule anywhere other than `rsc_defaults` or `op_defaults`; `score` or `score-attribute` in a rule anywhere other than a location constraint; `value-source` in a rule `expression` anywhere other than a location constraint; `monthdays`, `weekdays`, `weekyears`, or `moon` in a `duration`; `yearsdays` anywhere (the correct spelling `yeardays` is allowed only in a `time_spec`); an invalid `role` name in any `rule` * Changes in defaults ** `action` configured as a fence device parameter is now ignored rather than treated as a default fencing action ** The `concurrent-fencing` cluster option now defaults to true and is deprecated ** The `globally-unique` clone option now defaults to `true` when `clone-node-max` is greater than 1 * Location constraints may now have at most a single top-level rule (previously, they could have any number; use separate location constraints for each top-level rule instead). * `nvpair` elements must now contain a `value` attribute (it is no longer optional). * If an `nvpair` element has an `id-ref` attribute, it can no longer also have a `name` attribute to override the referenced element. * Pacemaker has always allowed custom names to be configured in cluster options and resource options, which are ignored by Pacemaker but can be used by users' own tooling. The user has to take care not to conflict with any option names that Pacemaker uses. As of Pacemaker 2.0.0, when support for certain deprecated options was dropped, the schema enforced that users could not use those names as custom names, to prevent misunderstandings. Those restrictions have now been dropped, and if those names are configured, they will be ignored as with any other custom name.

**This describes proposed configuration changes for a possible Pacemaker 3.0.0 release. Discussion should be directed to the [[ /w/clusterlabs/clusterlabs_mailing_lists/ | users mailing list]].** These changes are only proposed and may not make it into the final release. == The build process == === Dependencies === In general, Pacemaker 3 will support dependency versions that are available in supported releases of major Linux distributions (usually but not always including long-term support releases). The following are specifically targeted for support: * Debian 10 "buster" and later * RHEL 8.0 and later * SUSE 15.0 and later * Ubuntu 18 "bionic" and later The only changes are these new minimum dependency versions: * libdbus 1.5.12 * libgnutls 3.1.7 (also now required rather than optional) * libqb 1.0.1 * libxml2 2.9.2 * pkg-config 0.28 (or pkgconf) * Python 3.6 * rpm 4.14.0 (if building RPMs) === configure script === The following configure script options are no longer supported: * `--enable-legacy-links` //(deprecated in Pacemaker 2.1.0)// * `--enable-compat-2.0` //(introduced in 2.1.0 explicitly for the 2.1 series only)// * `--enable-upstart` //(deprecated in Pacemaker 2.1.0)// * `--with-gnutls` //(GnuTLS is now required and cannot be disabled)// * `--with-nagios` //(deprecated in Pacemaker 2.1.6)// === RPMs === * The `RPMDEST` Makefile variable may no longer be set to `toplevel`. The default of `subtree`, and explicit paths, are still supported. * The spec file now longer supports the `--with-doc` and `--without-doc` options. Documentation will always be built. * Pacemaker's logrotate script is now part of the pacemaker package instead of the pacemaker-cli package. * The `--with stonithd`/`--without stonithd` spec file option, which controls support for Linux-HA fencing agents, has been renamed to `--with linuxha/--without linuxha` --- == Resource agents == * The obsolete and deprecated **ocf:pacemaker:o2cb** resource agent (which manages the no-longer-used ocf_controld.pcmk daemon for OCFS2 file systems) has been dropped. --- == Start-up environment variables == These are typically set under /etc/sysconfig or /etc/default. * Pacemaker no longer uses the `HA_cib_timeout`, `HA_shutdown_delay`, `PCMK_cib_timeout`, `PCMK_dh_min_bits`, or `PCMK_shutdown_delay` variables. * Pacemaker now interprets the value of `PCMK_panic_action` case-sensitively, and invalid values will be logged as a warning. --- == Configuration Information Base (CIB) XML changes == * **XML syntax in the CIB must now be correctly formed.** Previously, some simple XML syntax errors would be ignored. Upgrades will not succeed if the XML syntax is invalid, so ensure it is valid //before// upgrading any node in the cluster (if you are running Pacemaker 2.1.8 or later, you can check the logs for deprecation warnings). * **Deprecated schemas have been dropped.** The `validate-with` attribute of the `cib` XML element may no longer be omitted or set to the deprecated values `pacemaker-0.6`, `transitional-0.6`, `pacemaker-0.7`, `pacemaker-1.1`, `pacemaker-next`, or any unknown value. Upgrades will not succeed unless the CIB has a supported value for `validate-with` //before// the upgrade, so ensure one is set before upgrading any node in the cluster. (Disabling validation by setting `validate-with` to `none` remains allowed and deprecated.) * **Schema names are now case-sensitive.** The `validate-with` attribute of the `cib` XML element must now match the schema file names by case (all lowercase). Upgrades will not succeed unless the CIB has a supported value for `validate-with` //before// the upgrade, so ensure one is set before upgrading any node in the cluster. * Support for the following deprecated (and mostly undocumented) features has been dropped. Any uses in existing configurations will be automatically and silently converted to current syntax when possible. They will not be allowed in new configurations, or in existing ones after running `cibadmin --upgrade`. ** cluster options: `crmd-finalization-timeout`, `crmd-integration-timeout`, `crmd-transition-delay`, `remove-after-stop`, and the `stonith-action` cluster option set to `poweroff` ** nodes: `type` set to `ping` ** `primitive` resources: `nagios` or `upstart` class ** `group` resources: `collocated` and `ordered` group options; groups with no members will be ignored (previously, they could have some effect if used in constraintsmaster` resources (use `clone` resource with `promotable` meta-attribute set to `true` instead) ** `masterbundle` resourcesurces: using `rkt` to launch containers; `masters` meta-attribute ** `bundle` resources: using `rkt` to launch containers** operation options: `can-fail`, `can_fail`, and `role_after_failure` ** resource meta-attributes: `can-fail`, `can_fail`,options: `restart-type`, and `role-after-failure`e` ** rules: `moon` in `date_spec` elements ** constraints: `lifetime` elements in any constraint, `ordering` attribute of resource sets in colocation constraints, and `score` in ordering constraints * Error checking is stricter (manymost of these errors are possible only with schema validation disabled) ** Clones using non-OCF resource agents and the `globally-unique` clone option set to `true` will be considered an error and ignored (currently, a warning is logged buand so will not affect the resource is allowedvast majority of users) ** Invalid fencing topology levels are now disallowed by the schema (rather than waiting until they are rejected by the fencer; valid levels are 1 through 9) ** The value of an `acl_target` element's `id` attribute must be of the XML ID type (noncompliant names will be given an auto-generated ID and moved to the `name` attribute) ** Resource history entries with an invalid `rc-code` (which should be possible only with manual mis-editing or a bug) will now be ignored rather than treated as action failures ** Location constraints with an invalid ``score`, `boolean-op` or ``, or `role`, colocation constraints with an invalid `score`, `rsc-role`, colocation constraints with an invalid `rsc-role` or `with-rsc-role`, and ticket constraints with an invalid `rsc-role` are now ignored (previously, the default value would be used). ** An `nvpair` elements with an invalid `id-ref` attribute is now ignored even if it has a `name` and/or `value` attribute as well ** An `nvpair` element with a valid `id-ref` attribute and also a`name` and/or `value` attributes now ignores thee `name` and `value` attributes ** If a `cluster_property_set`, `instance_attributes`, `meta_attributes`, or `utilization` element contains multiple top-level rules, only the first will now be interpreted ** A `node` with an invalid `type` is now ignored (previously, they were treated as `ping` nodes, contributing to quorum but unable to run resources) ** Any `instance_attributes` in `rsc_defaults` is now ignored * Rule syntax is now be more strictly interpreted ** A date expression is now treated as not passing if it is missing an `id`; contains an invalid `operation`; contains a `date_spec` or `duration` with a missing `id` or an attribute with an invalid range; has an `operation` of `in_range`, `gt`, or `lt` with an invalid or missing `start` or `end`; or uses `date_spec` without a `date_spec` element ** An attribute expression is now treated as not passing if it is missing an `id`; has an invalid `value-source` or `type`; or has `value` missing when it is required or present when it is unused ** A resource expression is now treated as not passing if it is missing an `id` ** An operation expression is now treated as not passing if it is missing an `id` or `interval` or has an invalid `interval` ** A rule is now treated as not passing if it is missing an `id` or has an invalid `boolean-op` ** An rule with a `boolean-op` of `or` with no conditions is now properly treated as passing. ** The schema will no longer allow rule syntax where it is unused, including: `op_expression` in a rule anywhere other than `op_defaults`; `rsc_expression` in a rule anywhere other than `rsc_defaults` or `op_defaults`; `score` or `score-attribute` in a rule anywhere other than a location constraint; `value-source` in a rule `expression` anywhere other than a location constraint; `monthdays`, `weekdays`, `weekyears`, or `moon` in a `duration`; `yearsdays` anywhere (the correct spelling `yeardays` is allowed only in a `time_spec`); an invalid `role` name in any `rule` * Changes in defaults ** The documentation states that the default for `action` in a `resource_set` is the enclosing constraint's action. However until now `start` has actually been used as the default. The behavior has changed to match the documentation. ** `action` configured as a fence device parameter is now ignored rather than treated as a default fencing action. ** The `concurrent-fencing` cluster option now defaults to true and is deprecated ** The `globally-unique` clone option now defaults to `true` when `clone-node-max` is greater than 1 * A negative `migration-threshold`* Location constraints may now have at most a single top-level rule (previously, which is invalid and was previously treated as 1,they could have any number; is now treated as 0 (no limituse separate location constraints for each top-level rule instead). * `nvpair` elements must now contain a `value` attribute (it is no longer optional). * Previously, a rule `expression` with the empty string as `value` would never be treated as passing. Now* If an `nvpair` element has an `id-ref` attribute, it passes when the corresponding nodeit can no longer also have a `name` attribute isto override the empty stringreferenced element. * Pacemaker has always allowed custom settingsnames to be configured in cluster options and resource options, which will beare ignored by Pacemaker but can be used by users' own tooling. The user has to take care not to conflict with any settingoption names that Pacemaker uses. As of Pacemaker 2.0.0, when support for certain deprecated options was dropped, the schema enforced that users could not use those names as custom settingsnames, to prevent misunderstandings. Those restrictions have now been dropped, and if those names are configured, they will be ignored as with any other custom settingname.