diff --git a/cts/cli/regression.rules.exp b/cts/cli/regression.rules.exp
index 86e937c314..6e35c44e7b 100644
--- a/cts/cli/regression.rules.exp
+++ b/cts/cli/regression.rules.exp
@@ -1,629 +1,629 @@
Created new pacemaker configuration
Setting up shadow instance
A new shadow instance was created. To begin using it paste the following into your shell:
CIB_shadow=cts-cli ; export CIB_shadow
=#=#=#= Begin test: Try to check a rule that doesn't exist =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
No rule found with ID=blahblah
Error checking rule: No such device or address
=#=#=#= Current cib after: Try to check a rule that doesn't exist =#=#=#=
=#=#=#= End test: Try to check a rule that doesn't exist - No such object (105) =#=#=#=
* Passed: crm_rule - Try to check a rule that doesn't exist
=#=#=#= Begin test: Try to check a rule that has too many date_expressions =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Can't check rule cli-rule-too-many-date-expressions because it does not have exactly one date_expression
Error checking rule: Operation not supported
=#=#=#= Current cib after: Try to check a rule that has too many date_expressions =#=#=#=
=#=#=#= End test: Try to check a rule that has too many date_expressions - Unimplemented (3) =#=#=#=
* Passed: crm_rule - Try to check a rule that has too many date_expressions
=#=#=#= Begin test: Verify basic rule is expired =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Rule cli-prefer-rule-dummy-expired is expired
=#=#=#= Current cib after: Verify basic rule is expired =#=#=#=
=#=#=#= End test: Verify basic rule is expired - Requested item has expired (110) =#=#=#=
* Passed: crm_rule - Verify basic rule is expired
=#=#=#= Begin test: Verify basic rule worked in the past =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Rule cli-prefer-rule-dummy-expired is still in effect
=#=#=#= Current cib after: Verify basic rule worked in the past =#=#=#=
=#=#=#= End test: Verify basic rule worked in the past - OK (0) =#=#=#=
* Passed: crm_rule - Verify basic rule worked in the past
=#=#=#= Begin test: Verify basic rule is not yet in effect =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Rule cli-prefer-rule-dummy-not-yet has not yet taken effect
=#=#=#= Current cib after: Verify basic rule is not yet in effect =#=#=#=
=#=#=#= End test: Verify basic rule is not yet in effect - Requested item is not yet in effect (111) =#=#=#=
* Passed: crm_rule - Verify basic rule is not yet in effect
=#=#=#= Begin test: Verify date_spec rule with years has expired =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Rule cli-prefer-rule-dummy-date_spec-only-years is expired
=#=#=#= Current cib after: Verify date_spec rule with years has expired =#=#=#=
=#=#=#= End test: Verify date_spec rule with years has expired - Requested item has expired (110) =#=#=#=
* Passed: crm_rule - Verify date_spec rule with years has expired
=#=#=#= Begin test: Verify date_spec rule with years is in effect =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Rule cli-prefer-rule-dummy-date_spec-only-years satisfies conditions
=#=#=#= Current cib after: Verify date_spec rule with years is in effect =#=#=#=
=#=#=#= End test: Verify date_spec rule with years is in effect - OK (0) =#=#=#=
* Passed: crm_rule - Verify date_spec rule with years is in effect
=#=#=#= Begin test: Try to check a rule whose date_spec does not contain years= =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Rule either must not use date_spec, or use date_spec with years= but not moon=
Error checking rule: No such device or address
=#=#=#= Current cib after: Try to check a rule whose date_spec does not contain years= =#=#=#=
=#=#=#= End test: Try to check a rule whose date_spec does not contain years= - No such object (105) =#=#=#=
* Passed: crm_rule - Try to check a rule whose date_spec does not contain years=
=#=#=#= Begin test: Try to check a rule whose date_spec contains years= and moon= =#=#=#=
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Rule either must not use date_spec, or use date_spec with years= but not moon=
Error checking rule: No such device or address
=#=#=#= Current cib after: Try to check a rule whose date_spec contains years= and moon= =#=#=#=
=#=#=#= End test: Try to check a rule whose date_spec contains years= and moon= - No such object (105) =#=#=#=
* Passed: crm_rule - Try to check a rule whose date_spec contains years= and moon=
=#=#=#= Begin test: Try to check a rule with no date_expression =#=#=#=
-unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
-unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
-unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
+unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
+unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
+unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Can't check rule cli-no-date_expression-rule because it does not have exactly one date_expression
Error checking rule: Operation not supported
=#=#=#= Current cib after: Try to check a rule with no date_expression =#=#=#=
=#=#=#= End test: Try to check a rule with no date_expression - Unimplemented (3) =#=#=#=
* Passed: crm_rule - Try to check a rule with no date_expression
diff --git a/cts/cli/regression.upgrade.exp b/cts/cli/regression.upgrade.exp
index d1435395c5..28ca057004 100644
--- a/cts/cli/regression.upgrade.exp
+++ b/cts/cli/regression.upgrade.exp
@@ -1,144 +1,147 @@
Created new pacemaker configuration
Setting up shadow instance
A new shadow instance was created. To begin using it paste the following into your shell:
CIB_shadow=cts-cli ; export CIB_shadow
=#=#=#= Begin test: Set stonith-enabled=false =#=#=#=
=#=#=#= Current cib after: Set stonith-enabled=false =#=#=#=
=#=#=#= End test: Set stonith-enabled=false - OK (0) =#=#=#=
* Passed: crm_attribute - Set stonith-enabled=false
=#=#=#= Begin test: Configure the initial resource =#=#=#=
=#=#=#= Current cib after: Configure the initial resource =#=#=#=
=#=#=#= End test: Configure the initial resource - OK (0) =#=#=#=
* Passed: cibadmin - Configure the initial resource
=#=#=#= Begin test: Upgrade to latest CIB schema (trigger 2.10.xsl + the wrapping) =#=#=#=
update_validation debug: Testing 'pacemaker-2.10' validation (13 of X)
update_validation debug: Upgrading pacemaker-2.10-style configuration to pacemaker-3.0 with upgrade-2.10.xsl
apply_upgrade debug: Upgrading pacemaker-2.10-style configuration, pre-upgrade phase with upgrade-2.10-enter.xsl
apply_upgrade debug: Upgrading pacemaker-2.10-style configuration, main phase with upgrade-2.10.xsl
INFO: Resources-operation instance_attributes: mySmartFuse-monitor-inputpower (rsc=mySmartFuse, meta=mySmartFuse-inputpower-instanceparams): dropping requires
INFO: Resources-operation instance_attributes: ... only start/promote operation taken into account
INFO: Resources-operation instance_attributes: mySmartFuse-monitor-outputpower (rsc=mySmartFuse, meta=mySmartFuse-outputpower-instanceparams): dropping requires
INFO: Resources-operation instance_attributes: ... only start/promote operation taken into account
apply_upgrade debug: Upgrading pacemaker-2.10-style configuration, post-upgrade phase with upgrade-2.10-leave.xsl
DEBUG: instance_attributes: original element pointed to with @id-ref (mySmartFuse-outputpower-instanceparams) disappeared during upgrade
update_validation info: Transformation upgrade-2.10.xsl successful
update_validation debug: Testing 'pacemaker-3.0' validation (14 of X)
update_validation debug: pacemaker-3.0-style configuration is also valid for pacemaker-3.1
update_validation debug: Testing 'pacemaker-3.1' validation (15 of X)
update_validation debug: Configuration valid for schema: pacemaker-3.1
update_validation debug: pacemaker-3.1-style configuration is also valid for pacemaker-3.2
update_validation debug: Testing 'pacemaker-3.2' validation (16 of X)
update_validation debug: Configuration valid for schema: pacemaker-3.2
-update_validation trace: Stopping at pacemaker-3.2
-update_validation info: Transformed the configuration from pacemaker-2.10 to pacemaker-3.2
+update_validation debug: pacemaker-3.2-style configuration is also valid for pacemaker-3.3
+update_validation debug: Testing 'pacemaker-3.3' validation (17 of X)
+update_validation debug: Configuration valid for schema: pacemaker-3.3
+update_validation trace: Stopping at pacemaker-3.3
+update_validation info: Transformed the configuration from pacemaker-2.10 to pacemaker-3.3
=#=#=#= Current cib after: Upgrade to latest CIB schema (trigger 2.10.xsl + the wrapping) =#=#=#=
=#=#=#= End test: Upgrade to latest CIB schema (trigger 2.10.xsl + the wrapping) - OK (0) =#=#=#=
* Passed: cibadmin - Upgrade to latest CIB schema (trigger 2.10.xsl + the wrapping)
=#=#=#= Begin test: Query a resource instance attribute (shall survive) =#=#=#=
outputpower
=#=#=#= Current cib after: Query a resource instance attribute (shall survive) =#=#=#=
=#=#=#= End test: Query a resource instance attribute (shall survive) - OK (0) =#=#=#=
* Passed: crm_resource - Query a resource instance attribute (shall survive)
diff --git a/cts/cli/regression.validity.exp b/cts/cli/regression.validity.exp
index bccea42087..46e54b5095 100644
--- a/cts/cli/regression.validity.exp
+++ b/cts/cli/regression.validity.exp
@@ -1,462 +1,476 @@
Created new pacemaker configuration
Setting up shadow instance
A new shadow instance was created. To begin using it paste the following into your shell:
CIB_shadow=cts-cli ; export CIB_shadow
=#=#=#= Begin test: Try to make resulting CIB invalid (enum violation) =#=#=#=
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Call failed: Update does not conform to the configured schema
=#=#=#= Current cib after: Try to make resulting CIB invalid (enum violation) =#=#=#=
=#=#=#= End test: Try to make resulting CIB invalid (enum violation) - Invalid configuration (78) =#=#=#=
* Passed: cibadmin - Try to make resulting CIB invalid (enum violation)
=#=#=#= Begin test: Run crm_simulate with invalid CIB (enum violation) =#=#=#=
update_validation debug: Testing 'pacemaker-1.2' validation (1 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-1.2 validation failed
update_validation debug: Testing 'pacemaker-1.3' validation (2 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-1.3 validation failed
update_validation debug: Testing 'pacemaker-2.0' validation (3 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.0 validation failed
update_validation debug: Testing 'pacemaker-2.1' validation (4 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.1 validation failed
update_validation debug: Testing 'pacemaker-2.2' validation (5 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.2 validation failed
update_validation debug: Testing 'pacemaker-2.3' validation (6 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.3 validation failed
update_validation debug: Testing 'pacemaker-2.4' validation (7 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.4 validation failed
update_validation debug: Testing 'pacemaker-2.5' validation (8 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.5 validation failed
update_validation debug: Testing 'pacemaker-2.6' validation (9 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.6 validation failed
update_validation debug: Testing 'pacemaker-2.7' validation (10 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.7 validation failed
update_validation debug: Testing 'pacemaker-2.8' validation (11 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.8 validation failed
update_validation debug: Testing 'pacemaker-2.9' validation (12 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.9 validation failed
update_validation debug: Testing 'pacemaker-2.10' validation (13 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-2.10 validation failed
update_validation debug: Testing 'pacemaker-3.0' validation (14 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-3.0 validation failed
update_validation debug: Testing 'pacemaker-3.1' validation (15 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-3.1 validation failed
update_validation debug: Testing 'pacemaker-3.2' validation (16 of X)
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
update_validation trace: pacemaker-3.2 validation failed
-Cannot upgrade configuration (claiming schema pacemaker-1.2) to at least pacemaker-3.0 because it does not validate with any schema from pacemaker-1.2 to pacemaker-3.2
+update_validation debug: Testing 'pacemaker-3.3' validation (17 of X)
+element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
+element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
+update_validation trace: pacemaker-3.3 validation failed
+Cannot upgrade configuration (claiming schema pacemaker-1.2) to at least pacemaker-3.0 because it does not validate with any schema from pacemaker-1.2 to pacemaker-3.3
=#=#=#= End test: Run crm_simulate with invalid CIB (enum violation) - Invalid configuration (78) =#=#=#=
* Passed: crm_simulate - Run crm_simulate with invalid CIB (enum violation)
=#=#=#= Begin test: Try to make resulting CIB invalid (unrecognized validate-with) =#=#=#=
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Call failed: Update does not conform to the configured schema
=#=#=#= Current cib after: Try to make resulting CIB invalid (unrecognized validate-with) =#=#=#=
=#=#=#= End test: Try to make resulting CIB invalid (unrecognized validate-with) - Invalid configuration (78) =#=#=#=
* Passed: cibadmin - Try to make resulting CIB invalid (unrecognized validate-with)
=#=#=#= Begin test: Run crm_simulate with invalid CIB (unrecognized validate-with) =#=#=#=
update_validation debug: Unknown validation schema
update_validation debug: Testing 'pacemaker-1.0' validation (0 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-1.0 validation failed
update_validation debug: Testing 'pacemaker-1.2' validation (1 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-1.2 validation failed
update_validation debug: Testing 'pacemaker-1.3' validation (2 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-1.3 validation failed
update_validation debug: Testing 'pacemaker-2.0' validation (3 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.0 validation failed
update_validation debug: Testing 'pacemaker-2.1' validation (4 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.1 validation failed
update_validation debug: Testing 'pacemaker-2.2' validation (5 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.2 validation failed
update_validation debug: Testing 'pacemaker-2.3' validation (6 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.3 validation failed
update_validation debug: Testing 'pacemaker-2.4' validation (7 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.4 validation failed
update_validation debug: Testing 'pacemaker-2.5' validation (8 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.5 validation failed
update_validation debug: Testing 'pacemaker-2.6' validation (9 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.6 validation failed
update_validation debug: Testing 'pacemaker-2.7' validation (10 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.7 validation failed
update_validation debug: Testing 'pacemaker-2.8' validation (11 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.8 validation failed
update_validation debug: Testing 'pacemaker-2.9' validation (12 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.9 validation failed
update_validation debug: Testing 'pacemaker-2.10' validation (13 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-2.10 validation failed
update_validation debug: Testing 'pacemaker-3.0' validation (14 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-3.0 validation failed
update_validation debug: Testing 'pacemaker-3.1' validation (15 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-3.1 validation failed
update_validation debug: Testing 'pacemaker-3.2' validation (16 of X)
element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
update_validation trace: pacemaker-3.2 validation failed
-Cannot upgrade configuration (claiming schema pacemaker-9999.0) to at least pacemaker-3.0 because it does not validate with any schema from unknown to pacemaker-3.2
+update_validation debug: Testing 'pacemaker-3.3' validation (17 of X)
+element cib: Relax-NG validity error : Invalid attribute validate-with for element cib
+update_validation trace: pacemaker-3.3 validation failed
+Cannot upgrade configuration (claiming schema pacemaker-9999.0) to at least pacemaker-3.0 because it does not validate with any schema from unknown to pacemaker-3.3
=#=#=#= End test: Run crm_simulate with invalid CIB (unrecognized validate-with) - Invalid configuration (78) =#=#=#=
* Passed: crm_simulate - Run crm_simulate with invalid CIB (unrecognized validate-with)
=#=#=#= Begin test: Try to make resulting CIB invalid, but possibly recoverable (valid with X.Y+1) =#=#=#=
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Call failed: Update does not conform to the configured schema
=#=#=#= Current cib after: Try to make resulting CIB invalid, but possibly recoverable (valid with X.Y+1) =#=#=#=
=#=#=#= End test: Try to make resulting CIB invalid, but possibly recoverable (valid with X.Y+1) - Invalid configuration (78) =#=#=#=
* Passed: cibadmin - Try to make resulting CIB invalid, but possibly recoverable (valid with X.Y+1)
=#=#=#= Begin test: Run crm_simulate with invalid, but possibly recoverable CIB (valid with X.Y+1) =#=#=#=
update_validation debug: Testing 'pacemaker-1.2' validation (1 of X)
element tags: Relax-NG validity error : Element configuration has extra content: tags
update_validation trace: pacemaker-1.2 validation failed
update_validation debug: Testing 'pacemaker-1.3' validation (2 of X)
update_validation debug: pacemaker-1.3-style configuration is also valid for pacemaker-2.0
update_validation debug: Testing 'pacemaker-2.0' validation (3 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.0
update_validation debug: pacemaker-2.0-style configuration is also valid for pacemaker-2.1
update_validation debug: Testing 'pacemaker-2.1' validation (4 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.1
update_validation debug: pacemaker-2.1-style configuration is also valid for pacemaker-2.2
update_validation debug: Testing 'pacemaker-2.2' validation (5 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.2
update_validation debug: pacemaker-2.2-style configuration is also valid for pacemaker-2.3
update_validation debug: Testing 'pacemaker-2.3' validation (6 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.3
update_validation debug: pacemaker-2.3-style configuration is also valid for pacemaker-2.4
update_validation debug: Testing 'pacemaker-2.4' validation (7 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.4
update_validation debug: pacemaker-2.4-style configuration is also valid for pacemaker-2.5
update_validation debug: Testing 'pacemaker-2.5' validation (8 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.5
update_validation debug: pacemaker-2.5-style configuration is also valid for pacemaker-2.6
update_validation debug: Testing 'pacemaker-2.6' validation (9 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.6
update_validation debug: pacemaker-2.6-style configuration is also valid for pacemaker-2.7
update_validation debug: Testing 'pacemaker-2.7' validation (10 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.7
update_validation debug: pacemaker-2.7-style configuration is also valid for pacemaker-2.8
update_validation debug: Testing 'pacemaker-2.8' validation (11 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.8
update_validation debug: pacemaker-2.8-style configuration is also valid for pacemaker-2.9
update_validation debug: Testing 'pacemaker-2.9' validation (12 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.9
update_validation debug: pacemaker-2.9-style configuration is also valid for pacemaker-2.10
update_validation debug: Testing 'pacemaker-2.10' validation (13 of X)
update_validation debug: Configuration valid for schema: pacemaker-2.10
update_validation debug: pacemaker-2.10-style configuration is also valid for pacemaker-3.0
update_validation debug: Testing 'pacemaker-3.0' validation (14 of X)
update_validation debug: Configuration valid for schema: pacemaker-3.0
update_validation debug: pacemaker-3.0-style configuration is also valid for pacemaker-3.1
update_validation debug: Testing 'pacemaker-3.1' validation (15 of X)
update_validation debug: Configuration valid for schema: pacemaker-3.1
update_validation debug: pacemaker-3.1-style configuration is also valid for pacemaker-3.2
update_validation debug: Testing 'pacemaker-3.2' validation (16 of X)
update_validation debug: Configuration valid for schema: pacemaker-3.2
-update_validation trace: Stopping at pacemaker-3.2
-update_validation info: Transformed the configuration from pacemaker-1.2 to pacemaker-3.2
+update_validation debug: pacemaker-3.2-style configuration is also valid for pacemaker-3.3
+update_validation debug: Testing 'pacemaker-3.3' validation (17 of X)
+update_validation debug: Configuration valid for schema: pacemaker-3.3
+update_validation trace: Stopping at pacemaker-3.3
+update_validation info: Transformed the configuration from pacemaker-1.2 to pacemaker-3.3
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Current cluster status:
dummy1 (ocf::pacemaker:Dummy): Stopped
dummy2 (ocf::pacemaker:Dummy): Stopped
Transition Summary:
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Executing cluster transition:
Revised cluster status:
dummy1 (ocf::pacemaker:Dummy): Stopped
dummy2 (ocf::pacemaker:Dummy): Stopped
=#=#=#= End test: Run crm_simulate with invalid, but possibly recoverable CIB (valid with X.Y+1) - OK (0) =#=#=#=
* Passed: crm_simulate - Run crm_simulate with invalid, but possibly recoverable CIB (valid with X.Y+1)
=#=#=#= Begin test: Make resulting CIB valid, although without validate-with attribute =#=#=#=
=#=#=#= Current cib after: Make resulting CIB valid, although without validate-with attribute =#=#=#=
=#=#=#= End test: Make resulting CIB valid, although without validate-with attribute - OK (0) =#=#=#=
* Passed: cibadmin - Make resulting CIB valid, although without validate-with attribute
=#=#=#= Begin test: Run crm_simulate with valid CIB, but without validate-with attribute =#=#=#=
Schema validation of configuration is disabled (enabling is encouraged and prevents common misconfigurations)
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Current cluster status:
dummy1 (ocf::pacemaker:Dummy): Stopped
dummy2 (ocf::pacemaker:Dummy): Stopped
Transition Summary:
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Executing cluster transition:
Revised cluster status:
dummy1 (ocf::pacemaker:Dummy): Stopped
dummy2 (ocf::pacemaker:Dummy): Stopped
=#=#=#= End test: Run crm_simulate with valid CIB, but without validate-with attribute - OK (0) =#=#=#=
* Passed: crm_simulate - Run crm_simulate with valid CIB, but without validate-with attribute
=#=#=#= Begin test: Make resulting CIB invalid, and without validate-with attribute =#=#=#=
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
+element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
+element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
=#=#=#= Current cib after: Make resulting CIB invalid, and without validate-with attribute =#=#=#=
=#=#=#= End test: Make resulting CIB invalid, and without validate-with attribute - OK (0) =#=#=#=
* Passed: cibadmin - Make resulting CIB invalid, and without validate-with attribute
=#=#=#= Begin test: Run crm_simulate with invalid CIB, also without validate-with attribute =#=#=#=
Schema validation of configuration is disabled (enabling is encouraged and prevents common misconfigurations)
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
+validity.bad.xml:10: element rsc_order: Relax-NG validity error : Invalid attribute first-action for element rsc_order
+validity.bad.xml:10: element rsc_order: Relax-NG validity error : Element constraints has extra content: rsc_order
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
unpack_simple_rsc_order error: Cannot invert constraint 'ord_1-2' (please specify inverse manually)
Current cluster status:
dummy1 (ocf::pacemaker:Dummy): Stopped
dummy2 (ocf::pacemaker:Dummy): Stopped
Transition Summary:
unpack_resources error: Resource start-up disabled since no STONITH resources have been defined
unpack_resources error: Either configure some or disable STONITH with the stonith-enabled option
unpack_resources error: NOTE: Clusters with shared data need STONITH to ensure data integrity
Executing cluster transition:
Revised cluster status:
dummy1 (ocf::pacemaker:Dummy): Stopped
dummy2 (ocf::pacemaker:Dummy): Stopped
=#=#=#= End test: Run crm_simulate with invalid CIB, also without validate-with attribute - OK (0) =#=#=#=
* Passed: crm_simulate - Run crm_simulate with invalid CIB, also without validate-with attribute
diff --git a/xml/Readme.md b/xml/Readme.md
index a8f414d2cb..65c331d93e 100644
--- a/xml/Readme.md
+++ b/xml/Readme.md
@@ -1,142 +1,143 @@
# Schema Reference
Pacemaker's XML schema has a version of its own, independent of the version of
Pacemaker itself.
## Versioned Schema Evolution
A versioned schema offers transparent backward and forward compatibility.
- It reflects the timeline of schema-backed features (introduction,
changes to the syntax, possibly deprecation) through the versioned
stable schema increments, while keeping schema versions used by default
by older Pacemaker versions untouched.
- Pacemaker internally uses the latest stable schema version, and relies on
supplemental transformations to promote cluster configurations based on
older, incompatible schema versions into the desired form.
- It allows experimental features with a possibly unstable configuration
interface to be developed using the special `next` version of the schema.
## Mapping Pacemaker Versions to Schema Versions
| Pacemaker | Latest Schema | Changed
| --------- | ------------- | ----------------------------------------------
+| `2.0.4` | `3.3` | `tags`
| `2.0.1` | `3.2` | `resources`
| `2.0.0` | `3.1` | `constraints`, `resources`
| `1.1.18` | `2.10` | `resources`, `alerts`
| `1.1.17` | `2.9` | `resources`, `rule`
| `1.1.16` | `2.6` | `constraints`
| `1.1.15` | `2.5` | `alerts`
| `1.1.14` | `2.4` | `fencing`
| `1.1.13` | `2.3` | `constraints`
| `1.1.12` | `2.0` | `nodes`, `nvset`, `resources`, `tags`, `acls`
| `1.1.8`+ | `1.2` |
## Schema generation
Each logical portion of the schema goes into its own RNG file, named like
`${base}-${X}.${Y}.rng`. `${base}` identifies the portion of the schema
(e.g. constraints, resources); ${X}.${Y} is the latest schema version that
contained changes in this portion of the schema.
The complete, overall schema, `pacemaker-${X}.${Y}.rng`, is automatically
generated from the other files via the Makefile.
# Updating schema files #
## Experimental features ##
Experimental features go into `${base}-next.rng` where `${base}` is the
affected portion of the schema. If such a file does not already exist,
create it by copying the most recent `${base}-${X}.${Y}.rng`.
Pacemaker will not use the experimental schema by default; the cluster
administrator must explicitly set the `validate-with` property appropriately to
use it.
## Stable features ##
The current stable version is determined at runtime when
crm\_schema\_init() scans the CRM\_SCHEMA\_DIRECTORY.
It will have the form `pacemaker-${X}.${Y}` and the highest
`${X}.${Y}` wins.
### Simple Additions
When the new syntax is a simple addition to the previous one, create a
new entry, incrementing `${Y}`.
### Feature Removal or otherwise Incompatible Changes
When the new syntax is not a simple addition to the previous one,
create a new entry, incrementing `${X}` and setting `${Y} = 0`.
An XSLT file is also required that converts an old syntax to the new
one and must be named `upgrade-${Xold}.${Yold}.xsl`.
See `xml/upgrade-1.3.xsl` for an example.
Since `xml/upgrade-2.10.xsl`, rather self-descriptive approach is taken,
separating metadata of the replacements and other modifications to
perform from the actual executive parts, which is leveraged, e.g., with
the on-the-fly overview as obtained with `./regression.sh -X test2to3`.
Also this was the first time particular key names of `nvpair`s,
i.e. below the granularity of the schemas so far, received attention,
and consequently, no longer expected names became systemically banned
in the after-upgrade schemas, using `` construct in the
data type specification pertaining the affected XML path.
The implied complexity also resulted in establishing a new compound,
stepwise transformation, alleviating the procedural burden from the
core upgrade recipe. In particular, `id-ref` based syntactic
simplification granted in the CIB format introduces nonnegligible
internal "noise" because of the extra indirection encumbered with
generally non-bijective character of such a scheme (context-dependent
interpretation). To reduce this strain, a symmetric arrangement is
introduced as a pair of _enter_/_leave_ (pre-upgrade/post-upgrade)
transformations where the latter is meant to eventually reversibly
restore what the former intentionally simplified (normalized) for
upgrade transformation's peruse. It's optional (even the post-upgrade
counterpart is optional alone) and depends on whether the suitable
files are found along the upgrade transformation itself: e.g., for
`upgrade-2.10.xsl`, such files are `upgrade-2.10-enter.xsl` and
`upgrade-2.10-leave.xsl`. Note that unfolding + refolding `id-ref`
shortcuts is just a practically imposed individual case of how to
reversibly make the configuration space tractable in the upgrade
itself, allowing for more sophistication down the road.
### General Procedure
1. Copy the most recent version of `${base}-*.rng` to `${base}-${X}.${Y}.rng`
1. Commit the copy, e.g. `"Low: xml: clone ${base} schema in preparation for
changes"`. This way, the actual change will be obvious in the commit history.
1. Modify `${base}-${X}.${Y}.rng` as required.
1. If required, add an XSLT file, and update `xslt\_SCRIPTS` in `xml/Makefile.am`.
1. Commit
1. `make -C xml clean; make -C xml all` to rebuild the schemas in the local
source directory.
1. The CIB validity regression tests will break after the schema is updated.
Run `cts/cts-cli -s` to make the referential outcomes reflect the transient
changes made so far, `git diff cts/cli/regression.validity.exp` to ensure
the these changes look sane, and `git add cts/cli/regression.validity.exp`
to enqueue this change for a subsequent commit.
1. Similarly, with the new major version `${X}`, it's advisable to refresh
scheduler tests at some point, see the instructions in `cts/README.md`.
## Using a New Schema
New features will not be available until the cluster administrator:
1. Updates all the nodes
1. Runs the equivalent of `cibadmin --upgrade --force`
## Random Notes
From the source directory, run `make -C xml diff` to see the changes
in the current schema (compared to the previous ones) and also the
pending changes in `pacemaker-next`.
Alternatively, if the intention is to grok the overall historical schema
evolution, use `make -C xml fulldiff`.
diff --git a/xml/tags-3.3.rng b/xml/tags-3.3.rng
new file mode 100644
index 0000000000..d94d73a106
--- /dev/null
+++ b/xml/tags-3.3.rng
@@ -0,0 +1,25 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+