Feature: crmd: modify feature set handling
Change the feature set interpretation:
- The intent of the previous handling was to prevent an older node from joining a cluster with a newer, incompatible DC. However, this didn't work. It was based on an initial CIB query, which would return the local on-disk CIB, which wouldn't contain the current DC's feature set. If the DC was incompatible, the local cib would not accept any changes from it, so the feature set in the local CIB would be wrong. The revision check would pass, but the cib would refuse updates, leading to trouble.
Now, the handling is done in the join process, rather than a CIB query. The DC sends its feature set with the join offer, and the joining node sends its feature set with its join request. Both sides verify compatibility, to handle the case where one of them has the previous logic.
- The intended policy was always to refuse to join if the major version differs at all. Even if the above problem did not exist, the previous code would have allowed the joining node's major version to be higher than the DC's. Now, the major version restriction is enforced.
- As was the intended policy before, the join will be rejected if the DC's minor version is greater than the joining node's.
- In a departure from the previous intended policy, changes in the minor-minor version will now be ignored, so that it can be used to advertise feature support to resource agents without affecting upgrades.
Bump the feature set (to 3.1.0) for this and the bundle "masters" to
"promoted-max" change, which required a schema change.