Phriction Welcome to the ClusterLabs Wiki Projects Pacemaker Pacemaker Release Checklist History Version 3 vs 4
Version 3 vs 4
Version 3 vs 4
Content Changes
Content Changes
== Pull request policies during release cycle ==
* After the rc1 release, only bug fixes should be allowed into the release branch (features go into the main branch).
* After some later release candidate, only regression fixes should be allowed into the release branch (all other bug fixes, and features, go into the main branch).
* After the final release, all pull requests should be against the main branch.
== Testing ==
All of this should be done on the main branch just before rc1, addressing any issues found. Selected parts can be repeated for later release candidates when appropriate.
* Run all **regression tests** (including cts-cli -t agents, cts-attrd, cts-fencing, cts-exec, and cts-exec -R; see `cts/README.md`).
* Run a long **CTS lab** (see `cts/README.md`) against a several-node cluster using `CTSlab.py` or a wrapper such as `cts/cluster_test` or your own.
* Perform **static analysis**. The devel directory has make targets for convenience (`cppcheck`, `clang`, and `coverity` or `coverity-corp`).
* Verify **OCF agent meta-data** is valid (replacing $OCF_RNG_SCHEMA with the path to ra-api.rng from a local checkout of the OCF-spec source repository):
** `make RNG="$OCF_RNG_SCHEMA" -C agents/ocf validate`
* Verify **formatted output** args using [[https://github.com/clumens/fosa/|fosa]].
* Locally generate all **documentation** that will be uploaded to the website, and proof changes since the last release.
** To see changes: `git diff Pacemaker-$LAST_RELEASE.. -- doc/sphinx`
* Perform a **rolling upgrade** from any supported previous version.
* Optionally, compare **scheduler timings** against the previous release to make sure there are no large unexplained regressions. From a checkout, run the first command below for the old and new builds, then run the second command to compare the two. Minimize variation however you can (use same host, don't do anything else while running it, maybe try to seed the disk read cache). There still is a good bit of variation from run to run so it is unclear at this point what is significant.
** `PCMK_schema_directory=xml tools/crm_simulate --profile cts/scheduler/xml --repeat 1000 > ~/crm_simulate.$VERSION.out`
** `tools/pcmk_simtimes ~/crm_simulate.*.out -s 0.1 -p 5`
* Optionally, check the current buildability in [[https://copr.fedorainfracloud.org/coprs/g/ClusterLabs/devel/builds/|COPR]] (default sort starts with the newest builds, optionally use `pacemaker` in the filter box, check at least `Git branch` field on the details page, optionally also git commit encoded in the `R`> part of package identifier)
== rc1 only ==
* **Pull main branch into release branch**
** If this is a new major or minor release, create the release branch, and ensure CI gets updated to check it.
* Things that don't need to be done every release but are helpful every now and then:
** `cts/cts-scheduler --update` (only if the tests already all pass!), to update whitespace usage in expected output (which is ignored by the regression tests, but can help manual comparisons)
** **(Do not do this until we can raise the minimum automake dependency to 1.14)** `make -C maint gnulib-update` to update the code we bundle from gnulib
* Make sure that the **[[/w/projects/pacemaker/pacemaker_feature_set/ | feature set]]** has been bumped appropriately for code changes in this release (the minor version if any new features require DC support including most schema changes, and the minor-minor version if any new features at all have been added including command-line tool arguments).
* Make sure that **PCMK__API_VERSION** in `include/crm/common/output_internal.h` is equal to the highest version in the source file names in `xml/api`.
* Update **translation files:** `make -C po update-po`
* Update **xml/README.md** for pacemaker-to-schema version mapping, if needed
** To see schema changes since last release: `git log --no-merges Pacemaker-$LAST_VERSION.. xml`
* Ensure that significant new features are **documented** in <cite>Pacemaker Explained</cite> (with "(since <version>)" for new syntax).
* Run **`maint/bumplibs`** to update shared library (libtool) versions for changes since last point release, which will show all library diffs since the last release and ask whether the changes involve compatible public API additions, incompatible public API additions/removals, or fixes. Verify the script's changes (which it displays at the end).
** Versions can be updated again for later release candidates if they break the public API (which should be avoided unless absolutely necessary) by running the script with `Pacemaker-$PREVIOUS_RC_TAG` as the argument.
** For details about library versioning, see [[http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html|Updating library version information]].
* Update **m4/version.m4** to new release version, without -rc suffix.
** When bumping the major or minor release number, `epub_identifier` in doc/sphinx/conf.py.in needs to be updated
** At some point, it might be worthwhile to include the -rc suffix for better differentiation of release candidates, but that will need a lot of testing to see how it affects RPMS, version comparisons, etc.
=== All releases ===
Repeat until final. All work is in the release branch.
* Update the **Changelog:**
** **release candidates:** `make -C maint LAST_RELEASE=Pacemaker-$LAST_VERSION changelog` and then edit manually to make sense from a user perspective
** **final:** merge the rc entries into a single entry, then update summary with the output of `make -C maint LAST_RELEASE=Pacemaker-$LAST_FINAL_VERSION NEXT_RELEASE=Pacemaker-$NEW_VERSION summary`
* Make a github release:
** Go to [[https://github.com/ClusterLabs/pacemaker/releases|Pacemaker releases on Github]] and select `Draft a new release`
** **Choose a tag:** Pacemaker-//X.Y.Z//-rc//N// or Pacemaker-//X.Y.Z//
** **Target:** release branch
** **Release title:** "Pacemaker //X.Y.Z// - Release Candidate //N//" or "Pacemaker //X.Y.Z// - Final"
** **Describe this release:** copy relevant part of ChangeLog
** **This is a pre-release:** check if this is an rc
** Can save as draft to preview if desired
* If this release fixed any regressions, update the descriptions on GitHub of the releases that introduced the regressions
* Publicize
** See for example https://lists.clusterlabs.org/pipermail/users/2019-October/026482.html (to generate a list of contributors, run `make -C maint LAST_RELEASE=Pacemaker-$PREVIOUS authors` or equivalently, `git log Pacemaker-$PREVIOUS..Pacemaker-$RC --format='%an'|sort -u`)
** If this is rc1, announce request policy on developers@clusterlabs.org (features into main, fixes into release branch), e.g. https://lists.clusterlabs.org/pipermail/developers/2019-October/002230.html
* Pull release branch back into main branch and merge.
* If this is an rc, wait for feedback, and address any reported issues. About 1-4 weeks between release candidates is good.
== After final release ==
* `export RSYNC_RSH="ssh -p $PORT"` appropriately for the ClusterLabs website host.
* `git fetch upstream --tags` so the make targets can use the new release tag instead of commit ID.
* Run `make -C doc www` to regenerate most documentation on ClusterLabs.org website (doxygen, indexed source, Clusters From Scratch, Pacemaker Explained, etc.).
* Run `make -C doc LAST_RELEASE=Pacemaker-$PREVIOUS_RELEASE TAG=Pacemaker-$NEW_RELEASE abi-www` to generate the ABI compatibility report.
* Update [[/w/projects/pacemaker/pacemaker_release_calendar/ | release calendar]] and [[/w/projects/pacemaker/pacemaker_feature_set/ | Pacemaker feature set]] on this wiki
* In the ClusterLabs project manager, update all relevant tasks from "Merged" status to "Released" (do [[https://projects.clusterlabs.org/maniphest/query/advanced/|Advanced Search]] for Merged status and release tag, then select all and bulk edit), and [[https://projects.clusterlabs.org/project/edit/?milestone=1|create the next milestone(s)]] (which will be the release(s) after next)
* Announce request policy on developers@clusterlabs.org (all requests into main).
* Edit any pull requests open against the release branch to submit against main.
== Policies ==
* All new development should be against the main branch. If a change is desired in a release, a PR backporting it to the desired release branch should be submitted separately.
* If a release is in progress (rc1 has been released but the final has not), only bug fix backports should be merged into the release branch. Near the end of the release cycle, only regression fixes should be merged.
== Testing ==
All of this should be done on the main branch just before rc1, addressing any issues found. Selected parts can be repeated for later release candidates when appropriate.
* Run all **regression tests** (including cts-cli -t agents, cts-attrd, cts-fencing, cts-exec, and cts-exec -R; see `cts/README.md`).
* Run a long **CTS lab** (see `cts/README.md`) against a several-node cluster using `CTSlab.py` or a wrapper such as `cts/cluster_test` or your own.
* Perform **static analysis**. The devel directory has make targets for convenience (`cppcheck`, `clang`, and `coverity` or `coverity-corp`).
* Verify **OCF agent meta-data** is valid (replacing $OCF_RNG_SCHEMA with the path to ra-api.rng from a local checkout of the OCF-spec source repository):
** `make RNG="$OCF_RNG_SCHEMA" -C agents/ocf validate`
* Verify **formatted output** args using [[https://github.com/clumens/fosa/|fosa]].
* Locally generate all **documentation** that will be uploaded to the website, and proof changes since the last release.
** To see changes: `git diff Pacemaker-$LAST_RELEASE.. -- doc/sphinx`
* Perform a **rolling upgrade** from any supported previous version.
* Optionally, compare **scheduler timings** against the previous release to make sure there are no large unexplained regressions. From a checkout, run the first command below for the old and new builds, then run the second command to compare the two. Minimize variation however you can (use same host, don't do anything else while running it, maybe try to seed the disk read cache). There still is a good bit of variation from run to run so it is unclear at this point what is significant.
** `PCMK_schema_directory=xml tools/crm_simulate --profile cts/scheduler/xml --repeat 1000 > ~/crm_simulate.$VERSION.out`
** `tools/pcmk_simtimes ~/crm_simulate.*.out -s 0.1 -p 5`
* Optionally, check the current buildability in [[https://copr.fedorainfracloud.org/coprs/g/ClusterLabs/devel/builds/|COPR]] (default sort starts with the newest builds, optionally use `pacemaker` in the filter box, check at least `Git branch` field on the details page, optionally also git commit encoded in the `R`> part of package identifier)
== rc1 only ==
* **Pull main branch into release branch**
** If this is a new major or minor release, create the release branch, and ensure CI gets updated to check it.
* Things that don't need to be done every release but are helpful every now and then:
** `cts/cts-scheduler --update` (only if the tests already all pass!), to update whitespace usage in expected output (which is ignored by the regression tests, but can help manual comparisons)
** **(Do not do this until we can raise the minimum automake dependency to 1.14)** `make -C maint gnulib-update` to update the code we bundle from gnulib
* Make sure that the **[[/w/projects/pacemaker/pacemaker_feature_set/ | feature set]]** has been bumped appropriately for code changes in this release (the minor version if any new features require DC support including most schema changes, and the minor-minor version if any new features at all have been added including command-line tool arguments).
* Make sure that **PCMK__API_VERSION** in `include/crm/common/output_internal.h` is equal to the highest version in the source file names in `xml/api`.
* Update **translation files:** `make -C po update-po`
* Update **xml/README.md** for pacemaker-to-schema version mapping, if needed
** To see schema changes since last release: `git log --no-merges Pacemaker-$LAST_VERSION.. xml`
* Ensure that significant new features are **documented** in <cite>Pacemaker Explained</cite> (with "(since <version>)" for new syntax).
* Run **`maint/bumplibs`** to update shared library (libtool) versions for changes since last point release, which will show all library diffs since the last release and ask whether the changes involve compatible public API additions, incompatible public API additions/removals, or fixes. Verify the script's changes (which it displays at the end).
** Versions can be updated again for later release candidates if they break the public API (which should be avoided unless absolutely necessary) by running the script with `Pacemaker-$PREVIOUS_RC_TAG` as the argument.
** For details about library versioning, see [[http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html|Updating library version information]].
* Update **m4/version.m4** to new release version, without -rc suffix.
** When bumping the major or minor release number, `epub_identifier` in doc/sphinx/conf.py.in needs to be updated
** At some point, it might be worthwhile to include the -rc suffix for better differentiation of release candidates, but that will need a lot of testing to see how it affects RPMS, version comparisons, etc.
=== All releases ===
Repeat until final. All work is in the release branch.
* Update the **Changelog:**
** **release candidates:** `make -C maint LAST_RELEASE=Pacemaker-$LAST_VERSION changelog` and then edit manually to make sense from a user perspective
** **final:** merge the rc entries into a single entry, then update summary with the output of `make -C maint LAST_RELEASE=Pacemaker-$LAST_FINAL_VERSION NEXT_RELEASE=Pacemaker-$NEW_VERSION summary`
* Make a github release:
** Go to [[https://github.com/ClusterLabs/pacemaker/releases|Pacemaker releases on Github]] and select `Draft a new release`
** **Choose a tag:** Pacemaker-//X.Y.Z//-rc//N// or Pacemaker-//X.Y.Z//
** **Target:** release branch
** **Release title:** "Pacemaker //X.Y.Z// - Release Candidate //N//" or "Pacemaker //X.Y.Z// - Final"
** **Describe this release:** copy relevant part of ChangeLog
** **This is a pre-release:** check if this is an rc
** Can save as draft to preview if desired
* If this release fixed any regressions, update the descriptions on GitHub of the releases that introduced the regressions
* Publicize
** See for example https://lists.clusterlabs.org/pipermail/users/2019-October/026482.html (to generate a list of contributors, run `make -C maint LAST_RELEASE=Pacemaker-$PREVIOUS authors` or equivalently, `git log Pacemaker-$PREVIOUS..Pacemaker-$RC --format='%an'|sort -u`)
** If this is rc1, announce request policy on developers@clusterlabs.org (features into main, fixes into release branch), e.g. https://lists.clusterlabs.org/pipermail/developers/2019-October/002230.html
* Pull release branch back into main branch and merge.
* If this is an rc, wait for feedback, and address any reported issues. About 1-4 weeks between release candidates is good.
== After final release ==
* `export RSYNC_RSH="ssh -p $PORT"` appropriately for the ClusterLabs website host.
* `git fetch upstream --tags` so the make targets can use the new release tag instead of commit ID.
* Run `make -C doc www` to regenerate most documentation on ClusterLabs.org website (doxygen, indexed source, Clusters From Scratch, Pacemaker Explained, etc.).
* Run `make -C doc LAST_RELEASE=Pacemaker-$PREVIOUS_RELEASE TAG=Pacemaker-$NEW_RELEASE abi-www` to generate the ABI compatibility report.
* In the ClusterLabs project manager (this site):
** Update [[/w/projects/pacemaker/pacemaker_release_calendar/ | release calendar]] and [[/w/projects/pacemaker/pacemaker_feature_set/ | Pacemaker feature set]].
** Update all relevant tasks from //Merged// status to //Released// (do [[https://projects.clusterlabs.org/maniphest/query/advanced/|Advanced Search]] for Merged status and release tag, then select all and bulk edit)
** Archive the release milestone(s) (go to milestone and select Manage then Archive)
** [[https://projects.clusterlabs.org/project/edit/?milestone=1|Create the next milestone(s)]] (which will be the release(s) after next)
== Pull request policies during release cycle ==ies ==
* After* All new development should be against the rc1main branch. If a change is desired in a release, only bug fixes should be allowed into thea PR backporting it to the desired release branch (features go into the main branch)should be submitted separately.
* After some later* If a release candidate, only reis in progression fixes should be allowed into the (rc1 has been release branch (all other bug fixesd but the final has not), and features,only bug fix backports should be merged into the release branch. go intoNear the main branch).
* Afterend of the final releaserelease cycle, all pull requests should be against the main branchonly regression fixes should be merged.
== Testing ==
All of this should be done on the main branch just before rc1, addressing any issues found. Selected parts can be repeated for later release candidates when appropriate.
* Run all **regression tests** (including cts-cli -t agents, cts-attrd, cts-fencing, cts-exec, and cts-exec -R; see `cts/README.md`).
* Run a long **CTS lab** (see `cts/README.md`) against a several-node cluster using `CTSlab.py` or a wrapper such as `cts/cluster_test` or your own.
* Perform **static analysis**. The devel directory has make targets for convenience (`cppcheck`, `clang`, and `coverity` or `coverity-corp`).
* Verify **OCF agent meta-data** is valid (replacing $OCF_RNG_SCHEMA with the path to ra-api.rng from a local checkout of the OCF-spec source repository):
** `make RNG="$OCF_RNG_SCHEMA" -C agents/ocf validate`
* Verify **formatted output** args using [[https://github.com/clumens/fosa/|fosa]].
* Locally generate all **documentation** that will be uploaded to the website, and proof changes since the last release.
** To see changes: `git diff Pacemaker-$LAST_RELEASE.. -- doc/sphinx`
* Perform a **rolling upgrade** from any supported previous version.
* Optionally, compare **scheduler timings** against the previous release to make sure there are no large unexplained regressions. From a checkout, run the first command below for the old and new builds, then run the second command to compare the two. Minimize variation however you can (use same host, don't do anything else while running it, maybe try to seed the disk read cache). There still is a good bit of variation from run to run so it is unclear at this point what is significant.
** `PCMK_schema_directory=xml tools/crm_simulate --profile cts/scheduler/xml --repeat 1000 > ~/crm_simulate.$VERSION.out`
** `tools/pcmk_simtimes ~/crm_simulate.*.out -s 0.1 -p 5`
* Optionally, check the current buildability in [[https://copr.fedorainfracloud.org/coprs/g/ClusterLabs/devel/builds/|COPR]] (default sort starts with the newest builds, optionally use `pacemaker` in the filter box, check at least `Git branch` field on the details page, optionally also git commit encoded in the `R`> part of package identifier)
== rc1 only ==
* **Pull main branch into release branch**
** If this is a new major or minor release, create the release branch, and ensure CI gets updated to check it.
* Things that don't need to be done every release but are helpful every now and then:
** `cts/cts-scheduler --update` (only if the tests already all pass!), to update whitespace usage in expected output (which is ignored by the regression tests, but can help manual comparisons)
** **(Do not do this until we can raise the minimum automake dependency to 1.14)** `make -C maint gnulib-update` to update the code we bundle from gnulib
* Make sure that the **[[/w/projects/pacemaker/pacemaker_feature_set/ | feature set]]** has been bumped appropriately for code changes in this release (the minor version if any new features require DC support including most schema changes, and the minor-minor version if any new features at all have been added including command-line tool arguments).
* Make sure that **PCMK__API_VERSION** in `include/crm/common/output_internal.h` is equal to the highest version in the source file names in `xml/api`.
* Update **translation files:** `make -C po update-po`
* Update **xml/README.md** for pacemaker-to-schema version mapping, if needed
** To see schema changes since last release: `git log --no-merges Pacemaker-$LAST_VERSION.. xml`
* Ensure that significant new features are **documented** in <cite>Pacemaker Explained</cite> (with "(since <version>)" for new syntax).
* Run **`maint/bumplibs`** to update shared library (libtool) versions for changes since last point release, which will show all library diffs since the last release and ask whether the changes involve compatible public API additions, incompatible public API additions/removals, or fixes. Verify the script's changes (which it displays at the end).
** Versions can be updated again for later release candidates if they break the public API (which should be avoided unless absolutely necessary) by running the script with `Pacemaker-$PREVIOUS_RC_TAG` as the argument.
** For details about library versioning, see [[http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html|Updating library version information]].
* Update **m4/version.m4** to new release version, without -rc suffix.
** When bumping the major or minor release number, `epub_identifier` in doc/sphinx/conf.py.in needs to be updated
** At some point, it might be worthwhile to include the -rc suffix for better differentiation of release candidates, but that will need a lot of testing to see how it affects RPMS, version comparisons, etc.
=== All releases ===
Repeat until final. All work is in the release branch.
* Update the **Changelog:**
** **release candidates:** `make -C maint LAST_RELEASE=Pacemaker-$LAST_VERSION changelog` and then edit manually to make sense from a user perspective
** **final:** merge the rc entries into a single entry, then update summary with the output of `make -C maint LAST_RELEASE=Pacemaker-$LAST_FINAL_VERSION NEXT_RELEASE=Pacemaker-$NEW_VERSION summary`
* Make a github release:
** Go to [[https://github.com/ClusterLabs/pacemaker/releases|Pacemaker releases on Github]] and select `Draft a new release`
** **Choose a tag:** Pacemaker-//X.Y.Z//-rc//N// or Pacemaker-//X.Y.Z//
** **Target:** release branch
** **Release title:** "Pacemaker //X.Y.Z// - Release Candidate //N//" or "Pacemaker //X.Y.Z// - Final"
** **Describe this release:** copy relevant part of ChangeLog
** **This is a pre-release:** check if this is an rc
** Can save as draft to preview if desired
* If this release fixed any regressions, update the descriptions on GitHub of the releases that introduced the regressions
* Publicize
** See for example https://lists.clusterlabs.org/pipermail/users/2019-October/026482.html (to generate a list of contributors, run `make -C maint LAST_RELEASE=Pacemaker-$PREVIOUS authors` or equivalently, `git log Pacemaker-$PREVIOUS..Pacemaker-$RC --format='%an'|sort -u`)
** If this is rc1, announce request policy on developers@clusterlabs.org (features into main, fixes into release branch), e.g. https://lists.clusterlabs.org/pipermail/developers/2019-October/002230.html
* Pull release branch back into main branch and merge.
* If this is an rc, wait for feedback, and address any reported issues. About 1-4 weeks between release candidates is good.
== After final release ==
* `export RSYNC_RSH="ssh -p $PORT"` appropriately for the ClusterLabs website host.
* `git fetch upstream --tags` so the make targets can use the new release tag instead of commit ID.
* Run `make -C doc www` to regenerate most documentation on ClusterLabs.org website (doxygen, indexed source, Clusters From Scratch, Pacemaker Explained, etc.).
* Run `make -C doc LAST_RELEASE=Pacemaker-$PREVIOUS_RELEASE TAG=Pacemaker-$NEW_RELEASE abi-www` to generate the ABI compatibility report.
* In the ClusterLabs project manager (this site):
** Update [[/w/projects/pacemaker/pacemaker_release_calendar/ | release calendar]] and [[/w/projects/pacemaker/pacemaker_feature_set/ | Pacemaker feature set]] on this wiki
* In the ClusterLabs project manager, u.
** Update all relevant tasks from "//Merged"// status to "//Released"// (do [[https://projects.clusterlabs.org/maniphest/query/advanced/|Advanced Search]] for Merged status and release tag, then select all and bulk edit), and [[https://projects.clusterlabs.org/project/edit/?milestone=1|create the next milestone(s)]] (which will be the release(s) after next)
* Announce request policy on developers@clusterlabs.org (all requests into main).
* Edit any pull requests open against the release branch to submit against main.then select all and bulk edit)
** Archive the release milestone(s) (go to milestone and select Manage then Archive)
** [[https://projects.clusterlabs.org/project/edit/?milestone=1|Create the next milestone(s)]] (which will be the release(s) after next)