Page MenuHomeClusterLabs Projects
IndexNew Document
Phriction Welcome to the ClusterLabs Wiki Projects Pacemaker Bundle Walk-through

Bundle Walk-through
Updated 3 Days AgoPublic
Actions

Overview

This walk-through demonstrates Pacemaker's bundle feature, which manages multiple instances of a container image, along with the networking and storage that each container needs.

For container management, Pacemaker supports podman (since 2.0.1) and docker (since 1.1.17). In order to keep things simple, this walk-through will focus on podman.

This walk-through uses the low-level Pacemaker command-line tools. Your higher-level tool of choice (such as crm shell or pcs) might provide an easier interface, but this will provide an understanding of what those are doing under the hood.

Prerequisites

  • Configure a test cluster of at least three cluster nodes. Two node clusters will also work, though in general two node clusters are not recommended. For a two node cluster, update the commands below to refer to the correct IP addresses and number of bundles. Pacemaker 3.0.1 or later is recommended for the most important features and bug fixes.
  • The test cluster must be connected to a network with three sequential, unused IP addresses available for the test. For this walk-through, we'll use 192.168.122.131 through 192.168.122.133, on the enp1s0 interface. If you use different values, replace them accordingly in the commands below.
  • Each node should have at least 450MB free disk space.

Preparation

Repeat on every cluster node:

  • Create a place to store logs from the container instances (otherwise the logs will be lost when the container is stopped):
mkdir -p /var/log/pacemaker/bundles/httpd-bundle-{0,1,2}
  • If SELinux is enabled on the cluster node, allow the new log directories to be used by containers:
semanage fcontext -a -t container_file_t "/var/log/pacemaker/bundles(/.*)?"
restorecon -Rv /var/log/pacemaker/bundles
  • For this walk-through, we'll use a web server container, so we need to create a sample web page for our container to serve. For a real web server, this would be identical on all nodes (often via shared storage), but here we will create a slightly different page for each host, so we can tell which container instance and underlying host are being used when we visit the web page:
mkdir -p /var/local/containers/httpd-bundle-{0,1,2}
for i in 0 1 2; do cat >/var/local/containers/httpd-bundle-$i/index.html <<EOF
<html>
<head><title>Bundle test</title></head>
<body>
<h1>httpd-bundle-$i @ $(hostname)</h1>
</body>
</html>
EOF
done
  • We'll use Apache httpd as our web server. To enable monitoring of the web server inside the container, we need a configuration snippet to enable its server-status URL. Note that Require all granted allows anyone to see the server status. This should never be done in production, but it's okay here for testing purposes.
cat <<-END >/var/local/containers/status.conf
  <Location /server-status>
    SetHandler server-status
    Require all granted
  </Location>
END

Create container image

Obtain or create the desired container image, containing:

  • Pacemaker 3.0.1 or later recommended.
  • Whatever service you wish to provide (including configuration).

podman

  • Install buildah and podman on each cluster node. The exact commands will vary by distribution; as an example, for CentOS Stream 10, use dnf -y group install "Container Management".
  • Choose a base image for the container OS. This walk-through uses CentOS Stream 10 as an example, but any distribution with a recent Pacemaker will work, and it doesn't have to be the same as the cluster nodes that will run it.
BASE="quay.io/centos/centos:stream10"
  • Choose any name you want for a working container. This walk-through uses "working-pcmktest-http".
CNAME=working-pcmktest-http
  • Choose any name you want for the final container image. This walk-through uses "pcmktest:http".
IMAGE="pcmktest:http"
  • On each cluster node, build the container image, ensuring Pacemaker, the service, and all dependencies are installed along with any configuration needed:
podman pull "$BASE"
buildah rm "$CNAME" # Remove any previous working container by the same name
CONTAINER="$(buildah from --name "$CNAME" "$BASE")"
buildah run "$CONTAINER" -- dnf update -y
buildah run "$CONTAINER" -- dnf install --enablerepo=highavailability -y bind-utils curl httpd lsof wget which pacemaker pacemaker-remote resource-agents
buildah add "$CONTAINER" /var/local/containers/status.conf /etc/httpd/conf.d/status.conf
buildah commit --format docker "$CONTAINER" "$IMAGE"
  • If desired, you can verify that the image was created:
buildah containers

The output should look something like:

CONTAINER ID  BUILDER  IMAGE ID     IMAGE NAME                          CONTAINER NAME
32ea198d9f89     *     4004aee01d2c quay.io/centos/centos:stream10      working-pcmktest-http

Configure the cluster

  • Start the cluster on all nodes.
  • On any one node, configure a bundle using the test image. Here, we will show the raw XML used by Pacemaker; see the documentation for your favorite higher-level tool for a more user-friendly interface. This example uses a <podman> tag, which you should replace with <docker> (possibly with slightly different options) according to your choice of container software.
# cibadmin --modify --allow-create --scope resources -X '<bundle id="httpd-bundle">
<podman image="localhost/pcmktest:http" replicas="3" />
  <network ip-range-start="192.168.122.131" host-interface="enp1s0" host-netmask="24">
    <port-mapping id="httpd-port" port="80"/>
  </network>
  <storage>
    <storage-mapping id="httpd-root" source-dir-root="/var/local/containers" target-dir="/var/www/html" options="rw,Z"/>
    <storage-mapping id="httpd-syslog" source-dir="/dev/log" target-dir="/dev/log" options="rw,Z"/>
  </storage>
  <primitive class="ocf" id="httpd" provider="heartbeat" type="apache">
    <instance_attributes id="httpd-attrs">
      <nvpair id="httpd-attrs-statusurl" name="statusurl" value="http://localhost/server-status"/>
    </instance_attributes>
    <operations>
      <op id="httpd-monitor" name="monitor" interval="30s"/>
    </operations>
  </primitive>
</bundle>'
  • Test away. Three containers should come up, and the Web page we created should be reachable at the specified IPs. You can also modify the bundle configuration to try different values. The container instances will be named like httpd-bundle-podman-0, so you can use standard podman commands with that (for example, podman inspect or podman exec).
Last Author
clumens
Last Edited
Tue, Feb 24, 12:12 PM