Page MenuHomeClusterLabs Projects

Bundle Walk-through
Updated 255 Days AgoPublic

Overview

This walk-through demonstrates Pacemaker's bundle feature, which manages multiple instances of a container image, along with the networking and storage that each container needs.

For container management, Pacemaker supports podman (since 2.0.1), docker (since 1.1.17), and rkt (since 1.1.18).

This walk-through uses the low-level Pacemaker command-line tools. Your higher-level tool of choice (such as crm shell or pcs) might provide an easier interface, but this will provide an understanding of what those are doing under the hood.

Prerequisites

  • Configure a test cluster of at least two cluster nodes (Pacemaker 2.0.1 or later recommended for the most important features and bug fixes).
  • The test cluster must be connected to a network with three sequential, unused IP addresses available for the test. For this walk-through, we'll use 192.168.122.131 through 192.168.122.133, on the eth0 interface. If you use different values, replace them accordingly in the commands below.
  • Each node should have at least 450MB free disk space.

Preparation

Repeat on every cluster node:

  • Create a place to store logs from the container instances (otherwise the logs will be lost when the container is stopped):
mkdir -p /var/log/pacemaker/bundles/httpd-bundle-{0,1,2}
  • If SELinux is enabled on the cluster node, allow the new log directories to be used by containers:
semanage fcontext -a -t container_file_t "/var/log/pacemaker/bundles(/.*)?"
restorecon -Rv /var/log/pacemaker/bundles
  • For this walk-through, we'll use a web server container, so we need to create a sample web page for our container to serve. For a real web server, this would be identical on all nodes (often via shared storage), but here we will create a slightly different page for each host, so we can tell which container instance and underlying host are being used when we visit the web page:
mkdir -p /var/local/containers/httpd-bundle-{0,1,2}
for i in 0 1 2; do cat >/var/local/containers/httpd-bundle-$i/index.html <<EOF
<html>
<head><title>Bundle test</title></head>
<body>
<h1>httpd-bundle-$i @ $(hostname)</h1>
</body>
</html>
EOF
done
  • We'll use Apache httpd as our web server. To enable monitoring of the web server inside the container, we need a configuration snippet to enable its server-status URL:
cat <<-END >/var/local/containers/status.conf

  <Location /server-status>

  SetHandler server-status

  Require local

  </Location>
END

Create container image

Obtain or create the desired container image, containing:

  • Pacemaker 1.1.12 or later (2.0.1 or later recommended)
  • Whatever service you wish to provide (including configuration).

Below are examples for podman and docker (only one is needed).

podman

  • Install buildah and podman on each cluster node. The exact commands will vary by distribution; as an example, for RHEL 9, use yum -y group install "Container Management".
  • Choose a base image for the container OS. This walk-through uses RHEL 9 as an example, but any distribution with a recent Pacemaker will work, and it doesn't have to be the same as the cluster nodes that will run it.
BASE="registry.access.redhat.com/ubi9/ubi"
  • Choose any name you want for a working container. This walk-through uses "working-pcmktest-http".
CNAME=working-pcmktest-http
  • Choose any name you want for the final container image. This walk-through uses "pcmktest:http".
IMAGE="pcmktest:http"
  • On each cluster node, build the container image, ensuring Pacemaker, the service, and all dependencies are installed along with any configuration needed:
podman pull "$BASE"
buildah rm "$CNAME" # Remove any previous working container by the same name
CONTAINER="$(buildah from --name "$CNAME" "$BASE")"
buildah run "$CONTAINER" -- yum update -y
buildah run "$CONTAINER" -- yum install -y bind-utils curl httpd lsof wget which pacemaker pacemaker-remote resource-agents
buildah add "$CONTAINER" /var/local/containers/status.conf /etc/httpd/conf.d/status.conf
buildah commit --format docker "$CONTAINER" "$IMAGE"
  • If desired, you can verify that the image was created:
buildah containers

The output should look something like:

CONTAINER ID  BUILDER  IMAGE ID     IMAGE NAME                          CONTAINER NAME
f9e04462b4e9     *     1e1148e4cc2c registry.access.redhat.com/ubi9/ubi working-pcmktest-http

Docker

On each cluster node:

  • Install Docker. If your distribution doesn't provide Docker packages, you can get them from Docker.
  • Enable the docker daemon to run at boot. For example, if systemd is used:
# systemctl enable --now docker
  • Pull a base image for the container OS (CentOS 7 is used here as an example):
# docker pull centos:centos7
  • Create a Dockerfile:
# cat >/var/local/containers/Dockerfile <<EOF
FROM centos:centos7
COPY status.conf /etc/httpd/conf.d/status.conf
RUN yum update -y
RUN yum install -y httpd bind-utils curl lsof wget which pacemaker pacemaker-remote resource-agents
EOF
  • Build the container image. Doing this once is sufficient, but you should repeat this step if you make changes to the Dockerfile.
# cd /var/local/containers

# docker rmi pcmktest:http # remove any previous image by the same name

# docker build -t pcmktest:http .
  • If desired, you can verify that the image was created:
# docker images

The output should look something like:

REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
pcmktest            http                aab04ad64ab0        About a minute ago   412 MB
centos              centos7             98d35105a391        12 days ago          192 MB

Configure the cluster

  • Start the cluster on all nodes.
  • On any one node, configure a bundle using the test image. Here, we will show the raw XML used by Pacemaker; see the documentation for your favorite higher-level tool for a more user-friendly interface. This example uses a <podman> tag, which you should replace with <docker> or <rkt> (possibly with slightly different options) according to your choice of container software.
# cibadmin --modify --allow-create --scope resources -X '<bundle id="httpd-bundle">
<podman image="localhost/pcmktest:http" replicas="3" />
  <network ip-range-start="192.168.122.131" host-interface="eth0" host-netmask="24">
    <port-mapping id="httpd-port" port="80"/>
  </network>
  <storage>
    <storage-mapping id="httpd-root" source-dir-root="/var/local/containers" target-dir="/var/www/html" options="rw,Z"/>
    <storage-mapping id="httpd-syslog" source-dir="/dev/log" target-dir="/dev/log" options="rw,Z"/>
  </storage>
  <primitive class="ocf" id="httpd" provider="heartbeat" type="apache">
    <instance_attributes id="httpd-attrs">
      <nvpair id="httpd-attrs-statusurl" name="statusurl" value="http://localhost/server-status"/>
    </instance_attributes>
    <operations>
      <op id="httpd-monitor" name="monitor" interval="30s"/>
    </operations>
  </primitive>
</bundle>'
  • Test away. Three containers should come up, and the Web page we created should be reachable at the specified IPs. You can also modify the bundle configuration to try different values. The container instances will be named like httpd-bundle-podman-0, so you can use standard podman commands with that (for example, podman inspect or podman exec).
Last Author
kgaillot
Last Edited
Oct 31 2023, 5:13 PM