---------------------------------------------- The Corosync Cluster Engine Installation Guide ---------------------------------------------- Please read LICENSE for a description of the licensing of this software. --------------------- * Building from git * --------------------- When building and installing from git, autoconf 2.61, automake 1.11, libtool 2.2.6 and pkgconfig 0.23 (or later versions) are required. Prior versions may result in build failures. Step 1: check out a read only copy of the repository git clone git://github.com/corosync/corosync.git Find the version you want to build. Usually this will be the "master" version. If you want to build a specific released version, use git checkout VERSION. Step 2: Generate the makefiles balance:~/corosync% ./autogen.sh Step 3: Run the configure script balance:~/corosync% ./configure Step 4: Install the binaries balance:~/corosync% su balance:~/corosync# make install ------------------------- * Building from tarball * ------------------------- The tarball is distributed with pregenerated makefiles. There is no need to run the autogen.sh script in this case. Step 1: Run the configure script balance:~/corosync% ./configure Step 2: Install the binaries balance:~/corosync% su balance:~/corosync# make install ------------------------------- * A notice about dependencies * ------------------------------- We have strived very hard to avoid dependencies as much as possible, but there are two required libraries: LibQB and NSS. Optional dependencies are support for Infiniband RDMA, DBUS, SNMP and libstatgrab. The nss packages required to build corosync can usually be found by installing the "nss-devel" and "nspr-devel" packages (names may vary, please use distribution's package manager). Also don't forget to install pkgconfig. ----------------------------------------- * Building with Infiniband RDMA support * ----------------------------------------- Infiniband RDMA build support is disabled by default. To enable rdma during building: balance:~/corosync% ./configure --enable-rdma Infiniband rdma support requires the libibverbs and librdmacm packages (and their -devel counterparts). Corosync requires the use of pkgconfig in these packages. The Corosync maintainers have sent patches for this feature upstream to the Infiniband maintainers. If your version of these packages don't have pkgconfig support, you will see an error during configure such as: checking for ibverbs... configure: error: Package requirements (ibverbs) were not met: No package 'ibverbs' found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables ibverbs_CFLAGS and ibverbs_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details. ** Follow the directions: balance:~/corosync/trunk% export ibverbs_CFLAGS=-I/usr/include/infiniband balance:~/corosync/trunk% export ibverbs_LIBS=-libverbs balance:~/corosync/trunk% export rdmacm_CFLAGS=-I/usr/include/rdma balance:~/corosync/trunk% export rdmacm_LIBS=-lrdmacm balance:~/corosync/trunk% ./configure --enable-rdma Hopefully pkgconfig support is added in your distribution's version of libibverbs and librdmacm soon if you run into this problem. ----------------------------------------- * Building with SNMP/DBUS support * ----------------------------------------- You can get SNMP traps on the following corosync events: 1) node joine/leave 2) application connect/dissconnect from corosync 3) quorum gain/lost There are 2 modes of achieving this DBUS + foghorn and snmp-agentx. Setting up to get dbus events. ------------------------------ foghorn (http://git.fedorahosted.org/git/foghorn.git) converts dbus signals into snmp traps. So install foghorn. $ ./configure --enable-dbus $ make && sudo make install $ /etc/init.d/corosync start $ echo "OPTIONS=\"-d\"" > /etc/sysconfig/corosync-notifyd $ /etc/init.d/corosync-notifyd start Start foghorn to see the dbus signals getting sent try: $ dbus-monitor --system Setting up snmp-agentx. ----------------------- If you don't want to use dbus then you can use snmp-agentx. $ ./configure --enable-snmp $ make && sudo make install $ /etc/init.d/corosync start $ vim /etc/snmp/snmptrapd.conf Add the following: authCommunity log,execute,net public $ /etc/init.d/snmptrapd start $ echo "OPTIONS=\"-s\"" > /etc/sysconfig/corosync-notifyd $ /etc/init.d/corosync-notifyd start I start up wireshark to see if there are any snmp traps been sent as I am too lazy to setup a manager to receive traps. run a program that talks to corosync e.g. $ corosync-cmapctl And you should get traps ------------------------ * Configuring Corosync * ------------------------ The corosync executive will automatically determine cluster membership by communicating on a specified multicast address and port. The directory conf contains the file corosync.conf.example # Please read the corosync.conf.5 manual page totem { version: 2 secauth: off threads: 0 interface { ringnumber: 0 bindnetaddr: 192.168.1.1 mcastaddr: 226.94.1.1 mcastport: 5405 } } logging { fileline: off to_stderr: yes to_file: yes to_syslog: yes logfile: /tmp/corosync.log debug: off timestamp: on } The totem section contains three values. All three values must be set or the corosync executive wll exit with an error. bindnetaddr specifies the address which the corosync Executive should bind to. This address should always end in zero. If the local interface taffic should routed over is 192.168.5.92, set bindnetaddr to 192.168.5.0. mcastaddr is a multicast address. The default should work but you may have a different network configuration. Avoid 224.x.x.x because this is a "config" multicast address. mcastport specifies the UDP port number. It is possible to use the same multicast address on a network with the corosync services configured for different UDP ports. The timeout section contains seven values. This section is not normally used, but rather used to override the program defaults for the purposes of fine tuning for a given networking/processor combination or for debugging purposes. Be careful to use the same timeout values on each of the nodes in the cluster or unpredictable results may occur. Generate a private key ---------------------- corosync uses cryptographic techniques to ensure authenticity and privacy of messages. A private key must be generated and shared by all processors for correct operation. First generate the key on one of the nodes: balance# corosync-keygen Corosync Authentication key generator. Gathering 1024 bits for key from /dev/random. Writing corosync key to /etc/corosync/authkey. After this is complete, a private key will be in the file /etc/corosync/authkey. This private key must be copied to every processor that will be a member of the cluster. If the private key isn't the same for every node, those nodes with nonmatching private keys will not be able to join the same configuration. Copy the key to some transportable storage or use ssh to transmit the key from node to node. Then install the key with the command: balance# install -D --group=0 --owner=0 --mode=0400 /path_to_authkey/authkey /etc/corosync/authkey If the message invalid digest appears, the keys are not the same on each node. Run the corosync executive ------------------------- Get one or more nodes and run the corosync executive on each node. Run the corosync daemon after following the previous directions. The daemon must be run as UID 0(root). please read SECURITY to understand the threat model assumed by corosync and the techniques corosync use to overcome these threats. Before running any of the test programs --------------------------------------- The corosync executive will ensure security by only allowing the UID 0(root) or GID 0(root) to connect to it. To allow other users to access the corosync executive, create a directory called /etc/corosync/uidgid.d and place a file in it named in some way that is identifiable to you. All files in this directory will be scanned and their contents added to the allowed uid gid database. The contents of this file should be uidgid { uid: username gid: groupname } Please note that these users then have full ability to transmit and receive messages in the cluster and are not bound by the threat model described in SECURITY. Try out the corosync cpg functionality -------------------------------------- After corosync is running Run test/testcpg on multiple nodes or on the same node. Messages can be typed which will then be sent to other testcpg applications in the cluster. To see a hashed verified output of data on all nodes, test/cpgverify can be run.