No OneTemporary
Actions

Size

42 KB

Referenced Files

None

Subscribers

None

View Options

	diff --git a/doc/Clusters_from_Scratch/en-US/Ch-Active-Active.txt b/doc/Clusters_from_Scratch/en-US/Ch-Active-Active.txt
	index f7692e1a6a..6d5f03fd05 100644
	--- a/doc/Clusters_from_Scratch/en-US/Ch-Active-Active.txt
	+++ b/doc/Clusters_from_Scratch/en-US/Ch-Active-Active.txt
	@@ -1,549 +1,537 @@
	= Conversion to Active/Active =

	== Requirements ==

	The primary requirement for an Active/Active cluster is that the data
	required for your services is available, simultaneously, on both
	machines. Pacemaker makes no requirement on how this is achieved, you
	could use a SAN if you had one available, however since DRBD supports
	multiple Primaries, we can also use that.

	The only hitch is that we need to use a cluster-aware filesystem. The
	one we used earlier with DRBD, ext4, is not one of those. Both OCFS2
	and GFS2 are supported, however here we will use GFS2 which comes with
	Fedora 17.

	=== Installing the required Software ===

	[source,Bash]
	-----
	-# yum install -y gfs2-utils dlm
	+# yum install -y gfs2-utils dlm kernel-modules-extra
	Loaded plugins: langpacks, presto, refresh-packagekit
	Resolving Dependencies
	--> Running transaction check
	----> Package dlm.x86_64 0:3.99.1-4.fc17 will be installed
	---> Processing Dependency: dlm-lib = 3.99.1-4.fc17 for package: dlm-3.99.1-4.fc17.x86_64
	---> Processing Dependency: libdlmcontrol.so.3()(64bit) for package: dlm-3.99.1-4.fc17.x86_64
	---> Processing Dependency: libdlm.so.3()(64bit) for package: dlm-3.99.1-4.fc17.x86_64
	+---> Package dlm.x86_64 0:3.99.4-1.fc17 will be installed
	---> Package gfs2-utils.x86_64 0:3.1.4-3.fc17 will be installed
	---> Running transaction check
	----> Package dlm-lib.x86_64 0:3.99.1-4.fc17 will be installed
	+---> Package kernel-modules-extra.x86_64 0:3.4.4-3.fc17 will be installed
	--> Finished Dependency Resolution

	Dependencies Resolved

	-=====================================================================================
	- Package Arch Version Repository Size
	-=====================================================================================
	+================================================================================
	+ Package Arch Version Repository Size
	+================================================================================
	Installing:
	- dlm x86_64 3.99.1-4.fc17 updates-testing 77 k
	- gfs2-utils x86_64 3.1.4-3.fc17 fedora 214 k
	-Installing for dependencies:
	- dlm-lib x86_64 3.99.1-4.fc17 updates-testing 19 k
	+ dlm x86_64 3.99.4-1.fc17 updates 83 k
	+ gfs2-utils x86_64 3.1.4-3.fc17 fedora 214 k
	+ kernel-modules-extra x86_64 3.4.4-3.fc17 updates 1.7 M

	Transaction Summary
	-=====================================================================================
	-Install 2 Packages (+1 Dependent package)
	+================================================================================
	+Install 3 Packages

	-Total download size: 309 k
	-Installed size: 879 k
	+Total download size: 1.9 M
	+Installed size: 7.7 M
	Downloading Packages:
	-(1/3): dlm-3.99.1-4.fc17.x86_64.rpm \| 77 kB 00:00
	-(2/3): dlm-lib-3.99.1-4.fc17.x86_64.rpm \| 19 kB 00:00
	-(3/3): gfs2-utils-3.1.4-3.fc17.x86_64.rpm \| 214 kB 00:00
	- -------------------------------------------------------------------------------------
	-Total 241 kB/s \| 309 kB 00:01
	+(1/3): dlm-3.99.4-1.fc17.x86_64.rpm \| 83 kB 00:00
	+(2/3): gfs2-utils-3.1.4-3.fc17.x86_64.rpm \| 214 kB 00:00
	+(3/3): kernel-modules-extra-3.4.4-3.fc17.x86_64.rpm \| 1.7 MB 00:01
	+--------------------------------------------------------------------------------
	+Total 615 kB/s \| 1.9 MB 00:03
	Running Transaction Check
	Running Transaction Test
	Transaction Test Succeeded
	Running Transaction
	- Installing : dlm-lib-3.99.1-4.fc17.x86_64 1/3
	- Installing : dlm-3.99.1-4.fc17.x86_64 2/3
	- Installing : gfs2-utils-3.1.4-3.fc17.x86_64 3/3
	- Verifying : dlm-lib-3.99.1-4.fc17.x86_64 1/3
	- Verifying : gfs2-utils-3.1.4-3.fc17.x86_64 2/3
	- Verifying : dlm-3.99.1-4.fc17.x86_64 3/3
	+ Installing : kernel-modules-extra-3.4.4-3.fc17.x86_64 1/3
	+ Installing : gfs2-utils-3.1.4-3.fc17.x86_64 2/3
	+ Installing : dlm-3.99.4-1.fc17.x86_64 3/3
	+ Verifying : dlm-3.99.4-1.fc17.x86_64 1/3
	+ Verifying : gfs2-utils-3.1.4-3.fc17.x86_64 2/3
	+ Verifying : kernel-modules-extra-3.4.4-3.fc17.x86_64 3/3

	Installed:
	- dlm.x86_64 0:3.99.1-4.fc17 gfs2-utils.x86_64 0:3.1.4-3.fc17
	-
	-Dependency Installed:
	- dlm-lib.x86_64 0:3.99.1-4.fc17
	+ dlm.x86_64 0:3.99.4-1.fc17
	+ gfs2-utils.x86_64 0:3.1.4-3.fc17
	+ kernel-modules-extra.x86_64 0:3.4.4-3.fc17

	Complete!
	-----

	== Create a GFS2 Filesystem ==

	[[GFS2_prep]]
	=== Preparation ===

	Before we do anything to the existing partition, we need to make sure it
	is unmounted. We do this by telling the cluster to stop the WebFS resource.
	This will ensure that other resources (in our case, Apache) using WebFS
	are not only stopped, but stopped in the correct order.

	[source,Bash]
	-----
	# crm resource stop WebFS
	# crm_mon -1
	============
	Last updated: Tue Apr 3 14:07:36 2012
	Last change: Tue Apr 3 14:07:15 2012 via cibadmin on pcmk-1
	Stack: corosync
	Current DC: pcmk-1 (1702537408) - partition with quorum
	Version: 1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff
	2 Nodes configured, unknown expected votes
	5 Resources configured.
	============

	Online: [ pcmk-1 pcmk-2 ]

	ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2
	Master/Slave Set: WebDataClone [WebData]
	Masters: [ pcmk-2 ]
	Slaves: [ pcmk-1 ]
	-----

	[NOTE]
	=======

	Note that both Apache and WebFS have been stopped.

	=======

	=== Create and Populate an GFS2 Partition ===

	Now that the cluster stack and integration pieces are running smoothly,
	we can create an GFS2 partition.

	[WARNING]
	=========

	This will erase all previous content stored on the DRBD device. Ensure
	you have a copy of any important data.

	=========

	We need to specify a number of additional parameters when creating a
	GFS2 partition.

	First we must use the -p option to specify that we want to use the the
	Kernel's DLM. Next we use -j to indicate that it should reserve enough
	space for two journals (one per node accessing the filesystem).

	Lastly, we use -t to specify the lock table name. The format for this
	field is +clustername:fsname+. For the +fsname+, we need to use the same
	value as specified in 'cluster.conf' for +cluster_name+. Just pick
	something unique and descriptive and add somewhere inside the +totem+
	block. For example:

	.....
	totem {
	version: 2

	# cypto_cipher and crypto_hash: Used for mutual node authentication.
	# If you choose to enable this, then do remember to create a shared
	# secret with "corosync-keygen".
	crypto_cipher: none
	crypto_hash: none
	cluster_name: webtest
	...
	.....

	[IMPORTANT]
	===========
	Do this on each node in the cluster and be sure to restart them before
	continuing.
	===========

	[IMPORTANT]
	===========
	We must run the next command on whichever node last had '/dev/drbd'
	mounted. Otherwise you will receive the message:

	-----
	/dev/drbd1: Read-only file system
	-----
	===========

	[source,Bash]
	-----
	-# ssh pcmk-2 -- mkfs.gfs2 -p lock_dlm -j 2 -t webtest /dev/drbd1
	+# ssh pcmk-2 -- mkfs.gfs2 -p lock_dlm -j 2 -t webtest:web /dev/drbd1
	This will destroy any data on /dev/drbd1.
	It appears to contain: Linux rev 1.0 ext4 filesystem data, UUID=dc45fff3-c47a-4db2-96f7-a8049a323fe4 (extents) (large files) (huge files)
	Are you sure you want to proceed? [y/n]y
	Device: /dev/drbd1
	Blocksize: 4096
	Device Size 0.97 GB (253935 blocks)
	Filesystem Size: 0.97 GB (253932 blocks)
	Journals: 2
	Resource Groups: 4
	Locking Protocol: "lock_dlm"
	Lock Table: "webtest"
	UUID: ed293a02-9eee-3fa3-ed1c-435ef1fd0116
	-----

	[source,Bash]
	-----
	# crm
	crm(live)# cib new dlm
	INFO: dlm shadow CIB created
	crm(dlm)# configure primitive dlm ocf:pacemaker:controld \
	op monitor interval=60s
	crm(dlm)# configure clone dlm_clone dlm meta clone-max=2 clone-node-max=1
	crm(dlm)# configure show
	node $id="1702537408" pcmk-1 \
	attributes standby="off"
	node $id="1719314624" pcmk-2
	primitive ClusterIP ocf:heartbeat:IPaddr2 \
	params ip="192.168.122.120" cidr_netmask="32" \
	op monitor interval="30s"
	primitive WebData ocf:linbit:drbd \
	params drbd_resource="wwwdata" \
	op monitor interval="60s"
	primitive WebFS ocf:heartbeat:Filesystem \
	params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="ext4" \
	meta target-role="Stopped"
	primitive WebSite ocf:heartbeat:apache \
	params configfile="/etc/httpd/conf/httpd.conf" \
	op monitor interval="1min"
	primitive dlm ocf:pacemaker:controld \
	op monitor interval="60s"
	ms WebDataClone WebData \
	meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
	clone dlm_clone dlm \
	meta clone-max="2" clone-node-max="1"
	location prefer-pcmk-1 WebSite 50: pcmk-1
	colocation WebSite-with-WebFS inf: WebSite WebFS
	colocation fs_on_drbd inf: WebFS WebDataClone:Master
	colocation website-with-ip inf: WebSite ClusterIP
	order WebFS-after-WebData inf: WebDataClone:promote WebFS:start
	order WebSite-after-WebFS inf: WebFS WebSite
	order apache-after-ip inf: ClusterIP WebSite
	property $id="cib-bootstrap-options" \
	dc-version="1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff" \
	cluster-infrastructure="corosync" \
	stonith-enabled="false" \
	no-quorum-policy="ignore" \
	last-lrm-refresh="1333446866"
	rsc_defaults $id="rsc-options" \
	resource-stickiness="100"
	op_defaults $id="op-options" \
	timeout="240s"
	crm(dlm)# cib commit dlm
	INFO: commited 'dlm' shadow CIB to the cluster
	crm(dlm)# quit
	bye
	# crm_mon -1
	============
	Last updated: Wed Apr 4 01:15:11 2012
	Last change: Wed Apr 4 00:50:11 2012 via crmd on pcmk-1
	Stack: corosync
	Current DC: pcmk-1 (1702537408) - partition with quorum
	Version: 1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff
	2 Nodes configured, unknown expected votes
	7 Resources configured.
	============

	Online: [ pcmk-1 pcmk-2 ]

	ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
	Master/Slave Set: WebDataClone [WebData]
	Masters: [ pcmk-1 ]
	Slaves: [ pcmk-2 ]
	Clone Set: dlm_clone [dlm]
	Started: [ pcmk-1 pcmk-2 ]
	-----

	-
	-[WARNING]
	-=========
	-Nothing has been updated past this point and is probably severely broken.
	-=========
	-
	Then (re)populate the new filesystem with data (web pages). For now we'll
	create another variation on our home page.

	[source,Bash]
	-----
	# mount /dev/drbd1 /mnt/
	# cat <<-END >/mnt/index.html
	<html>
	<body>My Test Site - GFS2</body>
	</html>
	END
	# umount /dev/drbd1
	# drbdadm verify wwwdata#
	-----

	== Reconfigure the Cluster for GFS2 ==

	[source,Bash]
	-----
	# crm
	crm(live) # cib new GFS2
	INFO: GFS2 shadow CIB created
	crm(GFS2) # configure delete WebFS
	crm(GFS2) # configure primitive WebFS ocf:heartbeat:Filesystem params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="gfs2"
	-----

	Now that we've recreated the resource, we also need to recreate all the
	constraints that used it. This is because the shell will automatically
	remove any constraints that referenced WebFS.

	[source,Bash]
	-----
	crm(GFS2) # configure colocation WebSite-with-WebFS inf: WebSite WebFS
	crm(GFS2) # configure colocation fs_on_drbd inf: WebFS WebDataClone:Master
	crm(GFS2) # configure order WebFS-after-WebData inf: WebDataClone:promote WebFS:start
	crm(GFS2) # configure order WebSite-after-WebFS inf: WebFS WebSite
	crm(GFS2) # configure show
	node pcmk-1
	node pcmk-2
	primitive WebData ocf:linbit:drbd \
	params drbd_resource="wwwdata" \
	op monitor interval="60s"
	primitive WebFS ocf:heartbeat:Filesystem \
	params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="gfs2"
	primitive WebSite ocf:heartbeat:apache \
	params configfile="/etc/httpd/conf/httpd.conf" \
	op monitor interval="1min"
	primitive ClusterIP ocf:heartbeat:IPaddr2 \
	params ip="192.168.122.101" cidr_netmask="32" \
	op monitor interval="30s"
	ms WebDataClone WebData \
	meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
	colocation WebSite-with-WebFS inf: WebSite WebFS
	colocation fs_on_drbd inf: WebFS WebDataClone:Master
	colocation website-with-ip inf: WebSite ClusterIP
	order WebFS-after-WebData inf: WebDataClone:promote WebFS:start
	order WebSite-after-WebFS inf: WebFS WebSite
	order apache-after-ip inf: ClusterIP WebSite
	property $id="cib-bootstrap-options" \
	dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \
	cluster-infrastructure="openais" \
	expected-quorum-votes="2" \
	stonith-enabled="false" \
	no-quorum-policy="ignore"
	rsc_defaults $id="rsc-options" \
	resource-stickiness="100"
	-----

	Review the configuration before uploading it to the cluster, quitting the
	shell and watching the cluster's response

	[source,Bash]
	-----
	crm(GFS2) # cib commit GFS2
	INFO: commited 'GFS2' shadow CIB to the cluster
	crm(GFS2) # quit
	bye
	# crm_mon
	============
	Last updated: Thu Sep 3 20:49:54 2009
	Stack: openais
	Current DC: pcmk-2 - partition with quorum
	Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f
	2 Nodes configured, 2 expected votes
	6 Resources configured.
	============

	Online: [ pcmk-1 pcmk-2 ]

	WebSite (ocf::heartbeat:apache): Started pcmk-2
	Master/Slave Set: WebDataClone
	Masters: [ pcmk-1 ]
	Slaves: [ pcmk-2 ]
	ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2WebFS (ocf::heartbeat:Filesystem): Started pcmk-1
	-----

	== Reconfigure Pacemaker for Active/Active ==

	Almost everything is in place. Recent versions of DRBD are capable of
	operating in Primary/Primary mode and the filesystem we're using is
	cluster aware. All we need to do now is reconfigure the cluster to take
	advantage of this.

	This will involve a number of changes, so we'll again use interactive
	mode.

	[source,Bash]
	-----
	# crm
	# cib new active
	-----

	There's no point making the services active on both locations if we can't
	reach them, so lets first clone the IP address. Cloned IPaddr2 resources
	use an iptables rule to ensure that each request only gets processed by one of
	the two clone instances. The additional meta options tell the cluster how
	many instances of the clone we want (one "request bucket" for each node)
	and that if all other nodes fail, then the remaining node should hold all
	of them. Otherwise the requests would be simply discarded.

	[source,Bash]
	-----
	# configure clone WebIP ClusterIP \
	meta globally-unique="true" clone-max="2" clone-node-max="2"
	-----

	Now we must tell the ClusterIP how to decide which requests are
	processed by which hosts. To do this we must specify the
	clusterip_hash parameter.

	Open the ClusterIP resource

	[source,Bash]
	-----
	# configure edit ClusterIP
	-----

	And add the following to the params line

	.....
	clusterip_hash="sourceip"
	.....

	So that the complete definition looks like:

	.....
	primitive ClusterIP ocf:heartbeat:IPaddr2 \
	params ip="192.168.122.101" cidr_netmask="32" clusterip_hash="sourceip" \
	op monitor interval="30s"
	.....

	Here is the full transcript

	[source,Bash]
	-----
	# crm crm(live)
	# cib new active
	INFO: active shadow CIB created
	crm(active) # configure clone WebIP ClusterIP \
	meta globally-unique="true" clone-max="2" clone-node-max="2"
	crm(active) # configure shownode pcmk-1
	node pcmk-2
	primitive WebData ocf:linbit:drbd \
	params drbd_resource="wwwdata" \
	op monitor interval="60s"
	primitive WebFS ocf:heartbeat:Filesystem \
	params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="gfs2"
	primitive WebSite ocf:heartbeat:apache \
	params configfile="/etc/httpd/conf/httpd.conf" \
	op monitor interval="1min"
	primitive ClusterIP ocf:heartbeat:IPaddr2 \
	params ip="192.168.122.101" cidr_netmask="32" clusterip_hash="sourceip" \
	op monitor interval="30s"
	ms WebDataClone WebData \
	meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
	clone WebIP ClusterIP \
	meta globally-unique="true" clone-max="2" clone-node-max="2"
	colocation WebSite-with-WebFS inf: WebSite WebFS
	colocation fs_on_drbd inf: WebFS WebDataClone:Master
	colocation website-with-ip inf: WebSite WebIPorder WebFS-after-WebData inf: WebDataClone:promote WebFS:start
	order WebSite-after-WebFS inf: WebFS WebSiteorder apache-after-ip inf: WebIP WebSite
	property $id="cib-bootstrap-options" \
	dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \
	cluster-infrastructure="openais" \
	expected-quorum-votes="2" \
	stonith-enabled="false" \
	no-quorum-policy="ignore"
	rsc_defaults $id="rsc-options" \
	resource-stickiness="100"
	-----

	Notice how any constraints that referenced ClusterIP have been updated
	to use WebIP instead. This is an additional benefit of using the crm
	shell.

	Next we need to convert the filesystem and Apache resources into
	clones. Again, the shell will automatically update any relevant
	constraints.

	[source,Bash]
	-----
	crm(active) # configure clone WebFSClone WebFS
	crm(active) # configure clone WebSiteClone WebSite
	-----

	The last step is to tell the cluster that it is now allowed to promote
	both instances to be Primary (aka. Master).

	[source,Bash]
	-----
	crm(active) # configure edit WebDataClone
	-----

	Change master-max to 2

	[source,Bash]
	-----
	crm(active) # configure show
	node pcmk-1
	node pcmk-2
	primitive WebData ocf:linbit:drbd \
	params drbd_resource="wwwdata" \
	op monitor interval="60s"
	primitive WebFS ocf:heartbeat:Filesystem \
	params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="gfs2"
	primitive WebSite ocf:heartbeat:apache \
	params configfile="/etc/httpd/conf/httpd.conf" \
	op monitor interval="1min"
	primitive ClusterIP ocf:heartbeat:IPaddr2 \
	params ip="192.168.122.101" cidr_netmask="32" clusterip_hash="sourceip" \
	op monitor interval="30s"
	ms WebDataClone WebData \
	meta master-max="2" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
	clone WebFSClone WebFSclone WebIP ClusterIP \
	meta globally-unique="true" clone-max="2" clone-node-max="2"
	clone WebSiteClone WebSitecolocation WebSite-with-WebFS inf: WebSiteClone WebFSClone
	colocation fs_on_drbd inf: WebFSClone WebDataClone:Master
	colocation website-with-ip inf: WebSiteClone WebIP
	order WebFS-after-WebData inf: WebDataClone:promote WebFSClone:start
	order WebSite-after-WebFS inf: WebFSClone WebSiteClone
	order apache-after-ip inf: WebIP WebSiteClone
	property $id="cib-bootstrap-options" \
	dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \
	cluster-infrastructure="openais" \
	expected-quorum-votes="2" \
	stonith-enabled="false" \
	no-quorum-policy="ignore"
	rsc_defaults $id="rsc-options" \
	resource-stickiness="100"
	-----

	Review the configuration before uploading it to the cluster, quitting the
	shell and watching the cluster's response

	[source,Bash]
	-----
	crm(active) # cib commit active
	INFO: commited 'active' shadow CIB to the cluster
	crm(active) # quit
	bye
	# crm_mon
	============
	Last updated: Thu Sep 3 21:37:27 2009
	Stack: openais
	Current DC: pcmk-2 - partition with quorum
	Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f
	2 Nodes configured, 2 expected votes
	6 Resources configured.
	============

	Online: [ pcmk-1 pcmk-2 ]

	Master/Slave Set: WebDataClone
	Masters: [ pcmk-1 pcmk-2 ]
	Clone Set: WebIP Started: [ pcmk-1 pcmk-2 ]
	Clone Set: WebFSClone Started: [ pcmk-1 pcmk-2 ]
	Clone Set: WebSiteClone Started: [ pcmk-1 pcmk-2 ]
	-----

	=== Testing Recovery ===

	[NOTE]
	=======
	TODO: Put one node into standby to demonstrate failover
	=======
	diff --git a/doc/Clusters_from_Scratch/en-US/Ch-Shared-Storage.txt b/doc/Clusters_from_Scratch/en-US/Ch-Shared-Storage.txt
	index 15a6d9d078..a526eeac00 100644
	--- a/doc/Clusters_from_Scratch/en-US/Ch-Shared-Storage.txt
	+++ b/doc/Clusters_from_Scratch/en-US/Ch-Shared-Storage.txt
	@@ -1,556 +1,556 @@
	= Replicated Storage with DRBD =

	== Background ==
	Even if you're serving up static websites, having to manually synchronize
	the contents of that website to all the machines in the cluster is not
	ideal. For dynamic websites, such as a wiki, it's not even an option. Not
	everyone care afford network-attached storage but somehow the data needs
	to be kept in sync. Enter DRBD which can be thought of as network based
	RAID-1. See http://www.drbd.org/ for more details.

	== Install the DRBD Packages ==

	Since its inclusion in the upstream 2.6.33 kernel, everything needed
	to use DRBD has shiped with Fedora since version 13. All you need to
	do is install it:

	[source,Bash]
	.....
	# yum install -y drbd-pacemaker drbd-udev
	Loaded plugins: langpacks, presto, refresh-packagekit
	Resolving Dependencies
	--> Running transaction check
	---> Package drbd-pacemaker.x86_64 0:8.3.11-5.fc17 will be installed
	--> Processing Dependency: drbd-utils = 8.3.11-5.fc17 for package: drbd-pacemaker-8.3.11-5.fc17.x86_64
	---> Package drbd-udev.x86_64 0:8.3.11-5.fc17 will be installed
	--> Running transaction check
	---> Package drbd-utils.x86_64 0:8.3.11-5.fc17 will be installed
	--> Finished Dependency Resolution

	Dependencies Resolved

	======================================================================================
	Package Arch Version Repository Size
	======================================================================================
	Installing:
	drbd-pacemaker x86_64 8.3.11-5.fc17 updates-testing 22 k
	drbd-udev x86_64 8.3.11-5.fc17 updates-testing 6.4 k
	Installing for dependencies:
	drbd-utils x86_64 8.3.11-5.fc17 updates-testing 183 k

	Transaction Summary
	======================================================================================
	Install 2 Packages (+1 Dependent package)

	Total download size: 212 k
	Installed size: 473 k
	Downloading Packages:
	(1/3): drbd-pacemaker-8.3.11-5.fc17.x86_64.rpm \| 22 kB 00:00
	(2/3): drbd-udev-8.3.11-5.fc17.x86_64.rpm \| 6.4 kB 00:00
	(3/3): drbd-utils-8.3.11-5.fc17.x86_64.rpm \| 183 kB 00:00
	--------------------------------------------------------------------------------------
	Total 293 kB/s \| 212 kB 00:00
	Running Transaction Check
	Running Transaction Test
	Transaction Test Succeeded
	Running Transaction
	Installing : drbd-utils-8.3.11-5.fc17.x86_64 1/3
	Installing : drbd-pacemaker-8.3.11-5.fc17.x86_64 2/3
	Installing : drbd-udev-8.3.11-5.fc17.x86_64 3/3
	Verifying : drbd-pacemaker-8.3.11-5.fc17.x86_64 1/3
	Verifying : drbd-udev-8.3.11-5.fc17.x86_64 2/3
	Verifying : drbd-utils-8.3.11-5.fc17.x86_64 3/3

	Installed:
	drbd-pacemaker.x86_64 0:8.3.11-5.fc17 drbd-udev.x86_64 0:8.3.11-5.fc17

	Dependency Installed:
	drbd-utils.x86_64 0:8.3.11-5.fc17

	Complete!
	.....

	== Configure DRBD ==

	Before we configure DRBD, we need to set aside some disk for it to use.

	=== Create A Partition for DRBD ===

	If you have more than 1Gb free, feel free to use it. For this guide
	however, 1Gb is plenty of space for a single html file and sufficient for
	later holding the GFS2 metadata.

	[source,Bash]
	----
	# vgdisplay \| grep -e Name - e Free
	- VG Name vg_fedora13
	+ VG Name vg_pcmk1
	Free PE / Size 31 / 992.00 MiB
	# lvs
	LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
	- lv_root vg_fedora13 -wi-ao-- 8.56g
	- lv_swap vg_fedora13 -wi-ao-- 960.00m
	-# lvcreate -n drbd-demo -L 1G vg_fedora13
	+ lv_root vg_pcmk1 -wi-ao-- 8.56g
	+ lv_swap vg_pcmk1 -wi-ao-- 960.00m
	+# lvcreate -n drbd-demo -L 1G vg_pcmk1
	Logical volume "drbd-demo" created
	# lvs
	LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
	- drbd-demo vg_fedora13 -wi-a--- 1.00G
	- lv_root vg_fedora13 -wi-ao-- 8.56g
	- lv_swap vg_fedora13 -wi-ao-- 960.00m
	+ drbd-demo vg_pcmk1 -wi-a--- 1.00G
	+ lv_root vg_pcmk1 -wi-ao-- 8.56g
	+ lv_swap vg_pcmk1 -wi-ao-- 960.00m
	----

	Repeat this on the second node, be sure to use the same size partition.

	[source,Bash]
	----
	# ssh pcmk-2 -- lvs
	LV VG Attr LSize Origin Snap% Move Log Copy% Convert
	- lv_root vg_fedora13 -wi-ao-- 8.56g
	- lv_swap vg_fedora13 -wi-ao-- 960.00m
	-# ssh pcmk-2 -- lvcreate -n drbd-demo -L 1G vg_fedora13
	+ lv_root vg_pcmk1 -wi-ao-- 8.56g
	+ lv_swap vg_pcmk1 -wi-ao-- 960.00m
	+# ssh pcmk-2 -- lvcreate -n drbd-demo -L 1G vg_pcmk1
	Logical volume "drbd-demo" created
	# ssh pcmk-2 -- lvs
	LV VG Attr LSize Origin Snap% Move Log Copy% Convert
	- drbd-demo vg_fedora13 -wi-a--- 1.00G
	- lv_root vg_fedora13 -wi-ao-- 8.56g
	- lv_swap vg_fedora13 -wi-ao-- 960.00m
	+ drbd-demo vg_pcmk1 -wi-a--- 1.00G
	+ lv_root vg_pcmk1 -wi-ao-- 8.56g
	+ lv_swap vg_pcmk1 -wi-ao-- 960.00m
	----

	=== Write the DRBD Config ===

	There is no series of commands for building a DRBD configuration, so simply
	copy the configuration below to /etc/drbd.conf

	Detailed information on the directives used in this configuration (and
	other alternatives) is available from
	http://www.drbd.org/users-guide/ch-configure.html

	[WARNING]
	=========

	Be sure to use the names and addresses of your nodes if they differ from
	the ones used in this guide.

	=========

	....
	global {
	usage-count yes;
	}
	common {
	protocol C;
	}
	resource wwwdata {
	meta-disk internal;
	device /dev/drbd1;
	syncer {
	verify-alg sha1;
	}
	net {
	allow-two-primaries;
	}
	on pcmk-1 {
	- disk /dev/vg_fedora13/drbd-demo;
	+ disk /dev/vg_pcmk1/drbd-demo;
	address 192.168.122.101:7789;
	}
	on pcmk-2 {
	- disk /dev/vg_fedora13/drbd-demo;
	+ disk /dev/vg_pcmk1/drbd-demo;
	address 192.168.122.102:7789;
	}
	}
	....

	[NOTE]
	=======

	TODO: Explain the reason for the allow-two-primaries option

	=======

	=== Initialize and Load DRBD ===

	With the configuration in place, we can now perform the DRBD
	initialization

	[source,Bash]
	----
	# drbdadm create-md wwwdata
	Writing meta data...
	initializing activity log
	NOT initialized bitmap
	New drbd meta data block successfully created.
	success
	----

	Now load the DRBD kernel module and confirm that everything is sane

	[source,Bash]
	----
	# modprobe drbd
	# drbdadm up wwwdata
	# cat /proc/drbd
	version: 8.3.11 (api:88/proto:86-96)
	srcversion: 0D2B62DEDB020A425130935

	1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
	ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:1015740
	----

	Repeat on the second node

	[source,Bash]
	----
	# ssh pcmk-2 -- drbdadm --force create-md wwwdata
	Writing meta data...
	initializing activity log
	NOT initialized bitmap
	New drbd meta data block successfully created.
	success
	# ssh pcmk-2 -- modprobe drbd
	WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/.
	# ssh pcmk-2 -- drbdadm up wwwdata
	# ssh pcmk-2 -- cat /proc/drbd
	version: 8.3.11 (api:88/proto:86-96)
	srcversion: 0D2B62DEDB020A425130935

	1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
	ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:1015740
	----

	Now we need to tell DRBD which set of data to use. Since both sides
	contain garbage, we can run the following on pcmk-1:

	[source,Bash]
	----
	# drbdadm -- --overwrite-data-of-peer primary wwwdata
	# cat /proc/drbd
	version: 8.3.11 (api:88/proto:86-96)
	srcversion: 0D2B62DEDB020A425130935

	1: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
	ns:8064 nr:0 dw:0 dr:8728 al:0 bm:0 lo:0 pe:1 ua:0 ap:0 ep:1 wo:f oos:1007804
	[>....................] sync'ed: 0.9% (1007804/1015740)K
	finish: 0:12:35 speed: 1,320 (1,320) K/sec
	----

	After a while, the sync should finish and you'll see:

	[source,Bash]
	----
	# cat /proc/drbd
	version: 8.3.11 (api:88/proto:86-96)
	srcversion: 0D2B62DEDB020A425130935

	1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
	ns:1015740 nr:0 dw:0 dr:1016404 al:0 bm:62 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
	----

	pcmk-1 is now in the Primary state which allows it to be written to.
	Which means it's a good point at which to create a filesystem and populate
	it with some data to serve up via our WebSite resource.


	=== Populate DRBD with Data ===

	[source,Bash]
	----
	# mkfs.ext4 /dev/drbd1
	mke2fs 1.42 (29-Nov-2011)
	Filesystem label=
	OS type: Linux
	Block size=4096 (log=2)
	Fragment size=4096 (log=2)
	Stride=0 blocks, Stripe width=0 blocks
	63488 inodes, 253935 blocks
	12696 blocks (5.00%) reserved for the super user
	First data block=0
	Maximum filesystem blocks=260046848
	8 block groups
	32768 blocks per group, 32768 fragments per group
	7936 inodes per group
	Superblock backups stored on blocks:
	32768, 98304, 163840, 229376

	Allocating group tables: done
	Writing inode tables: done
	Creating journal (4096 blocks): done
	Writing superblocks and filesystem accounting information: done
	----

	Now mount the newly created filesystem so we can create our index file

	[source,Bash]
	----
	# mount /dev/drbd1 /mnt/
	# cat <<-END >/mnt/index.html
	<html>
	<body>My Test Site - drbd</body>
	</html>
	END
	# umount /dev/drbd1
	----

	== Configure the Cluster for DRBD ==

	One handy feature of the crm shell is that you can use it in
	interactive mode to make several changes atomically.

	First we launch the shell. The prompt will change to indicate you're
	in interactive mode.

	[source,Bash]
	----
	# crm
	crm(live) #
	----

	Next we must create a working copy of the current configuration. This is
	where all our changes will go. The cluster will not see any of them until
	we say it's ok. Notice again how the prompt changes, this time to indicate
	that we're no longer looking at the live cluster.

	[source,Bash]
	----
	cib crm(live) # cib new drbd
	INFO: drbd shadow CIB created
	crm(drbd) #
	----

	Now we can create our DRBD clone and display the revised configuration.

	[source,Bash]
	----
	crm(drbd) # configure primitive WebData ocf:linbit:drbd params drbd_resource=wwwdata \
	op monitor interval=60s
	crm(drbd) # configure ms WebDataClone WebData meta master-max=1 master-node-max=1 \
	clone-max=2 clone-node-max=1 notify=true
	crm(drbd) # configure show
	node $id="1702537408" pcmk-1
	node $id="1719314624" pcmk-2
	primitive ClusterIP ocf:heartbeat:IPaddr2 \
	params ip="192.168.122.120" cidr_netmask="32" \
	op monitor interval="30s"
	primitive WebData ocf:linbit:drbd \
	params drbd_resource="wwwdata" \
	op monitor interval="60s"
	primitive WebSite ocf:heartbeat:apache \
	params configfile="/etc/httpd/conf/httpd.conf" \
	op monitor interval="1min"
	ms WebDataClone WebData \
	meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
	location prefer-pcmk-1 WebSite 50: pcmk-1
	colocation website-with-ip inf: WebSite ClusterIP
	order apache-after-ip inf: ClusterIP WebSite
	property $id="cib-bootstrap-options" \
	dc-version="1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff" \
	cluster-infrastructure="corosync" \
	stonith-enabled="false" \
	no-quorum-policy="ignore" \
	last-lrm-refresh="1333446866"
	rsc_defaults $id="rsc-options" \
	resource-stickiness="100"
	op_defaults $id="op-options" \
	timeout="240s"
	----

	Once we're happy with the changes, we can tell the cluster to start using
	them and use crm_mon to check everything is functioning.

	[source,Bash]
	----
	crm(drbd) # cib commit drbd
	INFO: commited 'drbd' shadow CIB to the cluster
	crm(drbd) # quit
	bye
	# crm_mon -1
	============
	Last updated: Tue Apr 3 13:50:01 2012
	Last change: Tue Apr 3 13:49:46 2012 via crm_shadow on pcmk-1
	Stack: corosync
	Current DC: pcmk-1 (1702537408) - partition with quorum
	Version: 1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff
	2 Nodes configured, unknown expected votes
	4 Resources configured.
	============

	Online: [ pcmk-1 pcmk-2 ]

	ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
	WebSite (ocf::heartbeat:apache): Started pcmk-1
	Master/Slave Set: WebDataClone [WebData]
	Masters: [ pcmk-1 ]
	Slaves: [ pcmk-2 ]
	----

	[NOTE]
	=======

	TODO: Include details on adding a second DRBD resource

	=======

	Now that DRBD is functioning we can configure a Filesystem resource to
	use it. In addition to the filesystem's definition, we also need to
	tell the cluster where it can be located (only on the DRBD Primary)
	and when it is allowed to start (after the Primary was promoted).

	Once again we'll use the shell's interactive mode

	[source,Bash]
	----
	# crm
	crm(live) # cib new fs
	INFO: fs shadow CIB created
	crm(fs) # configure primitive WebFS ocf:heartbeat:Filesystem \
	params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="ext4"
	crm(fs) # configure colocation fs_on_drbd inf: WebFS WebDataClone:Master
	crm(fs) # configure order WebFS-after-WebData inf: WebDataClone:promote WebFS:start
	----

	We also need to tell the cluster that Apache needs to run on the same
	machine as the filesystem and that it must be active before Apache can
	start.

	[source,Bash]
	----
	crm(fs) # configure colocation WebSite-with-WebFS inf: WebSite WebFS
	crm(fs) # configure order WebSite-after-WebFS inf: WebFS WebSite
	----

	Time to review the updated configuration:

	[source,Bash]
	----
	crm(fs) # configure show
	node $id="1702537408" pcmk-1
	node $id="1719314624" pcmk-2
	primitive ClusterIP ocf:heartbeat:IPaddr2 \
	params ip="192.168.122.120" cidr_netmask="32" \
	op monitor interval="30s"
	primitive WebData ocf:linbit:drbd \
	params drbd_resource="wwwdata" \
	op monitor interval="60s"
	primitive WebFS ocf:heartbeat:Filesystem \
	params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="ext4"
	primitive WebSite ocf:heartbeat:apache \
	params configfile="/etc/httpd/conf/httpd.conf" \
	op monitor interval="1min"
	ms WebDataClone WebData \
	meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
	location prefer-pcmk-1 WebSite 50: pcmk-1
	colocation WebSite-with-WebFS inf: WebSite WebFS
	colocation fs_on_drbd inf: WebFS WebDataClone:Master
	colocation website-with-ip inf: WebSite ClusterIP
	order WebFS-after-WebData inf: WebDataClone:promote WebFS:start
	order WebSite-after-WebFS inf: WebFS WebSite
	order apache-after-ip inf: ClusterIP WebSite
	property $id="cib-bootstrap-options" \
	dc-version="1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff" \
	cluster-infrastructure="corosync" \
	stonith-enabled="false" \
	no-quorum-policy="ignore" \
	last-lrm-refresh="1333446866"
	rsc_defaults $id="rsc-options" \
	resource-stickiness="100"
	op_defaults $id="op-options" \
	timeout="240s"
	----

	After reviewing the new configuration, we again upload it and watch the
	cluster put it into effect.

	[source,Bash]
	----
	crm(fs) # cib commit fs
	INFO: commited 'fs' shadow CIB to the cluster
	crm(fs) # quit
	bye
	# crm_mon -1
	============
	Last updated: Tue Apr 3 13:52:21 2012
	Last change: Tue Apr 3 13:52:06 2012 via crm_shadow on pcmk-1
	Stack: corosync
	Current DC: pcmk-1 (1702537408) - partition with quorum
	Version: 1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff
	2 Nodes configured, unknown expected votes
	5 Resources configured.
	============

	Online: [ pcmk-1 pcmk-2 ]

	ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
	WebSite (ocf::heartbeat:apache): Started pcmk-1
	Master/Slave Set: WebDataClone [WebData]
	Masters: [ pcmk-1 ]
	Slaves: [ pcmk-2 ]
	WebFS (ocf::heartbeat:Filesystem): Started pcmk-1
	----

	=== Testing Migration ===

	We could shut down the active node again, but another way to safely
	simulate recovery is to put the node into what is called "standby
	mode". Nodes in this state tell the cluster that they are not allowed
	to run resources. Any resources found active there will be moved
	elsewhere. This feature can be particularly useful when updating the
	resources' packages.

	Put the local node into standby mode and observe the cluster move all
	the resources to the other node. Note also that the node's status will
	change to indicate that it can no longer host resources.

	[source,Bash]
	----
	# crm node standby
	# crm_mon -1
	============
	Last updated: Tue Apr 3 13:59:14 2012
	Last change: Tue Apr 3 13:52:36 2012 via crm_attribute on pcmk-1
	Stack: corosync
	Current DC: pcmk-1 (1702537408) - partition with quorum
	Version: 1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff
	2 Nodes configured, unknown expected votes
	5 Resources configured.
	============

	Node pcmk-1 (1702537408): standby
	Online: [ pcmk-2 ]

	ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2
	WebSite (ocf::heartbeat:apache): Started pcmk-2
	Master/Slave Set: WebDataClone [WebData]
	Masters: [ pcmk-2 ]
	Stopped: [ WebData:1 ]
	WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
	----

	Once we've done everything we needed to on pcmk-1 (in this case nothing,
	we just wanted to see the resources move), we can allow the node to be a
	full cluster member again.

	[source,Bash]
	----
	# crm node online
	# crm_mon -1
	============
	Last updated: Tue Apr 3 14:00:06 2012
	Last change: Tue Apr 3 14:00:00 2012 via crm_attribute on pcmk-1
	Stack: corosync
	Current DC: pcmk-1 (1702537408) - partition with quorum
	Version: 1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff
	2 Nodes configured, unknown expected votes
	5 Resources configured.
	============

	Online: [ pcmk-1 pcmk-2 ]

	ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2
	WebSite (ocf::heartbeat:apache): Started pcmk-2
	Master/Slave Set: WebDataClone [WebData]
	Masters: [ pcmk-2 ]
	Slaves: [ pcmk-1 ]
	WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
	----

	Notice that our resource stickiness settings prevent the services from
	migrating back to pcmk-1.

	diff --git a/doc/Clusters_from_Scratch/en-US/Clusters_from_Scratch.ent b/doc/Clusters_from_Scratch/en-US/Clusters_from_Scratch.ent
	index 5ddd8fb610..ecc8c1303a 100644
	--- a/doc/Clusters_from_Scratch/en-US/Clusters_from_Scratch.ent
	+++ b/doc/Clusters_from_Scratch/en-US/Clusters_from_Scratch.ent
	@@ -1,6 +1,6 @@
	<!ENTITY PRODUCT "Pacemaker">
	<!ENTITY BOOKID "Clusters_from_Scratch">
	<!ENTITY YEAR "2012">
	<!ENTITY HOLDER "Andrew Beekhof">
	<!ENTITY DISTRO "Fedora">
	-<!ENTITY DISTRO_VERSION "13">
	+<!ENTITY DISTRO_VERSION "17">
	diff --git a/doc/Clusters_from_Scratch/en-US/Revision_History.xml b/doc/Clusters_from_Scratch/en-US/Revision_History.xml
	index 0fbc0fb833..4ee03dd096 100644
	--- a/doc/Clusters_from_Scratch/en-US/Revision_History.xml
	+++ b/doc/Clusters_from_Scratch/en-US/Revision_History.xml
	@@ -1,43 +1,49 @@
	<?xml version='1.0' encoding='utf-8' ?>
	<!DOCTYPE appendix PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
	<!ENTITY % BOOK_ENTITIES SYSTEM "Clusters_from_Scratch.ent">
	%BOOK_ENTITIES;
	]>
	<appendix id="appe-Clusters_from_Scratch-Revision_History">
	<title>Revision History</title>
	<simpara>
	<revhistory>
	<revision>
	<revnumber>1</revnumber>
	<date>Mon May 17 2010</date>
	<author><firstname>Andrew</firstname><surname>Beekhof</surname><email>andrew@beekhof.net</email></author>
	<revdescription><simplelist><member>Import from Pages.app</member></simplelist></revdescription>
	</revision>
	<revision>
	<revnumber>2</revnumber>
	<date>Wed Sep 22 2010</date>
	<author><firstname>Raoul</firstname><surname>Scarazzini</surname><email>rasca@miamammausalinux.org</email></author>
	<revdescription><simplelist><member>Italian translation</member></simplelist></revdescription>
	</revision>
	<revision>
	<revnumber>3</revnumber>
	<date>Wed Feb 9 2011</date>
	<author><firstname>Andrew</firstname><surname>Beekhof</surname><email>andrew@beekhof.net</email></author>
	<revdescription><simplelist><member>Updated for Fedora 13</member></simplelist></revdescription>
	</revision>
	<revision>
	<revnumber>4</revnumber>
	<date>Wed Oct 5 2011</date>
	<author><firstname>Andrew</firstname><surname>Beekhof</surname><email>andrew@beekhof.net</email></author>
	<revdescription><simplelist><member>Update the GFS2 section to use CMAN</member></simplelist></revdescription>
	</revision>
	<revision>
	<revnumber>5</revnumber>
	<date>Fri Feb 10 2012</date>
	<author><firstname>Andrew</firstname><surname>Beekhof</surname><email>andrew@beekhof.net</email></author>
	<revdescription><simplelist><member>Generate docbook content from asciidoc sources</member></simplelist></revdescription>
	</revision>
	+ <revision>
	+ <revnumber>6</revnumber>
	+ <date>Tues July 3 2012</date>
	+ <author><firstname>Andrew</firstname><surname>Beekhof</surname><email>andrew@beekhof.net</email></author>
	+ <revdescription><simplelist><member>Updated for Fedora 17</member></simplelist></revdescription>
	+ </revision>
	</revhistory>
	</simpara>
	</appendix>

File Metadata

Mime Type: text/x-diff
Expires: Sat, Nov 23, 7:27 AM (22 h, 15 m)
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 1018441
Default Alt Text: (42 KB)

No OneTemporaryActions

View Options

File Metadata

Event Timeline

No OneTemporary
Actions