How to install and run gulm and gfs.

Refer to the cluster project page for the latest information.
http://sources.redhat.com/cluster/

This document does not cover setting up a block device to run on.
Mostly because CLVM doesn't work with gulm yet.

Requires
--------

Gulm uses IPv6.  (see note at bottom of file)
CCS requires libxml2

Get source
----------

- download the source tarballs

  cluster               - http://sources.redhat.com/cluster/

- or to download source from cvs see

  http://sources.redhat.com/cluster/

  summary: after cvs login,
  cvs -d :pserver:cvs@sources.redhat.com:/cvs/cluster checkout cluster 


Build and install
-----------------

1. build and install from cluster tree
   cd cluster
   ./configure --kernel_src=/path/to/kernel
   make; make install

   - This builds and installs kernel modules, libraries and user programs.

   - Kernel modules can also be built within the original kernel source tree
     by applying the kernel patches from gfs-kernel/patches.


Load kernel modules
-------------------

depmod -a
modprobe gfs
modprobe lock_gulm

Modules that should be loaded: lock_gulm, gfs, lock_harness


Startup procedure
-----------------

Run these commands on each cluster node:

> ccsd                             - Starts the CCS daemon
> lock_gulmd --use_ccs             - Starts the gulm daemons.
> mount -t gfs /dev/foo /mnt       - Mounts a GFS file system


Shutdown procedure
------------------

Run these commands on each cluster node:

> umount /mnt			   - Unmounts a GFS file system
> gulm_tool shutdown localhost     - Stops the gulm daemons.
> killall ccsd                     - Stops the CCS daemon


Creating CCS config
-------------------

There is no GUI or command line program to create the config file yet.
The cluster config file "cluster.conf" must therefore be created manually.
Once created, cluster.conf should be placed in the /etc/cluster/ directory
on one cluster node.  CCS daemon (ccsd) will take care of transferring it
to other nodes where it's needed.  (FIXME: updating cluster.conf in a running
cluster is supported but not documented.)

A minimal cluster.conf example is shown below.


Creating GFS file systems
-------------------------

> gfs_mkfs -p lock_gulm -t <ClusterName>:<FSName> -j <Journals> <Device>

        <ClusterName> must match the cluster name used in CCS config
        <FSName> is a unique name chosen now to distinguish this fs from others
        <Journals> the number of journals in the fs, one for each node to mount
        <Device> a block device, usually an LVM logical volume


Config file
-----------

This example primarily illustrates the variety of fencing configurations.

The first node uses "cascade fencing"; if the first method fails (power cycling
with an APC Masterswitch), the second is tried (port disable on a Brocade FC
switch).  In this example, the node has dual paths to the storage so the port
on both paths must be disabled (the same idea applies to nodes with dual power
supplies.)

There is only one method of fencing the second node (via an APC Masterswitch)
so no cascade fencing is possible.

If no hardware is available for fencing, manual fencing can be used as shown
for the third node.  If a node with manual fencing fails, a human must take
notice (a message appears in the system log) and run fence_ack_manual after
resetting the failed node.  

With gulm, fencing happens on the server node that is the current master.

<?xml version="1.0"?>
<cluster name="alpha" config_version="1">

<gulm>
	<lockserver name="gulmserver1"/>
	<lockserver name="gulmserver2"/>
	<lockserver name="gulmserver3"/>
</gulm>

<clusternodes>
<clusternode name="gulmserver1">
	<fence>
		<method name="single">
			<device name="apc1" port="3"/>
		</method>
	</fence>
</clusternode>

<clusternode name="gulmserver2">
	<fence>
		<method name="single">
			<device name="apc1" port="4"/>
		</method>
	</fence>
</clusternode>

<clusternode name="gulmserver3">
	<fence>
		<method name="single">
			<device name="apc1" port="5"/>
		</method>
	</fence>
</clusternode>

<clusternode name="nd01">
	<fence>
		<method name="cascade1">
			<device name="apc1" port="1"/>
		</method>

		<method name="cascade2">
			<device name="brocade1" port="1"/>
			<device name="brocade2" port="1"/>
		</method>
	</fence>
</clusternode>

<clusternode name="nd02">
	<fence>
		<method name="single">
			<device name="apc1" port="2"/>
		</method>
	</fence>
</clusternode>

<clusternode name="nd03">
	<fence>
		<method name="single">
			<device name="human" ipaddr="nd03"/>
		</method>
	</fence>
</clusternode>
</clusternodes>

<fencedevices>
	<fencedevice name="apc1" agent="fence_apc" ipaddr="10.1.1.1" login="apc" passwd="apc"/>
	<fencedevice name="brocade1" agent="fence_brocade" ipaddr="10.1.1.2" login="user" passwd="pw"/>
	<fencedevice name="brocade2" agent="fence_brocade" ipaddr="10.1.1.3" login="user" passwd="pw"/>
	<fencedevice name="human" agent="fence_manual"/>
</fencedevices>

</cluster>


IPv6
----

All sockets in gulm are IPv6 sockets.  Since IPv6 supports encapsulation of
IPv4 addresses, your machines can be configured with either. (but you should
use one or the other, not both.)