Cluster Summit 20/6/2005

Red Hat Cluster Suite Infrastructure

Patrick Caulfield

(Cluster & Storage Group)

Not

I am not going to do last year’s presentation again

Hope you know what it’s all about

This is really just an update

Brief introduction

GFS is the poster-boy for Red Hat’s cluster suite

It is what most people want

Two infrastructures available to support it

Cluster infrastructures

Gulm

GFS-Specific.

Not really suitable for general clustering use

(though clvmd does work with it)

CMAN/DLM

Designed to be generic

Usable for other clustered applications

CMAN/DLM architecture

CMAN

DLM

GFS

VFS

lock_dlm

ccsd

fenced

App

App

libdlm

Shared

Storage

Other cluster nodes

kernel

userland

GULM architecture

GFS

VFS

lock_gulm

App

App

Shared

Storage

kernel

userland

lock_gulmd

lock_gulmd

lock_gulmd

lock_gulmd

Other nodes

New stuff

Moving cman out of the kernel

Re-engineering DLM for upstream submission

New architecture

DLM

GFS

VFS

lock_dlm

ccsd

fenced

App

App

libdlm

Shared

Storage

kernel

userland

cmand

groupd

lock_dlmd

dlm_controld

Why change?

Basically, no need for cluster manager to be in the kernel

Easier to maintain(coding and updates)

Hard to get upstream

Existing kernel code in RHEL4 so will be supported for some time.

New CMAN

Now a daemon

cman_tool behaves as before

libcman interface binary compatible

But some new features added

Current CVS is just a port of kernel

Will have virtual synchrony-type features.

New DLM

Re-organisation of code

Much of it rewritten

Controlled from userspace daemon

Upstream in -mm kernel

Not all features finished

Queries, hierarchical locks

New dlm(2)

Sysfs interface

May change, configfs

Dave working with ocfs people

API still compatible

Userland binary compatible

libdlm is the same anyway

What’s in CVS?

It all is!

Three main tags

RHEL4: stable

FC4: similar to RHEL4 but for kernel

HEAD: userland new code

Largely working but still unstable

APIs

Libdlm

A lot like OpenVMS dlm calls

Added lockspace calls

Added sync versions (for userspace)

libcman

Similar to openAIS but with tidier names.

Hello

I am (who am I ??)

This is not going to be a basic introduction to clusters or even Red Hat’s cluster suite & GFS.

I am going to assume you know what I mean by a cluster in this context (basically shared storage).

OK, just a quick overview, but that’s all you’re getting

Gulm is not deprecated as far as I know. It’s still suitable for several types of environment. Has been tested more in the field and with larger clusters than cman/dlm

This is what’s in RHEL4/Fedora.

I’ve missed off clvmd here to make it fit on a slide. It doesn’t change at all for gulm or the new stuff which I’ll come to soon.

This is also in RHEL4/Fedora and is similar to what’s also in RHEL3/Sistina GFS.

It’s recommended to run the lock servers off the main cluster machines, the lock_gulmd on the machine is just a forwarder

So, what are doing now?

Much more in userspace.

Groupd is what used to be “SM” (service manager) part of cman

Idea of “Closed Process Groups” see ISIS (says Dave!)

New daemon

You’ve never heard of libcman? Hmm surprise.

VS features are things like guaranteed order of delivery of messages, even across cluster transitions.

Thought about using openais core but not really happy about it – need Ipv6 and don’t much like the spinning token thing.

Current dev code uses the ring approach though. Needs a lot more work.

Remember – there are Oracle people here ......

Dave looking at OCFS nodemanager

“Mark” from Oracle.

“The ocfs group is working hard at getting a 1.0 release done, but after

that they're keen on getting rid of the kernel-based stuff they're doing

and will quite likely adopt the methods we're using for managing dlm/gfs

from userspace.”

We’ll do these in more detail later

or maybe...now...

Cluster Summit 2005

Patrick Caulfield

2004-01-21T10:15:53