Toolserver:Admin:HA cluster
This page was moved from the Toolserver wiki.
Toolserver has been replaced by Toolforge. As such, the instructions here may no longer work, but may still be of historical interest.
Please help by updating examples, links, template links, etc. If a page is still relevant, move it to a normal title and leave a redirect.
The HA cluster is an installation of Sun Cluster on two servers, turnera and damiana and their attached storage, ts-array5.
Cluster provides high availability for configured services. Each service is started on one node; if that node fails, the service is automatically migrated to the other node.
Currently, these services are hosted on the cluster:
- NFS
- LDAP
- Subversion
- sql-toolserver
- phpMyAdmin
- DNS
- tsbot
- HTTP proxy (ha-proxy.esi)
- Sun Grid Engine (qmaster only)
- Puppet
- Nagios
- Mail (Exim)
Cluster is based on shared storage. The entire array exports one LUN, which is configured as a VxVM disk group called disks. This has two volumes, home and misc. home contains a VxFS filesystem mounted at /global/home, and can only be mounted on one system at a time. misc contains a smaller filesystem mounted at /global/misc; this is a global filesystem and can be accessed from either cluster node.
Services on Cluster are called "resources", and are organised into "resource groups". For example, the "svn" resource group includes two services: "svn-lh", the logical hostname (IP address) for svn.toolserver.org, and "svn-apache", the Apache server that hosts the repositories. Failover between servers is done as resource groups, not individual resources.
DO NOT stop/start services on the cluster using the normal commands. Instead, use the Cluster-specific commands.
(Make sure, that /usr/cluster/bin is in your PATH)
- List resources:
clrs list -v - List resource groups:
clrg list -v - Take a resource group offline:
clrg offline svn - Bring it back online:
clrg online svn - Move the resource group to a different server:
clrg switch -n turnera svn - Display resource status:
scstat
Most cluster operations can be performed on either server, regardless of where the service is running. The exception is changes to the VxVM configuration, which can only be made on the server where the VxVM diskgroup is imported (usually the active node for nfs).