Contents
- Introduction
- Extra Options
- Simple Example
- What if?
- FAQ
- Writing a Clone Resource Agent
- Advanced Example
- Location Constraints for Masters
- Location Constraints for Slaves
- Co-locational and Ordering Constraints related to Masters
- Co-locational and Ordering Constraints related to Slaves
- Finding Masters in the Status Section
Related Pages: v2/Concepts v2/Concepts/Clones outdated documentation
Introduction
Superset of a cloned resource.
- Can be either anonymous or unique
Currently supports two states
- The states mean whatever you want them to mean
Extra Options
master_max (default: 1)
- The maximum number of resource masters that should be run in the cluster.
master_node_max (default: 1)
- The maximum number of resource masters that should be run on any given node.
Simple Example
In the example below we have specified that the WebServerIP resource should be a ClusteredIP. It will be started on at most 3 nodes and at most there can be 2 clone active on any given node. Instance 1 will serve the first "bucket", instance 2 the second and so on.
<master_slave id="MyCorpWWW" ordered="false" interleave="false" notify="false"> <instance_attributes> <attributes> <nvpair name="clone_max" value="3"/> <nvpair name="clone_node_max" value="2"/> <nvpair name="master_max" value="2"/> <nvpair name="master_node_max" value="1"/> </attributes> </instance_attributes> <primitive id="WebServerIP" class="ocf" type="IPaddr" provider="heartbeat"> <instance_attributes> <attributes> <nvpair name="ip" value="127.0.0.26"/> </attributes> </instance_attributes> </primitive> </master_slave>
What if?
If there are >=1 nodes available:
- Two nodes will be chosen and each node will run 1 master
- If there are 2 nodes available:
- 1 node will run 2 clones (1 master, 1 slave), the other 1 master
- If there is 1 nodes available:
- 1 node will run 2 clones (1 master, 1 slave), the clone will not be started
FAQ
- Will the master clone instances always be an unbroken sequence?
- No.
- Will the master clone instances always start at 0?
- No.
- Does the CRM keep track of the clone instances' state?
- No.
It is the responsibility of the OCFResourceAgent to know what state the clone instance is in. The CRM knows what state it thinks the resource should be in, but in all cases it is the script which must understand the resource and determine it's true state.
- No.
- Why does monitoring stop after my resource is promoted to master?
You must define a monitor action for every role (e.g. Master, Slave, Started, Stopped) you wish to monitor. Please refer to the examples given in ClusterInformationBase/Actions.
Writing a Clone Resource Agent
Which Resource Instance is Promoted
During the start operation, Resource Agent scripts should call the crm_master command-line tool. This tool automatically detects both the resource and host and should be used to set a preference for being promoted. Based on this, master_max, and master_node_max, the instance(s) with the highest preference will be promoted.
BeekhofG5::beekhof pengine # crm_master
usage: crm_master [-?VQ] -(D|G|v) [-l]
Options
--help (-?) : this help message
--verbose (-V) : turn on debug info. additional instances increase verbosity
--quiet (-Q) : Print only the value on stdout (use with -G)
--get-value (-G) : Retrieve rather than set the attribute
--delete-attr (-D) : Delete rather than set the attribute
--attr-value (-v) <string> : Value to use (ignored with -G)
--lifetime (-l) <string> : How long the preference lasts (reboot|forever)
Requirements for ResourceAgents
This items listed here are in addition to the requirements for Clones.
ResourceAgents are required to accurately report how many instances are active, inactive, masters and slaves. In monitor operations:
- Active masters should return $OCF_RUNNING_MASTER
- Failed masters should return $OCF_FAILED_MASTER
- Inactive instances should return $OCF_NOT_RUNNING
- All other active instances should use
- $OCF_SUCCESS if they are active, or
- one of the $OCF_ERR_* values if they have failed
All OCF variables can normally be found in /usr/lib/ocf/resource.d/heartbeat/ocf-shellfuncs which should be the first thing included by a resource agent.
Extra Operations
A Master/Slave Resource Agent must implement the promote and demote operations. The implementation is up to the user.
Extra Environment Variables
In addition to the environment variables added for clones, these variables are available to notify actions.
OCF_RESKEY_notify_operation (start|stop|promote|demote)
OCF_RESKEY_notify_{x}_resource
OCF_RESKEY_notify_ {x}_uname
- {x} : promote, demote, master, slave
Starting with 2.0.4, they will also be available to start, stop, promote and demote actions if notify="true" is set.
Proper Interpretation of Master Environment Variables
- Pre-notification (demote)
- Active resources: $OCF_RESKEY_notify_active_resource
- Master resources: $OCF_RESKEY_notify_master_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: NA
- Resources that were promoted: NA
- Resources that were demoted: NA
- Resources that were stopped: NA
- Post-notification (demote) / Pre-notification (stop)
- Active resources: $OCF_RESKEY_notify_active_resource
- Master resources: $OCF_RESKEY_notify_master_resource
minus $OCF_RESKEY_notify_demote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: NA
- Resources that were promoted: NA
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: NA
- Post-notification (stop) / Pre-notification (start)
- Active resources: $OCF_RESKEY_notify_active_resource
minus $OCF_RESKEY_notify_stop_resource
- Master resources: $OCF_RESKEY_notify_master_resource
minus $OCF_RESKEY_notify_demote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
minus $OCF_RESKEY_notify_stop_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
plus $OCF_RESKEY_notify_stop_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: NA
- Resources that were promoted: NA
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: $OCF_RESKEY_notify_stop_resource
- Active resources: $OCF_RESKEY_notify_active_resource
- Post-notification (start) / Pre-notification (promote)
- Active resources: $OCF_RESKEY_notify_active_resource
minus $OCF_RESKEY_notify_stop_resource
plus $OCF_RESKEY_notify_start_resource
- Master resources: $OCF_RESKEY_notify_master_resource
minus $OCF_RESKEY_notify_demote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
minus $OCF_RESKEY_notify_stop_resource
plus $OCF_RESKEY_notify_start_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
plus $OCF_RESKEY_notify_stop_resource
minus $OCF_RESKEY_notify_start_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: $OCF_RESKEY_notify_start_resource
- Resources that were promoted: NA
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: $OCF_RESKEY_notify_stop_resource
- Active resources: $OCF_RESKEY_notify_active_resource
- Post-notification (promote)
- Active resources: $OCF_RESKEY_notify_active_resource
minus $OCF_RESKEY_notify_stop_resource
plus $OCF_RESKEY_notify_start_resource
- Master resources: $OCF_RESKEY_notify_master_resource
minus $OCF_RESKEY_notify_demote_resource
plus $OCF_RESKEY_notify_promote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
minus $OCF_RESKEY_notify_stop_resource
plus $OCF_RESKEY_notify_start_resource
minus $OCF_RESKEY_notify_promote_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
plus $OCF_RESKEY_notify_stop_resource
minus $OCF_RESKEY_notify_start_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: $OCF_RESKEY_notify_start_resource
- Resources that were promoted: $OCF_RESKEY_notify_promote_resource
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: $OCF_RESKEY_notify_stop_resource
- Active resources: $OCF_RESKEY_notify_active_resource
Advanced Example
Location Constraints for Masters
It is sometimes desirable to preferentially constrain the master instance of a master/slave resource to run on a particular node or set of nodes in the cluster.
The CIB snippet below will add prefer the node ace to be master of the resource ms-drbd1 by 100 points.
<rsc_location id="loc:ms-drbd1_likes_ace" rsc="ms-drbd1">
<rule id="rule:ms-drbd1_likes_ace" role="master" score="100">
<expression attribute="#uname" operation="eq" value="ace"/>
</rule>
</rsc_location>
Location Constraints for Slaves
It is sometimes desirable to preferentially constrain the slave instance of a master/slave resource to run on a particular node or set of nodes in the cluster.
However, since all resources go through slave status before becoming promoted, what we have to do is constrain it to avoid becoming master.
The CIB snippet below will prefer to promote any node but fred to be master of the resource drbd1 by 100 points. This is effectively the same as saying "we want fred to avoid becoming master if possible".
<rsc_location id="loc:drbd1_likes_fred" rsc="drbd1">
<rule id="rule:drbd1_likes_fred" role="master" score="100">
<expression attribute="#uname" operation="ne" value="fred">
</rule>
</rsc_location>
Co-locational and Ordering Constraints related to Masters
It is very common to want to have a certain resource run only on a node which is master for a given resource.
You can either make sure the resource is run on the master, which is the propably more common approach:
<rsc_order id="ms-drbd0_before_fs0" from="fs0" action="start" to="ms-drbd0" to_action="promote"/> <rsc_colocation id="fs0_on_ms-drbd0" to="ms-drbd0" to_role="master" from="fs0" score="infinity"/>
Translation:
- Start fs0 after ms-drbd0 has been promoted
- Only start fs0 when ms-drbd0 is running as master
Or you can make sure the resource is not run on the slave:
<rsc_order id="ms-drbd0_before_fs0" from="fs0" action="start" to="ms-drbd0" to_action="promote"/> <rsc_colocation id="fs0_not_on_stopped_ms-drbd0" to="ms-drbd0" to_role="stopped" from="fs0" score="-infinity"/> <rsc_colocation id="fs0_not_on_slave_ms-drbd0" to="ms-drbd0" to_role="slave" from="fs0" score="-infinity"/>
Translation:
Never run fs0 on a node where ms-drbd0 is stopped.
Never run fs0 on a node where ms-drbd0 is running as slave.
Only start fs0 after promoting ms-drbd0 to master.
Since the the total set of states for a master/slave resource is {stopped, slave, master}, this only allows it to run on a node which is running as master.
Co-locational and Ordering Constraints related to Slaves
The idiom to use if you have a resource which you want to run on the slave node is similar to the colocation with master case:
<rsc_colocation id="fs_on_drbd0" to="drbd0-partition" to_role="stopped" from="mount-drbd0" score="-infinity"/> <rsc_colocation id="fs_on_drbd0" to="drbd0-partition" to_role="master" from="mount-drbd0" score="-infinity"/>
Translation:
Never run fs_on_drbd0 on a node where drbd0-partition is stopped.
Never run fs_on_drbd0 on a node where drbd0-partition is running as master.
Since the the total set of states for a master/slave resource is {stopped, slave, master}, this only allows the resource to run on a node which is running as slave.
Question:
I don't know how to write the corresponding ordering rule for this case - to ensure that it only runs after being started OR after being demoted from master. Is it necessary? If it isn't necessary, then probably the ordering rule for the master case isn't needed either.