Configuration Management
A Momentum cluster installation is configured using the ecelerity.conf file just like a Momentum single node configuration. However, the ecelerity.conf file in a cluster configuration also includes the ecelerity-cluster.conf file. This file is typically used to specify options common to all nodes—the cluster topology for example.
Note
The eccluster.conf file configures the behavior of the Momentum Cluster Manager, eccmgr only, not cluster nodes. For more information about this file see eccluster.conf.
There are numerous options that are cluster-specific. Some of these options such as cluster_max_outbound_connections are visible in various scopes but most options specific to cluster configuration are defined in the cluster module. The following table shows the options that are exclusive to a cluster configuration:
| Option/Description | Type | Default | Scopes |
|---|---|---|---|
| cluster_group – The DuraVIP™ system coordinates IP ownership responsibilities via the cluster_group extended virtual synchrony group | na | cluster | |
| cluster_max_outbound_connections – Set the maximum number of outbound connections for a domain | sending | binding, domain, global, host | |
| cluster_outbound_throttle_connections – Limit the rate at which connections are established | sending | binding_group, domain, global | |
| cluster_outbound_throttle_messages – Limit the rate at which messages are delivered | sending | binding_group, domain, global | |
| cluster_server_max_outbound_connections – Set the maximum number of outbound connections | sending | binding, binding_group, global | |
| control_group – The cluster console manager utilizes this group to issue cluster-wide configuration commands | na | cluster | |
| “Module Overview” – Set the module debug level (applicable to all modules) | na | cluster | |
| duravip_balance_set_size – When balancing DuraVIP™s, how many to process as a batch in response to a balance request | na | cluster | |
| “DuraVIP™: IP Fail over” – Specify the binding a listener should follow | receiving | listen | |
| “DuraVIP™: IP Fail over” – Specify a DuraVIP™ preference | both | binding, listen | |
| “DuraVIP™: IP Fail over” – Whether to enable Durable MultiVIP© bindings | both | binding, listen | |
| Modules – Whether or not the module is enabled | na | cluster | |
| heartbeat_start_delay – Seconds to wait after startup before the cluster heartbeat is activated | na | cluster | |
| heartbeats_per_sec – How often to send a heartbeat | na | cluster | |
| if_check_interval – How often to run through a maintenance cycle | na | cluster | |
| if_down_limit – How long to wait before deciding to bring an IP online | na | cluster | |
| log_active_interval – Used to tune centralized logging | na | cluster | |
| log_group – Whether or not panic log messages are broadcast over spread | na | cluster | |
| log_idle_interval – The amount of time to sleep before looking for another segment | na | cluster | |
| ecelerity-cluster.conf – Define the location of the cluster manager logs | na | cluster | |
| ecelerity-cluster.conf – The port that the messaging bus listens on | na | cluster | |
| nodeaddr – The canonical cluster address for the node | na | cluster | |
| nodename – Override the node name that is used to canonically identify this cluster node | na | cluster | |
| ecelerity-cluster.conf – Define the cluster replication framework | na | cluster | |
| mbus.conf – The name of the subcluster | na | cluster | |
| “DuraVIP™: IP Fail over” – Define the cluster network topology | na | cluster | |
| unconditional_rebind – Whether the full set_binding logic is invoked or not | na | cluster | |
| view_balance_interval – How often DuraVIP™ views are subject to balancing | na | cluster | |
| view_broadcast_interval – How often to speculatively broadcast a view announcement to the cluster | na | cluster | |
| view_mature_time – How long a DuraVIP™ view needs to remain unchanged before considering it "mature" | na | cluster |
Momentum configuration files are maintained in a version control repository and exported to your cluster network via the ecconfigd service running on the cluster manager. For a detailed description of this service see ecconfigd.
In some situations it is possible to put the configuration replication into a conflicted state. This is usually happens when conflicting changes are made to the configuration for a subcluster on both sides of a network partition. In other words, in a two node cluster, if one of the nodes is unplugged from the network and configuration changes are made and committed on both nodes, when the network cable is re-connected, the configuration will attempt to sync but will notice that conflicting changes have been made.
If such a conflict occurs, you will need to resolve it by manually changing configuration files. For a description of this process see “Best Practices for Manually Changing Configuration Files”.
While ecconfigd is in a conflicted state, no configuration changes will be able to be committed.
On the client side of the configuration management, each node has a working copy checkout of the repository for its subcluster. This checkout is found at /opt/msys/ecelerity/etc/conf.
The tools that operate on the conf directory try very hard to avoid leaving it in a broken state. Every minute, each node will attempt to update its conf directory to match the repository. The update process will only modify the conf directory if the complete revision was able to be pulled—in other words, if the update could not complete, it will never leave a conf directory with a half-applied update.
If you have made local changes to the conf directory, the update will attempt to merge updates from the repository with your changes and will only update the conf directory if there were no conflicting changes.
If conflicting changes were found, ecconfigd will warn you and provide you with instructions on how to resolve the conflict.
In addition to managing the configuration of subclusters, ecconfigd will also manage a special global subcluster. This is mapped in as a sub-directory of the conf directory (/opt/msys/ecelerity/etc/conf/global), a directory that every node has access to. This is the recommended location for sharing cluster-wide configuration information between nodes.
The conf directory is created with the sub-directory, /opt/msys/ecelerity/etc/conf/default. This represents the default configuration for nodes in that subcluster. We recommend that you place your default configuration files within this directory.
If you need to configure a node differently from other nodes, the recommended approach is to create a separate ecelerity.conf file as described in “Copy node configuration”. Doing this creates a copy of the ecelerity.conf file in a sub-directory below /opt/msys/ecelerity/conf that bears the name of the node. The name of this copy is ecelerity.conf.
If you wish, you can also manually create this file by creating a directory below the /opt/msys/ecelerity/etc/conf and copying the ecelerity.conf file from the default directory. You must do this on the cluster manager and use eccfg to commit your changes to the repository.
Note
When you create a node-specific configuration file, a directory bearing the node name and a node-specific ecelerity.conf file are created on all nodes in the cluster.
When manually changing configuration files be sure to follow the best practices described in “Best Practices for Manually Changing Configuration Files”.
After creating node-specific configurations you must stop and restart the ecelerity process. To do this see ec_ctl. config reload will not load configuration changes.
When nodes use common values for a number of options, if you wish you can put these options in a configuration file stored in the global directory rather than repeating them in each /opt/msys/ecelerity/etc/conf/nodename/ecelerity.conf file. However, you must add include statements to the /opt/msys/ecelerity/etc/conf/nodename/ecelerity.conf file on each node.
If you have any configurations specific to a particular node, fallback values for configuration options in that node-local configuration file cannot be included via the /opt/msys/ecelerity/etc/conf/ecelerity.conf file. For an included file, the parent file's path is added to the search path, so if a file is included from /opt/msys/ecelerity/etc/conf/default/ecelerity.conf, the search path becomes:
/opt/msys/ecelerity/etc/conf/default:/opt/msys/ecelerity/etc:/opt/msys/ecelerity/etc/conf/global:»
/opt/msys/ecelerity/etc/conf/nodename:/opt/msys/ecelerity/etc/conf/default
If there are minor differences between nodes, you can include a local node configuration file in the following way. If, for example, you want OPTION = "FOO" on one node but OPTION = "BAR" on another, be sure that the option is left "undefined" in the ecelerity.conf file and OPTION is set in a node-local.conf file in all the /opt/msys/ecelerity/etc/conf/{NODENAME} directories. Be sure to also add an "include node-local.conf" statement to /opt/msys/ecelerity/etc/default/ecelerity.conf.
If there are major differences between node configurations, it is probably simpler to create a separate configuration file for each node as described in “Node-specific Configuration”.