Removing an Analytics Node

To remove an Analytics node (Vertica+Application) from a cluster, follow the documentation provided by HP on removing a Vertica node, removing all NGINX upstream references, and removing references in API configurations.

You cannot remove nodes that are critical for K-safety. For more information, see HP Vertica documentation.

These instructions apply to Momentum 4.2.1.x, where x > or = 11

1. From the first platform node, remove the application node from the "upstream api_webhooks" stanza in the /opt/msys/3rdParty/nginx/conf.d/api_webhooks_upstream.conf file.

2. Set the node variable to the FQDN of the application node being removed. Do the same for the other MTA nodes in the cluster. (This example assumes there are three platform nodes in the cluster, but all should be provided here.)

   export NODE=remove.app.node.fully.qualified.name;
   export MTA2=plat2.fully.qualified.name;
   export MTA3=plat3.fully.qualified.name;
   

3. Update the production.json files on all platform nodes.

   export PATH=/opt/msys/3rdParty/bin/:$PATH
   for f in \
     /opt/msys/app/metrics-etl/config/production.json \
     /opt/msys/app/webhooks-etl/config/production.json \
     /opt/msys/app/adaptive-delivery-etl/config/production.json \
     ; do
     tmpf=$(mktemp --tmpdir=$(dirname $f) -t)
     jq "del(.vertica.hosts[] | select(.==\"$NODE\"))" -a -M $f > $tmpf
     if [ $? = 0 ]; then
       mv -f $tmpf $f
       echo "$f updated";
     fi
   done
   

4. Copy the modified configuration files to all remaining platform nodes (provide more or fewer hostname variables as needed).

   for f in \
     /opt/msys/3rdParty/nginx/conf.d/api_webhooks_upstream.conf \
     /opt/msys/app/metrics-etl/config/production.json \
     /opt/msys/app/webhooks-etl/config/production.json \
     /opt/msys/app/adaptive-delivery-etl/config/production.json \
     ; do
     scp $f $MTA2:$f
     scp $f $MTA3:$f
   done
   

5. Restart all affected services on each platform node.

   service msys-app-metrics-etl restart
   service msys-app-webhooks-etl restart
   service msys-nginx reload
   

1. Remove the application node from the "upstream" stanzas in the following nginx/conf.d files on the first application node.

   • /opt/msys/3rdParty/nginx/conf.d/api_adaptive_delivery_upstream.conf

   • /opt/msys/3rdParty/nginx/conf.d/api_webhooks_upstream.conf

   • /opt/msys/3rdParty/nginx/conf.d/api_users_upstream.conf

   • /opt/msys/3rdParty/nginx/conf.d/api_metrics_upstream.conf

   • /opt/msys/3rdParty/nginx/conf.d/webui_upstream.conf

2. Set the node variable to the FQDN of the application node being removed. Do the same for the other application nodes in the cluster. (This example assumes there are currently four application nodes in the cluster, but all should be provided here.)

   export NODE=remove.app.node.fully.qualified.name; 
   export NODE2=app2.fully.qualified.name; 
   export NODE3=app3.fully.qualified.name;
   

3. Update the production.json files on all application nodes.

   export PATH=/opt/msys/3rdParty/bin/:$PATH
   for f in \
     /opt/msys/app/webhooks-api/config/production.json \
     /opt/msys/app/adaptive-delivery-api/config/production.json \
     /opt/msys/app/adaptive-delivery-api/cron/config/production.json \
     /opt/msys/app/metrics-api/config/production.json \
     /opt/msys/app/metrics-api/cron/config/production.json \
     ; do
     tmpf=$(mktemp --tmpdir=$(dirname $f) -t)
     jq "del(.vertica.hosts[] | select(.==\"$NODE\"))" -a -M $f > $tmpf
     if [ $? = 0 ]; then
       mv -f $tmpf $f
       echo "$f updated";
     fi
   done
   

4. Copy the modified configuration files to the original Analytics nodes.

   for f in \
     /opt/msys/3rdParty/nginx/conf.d/api_adaptive_delivery_upstream.conf \
     /opt/msys/3rdParty/nginx/conf.d/api_webhooks_upstream.conf \
     /opt/msys/3rdParty/nginx/conf.d/api_users_upstream.conf \
     /opt/msys/3rdParty/nginx/conf.d/api_metrics_upstream.conf \
     /opt/msys/3rdParty/nginx/conf.d/webui_upstream.conf \
     /opt/msys/app/webhooks-api/config/production.json \
     /opt/msys/app/adaptive-delivery-api/config/production.json \
     /opt/msys/app/adaptive-delivery-api/cron/config/production.json \
     /opt/msys/app/metrics-api/config/production.json \
     /opt/msys/app/metrics-api/cron/config/production.json \
     ; do
     scp $f $NODE2:$f
     scp $f $NODE3:$f
   done
   

5. Restart the services on all remaining application nodes.

   service msys-app-metrics-api restart
   service msys-app-users-api restart
   service msys-app-webhooks-api restart
   service msys-nginx reload
   

1. Follow instructions provided by HP to remove a Vertica node from a cluster.

2. Perform backups as necessary.