Upscaling OpenShift application nodes (#571)

* scale-up: playbook for upscaling app nodes

* scale-up: removed debug

* scale-up: made suggested changes

* scale-up: indentation fix

* upscaling: process split into two playbooks that are executed by a bash script

- upscaling_run.sh: bash script, usage displayed using -h parameter
- upscaling_pre-tasks: check that new value is higher, change inventory variable
- upscaling_scale-up: rerun provisioning and installation, verify change

* upscaling_run: fixed openshift-ansible-contrib directory name

* upscaling_run: inventory can be entered as relative path

* upscaling_scale-up: fixed formatting

* upscaling: minor changes

* upscaling: moved to .../provisioning/openstack directory, README updated, minor changes made

* README: minor changes

* README: formatting

* uspcaling: minor fix

* upscaling: fix

* upscaling: added customisations, fixes

- openshift-ansible-contrib and openshift-ansible paths are customisable
- fixed implicit incrementation by 1

* upscaling: fixes

* upscaling: fixes

* upscaling: another fix

* upscaling: another fix

* upscaling: fix

* upscaling: back to a single playbook, README updated

* minor fix

* pre_tasks: added labels for autoscaling

* scale-up: fixes

* scale-up: fixed host variables, post-verification is only based on labels

* scale-up: added openshift-ansible path customisation

- path has to be absolute, cannot contain '/' at the end

* scale-up: fix

* scale-up: debug removed

* README: added docs on openshift_ansible_dir, note about bastion

* static_inventory: newly added nodes are added to new_nodes group

- note: re-running provisioning fails when trying to install docker

* removing new line

* scale-up: running byo/config.yml or scaleup.yml based on the situation

- (whether there is an existing deployment or not)

* openstack.yml: indentation fix

* added refresh inventory

* upscaling: new_nodes only contains new does, it is not used during the first deployment

* static_inventory: make sure that new nodes end up only in their new_nodes group

* bug fixes

* another fix

* fixed condition

* scale-up, static_inventory role: all app node data gathered before provisioning

* upscaling: bug fixes

* upscaling: another fixes

* fixes

* upscaling: fix

* upscaling: fix

* upscaling: another logic fix

* bug fix for non-scaling deployments

1 files changed, 75 insertions, 0 deletions

diff --git a/playbooks/provisioning/openstack/scale-up.yaml b/playbooks/provisioning/openstack/scale-up.yaml
new file mode 100644
index 000000000..79fc09050
--- /dev/null
+++ b/playbooks/provisioning/openstack/scale-up.yaml
@@ -0,0 +1,75 @@
+---
+# Get the needed information about the current deployment
+- hosts: masters[0]
+  tasks:
+  - name: Get number of app nodes
+    shell: oc get nodes -l autoscaling=app --no-headers=true | wc -l
+    register: oc_old_num_nodes
+  - name: Get names of app nodes
+    shell: oc get nodes -l autoscaling=app --no-headers=true | cut -f1 -d " "
+    register: oc_old_app_nodes
+
+- hosts: localhost
+  tasks:
+  # Since both number and names of app nodes are to be removed
+  # localhost variables for these values need to be set
+  - name: Store old number and names of app nodes locally (if there is an existing deployment)
+    when: '"masters" in groups'
+    register: set_fact_result
+    set_fact:
+      oc_old_num_nodes: "{{ hostvars[groups['masters'][0]]['oc_old_num_nodes'].stdout }}"
+      oc_old_app_nodes: "{{ hostvars[groups['masters'][0]]['oc_old_app_nodes'].stdout_lines }}"
+
+  - name: Set default values for old app nodes (if there is no existing deployment)
+    when: 'set_fact_result | skipped'
+    set_fact:
+      oc_old_num_nodes: 0
+      oc_old_app_nodes: []
+
+  # Set how many nodes are to be added (1 by default)
+  - name: Set how many nodes are to be added
+    set_fact:
+      increment_by: 1
+  - name: Check that the number corresponds to scaling up (not down)
+    assert:
+      that: 'increment_by | int >= 1'
+      msg: >
+        FAIL: The value of increment_by must be at least 1
+        (but it is {{ increment_by | int }}).
+  - name: Update openstack_num_nodes variable
+    set_fact:
+      openstack_num_nodes: "{{ oc_old_num_nodes | int + increment_by | int }}"
+
+# Run provision.yaml with higher number of nodes to create a new app-node VM
+- include: provision.yaml
+
+# Run config.yml to perform openshift installation
+# Path to openshift-ansible can be customised:
+# - the value of openshift_ansible_dir has to be an absolute path
+# - the path cannot contain the '/' symbol at the end
+
+# Creating a new deployment by the full installation
+- include: "{{ openshift_ansible_dir }}/playbooks/byo/config.yml"
+  vars:
+    openshift_ansible_dir: ../../../../openshift-ansible
+  when: 'not groups["new_nodes"] | list'
+
+# Scaling up existing deployment
+- include: "{{ openshift_ansible_dir }}/playbooks/byo/openshift-node/scaleup.yml"
+  vars:
+    openshift_ansible_dir: ../../../../openshift-ansible
+  when: 'groups["new_nodes"] | list'
+
+# Post-verification: Verify new number of nodes
+- hosts: masters[0]
+  tasks:
+  - name: Get number of nodes
+    shell: oc get nodes -l autoscaling=app --no-headers=true | wc -l
+    register: oc_new_num_nodes
+  - name: Check that the actual result matches the defined value
+    assert:
+      that: 'oc_new_num_nodes.stdout | int == (hostvars["localhost"]["oc_old_num_nodes"] | int + hostvars["localhost"]["increment_by"] | int)'
+      msg: >
+        FAIL: Number of application nodes has not been increased accordingly
+        (it should be {{ hostvars["localhost"]["oc_old_num_nodes"] | int + hostvars["localhost"]["increment_by"] | int }}
+        but it is {{ oc_new_num_nodes.stdout | int }}).