Merge pull request #728 from tomassedovic/prebuild-cinder-registry

Pre-create a Cinder registry volume

6 files changed, 123 insertions, 91 deletions

diff --git a/playbooks/provisioning/openstack/README.md b/playbooks/provisioning/openstack/README.md
index 267176eec..ab1513a73 100644
--- a/playbooks/provisioning/openstack/README.md
+++ b/playbooks/provisioning/openstack/README.md
@@ -305,82 +305,105 @@ In order to set a custom entrypoint, update `openshift_master_cluster_public_hos
 Note than an empty hostname does not work, so if your domain is `openshift.example.com`,
 you cannot set this value to simply `openshift.example.com`.
 
-### Use an existing Cinder volume for the OpenShift registry
+### Creating and using a Cinder volume for the OpenShift registry
 
-You can optionally use an existing Cinder volume for the storage of
-your OpenShift registry.
+You can optionally have the playbooks create a Cinder volume and set
+it up as the OpenShift hosted registry.
 
-To do that, you need to have a Cinder volume (you can create one by
-running:
+To do that you need specify the desired Cinder volume name and size in
+Gigabytes in `inventory/group_vars/all.yml`:
 
-    openstack volume create --size <volume size in gb> <volume name>
+    cinder_hosted_registry_name: cinder-registry
+    cinder_hosted_registry_size_gb: 10
 
-The volume needs to have a file system created before you put it to
-use. We can do prepare it for you if you put this in inventory/group_vars/all.yml:
+With this, the playbooks will create the volume and set up its
+filesystem. If there is an existing volume of the same name, we will
+use it but keep the existing data on it.
 
-    prepare_and_format_registry_volume: true
-
-**NOTE:** doing so **will destroy any data that's currently on the volume**!
-
-You can also run the registry setup playbook directly:
-
-   ansible-playbook -i inventory playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml
-
-(the provisioning phase must be completed, first)
+To use the volume for the registry, you must first configure it with
+the OpenStack credentials by putting the following to `OSEv3.yml`:
 
+    openshift_cloudprovider_openstack_username: "{{ lookup('env','OS_USERNAME') }}"
+    openshift_cloudprovider_openstack_password: "{{ lookup('env','OS_PASSWORD') }}"
+    openshift_cloudprovider_openstack_auth_url: "{{ lookup('env','OS_AUTH_URL') }}"
+    openshift_cloudprovider_openstack_tenant_name: "{{ lookup('env','OS_TENANT_NAME') }}"
 
-To instruct OpenShift to actually use the volume, you must first configure it
-with the OpenStack credentials by putting the following to `OSEv3.yml`:
-
-    ## Openstack credentials
-    #openshift_cloudprovider_kind=openstack
-    #openshift_cloudprovider_openstack_auth_url=http://openstack.example.com:35357/v2.0/
-    #openshift_cloudprovider_openstack_username=username
-    #openshift_cloudprovider_openstack_password=password
-    #openshift_cloudprovider_openstack_domain_id=domain_id
-    #openshift_cloudprovider_openstack_domain_name=domain_name
-    #openshift_cloudprovider_openstack_tenant_id=tenant_id
-    #openshift_cloudprovider_openstack_tenant_name=tenant_name
-    #openshift_cloudprovider_openstack_region=region
-
-Note that these credentials may be different from the ones you used for
-provisioning (say for quota or access control reasons). To use the same
-OpenStack credentials for both, take a look at the `sample-inventory`. It shows
-how to read the values from your shell environment.
-
-Make sure to only set the values you need from (e.g. your keystonerc or
-clouds.yaml). Some of the options ar keystone V2 or V3 specific.
+This will use the credentials from your shell environment. If you want
+to enter them explicitly, you can. You can also use credentials
+different from the provisioning ones (say for quota or access control
+reasons).
 
 **NOTE**: If you're testing this on (DevStack)[devstack], you must
 explicitly set your Keystone API version to v2 (e.g.
-`OS_AUTH_URL=http://10.20.30.40/identity/v2.0`) instead of the default
+`OS_AUTH_URL=http://10.34.37.47/identity/v2.0`) instead of the default
 value provided by `openrc`. You may also encounter the following issue
 with Cinder:
 
 https://github.com/kubernetes/kubernetes/issues/50461
 
+You can read the (OpenShift documentation on configuring
+OpenStack)[openstack] for more information.
 
 [devstack]: https://docs.openstack.org/devstack/latest/
+[openstack]: https://docs.openshift.org/latest/install_config/configuring_openstack.html
 
 
-You can read the (OpenShift documentation on configuring
-OpenStack)[openstack] for more information.
+Next, we need to instruct OpenShift to use the Cinder volume for it's
+registry. Again in `OSEv3.yml`:
 
-[openstack]: https://docs.openshift.org/latest/install_config/configuring_openstack.html
+    #openshift_hosted_registry_storage_kind: openstack
+    #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce']
+    #openshift_hosted_registry_storage_openstack_filesystem: xfs
 
+The filesystem value here will be used in the initial formatting of
+the volume.
 
-Next we need to instruct openshift-ansible to use the Cinder volume
-for it's registry. Again in `OSEv3.yml`:
 
-    ## Use Cinder volume for Openshift registry:
+### Use an existing Cinder volume for the OpenShift registry
+
+You can also use a pre-existing Cinder volume for the storage of your
+OpenShift registry.
+
+To do that, you need to have a Cinder volume. You can create one by
+running:
+
+    openstack volume create --size <volume size in gb> <volume name>
+
+The volume needs to have a file system created before you put it to
+use.
+
+As with the automatically-created volume, you have to set up the
+OpenStack credentials in `inventory/group_vars/OSEv3.yml` as well as
+registry values:
+
     #openshift_hosted_registry_storage_kind: openstack
     #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce']
     #openshift_hosted_registry_storage_openstack_filesystem: xfs
     #openshift_hosted_registry_storage_openstack_volumeID: e0ba2d73-d2f9-4514-a3b2-a0ced507fa05
     #openshift_hosted_registry_storage_volume_size: 10Gi
 
-The **Cinder volume ID**, **filesystem** and **volume size** variables must
-correspond to the values in your volume.
+Note the `openshift_hosted_registry_storage_openstack_volumeID` and
+`openshift_hosted_registry_storage_volume_size` values: these need to
+be added in addition to the previous variables.
+
+The **Cinder volume ID**, **filesystem** and **volume size** variables
+must correspond to the values in your volume. The volume ID must be
+the **UUID** of the Cinder volume, *not its name*.
+
+We can do formate the volume for you if you ask for it in
+`inventory/group_vars/all.yml`:
+
+    prepare_and_format_registry_volume: true
+
+**NOTE:** doing so **will destroy any data that's currently on the volume**!
+
+You can also run the registry setup playbook directly:
+
+   ansible-playbook -i inventory playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml
+
+(the provisioning phase must be completed, first)
+
+
 
 ### Configure static inventory and access via a bastion node
 
diff --git a/playbooks/provisioning/openstack/post-provision-openstack.yml b/playbooks/provisioning/openstack/post-provision-openstack.yml
index 116eb1244..61f950c14 100644
--- a/playbooks/provisioning/openstack/post-provision-openstack.yml
+++ b/playbooks/provisioning/openstack/post-provision-openstack.yml
@@ -86,4 +86,7 @@
     - node-network-manager
 
 - include: prepare-and-format-cinder-volume.yaml
-  when: prepare_and_format_registry_volume|default(False)
+  when: >
+    prepare_and_format_registry_volume|default(False) or
+    (cinder_registry_volume is defined and
+      cinder_registry_volume.changed|default(False))
diff --git a/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml b/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml
index 2d630f79d..30e094459 100644
--- a/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml
+++ b/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml
@@ -18,56 +18,48 @@
       attached_device: >-
         {{ volume_attachment['attachments']|json_query("[?volume_id=='" + cinder_volume + "'].device | [0]") }}
 
+  - delegate_to: "{{ groups['masters'][0] }}"
+    block:
+    - name: Wait for the device to appear
+      wait_for: path={{ attached_device }}
 
-- hosts: masters[0]
-  gather_facts: False
-  become: True
-  tasks:
-  - name: Wait for the device to appear
-    wait_for: path={{ hostvars['localhost'].attached_device }}
-
-  - name: Create a temp directory for mounting the volume
-    tempfile:
-      prefix: cinder-volume
-      state: directory
-    register: cinder_mount_dir
+    - name: Create a temp directory for mounting the volume
+      tempfile:
+        prefix: cinder-volume
+        state: directory
+      register: cinder_mount_dir
 
-  - name: Format the device
-    filesystem:
-      fstype: "{{ openshift_hosted_registry_storage_openstack_filesystem }}"
-      dev: "{{ hostvars['localhost'].attached_device }}"
+    - name: Format the device
+      filesystem:
+        fstype: "{{ cinder_fs }}"
+        dev: "{{ attached_device }}"
 
-  - name: Mount the device
-    mount:
-      name: "{{ cinder_mount_dir.path }}"
-      src: "{{ hostvars['localhost'].attached_device }}"
-      state: mounted
-      fstype: "{{ openshift_hosted_registry_storage_openstack_filesystem }}"
+    - name: Mount the device
+      mount:
+        name: "{{ cinder_mount_dir.path }}"
+        src: "{{ attached_device }}"
+        state: mounted
+        fstype: "{{ cinder_fs }}"
 
-  - name: Change mode on the filesystem
-    file:
-      path: "{{ cinder_mount_dir.path }}"
-      state: directory
-      recurse: true
-      mode: 0777
-
-  - name: Unmount the device
-    mount:
-      name: "{{ cinder_mount_dir.path }}"
-      src: "{{ hostvars['localhost'].attached_device }}"
-      state: absent
-      fstype: "{{ openshift_hosted_registry_storage_openstack_filesystem }}"
+    - name: Change mode on the filesystem
+      file:
+        path: "{{ cinder_mount_dir.path }}"
+        state: directory
+        recurse: true
+        mode: 0777
 
-  - name: Delete the temp directory
-    file:
-      name: "{{ cinder_mount_dir.path }}"
-      state: absent
+    - name: Unmount the device
+      mount:
+        name: "{{ cinder_mount_dir.path }}"
+        src: "{{ attached_device }}"
+        state: absent
+        fstype: "{{ cinder_fs }}"
 
+    - name: Delete the temp directory
+      file:
+        name: "{{ cinder_mount_dir.path }}"
+        state: absent
 
-- hosts: localhost
-  gather_facts: False
-  become: False
-  tasks:
   - name: Detach the volume from the VM
     os_server_volume:
       state: absent
diff --git a/playbooks/provisioning/openstack/provision-openstack.yml b/playbooks/provisioning/openstack/provision-openstack.yml
index 6ec944d56..e4705bd2c 100644
--- a/playbooks/provisioning/openstack/provision-openstack.yml
+++ b/playbooks/provisioning/openstack/provision-openstack.yml
@@ -8,6 +8,10 @@
     - include: pre_tasks.yml
   roles:
     - role: openstack-stack
+    - role: openstack-create-cinder-registry
+      when:
+        - cinder_hosted_registry_name is defined
+        - cinder_hosted_registry_size_gb is defined
     - role: static_inventory
       when: openstack_inventory|default('static') == 'static'
       inventory_path: "{{ openstack_inventory_path|default(inventory_dir) }}"
diff --git a/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml b/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml
index 874ea7126..7d7683c62 100644
--- a/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml
+++ b/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml
@@ -26,6 +26,10 @@ openshift_hosted_registry_wait: True
 #openshift_hosted_registry_storage_kind: openstack
 #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce']
 #openshift_hosted_registry_storage_openstack_filesystem: xfs
+
+## Configure this if you're attaching a Cinder volume you've set up.
+## If you're using the `cinder_hosted_registry_name` option from
+## `all.yml`, this will be configured automaticaly.
 #openshift_hosted_registry_storage_openstack_volumeID: e0ba2d73-d2f9-4514-a3b2-a0ced507fa05
 #openshift_hosted_registry_storage_volume_size: 10Gi
 
diff --git a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml
index 2e73d2e26..bc186a6b8 100644
--- a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml
+++ b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml
@@ -62,6 +62,12 @@ openstack_default_flavor: "m1.medium"
 #docker_lb_volume_size: "5"
 docker_volume_size: "15"
 
+
+## Create a Cinder volume and use it for the OpenShift registry.
+## NOTE: the openstack credentials and hosted registry options must be set in OSEv3.yml!
+#cinder_hosted_registry_name: cinder-registry
+#cinder_hosted_registry_size_gb: 10
+
 ## Set up a filesystem on the cinder volume specified in `OSEv3.yaml`.
 ## You need to specify the file system and volume ID in OSEv3 via
 ## `openshift_hosted_registry_storage_openstack_filesystem` and