diff options
8 files changed, 135 insertions, 92 deletions
diff --git a/playbooks/provisioning/openstack/README.md b/playbooks/provisioning/openstack/README.md index 267176eec..ab1513a73 100644 --- a/playbooks/provisioning/openstack/README.md +++ b/playbooks/provisioning/openstack/README.md @@ -305,82 +305,105 @@ In order to set a custom entrypoint, update `openshift_master_cluster_public_hos Note than an empty hostname does not work, so if your domain is `openshift.example.com`, you cannot set this value to simply `openshift.example.com`. -### Use an existing Cinder volume for the OpenShift registry +### Creating and using a Cinder volume for the OpenShift registry -You can optionally use an existing Cinder volume for the storage of -your OpenShift registry. +You can optionally have the playbooks create a Cinder volume and set +it up as the OpenShift hosted registry. -To do that, you need to have a Cinder volume (you can create one by -running: +To do that you need specify the desired Cinder volume name and size in +Gigabytes in `inventory/group_vars/all.yml`: - openstack volume create --size <volume size in gb> <volume name> + cinder_hosted_registry_name: cinder-registry + cinder_hosted_registry_size_gb: 10 -The volume needs to have a file system created before you put it to -use. We can do prepare it for you if you put this in inventory/group_vars/all.yml: +With this, the playbooks will create the volume and set up its +filesystem. If there is an existing volume of the same name, we will +use it but keep the existing data on it. - prepare_and_format_registry_volume: true - -**NOTE:** doing so **will destroy any data that's currently on the volume**! - -You can also run the registry setup playbook directly: - - ansible-playbook -i inventory playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml - -(the provisioning phase must be completed, first) +To use the volume for the registry, you must first configure it with +the OpenStack credentials by putting the following to `OSEv3.yml`: + openshift_cloudprovider_openstack_username: "{{ lookup('env','OS_USERNAME') }}" + openshift_cloudprovider_openstack_password: "{{ lookup('env','OS_PASSWORD') }}" + openshift_cloudprovider_openstack_auth_url: "{{ lookup('env','OS_AUTH_URL') }}" + openshift_cloudprovider_openstack_tenant_name: "{{ lookup('env','OS_TENANT_NAME') }}" -To instruct OpenShift to actually use the volume, you must first configure it -with the OpenStack credentials by putting the following to `OSEv3.yml`: - - ## Openstack credentials - #openshift_cloudprovider_kind=openstack - #openshift_cloudprovider_openstack_auth_url=http://openstack.example.com:35357/v2.0/ - #openshift_cloudprovider_openstack_username=username - #openshift_cloudprovider_openstack_password=password - #openshift_cloudprovider_openstack_domain_id=domain_id - #openshift_cloudprovider_openstack_domain_name=domain_name - #openshift_cloudprovider_openstack_tenant_id=tenant_id - #openshift_cloudprovider_openstack_tenant_name=tenant_name - #openshift_cloudprovider_openstack_region=region - -Note that these credentials may be different from the ones you used for -provisioning (say for quota or access control reasons). To use the same -OpenStack credentials for both, take a look at the `sample-inventory`. It shows -how to read the values from your shell environment. - -Make sure to only set the values you need from (e.g. your keystonerc or -clouds.yaml). Some of the options ar keystone V2 or V3 specific. +This will use the credentials from your shell environment. If you want +to enter them explicitly, you can. You can also use credentials +different from the provisioning ones (say for quota or access control +reasons). **NOTE**: If you're testing this on (DevStack)[devstack], you must explicitly set your Keystone API version to v2 (e.g. -`OS_AUTH_URL=http://10.20.30.40/identity/v2.0`) instead of the default +`OS_AUTH_URL=http://10.34.37.47/identity/v2.0`) instead of the default value provided by `openrc`. You may also encounter the following issue with Cinder: https://github.com/kubernetes/kubernetes/issues/50461 +You can read the (OpenShift documentation on configuring +OpenStack)[openstack] for more information. [devstack]: https://docs.openstack.org/devstack/latest/ +[openstack]: https://docs.openshift.org/latest/install_config/configuring_openstack.html -You can read the (OpenShift documentation on configuring -OpenStack)[openstack] for more information. +Next, we need to instruct OpenShift to use the Cinder volume for it's +registry. Again in `OSEv3.yml`: -[openstack]: https://docs.openshift.org/latest/install_config/configuring_openstack.html + #openshift_hosted_registry_storage_kind: openstack + #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce'] + #openshift_hosted_registry_storage_openstack_filesystem: xfs +The filesystem value here will be used in the initial formatting of +the volume. -Next we need to instruct openshift-ansible to use the Cinder volume -for it's registry. Again in `OSEv3.yml`: - ## Use Cinder volume for Openshift registry: +### Use an existing Cinder volume for the OpenShift registry + +You can also use a pre-existing Cinder volume for the storage of your +OpenShift registry. + +To do that, you need to have a Cinder volume. You can create one by +running: + + openstack volume create --size <volume size in gb> <volume name> + +The volume needs to have a file system created before you put it to +use. + +As with the automatically-created volume, you have to set up the +OpenStack credentials in `inventory/group_vars/OSEv3.yml` as well as +registry values: + #openshift_hosted_registry_storage_kind: openstack #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce'] #openshift_hosted_registry_storage_openstack_filesystem: xfs #openshift_hosted_registry_storage_openstack_volumeID: e0ba2d73-d2f9-4514-a3b2-a0ced507fa05 #openshift_hosted_registry_storage_volume_size: 10Gi -The **Cinder volume ID**, **filesystem** and **volume size** variables must -correspond to the values in your volume. +Note the `openshift_hosted_registry_storage_openstack_volumeID` and +`openshift_hosted_registry_storage_volume_size` values: these need to +be added in addition to the previous variables. + +The **Cinder volume ID**, **filesystem** and **volume size** variables +must correspond to the values in your volume. The volume ID must be +the **UUID** of the Cinder volume, *not its name*. + +We can do formate the volume for you if you ask for it in +`inventory/group_vars/all.yml`: + + prepare_and_format_registry_volume: true + +**NOTE:** doing so **will destroy any data that's currently on the volume**! + +You can also run the registry setup playbook directly: + + ansible-playbook -i inventory playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml + +(the provisioning phase must be completed, first) + + ### Configure static inventory and access via a bastion node diff --git a/playbooks/provisioning/openstack/post-provision-openstack.yml b/playbooks/provisioning/openstack/post-provision-openstack.yml index 116eb1244..61f950c14 100644 --- a/playbooks/provisioning/openstack/post-provision-openstack.yml +++ b/playbooks/provisioning/openstack/post-provision-openstack.yml @@ -86,4 +86,7 @@ - node-network-manager - include: prepare-and-format-cinder-volume.yaml - when: prepare_and_format_registry_volume|default(False) + when: > + prepare_and_format_registry_volume|default(False) or + (cinder_registry_volume is defined and + cinder_registry_volume.changed|default(False)) diff --git a/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml b/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml index 2d630f79d..30e094459 100644 --- a/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml +++ b/playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml @@ -18,56 +18,48 @@ attached_device: >- {{ volume_attachment['attachments']|json_query("[?volume_id=='" + cinder_volume + "'].device | [0]") }} + - delegate_to: "{{ groups['masters'][0] }}" + block: + - name: Wait for the device to appear + wait_for: path={{ attached_device }} -- hosts: masters[0] - gather_facts: False - become: True - tasks: - - name: Wait for the device to appear - wait_for: path={{ hostvars['localhost'].attached_device }} - - - name: Create a temp directory for mounting the volume - tempfile: - prefix: cinder-volume - state: directory - register: cinder_mount_dir + - name: Create a temp directory for mounting the volume + tempfile: + prefix: cinder-volume + state: directory + register: cinder_mount_dir - - name: Format the device - filesystem: - fstype: "{{ openshift_hosted_registry_storage_openstack_filesystem }}" - dev: "{{ hostvars['localhost'].attached_device }}" + - name: Format the device + filesystem: + fstype: "{{ cinder_fs }}" + dev: "{{ attached_device }}" - - name: Mount the device - mount: - name: "{{ cinder_mount_dir.path }}" - src: "{{ hostvars['localhost'].attached_device }}" - state: mounted - fstype: "{{ openshift_hosted_registry_storage_openstack_filesystem }}" + - name: Mount the device + mount: + name: "{{ cinder_mount_dir.path }}" + src: "{{ attached_device }}" + state: mounted + fstype: "{{ cinder_fs }}" - - name: Change mode on the filesystem - file: - path: "{{ cinder_mount_dir.path }}" - state: directory - recurse: true - mode: 0777 - - - name: Unmount the device - mount: - name: "{{ cinder_mount_dir.path }}" - src: "{{ hostvars['localhost'].attached_device }}" - state: absent - fstype: "{{ openshift_hosted_registry_storage_openstack_filesystem }}" + - name: Change mode on the filesystem + file: + path: "{{ cinder_mount_dir.path }}" + state: directory + recurse: true + mode: 0777 - - name: Delete the temp directory - file: - name: "{{ cinder_mount_dir.path }}" - state: absent + - name: Unmount the device + mount: + name: "{{ cinder_mount_dir.path }}" + src: "{{ attached_device }}" + state: absent + fstype: "{{ cinder_fs }}" + - name: Delete the temp directory + file: + name: "{{ cinder_mount_dir.path }}" + state: absent -- hosts: localhost - gather_facts: False - become: False - tasks: - name: Detach the volume from the VM os_server_volume: state: absent diff --git a/playbooks/provisioning/openstack/provision-openstack.yml b/playbooks/provisioning/openstack/provision-openstack.yml index 6ec944d56..e4705bd2c 100644 --- a/playbooks/provisioning/openstack/provision-openstack.yml +++ b/playbooks/provisioning/openstack/provision-openstack.yml @@ -8,6 +8,10 @@ - include: pre_tasks.yml roles: - role: openstack-stack + - role: openstack-create-cinder-registry + when: + - cinder_hosted_registry_name is defined + - cinder_hosted_registry_size_gb is defined - role: static_inventory when: openstack_inventory|default('static') == 'static' inventory_path: "{{ openstack_inventory_path|default(inventory_dir) }}" diff --git a/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml b/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml index 874ea7126..7d7683c62 100644 --- a/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml +++ b/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml @@ -26,6 +26,10 @@ openshift_hosted_registry_wait: True #openshift_hosted_registry_storage_kind: openstack #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce'] #openshift_hosted_registry_storage_openstack_filesystem: xfs + +## Configure this if you're attaching a Cinder volume you've set up. +## If you're using the `cinder_hosted_registry_name` option from +## `all.yml`, this will be configured automaticaly. #openshift_hosted_registry_storage_openstack_volumeID: e0ba2d73-d2f9-4514-a3b2-a0ced507fa05 #openshift_hosted_registry_storage_volume_size: 10Gi diff --git a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml index 2e73d2e26..bc186a6b8 100644 --- a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml +++ b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml @@ -62,6 +62,12 @@ openstack_default_flavor: "m1.medium" #docker_lb_volume_size: "5" docker_volume_size: "15" + +## Create a Cinder volume and use it for the OpenShift registry. +## NOTE: the openstack credentials and hosted registry options must be set in OSEv3.yml! +#cinder_hosted_registry_name: cinder-registry +#cinder_hosted_registry_size_gb: 10 + ## Set up a filesystem on the cinder volume specified in `OSEv3.yaml`. ## You need to specify the file system and volume ID in OSEv3 via ## `openshift_hosted_registry_storage_openstack_filesystem` and diff --git a/roles/openstack-create-cinder-registry/tasks/main.yaml b/roles/openstack-create-cinder-registry/tasks/main.yaml new file mode 100644 index 000000000..6e9d1c2e7 --- /dev/null +++ b/roles/openstack-create-cinder-registry/tasks/main.yaml @@ -0,0 +1,5 @@ +--- +- os_volume: + display_name: "{{ cinder_hosted_registry_name }}" + size: "{{ cinder_hosted_registry_size_gb }}" + register: cinder_registry_volume diff --git a/roles/static_inventory/templates/inventory.j2 b/roles/static_inventory/templates/inventory.j2 index 987c98ec6..640a46ba2 100644 --- a/roles/static_inventory/templates/inventory.j2 +++ b/roles/static_inventory/templates/inventory.j2 @@ -42,10 +42,16 @@ etcd lb # Set variables common for all OSEv3 hosts -#[OSEv3:vars] +[OSEv3:vars] # For OSEv3 normal group vars, see ./group_vars/OSEv3.yml +{% if cinder_registry_volume is defined %} +openshift_hosted_registry_storage_openstack_volumeID="{{ cinder_registry_volume.id }}" +openshift_hosted_registry_storage_volume_size="{{ cinder_registry_volume.volume.size }}Gi" +{% endif %} + + # Host Groups [masters:children] |