diff options
Diffstat (limited to 'playbooks/provisioning')
9 files changed, 181 insertions, 135 deletions
diff --git a/playbooks/provisioning/openstack/README.md b/playbooks/provisioning/openstack/README.md index 1ff586b49..ae572f9b6 100644 --- a/playbooks/provisioning/openstack/README.md +++ b/playbooks/provisioning/openstack/README.md @@ -10,6 +10,7 @@ etc.). The result is an environment ready for openshift-ansible. * [Ansible-galaxy](https://pypi.python.org/pypi/ansible-galaxy-local-deps) * [jinja2](http://jinja.pocoo.org/docs/2.9/) * [shade](https://pypi.python.org/pypi/shade) +* python-jmespath / [jmespath](https://pypi.python.org/pypi/jmespath) * python-dns / [dnspython](https://pypi.python.org/pypi/dnspython) * Become (sudo) is not required. @@ -40,7 +41,7 @@ Alternatively you can install directly from github: -p openshift-ansible-contrib/roles Notes: -* This assumes we're in the directory that contains the clonned +* This assumes we're in the directory that contains the clonned openshift-ansible-contrib repo in its root path. * When trying to install a different version, the previous one must be removed first (`infra-ansible` directory from [roles](https://github.com/openshift/openshift-ansible-contrib/tree/master/roles)). @@ -62,10 +63,6 @@ Otherwise, even if there are differences between the two versions, installation cp -r openshift-ansible-contrib/playbooks/provisioning/openstack/sample-inventory inventory -### Copy clouds.yaml - - cp openshift-ansible-contrib/playbooks/provisioning/openstack/sample-inventory/clouds.yaml clouds.yaml - ### Copy ansible config cp openshift-ansible-contrib/playbooks/provisioning/openstack/sample-inventory/ansible.cfg ansible.cfg @@ -81,6 +78,10 @@ your servers will be under. With the default values, this will be That sudomain can be set as well by the `openshift_app_domain` variable in the inventory. +The `openstack_<role name>_hostname` is a set of variables used for customising +hostnames of servers with a given role. When such a variable stays commented, +default hostname (usually the role name) is used. + The `public_dns_nameservers` is a list of DNS servers accessible from all the created Nova servers. These will be serving as your DNS forwarders for external FQDNs that do not belong to the cluster's DNS domain and its subdomains. @@ -101,18 +102,29 @@ as a default nameserver that comes from the NetworkManager and cloud-init. `openstack keypair list`. This guide assumes that its corresponding private key is `~/.ssh/openshift`, stored on the ansible admin (control) node. -`openstack_default_image_name` is the name of the Glance image the -servers will use. You can -see your images with `openstack image list`. +`openstack_default_image_name` is the default name of the Glance image the +servers will use. You can see your images with `openstack image list`. +In order to set a different image for a role, uncomment the line with the +corresponding variable (e.g. `openstack_lb_image_name` for load balancer) and +set its value to another available image name. `openstack_default_image_name` +must stay defined as it is used as a default value for the rest of the roles. -`openstack_default_flavor` is the Nova flavor the servers will use. +`openstack_default_flavor` is the default Nova flavor the servers will use. You can see your flavors with `openstack flavor list`. +In order to set a different flavor for a role, uncomment the line with the +corresponding variable (e.g. `openstack_lb_flavor` for load balancer) and +set its value to another available flavor. `openstack_default_flavor` must +stay defined as it is used as a default value for the rest of the roles. `openstack_external_network_name` is the name of the Neutron network providing external connectivity. It is often called `public`, `external` or `ext-net`. You can see your networks with `openstack network list`. +`openstack_private_network_name` is the name of the private Neutron network +providing admin/control access for ansible. It can be merged with other +cluster networks, there are no special requirements for networking. + The `openstack_num_masters`, `openstack_num_infra` and `openstack_num_nodes` values specify the number of Master, Infra and App nodes to create. @@ -123,6 +135,16 @@ cluster node groups, like app or infra nodes. For example: `{'region': 'infra'}` The `openstack_nodes_to_remove` allows you to specify the numerical indexes of App nodes that should be removed; for example, ['0', '2'], +The `docker_volume_size` is the default Docker volume size the servers will use. +In order to set a different volume size for a role, +uncomment the line with the corresponding variable (e. g. `docker_master_volume_size` +for master) and change its value. `docker_volume_size` must stay defined as it is +used as a default value for some of the servers (master, infra, app node). +The rest of the roles (etcd, load balancer, dns) have their defaults hard-coded. + +**Note**: If the `ephemeral_volumes` is set to `true`, the `*_volume_size` variables +will be ignored and the deployment will not create any cinder volumes. + The `openstack_flat_secgrp`, controls Neutron security groups creation for Heat stacks. Set it to true, if you experience issues with sec group rules quotas. It trades security for number of rules, by sharing the same set @@ -177,16 +199,30 @@ variables for the `inventory/group_vars/OSEv3.yml`, `all.yml`: origin_release: 1.5.1 openshift_deployment_type: "{{ deployment_type }}" -### Configure static inventory +### Configure static inventory and access via a bastion node Example inventory variables: + openstack_use_bastion: true + bastion_ingress_cidr: "{{openstack_subnet_prefix}}.0/24" openstack_private_ssh_key: ~/.ssh/openshift openstack_inventory: static openstack_inventory_path: ../../../../inventory + openstack_ssh_config_path: /tmp/ssh.config.openshift.ansible.openshift.example.com + +The `openstack_subnet_prefix` is the openstack private network for your cluster. +And the `bastion_ingress_cidr` defines accepted range for SSH connections to nodes +additionally to the `ssh_ingress_cidr`` (see the security notes above). + +The SSH config will be stored on the ansible control node by the +gitven path. Ansible uses it automatically. To access the cluster nodes with +that ssh config, use the `-F` prefix, f.e.: + ssh -F /tmp/ssh.config.openshift.ansible.openshift.example.com master-0.openshift.example.com echo OK -In this guide, the latter points to the current directory, where you run ansible commands +Note, relative paths will not work for the `openstack_ssh_config_path`, but it +works for the `openstack_private_ssh_key` and `openstack_inventory_path`. In this +guide, the latter points to the current directory, where you run ansible commands from. To verify nodes connectivity, use the command: @@ -194,7 +230,7 @@ To verify nodes connectivity, use the command: ansible -v -i inventory/hosts -m ping all If something is broken, double-check the inventory variables, paths and the -generated `<openstack_inventory_path>/hosts` file. +generated `<openstack_inventory_path>/hosts` and `openstack_ssh_config_path` files. The `inventory: dynamic` can be used instead to access cluster nodes directly via floating IPs. In this mode you can not use a bastion node and should specify @@ -213,6 +249,15 @@ this is how you stat the provisioning process from your ansible control node: Note, here you start with an empty inventory. The static inventory will be populated with data so you can omit providing additional arguments for future ansible commands. +If bastion enabled, the generates SSH config must be applied for ansible. +Otherwise, it is auto included by the previous step. In order to execute it +as a separate playbook, use the following command: + + ansible-playbook openshift-ansible-contrib/playbooks/provisioning/openstack/post-provision-openstack.yml + +The first infra node then becomes a bastion node as well and proxies access +for future ansible commands. The post-provision step also configures Satellite, +if requested, and DNS server, and ensures other OpenShift requirements to be met. ### Install OpenShift @@ -220,6 +265,24 @@ Once it succeeds, you can install openshift by running: ansible-playbook openshift-ansible/playbooks/byo/config.yml +### Access UI + +OpenShift UI may be accessed via the 1st master node FQDN, port 8443. + +When using a bastion, you may want to make an SSH tunnel from your control node +to access UI on the `https://localhost:8443`, with this inventory variable: + + openshift_ui_ssh_tunnel: True + +Note, this requires sudo rights on the ansible control node and an absolute path +for the `openstack_private_ssh_key`. You should also update the control node's +`/etc/hosts`: + + 127.0.0.1 master-0.openshift.example.com + +In order to access UI, the ssh-tunnel service will be created and started on the +control node. Make sure to remove these changes and the service manually, when not +needed anymore. ## License diff --git a/playbooks/provisioning/openstack/openstack_dns_records.yml b/playbooks/provisioning/openstack/openstack_dns_records.yml deleted file mode 100644 index b5f0840c5..000000000 --- a/playbooks/provisioning/openstack/openstack_dns_records.yml +++ /dev/null @@ -1,75 +0,0 @@ ---- -- name: "Generate list of private A records" - set_fact: - private_records: "{{ private_records | default([]) + [ { 'type': 'A', 'hostname': hostvars[item]['ansible_hostname'], 'ip': hostvars[item]['private_v4'] } ] }}" - with_items: "{{ groups['cluster_hosts'] }}" - -- name: "Set the private DNS server to use the external value (if provided)" - set_fact: - nsupdate_server_private: "{{ external_nsupdate_keys['private']['server'] }}" - nsupdate_key_secret_private: "{{ external_nsupdate_keys['private']['key_secret'] }}" - nsupdate_key_algorithm_private: "{{ external_nsupdate_keys['private']['key_algorithm'] }}" - when: - - external_nsupdate_keys is defined - - external_nsupdate_keys['private'] is defined - -- name: "Set the private DNS server to use the provisioned value" - set_fact: - nsupdate_server_private: "{{ hostvars[groups['dns'][0]].public_v4 }}" - nsupdate_key_secret_private: "{{ hostvars[groups['dns'][0]].nsupdate_keys['private-' + full_dns_domain].key_secret }}" - nsupdate_key_algorithm_private: "{{ hostvars[groups['dns'][0]].nsupdate_keys['private-' + full_dns_domain].key_algorithm }}" - when: - - nsupdate_server_private is undefined - -- name: "Generate the private Add section for DNS" - set_fact: - private_named_records: - - view: "private" - zone: "{{ full_dns_domain }}" - server: "{{ nsupdate_server_private }}" - key_name: "{{ ( 'private-' + full_dns_domain ) }}" - key_secret: "{{ nsupdate_key_secret_private }}" - key_algorithm: "{{ nsupdate_key_algorithm_private | lower }}" - entries: "{{ private_records }}" - -- name: "Generate list of public A records" - set_fact: - public_records: "{{ public_records | default([]) + [ { 'type': 'A', 'hostname': hostvars[item]['ansible_hostname'], 'ip': hostvars[item]['public_v4'] } ] }}" - with_items: "{{ groups['cluster_hosts'] }}" - -- name: "Add wildcard records to the public A records" - set_fact: - public_records: "{{ public_records | default([]) + [ { 'type': 'A', 'hostname': '*.' + openshift_app_domain, 'ip': hostvars[item]['public_v4'] } ] }}" - with_items: "{{ groups['infra_hosts'] }}" - -- name: "Set the public DNS server details to use the external value (if provided)" - set_fact: - nsupdate_server_public: "{{ external_nsupdate_keys['public']['server'] }}" - nsupdate_key_secret_public: "{{ external_nsupdate_keys['public']['key_secret'] }}" - nsupdate_key_algorithm_public: "{{ external_nsupdate_keys['public']['key_algorithm'] }}" - when: - - external_nsupdate_keys is defined - - external_nsupdate_keys['public'] is defined - -- name: "Set the public DNS server details to use the provisioned value" - set_fact: - nsupdate_server_public: "{{ hostvars[groups['dns'][0]].public_v4 }}" - nsupdate_key_secret_public: "{{ hostvars[groups['dns'][0]].nsupdate_keys['public-' + full_dns_domain].key_secret }}" - nsupdate_key_algorithm_public: "{{ hostvars[groups['dns'][0]].nsupdate_keys['public-' + full_dns_domain].key_algorithm }}" - when: - - nsupdate_server_public is undefined - -- name: "Generate the public Add section for DNS" - set_fact: - public_named_records: - - view: "public" - zone: "{{ full_dns_domain }}" - server: "{{ nsupdate_server_public }}" - key_name: "{{ ( 'public-' + full_dns_domain ) }}" - key_secret: "{{ nsupdate_key_secret_public }}" - key_algorithm: "{{ nsupdate_key_algorithm_public | lower }}" - entries: "{{ public_records }}" - -- name: "Generate the final dns_records_add" - set_fact: - dns_records_add: "{{ private_named_records + public_named_records }}" diff --git a/playbooks/provisioning/openstack/openstack_dns_views.yml b/playbooks/provisioning/openstack/openstack_dns_views.yml deleted file mode 100644 index 7165b4269..000000000 --- a/playbooks/provisioning/openstack/openstack_dns_views.yml +++ /dev/null @@ -1,25 +0,0 @@ ---- -- name: "Generate ACL list for DNS server" - set_fact: - acl_list: "{{ acl_list | default([]) + [ (hostvars[item]['private_v4'] + '/32') ] }}" - with_items: "{{ groups['cluster_hosts'] }}" - -- name: "Generate the private view" - set_fact: - private_named_view: - - name: "private" - acl_entry: "{{ acl_list }}" - zone: - - dns_domain: "{{ full_dns_domain }}" - -- name: "Generate the public view" - set_fact: - public_named_view: - - name: "public" - zone: - - dns_domain: "{{ full_dns_domain }}" - forwarder: "{{ public_dns_nameservers }}" - -- name: "Generate the final named_config_views" - set_fact: - named_config_views: "{{ private_named_view + public_named_view }}" diff --git a/playbooks/provisioning/openstack/post-provision-openstack.yml b/playbooks/provisioning/openstack/post-provision-openstack.yml index a807c4d2f..f683b77be 100644 --- a/playbooks/provisioning/openstack/post-provision-openstack.yml +++ b/playbooks/provisioning/openstack/post-provision-openstack.yml @@ -4,7 +4,11 @@ become: False gather_facts: False tasks: - - wait_for_connection: + - when: not openstack_use_bastion|default(False)|bool + wait_for_connection: + - when: openstack_use_bastion|default(False)|bool + delegate_to: bastion + wait_for_connection: - hosts: cluster_hosts gather_facts: True @@ -48,9 +52,8 @@ become: true pre_tasks: - include: pre_tasks.yml - - name: "Generate dns-server views" - include: openstack_dns_views.yml roles: + - role: dns-views - role: infra-ansible/roles/dns-server - name: Build and process DNS Records @@ -59,9 +62,8 @@ become: False pre_tasks: - include: pre_tasks.yml - - name: "Generate dns records" - include: openstack_dns_records.yml roles: + - role: dns-records - role: infra-ansible/roles/dns - name: Switch the stack subnet to the configured private DNS server diff --git a/playbooks/provisioning/openstack/prerequisites.yml b/playbooks/provisioning/openstack/prerequisites.yml index 71a99fc82..dd4f980b2 100644 --- a/playbooks/provisioning/openstack/prerequisites.yml +++ b/playbooks/provisioning/openstack/prerequisites.yml @@ -20,6 +20,16 @@ that: 'shade_result.rc == 0' msg: "Python module shade is not installed" + # Check jmespath + - name: Try to import python module shade + command: python -c "import jmespath" + ignore_errors: yes + register: jmespath_result + - name: Check if jmespath is installed + assert: + that: 'jmespath_result.rc == 0' + msg: "Python module jmespath is not installed" + # Check python-dns - name: Try to import python DNS module command: python -c "import dns" diff --git a/playbooks/provisioning/openstack/provision-openstack.yml b/playbooks/provisioning/openstack/provision-openstack.yml index 0cac37aaf..6ec944d56 100644 --- a/playbooks/provisioning/openstack/provision-openstack.yml +++ b/playbooks/provisioning/openstack/provision-openstack.yml @@ -12,13 +12,20 @@ when: openstack_inventory|default('static') == 'static' inventory_path: "{{ openstack_inventory_path|default(inventory_dir) }}" private_ssh_key: "{{ openstack_private_ssh_key|default('~/.ssh/id_rsa') }}" + ssh_config_path: "{{ openstack_ssh_config_path|default('/tmp/ssh.config.openshift.ansible' + '.' + stack_name) }}" + ssh_user: "{{ ansible_user }}" -- name: Refresh Server inventory +- name: Refresh Server inventory or exit to apply SSH config hosts: localhost connection: local become: False gather_facts: False tasks: - - meta: refresh_inventory + - name: Exit to apply SSH config for a bastion + meta: end_play + when: openstack_use_bastion|default(False)|bool + - name: Refresh Server inventory + meta: refresh_inventory - include: post-provision-openstack.yml + when: not openstack_use_bastion|default(False)|bool diff --git a/playbooks/provisioning/openstack/sample-inventory/clouds.yaml b/playbooks/provisioning/openstack/sample-inventory/clouds.yaml deleted file mode 100644 index 8182d2995..000000000 --- a/playbooks/provisioning/openstack/sample-inventory/clouds.yaml +++ /dev/null @@ -1,5 +0,0 @@ ---- -ansible: - use_hostnames: True - expand_hostvars: True - fail_on_errors: True diff --git a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml index 9eb36ab13..bdd98d239 100644 --- a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml +++ b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml @@ -3,18 +3,57 @@ env_id: "openshift" public_dns_domain: "example.com" public_dns_nameservers: [] +# # Used Hostnames +# # - set custom hostnames for roles by uncommenting corresponding lines +#openstack_master_hostname: "master" +#openstack_infra_hostname: "infra-node" +#openstack_node_hostname: "app-node" +#openstack_lb_hostname: "lb" +#openstack_etcd_hostname: "etcd" +#openstack_dns_hostname: "dns" + openstack_ssh_public_key: "openshift" -openstack_default_image_name: "centos7" -openstack_default_flavor: "m1.medium" openstack_external_network_name: "public" +#openstack_private_network_name: "openshift-ansible-{{ stack_name }}-net" + +# # Used Images +# # - set specific images for roles by uncommenting corresponding lines +# # - note: do not remove openstack_default_image_name definition +#openstack_master_image_name: "centos7" +#openstack_infra_image_name: "centos7" +#openstack_node_image_name: "centos7" +#openstack_lb_image_name: "centos7" +#openstack_etcd_image_name: "centos7" +#openstack_dns_image_name: "centos7" +openstack_default_image_name: "centos7" openstack_num_masters: 1 openstack_num_infra: 1 openstack_num_nodes: 2 +# # Used Flavors +# # - set specific flavors for roles by uncommenting corresponding lines +# # - note: do note remove openstack_default_flavor definition +#openstack_master_flavor: "m1.medium" +#openstack_infra_flavor: "m1.medium" +#openstack_node_flavor: "m1.medium" +#openstack_lb_flavor: "m1.medium" +#openstack_etcd_flavor: "m1.medium" +#openstack_dns_flavor: "m1.medium" +openstack_default_flavor: "m1.medium" + # # Numerical index of nodes to remove # openstack_nodes_to_remove: [] +# # Docker volume size +# # - set specific volume size for roles by uncommenting corresponding lines +# # - note: do not remove docker_default_volume_size definition +#docker_master_volume_size: "15" +#docker_infra_volume_size: "15" +#docker_node_volume_size: "15" +#docker_etcd_volume_size: "2" +#docker_dns_volume_size: "1" +#docker_lb_volume_size: "5" docker_volume_size: "15" openstack_subnet_prefix: "192.168.99" @@ -69,5 +108,17 @@ ansible_user: openshift # # The path to checkpoint the static inventory from the in-memory one #openstack_inventory_path: ../../../../inventory +# # Use bastion node to access cluster nodes (Defaults to False). +# # Requires a static inventory. +#openstack_use_bastion: False +#bastion_ingress_cidr: "{{openstack_subnet_prefix}}.0/24" +# # # The Nova key-pair's private SSH key to access inventory nodes #openstack_private_ssh_key: ~/.ssh/openshift +# # The path for the SSH config to access all nodes +#openstack_ssh_config_path: /tmp/ssh.config.openshift.ansible.{{ env_id }}.{{ public_dns_domain }} + + +# If you want to use the VM storage instead of Cinder volumes, set this to `true`. +# NOTE: this is for testing only! Your data will be gone once the VM disappears! +# ephemeral_volumes: false diff --git a/playbooks/provisioning/openstack/stack_params.yaml b/playbooks/provisioning/openstack/stack_params.yaml index 9c0b09b45..60e9bcf45 100644 --- a/playbooks/provisioning/openstack/stack_params.yaml +++ b/playbooks/provisioning/openstack/stack_params.yaml @@ -3,21 +3,39 @@ stack_name: "{{ env_id }}.{{ public_dns_domain }}" dns_domain: "{{ public_dns_domain }}" dns_nameservers: "{{ public_dns_nameservers }}" subnet_prefix: "{{ openstack_subnet_prefix }}" +master_hostname: "{{ openstack_master_hostname | default('master') }}" +infra_hostname: "{{ openstack_infra_hostname | default('infra-node') }}" +node_hostname: "{{ openstack_node_hostname | default('app-node') }}" +lb_hostname: "{{ openstack_lb_hostname | default('lb') }}" +etcd_hostname: "{{ openstack_etcd_hostname | default('etcd') }}" +dns_hostname: "{{ openstack_dns_hostname | default('dns') }}" ssh_public_key: "{{ openstack_ssh_public_key }}" openstack_image: "{{ openstack_default_image_name }}" -lb_flavor: "{{ openstack_default_flavor | default('m1.small') }}" -etcd_flavor: "{{ openstack_default_flavor | default('m1.small') }}" -master_flavor: "{{ openstack_default_flavor | default('m1.medium') }}" -node_flavor: "{{ openstack_default_flavor | default('m1.medium') }}" -infra_flavor: "{{ openstack_default_flavor | default('m1.medium') }}" -dns_flavor: "{{ openstack_default_flavor | default('m1.small') }}" +lb_flavor: "{{ openstack_lb_flavor | default(openstack_default_flavor) }}" +etcd_flavor: "{{ openstack_etcd_flavor | default(openstack_default_flavor) }}" +master_flavor: "{{ openstack_master_flavor | default(openstack_default_flavor) }}" +node_flavor: "{{ openstack_node_flavor | default(openstack_default_flavor) }}" +infra_flavor: "{{ openstack_infra_flavor | default(openstack_default_flavor) }}" +dns_flavor: "{{ openstack_dns_flavor | default(openstack_default_flavor) }}" +openstack_master_image: "{{ openstack_master_image_name | default(openstack_default_image_name) }}" +openstack_infra_image: "{{ openstack_infra_image_name | default(openstack_default_image_name) }}" +openstack_node_image: "{{ openstack_node_image_name | default(openstack_default_image_name) }}" +openstack_lb_image: "{{ openstack_lb_image_name | default(openstack_default_image_name) }}" +openstack_etcd_image: "{{ openstack_etcd_image_name | default(openstack_default_image_name) }}" +openstack_dns_image: "{{ openstack_dns_image_name | default(openstack_default_image_name) }}" +openstack_private_network: "{{ openstack_private_network_name | default ('openshift-ansible-' + stack_name + '-net') }}" external_network: "{{ openstack_external_network_name }}" num_etcd: "{{ openstack_num_etcd | default(0) }}" num_masters: "{{ openstack_num_masters }}" num_nodes: "{{ openstack_num_nodes }}" num_infra: "{{ openstack_num_infra }}" num_dns: "{{ openstack_num_dns | default(1) }}" -master_volume_size: "{{ docker_volume_size }}" -app_volume_size: "{{ docker_volume_size }}" -infra_volume_size: "{{ docker_volume_size }}" +master_volume_size: "{{ docker_master_volume_size | default(docker_volume_size) }}" +infra_volume_size: "{{ docker_infra_volume_size | default(docker_volume_size) }}" +node_volume_size: "{{ docker_node_volume_size | default(docker_volume_size) }}" +etcd_volume_size: "{{ docker_etcd_volume_size | default('2') }}" +dns_volume_size: "{{ docker_dns_volume_size | default('1') }}" +lb_volume_size: "{{ docker_lb_volume_size | default('5') }}" nodes_to_remove: "{{ openstack_nodes_to_remove | default([]) | to_yaml }}" +use_bastion: "{{ openstack_use_bastion|default(False) }}" +ui_ssh_tunnel: "{{ openshift_ui_ssh_tunnel|default(False) }}" |