diff options
Diffstat (limited to 'playbooks')
| -rw-r--r-- | playbooks/openstack/README.md | 262 | ||||
| -rw-r--r-- | playbooks/openstack/advanced-configuration.md | 772 | ||||
| -rw-r--r-- | playbooks/openstack/openshift-cluster/install.yml | 18 | ||||
| -rw-r--r-- | playbooks/openstack/openshift-cluster/prerequisites.yml | 12 | ||||
| -rw-r--r-- | playbooks/openstack/openshift-cluster/provision.yml | 61 | ||||
| -rw-r--r-- | playbooks/openstack/openshift-cluster/provision_install.yml | 9 | ||||
| l--------- | playbooks/openstack/openshift-cluster/roles | 1 | ||||
| -rw-r--r-- | playbooks/openstack/sample-inventory/group_vars/OSEv3.yml | 59 | ||||
| -rw-r--r-- | playbooks/openstack/sample-inventory/group_vars/all.yml | 147 | ||||
| -rwxr-xr-x | playbooks/openstack/sample-inventory/inventory.py | 96 |
10 files changed, 1437 insertions, 0 deletions
diff --git a/playbooks/openstack/README.md b/playbooks/openstack/README.md new file mode 100644 index 000000000..f3fe13530 --- /dev/null +++ b/playbooks/openstack/README.md @@ -0,0 +1,262 @@ +# OpenStack Provisioning + +This directory contains [Ansible][ansible] playbooks and roles to create +OpenStack resources (servers, networking, volumes, security groups, +etc.). The result is an environment ready for OpenShift installation +via [openshift-ansible]. + +We provide everything necessary to be able to install OpenShift on +OpenStack (including the DNS and load balancer servers when +necessary). In addition, we work on providing integration with the +OpenStack-native services (storage, lbaas, baremetal as a service, +dns, etc.). + + +## OpenStack Requirements + +Before you start the installation, you need to have an OpenStack +environment to connect to. You can use a public cloud or an OpenStack +within your organisation. It is also possible to +use [Devstack][devstack] or [TripleO][tripleo]. In the case of +TripleO, we will be running on top of the **overcloud**. + +The OpenStack release must be Newton (for Red Hat OpenStack this is +version 10) or newer. It must also satisfy these requirements: + +* Heat (Orchestration) must be available +* The deployment image (CentOS 7 or RHEL 7) must be loaded +* The deployment flavor must be available to your user + - `m1.medium` / 4GB RAM + 40GB disk should be enough for testing + - look at + the [Minimum Hardware Requirements page][hardware-requirements] + for production +* The keypair for SSH must be available in openstack +* `keystonerc` file that lets you talk to the openstack services + * NOTE: only Keystone V2 is currently supported + +Optional: +* External Neutron network with a floating IP address pool + + +## DNS Requirements + +OpenShift requires DNS to operate properly. OpenStack supports DNS-as-a-service +in the form of the Designate project, but the playbooks here don't support it +yet. Until we do, you will need to provide a DNS solution yourself (or in case +you are not running Designate when we do). + +If your server supports nsupdate, we will use it to add the necessary records. + +TODO(shadower): describe how to build a sample DNS server and how to configure +our playbooks for nsupdate. + + +## Installation + +There are four main parts to the installation: + +1. [Preparing Ansible and dependencies](#1-preparing-ansible-and-dependencies) +2. [Configuring the desired OpenStack environment and OpenShift cluster](#2-configuring-the-openstack-environment-and-openshift-cluster) +3. [Creating the OpenStack resources (VMs, networking, etc.)](#3-creating-the-openstack-resources-vms-networking-etc) +4. [Installing OpenShift](#4-installing-openshift) + +This guide is going to install [OpenShift Origin][origin] +with [CentOS 7][centos7] images with minimal customisation. + +We will create the VMs for running OpenShift, in a new Neutron +network, assign Floating IP addresses and configure DNS. + +The OpenShift cluster will have a single Master node that will run +`etcd`, a single Infra node and two App nodes. + +You can look at +the [Advanced Configuration page][advanced-configuration] for +additional options. + + + +### 1. Preparing Ansible and dependencies + +First, you need to select where to run [Ansible][ansible] from (the +*Ansible host*). This can be the computer you read this guide on or an +OpenStack VM you'll create specifically for this purpose. + +We will use +a +[Docker image that has all the dependencies installed][control-host-image] to +make things easier. If you don't want to use Docker, take a look at +the [Ansible host dependencies][ansible-dependencies] and make sure +they're installed. + +Your *Ansible host* needs to have the following: + +1. Docker +2. `keystonerc` file with your OpenStack credentials +3. SSH private key for logging in to your OpenShift nodes + +Assuming your private key is `~/.ssh/id_rsa` and `keystonerc` in your +current directory: + +```bash +$ sudo docker run -it -v ~/.ssh:/mnt/.ssh:Z \ + -v $PWD/keystonerc:/root/.config/openstack/keystonerc.sh:Z \ + redhatcop/control-host-openstack bash +``` + +This will create the container, add your SSH key and source your +`keystonerc`. It should be set up for the installation. + +You can verify that everything is in order: + + +```bash +$ less .ssh/id_rsa +$ ansible --version +$ openstack image list +``` + + +### 2. Configuring the OpenStack Environment and OpenShift Cluster + +The configuration is all done in an Ansible inventory directory. We +will clone the [openshift-ansible][openshift-ansible] repository and set +things up for a minimal installation. + + +``` +$ git clone https://github.com/openshift/openshift-ansible +$ cp -r openshift-ansible/playbooks/openstack/sample-inventory/ inventory +``` + +If you're testing multiple configurations, you can have multiple +inventories and switch between them. + +#### OpenStack Configuration + +The OpenStack configuration is in `inventory/group_vars/all.yml`. + +Open the file and plug in the image, flavor and network configuration +corresponding to your OpenStack installation. + +```bash +$ vi inventory/group_vars/all.yml +``` + +1. Set the `openshift_openstack_keypair_name` to your OpenStack keypair name. + - See `openstack keypair list` to find the keypairs registered with + OpenShift. + - This must correspond to your private SSH key in `~/.ssh/id_rsa` +2. Set the `openshift_openstack_external_network_name` to the floating IP + network of your openstack. + - See `openstack network list` for the list of networks. + - It's often called `public`, `external` or `ext-net`. +3. Set the `openshift_openstack_default_image_name` to the image you want your + OpenShift VMs to run. + - See `openstack image list` for the list of available images. +4. Set the `openshift_openstack_default_flavor` to the flavor you want your + OpenShift VMs to use. + - See `openstack flavor list` for the list of available flavors. +5. Set the `openshift_openstack_dns_nameservers` to the list of the IP addresses + of the DNS servers used for the **private** address resolution. + +**NOTE ON DNS**: at minimum, the OpenShift nodes need to be able to access each +other by their hostname. OpenStack doesn't provide this by default, so you +need to provide a DNS server. Put the address of that DNS server in +`openshift_openstack_dns_nameservers` variable. + + + + +#### OpenShift configuration + +The OpenShift configuration is in `inventory/group_vars/OSEv3.yml`. + +The default options will mostly work, but unless you used the large +flavors for a production-ready environment, openshift-ansible's +hardware check will fail. + +Let's disable those checks by putting this in +`inventory/group_vars/OSEv3.yml`: + +```yaml +openshift_disable_check: disk_availability,memory_availability +``` + +**NOTE**: The default authentication method will allow **any username +and password** in! If you're running this in a public place, you need +to set up access control. + +Feel free to look at +the [Sample OpenShift Inventory][sample-openshift-inventory] and +the [advanced configuration][advanced-configuration]. + + +### 3. Creating the OpenStack resources (VMs, networking, etc.) + +We provide an `ansible.cfg` file which has some useful defaults -- you should +copy it to the directory you're going to run `ansible-playbook` from. + +```bash +$ cp openshift-ansible/ansible.cfg ansible.cfg +``` + +Then run the provisioning playbook -- this will create the OpenStack +resources: + +```bash +$ ansible-playbook --user openshift -i inventory openshift-ansible/playbooks/openstack/openshift-cluster/provision.yaml +``` + +If you're using multiple inventories, make sure you pass the path to +the right one to `-i`. + +If your SSH private key is not in `~/.ssh/id_rsa` use the `--private-key` +option to specify the correct path. + + +### 4. Installing OpenShift + +Run the `byo/config.yml` playbook on top of the OpenStack nodes we have +prepared. + +```bash +$ ansible-playbook -i inventory openshift-ansible/playbooks/byo/config.yml +``` + + +### Next Steps + +And that's it! You should have a small but functional OpenShift +cluster now. + +Take a look at [how to access the cluster][accessing-openshift] +and [how to remove it][uninstall-openshift] as well as the more +advanced configuration: + +* [Accessing the OpenShift cluster][accessing-openshift] +* [Removing the OpenShift cluster][uninstall-openshift] +* Set Up Authentication (TODO) +* [Multiple Masters with a load balancer][loadbalancer] +* [External Dns][external-dns] +* Multiple Clusters (TODO) +* [Cinder Registry][cinder-registry] +* [Bastion Node][bastion] + + +[ansible]: https://www.ansible.com/ +[openshift-ansible]: https://github.com/openshift/openshift-ansible +[devstack]: https://docs.openstack.org/devstack/ +[tripleo]: http://tripleo.org/ +[ansible-dependencies]: ./advanced-configuration.md#dependencies-for-localhost-ansible-controladmin-node +[control-host-image]: https://hub.docker.com/r/redhatcop/control-host-openstack/ +[hardware-requirements]: https://docs.openshift.org/latest/install_config/install/prerequisites.html#hardware +[origin]: https://www.openshift.org/ +[centos7]: https://www.centos.org/ +[sample-openshift-inventory]: https://github.com/openshift/openshift-ansible/blob/master/inventory/byo/hosts.example +[advanced-configuration]: ./advanced-configuration.md +[accessing-openshift]: ./advanced-configuration.md#accessing-the-openshift-cluster +[uninstall-openshift]: ./advanced-configuration.md#removing-the-openshift-cluster +[loadbalancer]: ./advanced-configuration.md#multi-master-configuration +[external-dns]: ./advanced-configuration.md#dns-configuration-variables +[cinder-registry]: ./advanced-configuration.md#creating-and-using-a-cinder-volume-for-the-openshift-registry +[bastion]: ./advanced-configuration.md#configure-static-inventory-and-access-via-a-bastion-node diff --git a/playbooks/openstack/advanced-configuration.md b/playbooks/openstack/advanced-configuration.md new file mode 100644 index 000000000..90cc20b98 --- /dev/null +++ b/playbooks/openstack/advanced-configuration.md @@ -0,0 +1,772 @@ +## Dependencies for localhost (ansible control/admin node) + +* [Ansible 2.3](https://pypi.python.org/pypi/ansible) +* [Ansible-galaxy](https://pypi.python.org/pypi/ansible-galaxy-local-deps) +* [jinja2](http://jinja.pocoo.org/docs/2.9/) +* [shade](https://pypi.python.org/pypi/shade) +* python-jmespath / [jmespath](https://pypi.python.org/pypi/jmespath) +* python-dns / [dnspython](https://pypi.python.org/pypi/dnspython) +* Become (sudo) is not required. + +**NOTE**: You can use a Docker image with all dependencies set up. +Find more in the [Deployment section](#deployment). + +### Optional Dependencies for localhost +**Note**: When using rhel images, `rhel-7-server-openstack-10-rpms` repository is required in order to install these packages. + +* `python-openstackclient` +* `python-heatclient` + +## Dependencies for OpenStack hosted cluster nodes (servers) + +There are no additional dependencies for the cluster nodes. Required +configuration steps are done by Heat given a specific user data config +that normally should not be changed. + +## Required galaxy modules + +In order to pull in external dependencies for DNS configuration steps, +the following commads need to be executed: + + ansible-galaxy install \ + -r openshift-ansible-contrib/playbooks/provisioning/openstack/galaxy-requirements.yaml \ + -p openshift-ansible-contrib/roles + +Alternatively you can install directly from github: + + ansible-galaxy install git+https://github.com/redhat-cop/infra-ansible,master \ + -p openshift-ansible-contrib/roles + +Notes: +* This assumes we're in the directory that contains the clonned +openshift-ansible-contrib repo in its root path. +* When trying to install a different version, the previous one must be removed first +(`infra-ansible` directory from [roles](https://github.com/openshift/openshift-ansible-contrib/tree/master/roles)). +Otherwise, even if there are differences between the two versions, installation of the newer version is skipped. + + +## Accessing the OpenShift Cluster + +### Use the Cluster DNS + +In addition to the OpenShift nodes, we created a DNS server with all +the necessary entries. We will configure your *Ansible host* to use +this new DNS and talk to the deployed OpenShift. + +First, get the DNS IP address: + +```bash +$ openstack server show dns-0.openshift.example.com --format value --column addresses +openshift-ansible-openshift.example.com-net=192.168.99.11, 10.40.128.129 +``` + +Note the floating IP address (it's `10.40.128.129` in this case) -- if +you're not sure, try pinging them both -- it's the one that responds +to pings. + +Next, edit your `/etc/resolv.conf` as root and put `nameserver DNS_IP` as your +**first entry**. + +If your `/etc/resolv.conf` currently looks like this: + +``` +; generated by /usr/sbin/dhclient-script +search openstacklocal +nameserver 192.168.0.3 +nameserver 192.168.0.2 +``` + +Change it to this: + +``` +; generated by /usr/sbin/dhclient-script +search openstacklocal +nameserver 10.40.128.129 +nameserver 192.168.0.3 +nameserver 192.168.0.2 +``` + +### Get the `oc` Client + +**NOTE**: You can skip this section if you're using the Docker image +-- it already has the `oc` binary. + +You need to download the OpenShift command line client (called `oc`). +You can download and extract `openshift-origin-client-tools` from the +OpenShift release page: + +https://github.com/openshift/origin/releases/latest/ + +Or you can now copy it from the master node: + + $ ansible -i inventory masters[0] -m fetch -a "src=/bin/oc dest=oc" + +Either way, find the `oc` binary and put it in your `PATH`. + + +### Logging in Using the Command Line + + +``` +oc login --insecure-skip-tls-verify=true https://master-0.openshift.example.com:8443 -u user -p password +oc new-project test +oc new-app --template=cakephp-mysql-example +oc status -v +curl http://cakephp-mysql-example-test.apps.openshift.example.com +``` + +This will trigger an image build. You can run `oc logs -f +bc/cakephp-mysql-example` to follow its progress. + +Wait until the build has finished and both pods are deployed and running: + +``` +$ oc status -v +In project test on server https://master-0.openshift.example.com:8443 + +http://cakephp-mysql-example-test.apps.openshift.example.com (svc/cakephp-mysql-example) + dc/cakephp-mysql-example deploys istag/cakephp-mysql-example:latest <- + bc/cakephp-mysql-example source builds https://github.com/openshift/cakephp-ex.git on openshift/php:7.0 + deployment #1 deployed about a minute ago - 1 pod + +svc/mysql - 172.30.144.36:3306 + dc/mysql deploys openshift/mysql:5.7 + deployment #1 deployed 3 minutes ago - 1 pod + +Info: + * pod/cakephp-mysql-example-1-build has no liveness probe to verify pods are still running. + try: oc set probe pod/cakephp-mysql-example-1-build --liveness ... +View details with 'oc describe <resource>/<name>' or list everything with 'oc get all'. + +``` + +You can now look at the deployed app using its route: + +``` +$ curl http://cakephp-mysql-example-test.apps.openshift.example.com +``` + +Its `title` should say: "Welcome to OpenShift". + + +### Accessing the UI + +You can also access the OpenShift cluster with a web browser by going to: + +https://master-0.openshift.example.com:8443 + +Note that for this to work, the OpenShift nodes must be accessible +from your computer and it's DNS configuration must use the cruster's +DNS. + + +## Removing the OpenShift Cluster + +Everything in the cluster is contained within a Heat stack. To +completely remove the cluster and all the related OpenStack resources, +run this command: + +```bash +openstack stack delete --wait --yes openshift.example.com +``` + + +## DNS configuration variables + +Pay special attention to the values in the first paragraph -- these +will depend on your OpenStack environment. + +Note that the provsisioning playbooks update the original Neutron subnet +created with the Heat stack to point to the configured DNS servers. +So the provisioned cluster nodes will start using those natively as +default nameservers. Technically, this allows to deploy OpenShift clusters +without dnsmasq proxies. + +The `openshift_openstack_clusterid` and `openshift_openstack_public_dns_domain` will form the cluster's DNS domain all +your servers will be under. With the default values, this will be +`openshift.example.com`. For workloads, the default subdomain is 'apps'. +That sudomain can be set as well by the `openshift_openstack_app_subdomain` variable in +the inventory. + +The `openstack_<role name>_hostname` is a set of variables used for customising +hostnames of servers with a given role. When such a variable stays commented, +default hostname (usually the role name) is used. + +The `openshift_openstack_dns_nameservers` is a list of DNS servers accessible from all +the created Nova servers. These will provide the internal name resolution for +your OpenShift nodes (as well as upstream name resolution for installing +packages, etc.). + +The `openshift_use_dnsmasq` controls either dnsmasq is deployed or not. +By default, dnsmasq is deployed and comes as the hosts' /etc/resolv.conf file +first nameserver entry that points to the local host instance of the dnsmasq +daemon that in turn proxies DNS requests to the authoritative DNS server. +When Network Manager is enabled for provisioned cluster nodes, which is +normally the case, you should not change the defaults and always deploy dnsmasq. + +`openshift_openstack_external_nsupdate_keys` describes an external authoritative DNS server(s) +processing dynamic records updates in the public and private cluster views: + + openshift_openstack_external_nsupdate_keys: + public: + key_secret: <some nsupdate key> + key_algorithm: 'hmac-md5' + key_name: 'update-key' + server: <public DNS server IP> + private: + key_secret: <some nsupdate key 2> + key_algorithm: 'hmac-sha256' + server: <public or private DNS server IP> + +Here, for the public view section, we specified another key algorithm and +optional `key_name`, which normally defaults to the cluster's DNS domain. +This just illustrates a compatibility mode with a DNS service deployed +by OpenShift on OSP10 reference architecture, and used in a mixed mode with +another external DNS server. + +Another example defines an external DNS server for the public view +additionally to the in-stack DNS server used for the private view only: + + openshift_openstack_external_nsupdate_keys: + public: + key_secret: <some nsupdate key> + key_algorithm: 'hmac-sha256' + server: <public DNS server IP> + +Here, updates matching the public view will be hitting the given public +server IP. While updates matching the private view will be sent to the +auto evaluated in-stack DNS server's **public** IP. + +Note, for the in-stack DNS server, private view updates may be sent only +via the public IP of the server. You can not send updates via the private +IP yet. This forces the in-stack private server to have a floating IP. +See also the [security notes](#security-notes) + +## Flannel networking + +In order to configure the +[flannel networking](https://docs.openshift.com/container-platform/3.6/install_config/configuring_sdn.html#using-flannel), +uncomment and adjust the appropriate `inventory/group_vars/OSEv3.yml` group vars. +Note that the `osm_cluster_network_cidr` must not overlap with the default +Docker bridge subnet of 172.17.0.0/16. Or you should change the docker0 default +CIDR range otherwise. For example, by adding `--bip=192.168.2.1/24` to +`DOCKER_NETWORK_OPTIONS` located in `/etc/sysconfig/docker-network`. + +Also note that the flannel network will be provisioned on a separate isolated Neutron +subnet defined from `osm_cluster_network_cidr` and having ports security disabled. +Use the `openstack_private_data_network_name` variable to define the network +name for the heat stack resource. + +After the cluster deployment done, you should run an additional post installation +step for flannel and docker iptables configuration: + + ansible-playbook openshift-ansible-contrib/playbooks/provisioning/openstack/post-install.yml + +## Other configuration variables + +`openshift_openstack_keypair_name` is a Nova keypair - you can see your +keypairs with `openstack keypair list`. It must correspond to the +private SSH key Ansible will use to log into the created VMs. This is +`~/.ssh/id_rsa` by default, but you can use a different key by passing +`--private-key` to `ansible-playbook`. + +`openshift_openstack_default_image_name` is the default name of the Glance image the +servers will use. You can see your images with `openstack image list`. +In order to set a different image for a role, uncomment the line with the +corresponding variable (e.g. `openshift_openstack_lb_image_name` for load balancer) and +set its value to another available image name. `openshift_openstack_default_image_name` +must stay defined as it is used as a default value for the rest of the roles. + +`openshift_openstack_default_flavor` is the default Nova flavor the servers will use. +You can see your flavors with `openstack flavor list`. +In order to set a different flavor for a role, uncomment the line with the +corresponding variable (e.g. `openshift_openstack_lb_flavor` for load balancer) and +set its value to another available flavor. `openshift_openstack_default_flavor` must +stay defined as it is used as a default value for the rest of the roles. + +`openshift_openstack_external_network_name` is the name of the Neutron network +providing external connectivity. It is often called `public`, +`external` or `ext-net`. You can see your networks with `openstack +network list`. + +`openshift_openstack_private_network_name` is the name of the private Neutron network +providing admin/control access for ansible. It can be merged with other +cluster networks, there are no special requirements for networking. + +The `openshift_openstack_num_masters`, `openshift_openstack_num_infra` and +`openshift_openstack_num_nodes` values specify the number of Master, Infra and +App nodes to create. + +The `openshift_openstack_cluster_node_labels` defines custom labels for your openshift +cluster node groups. It currently supports app and infra node groups. +The default value of this variable sets `region: primary` to app nodes and +`region: infra` to infra nodes. +An example of setting a customised label: +``` +openshift_openstack_cluster_node_labels: + app: + mylabel: myvalue +``` + +The `openshift_openstack_nodes_to_remove` allows you to specify the numerical indexes +of App nodes that should be removed; for example, ['0', '2'], + +The `docker_volume_size` is the default Docker volume size the servers will use. +In order to set a different volume size for a role, +uncomment the line with the corresponding variable (e. g. `docker_master_volume_size` +for master) and change its value. `docker_volume_size` must stay defined as it is +used as a default value for some of the servers (master, infra, app node). +The rest of the roles (etcd, load balancer, dns) have their defaults hard-coded. + +**Note**: If the `openshift_openstack_ephemeral_volumes` is set to `true`, the `*_volume_size` variables +will be ignored and the deployment will not create any cinder volumes. + +The `openshift_openstack_flat_secgrp`, controls Neutron security groups creation for Heat +stacks. Set it to true, if you experience issues with sec group rules +quotas. It trades security for number of rules, by sharing the same set +of firewall rules for master, node, etcd and infra nodes. + +The `openshift_openstack_required_packages` variable also provides a list of the additional +prerequisite packages to be installed before to deploy an OpenShift cluster. +Those are ignored though, if the `manage_packages: False`. + +The `openstack_inventory` controls either a static inventory will be created after the +cluster nodes provisioned on OpenStack cloud. Note, the fully dynamic inventory +is yet to be supported, so the static inventory will be created anyway. + +The `openstack_inventory_path` points the directory to host the generated static inventory. +It should point to the copied example inventory directory, otherwise ti creates +a new one for you. + +## Multi-master configuration + +Please refer to the official documentation for the +[multi-master setup](https://docs.openshift.com/container-platform/3.6/install_config/install/advanced_install.html#multiple-masters) +and define the corresponding [inventory +variables](https://docs.openshift.com/container-platform/3.6/install_config/install/advanced_install.html#configuring-cluster-variables) +in `inventory/group_vars/OSEv3.yml`. For example, given a load balancer node +under the ansible group named `ext_lb`: + + openshift_master_cluster_method: native + openshift_master_cluster_hostname: "{{ groups.ext_lb.0 }}" + openshift_master_cluster_public_hostname: "{{ groups.ext_lb.0 }}" + +## Provider Network + +Normally, the playbooks create a new Neutron network and subnet and attach +floating IP addresses to each node. If you have a provider network set up, this +is all unnecessary as you can just access servers that are placed in the +provider network directly. + +To use a provider network, set its name in `openshift_openstack_provider_network_name` in +`inventory/group_vars/all.yml`. + +If you set the provider network name, the `openshift_openstack_external_network_name` and +`openshift_openstack_private_network_name` fields will be ignored. + +**NOTE**: this will not update the nodes' DNS, so running openshift-ansible +right after provisioning will fail (unless you're using an external DNS server +your provider network knows about). You must make sure your nodes are able to +resolve each other by name. + +## Security notes + +Configure required `*_ingress_cidr` variables to restrict public access +to provisioned servers from your laptop (a /32 notation should be used) +or your trusted network. The most important is the `openshift_openstack_node_ingress_cidr` +that restricts public access to the deployed DNS server and cluster +nodes' ephemeral ports range. + +Note, the command ``curl https://api.ipify.org`` helps fiding an external +IP address of your box (the ansible admin node). + +There is also the `manage_packages` variable (defaults to True) you +may want to turn off in order to speed up the provisioning tasks. This may +be the case for development environments. When turned off, the servers will +be provisioned omitting the ``yum update`` command. This brings security +implications though, and is not recommended for production deployments. + +### DNS servers security options + +Aside from `openshift_openstack_node_ingress_cidr` restricting public access to in-stack DNS +servers, there are following (bind/named specific) DNS security +options available: + + named_public_recursion: 'no' + named_private_recursion: 'yes' + +External DNS servers, which is not included in the 'dns' hosts group, +are not managed. It is up to you to configure such ones. + +## Configure the OpenShift parameters + +Finally, you need to update the DNS entry in +`inventory/group_vars/OSEv3.yml` (look at +`openshift_master_default_subdomain`). + +In addition, this is the place where you can customise your OpenShift +installation for example by specifying the authentication. + +The full list of options is available in this sample inventory: + +https://github.com/openshift/openshift-ansible/blob/master/inventory/byo/hosts.ose.example + +Note, that in order to deploy OpenShift origin, you should update the following +variables for the `inventory/group_vars/OSEv3.yml`, `all.yml`: + + deployment_type: origin + openshift_deployment_type: "{{ deployment_type }}" + + +## Setting a custom entrypoint + +In order to set a custom entrypoint, update `openshift_master_cluster_public_hostname` + + openshift_master_cluster_public_hostname: api.openshift.example.com + +Note than an empty hostname does not work, so if your domain is `openshift.example.com`, +you cannot set this value to simply `openshift.example.com`. + +## Creating and using a Cinder volume for the OpenShift registry + +You can optionally have the playbooks create a Cinder volume and set +it up as the OpenShift hosted registry. + +To do that you need specify the desired Cinder volume name and size in +Gigabytes in `inventory/group_vars/all.yml`: + + openshift_openstack_cinder_hosted_registry_name: cinder-registry + openshift_openstack_cinder_hosted_registry_size_gb: 10 + +With this, the playbooks will create the volume and set up its +filesystem. If there is an existing volume of the same name, we will +use it but keep the existing data on it. + +To use the volume for the registry, you must first configure it with +the OpenStack credentials by putting the following to `OSEv3.yml`: + + openshift_cloudprovider_openstack_username: "{{ lookup('env','OS_USERNAME') }}" + openshift_cloudprovider_openstack_password: "{{ lookup('env','OS_PASSWORD') }}" + openshift_cloudprovider_openstack_auth_url: "{{ lookup('env','OS_AUTH_URL') }}" + openshift_cloudprovider_openstack_tenant_name: "{{ lookup('env','OS_TENANT_NAME') }}" + +This will use the credentials from your shell environment. If you want +to enter them explicitly, you can. You can also use credentials +different from the provisioning ones (say for quota or access control +reasons). + +**NOTE**: If you're testing this on (DevStack)[devstack], you must +explicitly set your Keystone API version to v2 (e.g. +`OS_AUTH_URL=http://10.34.37.47/identity/v2.0`) instead of the default +value provided by `openrc`. You may also encounter the following issue +with Cinder: + +https://github.com/kubernetes/kubernetes/issues/50461 + +You can read the (OpenShift documentation on configuring +OpenStack)[openstack] for more information. + +[devstack]: https://docs.openstack.org/devstack/latest/ +[openstack]: https://docs.openshift.org/latest/install_config/configuring_openstack.html + + +Next, we need to instruct OpenShift to use the Cinder volume for it's +registry. Again in `OSEv3.yml`: + + #openshift_hosted_registry_storage_kind: openstack + #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce'] + #openshift_hosted_registry_storage_openstack_filesystem: xfs + +The filesystem value here will be used in the initial formatting of +the volume. + +If you're using the dynamic inventory, you must uncomment these two values as +well: + + #openshift_hosted_registry_storage_openstack_volumeID: "{{ lookup('os_cinder', openshift_openstack_cinder_hosted_registry_name).id }}" + #openshift_hosted_registry_storage_volume_size: "{{ openshift_openstack_cinder_hosted_registry_size_gb }}Gi" + +But note that they use the `os_cinder` lookup plugin we provide, so you must +tell Ansible where to find it either in `ansible.cfg` (the one we provide is +configured properly) or by exporting the +`ANSIBLE_LOOKUP_PLUGINS=openshift-ansible-contrib/lookup_plugins` environment +variable. + + + +## Use an existing Cinder volume for the OpenShift registry + +You can also use a pre-existing Cinder volume for the storage of your +OpenShift registry. + +To do that, you need to have a Cinder volume. You can create one by +running: + + openstack volume create --size <volume size in gb> <volume name> + +The volume needs to have a file system created before you put it to +use. + +As with the automatically-created volume, you have to set up the +OpenStack credentials in `inventory/group_vars/OSEv3.yml` as well as +registry values: + + #openshift_hosted_registry_storage_kind: openstack + #openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce'] + #openshift_hosted_registry_storage_openstack_filesystem: xfs + #openshift_hosted_registry_storage_openstack_volumeID: e0ba2d73-d2f9-4514-a3b2-a0ced507fa05 + #openshift_hosted_registry_storage_volume_size: 10Gi + +Note the `openshift_hosted_registry_storage_openstack_volumeID` and +`openshift_hosted_registry_storage_volume_size` values: these need to +be added in addition to the previous variables. + +The **Cinder volume ID**, **filesystem** and **volume size** variables +must correspond to the values in your volume. The volume ID must be +the **UUID** of the Cinder volume, *not its name*. + +We can do formate the volume for you if you ask for it in +`inventory/group_vars/all.yml`: + + openshift_openstack_prepare_and_format_registry_volume: true + +**NOTE:** doing so **will destroy any data that's currently on the volume**! + +You can also run the registry setup playbook directly: + + ansible-playbook -i inventory playbooks/provisioning/openstack/prepare-and-format-cinder-volume.yaml + +(the provisioning phase must be completed, first) + + + +## Configure static inventory and access via a bastion node + +Example inventory variables: + + openshift_openstack_use_bastion: true + openshift_openstack_bastion_ingress_cidr: "{{openshift_openstack_subnet_prefix}}.0/24" + openstack_private_ssh_key: ~/.ssh/id_rsa + openstack_inventory: static + openstack_inventory_path: ../../../../inventory + openstack_ssh_config_path: /tmp/ssh.config.openshift.ansible.openshift.example.com + +The `openshift_openstack_subnet_prefix` is the openstack private network for your cluster. +And the `openshift_openstack_bastion_ingress_cidr` defines accepted range for SSH connections to nodes +additionally to the `openshift_openstack_ssh_ingress_cidr`` (see the security notes above). + +The SSH config will be stored on the ansible control node by the +gitven path. Ansible uses it automatically. To access the cluster nodes with +that ssh config, use the `-F` prefix, f.e.: + + ssh -F /tmp/ssh.config.openshift.ansible.openshift.example.com master-0.openshift.example.com echo OK + +Note, relative paths will not work for the `openstack_ssh_config_path`, but it +works for the `openstack_private_ssh_key` and `openstack_inventory_path`. In this +guide, the latter points to the current directory, where you run ansible commands +from. + +To verify nodes connectivity, use the command: + + ansible -v -i inventory/hosts -m ping all + +If something is broken, double-check the inventory variables, paths and the +generated `<openstack_inventory_path>/hosts` and `openstack_ssh_config_path` files. + +The `inventory: dynamic` can be used instead to access cluster nodes directly via +floating IPs. In this mode you can not use a bastion node and should specify +the dynamic inventory file in your ansible commands , like `-i openstack.py`. + +## Using Docker on the Ansible host + +If you don't want to worry about the dependencies, you can use the +[OpenStack Control Host image][control-host-image]. + +[control-host-image]: https://hub.docker.com/r/redhatcop/control-host-openstack/ + +It has all the dependencies installed, but you'll need to map your +code and credentials to it. Assuming your SSH keys live in `~/.ssh` +and everything else is in your current directory (i.e. `ansible.cfg`, +`keystonerc`, `inventory`, `openshift-ansible`, +`openshift-ansible-contrib`), this is how you run the deployment: + + sudo docker run -it -v ~/.ssh:/mnt/.ssh:Z \ + -v $PWD:/root/openshift:Z \ + -v $PWD/keystonerc:/root/.config/openstack/keystonerc.sh:Z \ + redhatcop/control-host-openstack bash + +(feel free to replace `$PWD` with an actual path to your inventory and +checkouts, but note that relative paths don't work) + +The first run may take a few minutes while the image is being +downloaded. After that, you'll be inside the container and you can run +the playbooks: + + cd openshift + ansible-playbook openshift-ansible-contrib/playbooks/provisioning/openstack/provision.yaml + + +### Run the playbook + +Assuming your OpenStack (Keystone) credentials are in the `keystonerc` +this is how you stat the provisioning process from your ansible control node: + + . keystonerc + ansible-playbook openshift-ansible-contrib/playbooks/provisioning/openstack/provision.yaml + +Note, here you start with an empty inventory. The static inventory will be populated +with data so you can omit providing additional arguments for future ansible commands. + +If bastion enabled, the generates SSH config must be applied for ansible. +Otherwise, it is auto included by the previous step. In order to execute it +as a separate playbook, use the following command: + + ansible-playbook openshift-ansible-contrib/playbooks/provisioning/openstack/post-provision-openstack.yml + +The first infra node then becomes a bastion node as well and proxies access +for future ansible commands. The post-provision step also configures Satellite, +if requested, and DNS server, and ensures other OpenShift requirements to be met. + + +## Running Custom Post-Provision Actions + +A custom playbook can be run like this: + +``` +ansible-playbook --private-key ~/.ssh/openshift -i inventory/ openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/custom-playbook.yml +``` + +If you'd like to limit the run to one particular host, you can do so as follows: + +``` +ansible-playbook --private-key ~/.ssh/openshift -i inventory/ openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/custom-playbook.yml -l app-node-0.openshift.example.com +``` + +You can also create your own custom playbook. Here are a few examples: + +### Adding additional YUM repositories + +``` +--- +- hosts: app + tasks: + + # enable EPL + - name: Add repository + yum_repository: + name: epel + description: EPEL YUM repo + baseurl: https://download.fedoraproject.org/pub/epel/$releasever/$basearch/ +``` + +This example runs against app nodes. The list of options include: + + - cluster_hosts (all hosts: app, infra, masters, dns, lb) + - OSEv3 (app, infra, masters) + - app + - dns + - masters + - infra_hosts + +### Attaching additional RHN pools + +``` +--- +- hosts: cluster_hosts + tasks: + - name: Attach additional RHN pool + become: true + command: "/usr/bin/subscription-manager attach --pool=<pool ID>" + register: attach_rhn_pool_result + until: attach_rhn_pool_result.rc == 0 + retries: 10 + delay: 1 +``` + +This playbook runs against all cluster nodes. In order to help prevent slow connectivity +problems, the task is retried 10 times in case of initial failure. +Note that in order for this example to work in your deployment, your servers must use the RHEL image. + +### Adding extra Docker registry URLs + +This playbook is located in the [custom-actions](https://github.com/openshift/openshift-ansible-contrib/tree/master/playbooks/provisioning/openstack/custom-actions) directory. + +It adds URLs passed as arguments to the docker configuration program. +Going into more detail, the configuration program (which is in the YAML format) is loaded into an ansible variable +([lines 27-30](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml#L27-L30)) +and in its structure, `registries` and `insecure_registries` sections are expanded with the newly added items +([lines 56-76](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml#L56-L76)). +The new content is then saved into the original file +([lines 78-82](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml#L78-L82)) +and docker is restarted. + +Example usage: +``` +ansible-playbook -i <inventory> openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml --extra-vars '{"registries": "reg1", "insecure_registries": ["ins_reg1","ins_reg2"]}' +``` + +### Adding extra CAs to the trust chain + +This playbook is also located in the [custom-actions](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions) directory. +It copies passed CAs to the trust chain location and updates the trust chain on each selected host. + +Example usage: +``` +ansible-playbook -i <inventory> openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/add-cas.yml --extra-vars '{"ca_files": [<absolute path to ca1 file>, <absolute path to ca2 file>]}' +``` + +Please consider contributing your custom playbook back to openshift-ansible-contrib! + +A library of custom post-provision actions exists in `openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions`. Playbooks include: + +* [add-yum-repos.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-yum-repos.yml): adds a list of custom yum repositories to every node in the cluster +* [add-rhn-pools.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-rhn-pools.yml): attaches a list of additional RHN pools to every node in the cluster +* [add-docker-registry.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml): adds a list of docker registries to the docker configuration on every node in the cluster +* [add-cas.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-rhn-pools.yml): adds a list of CAs to the trust chain on every node in the cluster + + +## Install OpenShift + +Once it succeeds, you can install openshift by running: + + ansible-playbook openshift-ansible/playbooks/byo/config.yml + +## Access UI + +OpenShift UI may be accessed via the 1st master node FQDN, port 8443. + +When using a bastion, you may want to make an SSH tunnel from your control node +to access UI on the `https://localhost:8443`, with this inventory variable: + + openshift_openstack_ui_ssh_tunnel: True + +Note, this requires sudo rights on the ansible control node and an absolute path +for the `openstack_private_ssh_key`. You should also update the control node's +`/etc/hosts`: + + 127.0.0.1 master-0.openshift.example.com + +In order to access UI, the ssh-tunnel service will be created and started on the +control node. Make sure to remove these changes and the service manually, when not +needed anymore. + +## Scale Deployment up/down + +### Scaling up + +One can scale up the number of application nodes by executing the ansible playbook +`openshift-ansible-contrib/playbooks/provisioning/openstack/scale-up.yaml`. +This process can be done even if there is currently no deployment available. +The `increment_by` variable is used to specify by how much the deployment should +be scaled up (if none exists, it serves as a target number of application nodes). +The path to `openshift-ansible` directory can be customised by the `openshift_ansible_dir` +variable. Its value must be an absolute path to `openshift-ansible` and it cannot +contain the '/' symbol at the end. + +Usage: + +``` +ansible-playbook -i <path to inventory> openshift-ansible-contrib/playbooks/provisioning/openstack/scale-up.yaml` [-e increment_by=<number>] [-e openshift_ansible_dir=<path to openshift-ansible>] +``` + +Note: This playbook works only without a bastion node (`openshift_openstack_use_bastion: False`). diff --git a/playbooks/openstack/openshift-cluster/install.yml b/playbooks/openstack/openshift-cluster/install.yml new file mode 100644 index 000000000..40d4767ba --- /dev/null +++ b/playbooks/openstack/openshift-cluster/install.yml @@ -0,0 +1,18 @@ +--- +# NOTE(shadower): the AWS playbook builds an in-memory inventory of +# all the EC2 instances here. We don't need to as that's done by the +# dynamic inventory. + +# TODO(shadower): the AWS playbook sets the +# `openshift_master_cluster_hostname` and `osm_custom_cors_origins` +# values here. We do it in the OSEv3 group vars. Do we need to add +# some logic here? + +- name: normalize groups + include: ../../byo/openshift-cluster/initialize_groups.yml + +- name: run the std_include + include: ../../common/openshift-cluster/std_include.yml + +- name: run the config + include: ../../common/openshift-cluster/config.yml diff --git a/playbooks/openstack/openshift-cluster/prerequisites.yml b/playbooks/openstack/openshift-cluster/prerequisites.yml new file mode 100644 index 000000000..0356b37dd --- /dev/null +++ b/playbooks/openstack/openshift-cluster/prerequisites.yml @@ -0,0 +1,12 @@ +--- +- hosts: localhost + tasks: + - name: Check dependencies and OpenStack prerequisites + include_role: + name: openshift_openstack + tasks_from: check-prerequisites.yml + + - name: Check network configuration + include_role: + name: openshift_openstack + tasks_from: net_vars_check.yaml diff --git a/playbooks/openstack/openshift-cluster/provision.yml b/playbooks/openstack/openshift-cluster/provision.yml new file mode 100644 index 000000000..fe3057158 --- /dev/null +++ b/playbooks/openstack/openshift-cluster/provision.yml @@ -0,0 +1,61 @@ +--- +- name: Create the OpenStack resources for cluster installation + hosts: localhost + tasks: + - name: provision cluster + include_role: + name: openshift_openstack + tasks_from: provision.yml + + +# NOTE(shadower): Bring in the host groups: +- name: normalize groups + include: ../../byo/openshift-cluster/initialize_groups.yml +- name: evaluate groups + include: ../../common/openshift-cluster/evaluate_groups.yml + + +- name: Wait for the nodes and gather their facts + hosts: oo_all_hosts + become: yes + # NOTE: The nodes may not be up yet, don't gather facts here. + # They'll be collected after `wait_for_connection`. + gather_facts: no + tasks: + - name: Wait for the the nodes to come up + wait_for_connection: + + - name: Gather facts for the new nodes + setup: + + +# NOTE(shadower): the (internal) DNS must be functional at this point!! +# That will have happened in provision.yml if nsupdate was configured. + +# TODO(shadower): consider splitting this up so people can stop here +# and configure their DNS if they have to. +- name: Populate the DNS entries + hosts: localhost + tasks: + - name: Populate DNS entries + include_role: + name: openshift_openstack + tasks_from: populate-dns.yml + when: + - openshift_openstack_external_nsupdate_keys is defined + - openshift_openstack_external_nsupdate_keys.private is defined or openshift_openstack_external_nsupdate_keys.public is defined + +- name: Prepare the Nodes in the cluster for installation + hosts: oo_all_hosts + become: yes + gather_facts: yes + tasks: + - name: Install dependencies + include_role: + name: openshift_openstack + tasks_from: node-packages.yml + + - name: Configure Node + include_role: + name: openshift_openstack + tasks_from: node-configuration.yml diff --git a/playbooks/openstack/openshift-cluster/provision_install.yml b/playbooks/openstack/openshift-cluster/provision_install.yml new file mode 100644 index 000000000..5d88c105f --- /dev/null +++ b/playbooks/openstack/openshift-cluster/provision_install.yml @@ -0,0 +1,9 @@ +--- +- name: Check the prerequisites for cluster provisioning in OpenStack + include: prerequisites.yml + +- name: Include the provision.yml playbook to create cluster + include: provision.yml + +- name: Include the install.yml playbook to install cluster + include: install.yml diff --git a/playbooks/openstack/openshift-cluster/roles b/playbooks/openstack/openshift-cluster/roles new file mode 120000 index 000000000..e2b799b9d --- /dev/null +++ b/playbooks/openstack/openshift-cluster/roles @@ -0,0 +1 @@ +../../../roles/
\ No newline at end of file diff --git a/playbooks/openstack/sample-inventory/group_vars/OSEv3.yml b/playbooks/openstack/sample-inventory/group_vars/OSEv3.yml new file mode 100644 index 000000000..1e55adb9e --- /dev/null +++ b/playbooks/openstack/sample-inventory/group_vars/OSEv3.yml @@ -0,0 +1,59 @@ +--- +openshift_deployment_type: origin +#openshift_deployment_type: openshift-enterprise +#openshift_release: v3.5 +openshift_master_default_subdomain: "apps.{{ openshift_openstack_clusterid }}.{{ openshift_openstack_public_dns_domain }}" + +openshift_master_cluster_method: native +openshift_master_cluster_hostname: "console.{{ openshift_openstack_clusterid }}.{{ openshift_openstack_public_dns_domain }}" +openshift_master_cluster_public_hostname: "{{ openshift_master_cluster_hostname }}" + +osm_default_node_selector: 'region=primary' + +openshift_hosted_router_wait: True +openshift_hosted_registry_wait: True + +## Openstack credentials +#openshift_cloudprovider_kind=openstack +#openshift_cloudprovider_openstack_auth_url: "{{ lookup('env','OS_AUTH_URL') }}" +#openshift_cloudprovider_openstack_username: "{{ lookup('env','OS_USERNAME') }}" +#openshift_cloudprovider_openstack_password: "{{ lookup('env','OS_PASSWORD') }}" +#openshift_cloudprovider_openstack_tenant_name: "{{ lookup('env','OS_TENANT_NAME') }}" +#openshift_cloudprovider_openstack_region="{{ lookup('env', 'OS_REGION_NAME') }}" + + +## Use Cinder volume for Openshift registry: +#openshift_hosted_registry_storage_kind: openstack +#openshift_hosted_registry_storage_access_modes: ['ReadWriteOnce'] +#openshift_hosted_registry_storage_openstack_filesystem: xfs + +## NOTE(shadower): This won't work until the openshift-ansible issue #5657 is fixed: +## https://github.com/openshift/openshift-ansible/issues/5657 +## If you're using the `openshift_openstack_cinder_hosted_registry_name` option from +## `all.yml`, uncomment these lines: +#openshift_hosted_registry_storage_openstack_volumeID: "{{ lookup('os_cinder', openshift_openstack_cinder_hosted_registry_name).id }}" +#openshift_hosted_registry_storage_volume_size: "{{ openshift_openstack_cinder_hosted_registry_size_gb }}Gi" + +## If you're using a Cinder volume you've set up yourself, uncomment these lines: +#openshift_hosted_registry_storage_openstack_volumeID: e0ba2d73-d2f9-4514-a3b2-a0ced507fa05 +#openshift_hosted_registry_storage_volume_size: 10Gi + + +# NOTE(shadower): the hostname check seems to always fail because the +# host's floating IP address doesn't match the address received from +# inside the host. +openshift_override_hostname_check: true + +# For POCs or demo environments that are using smaller instances than +# the official recommended values for RAM and DISK, uncomment the line below. +#openshift_disable_check: disk_availability,memory_availability + +# NOTE(shadower): Always switch to root on the OSEv3 nodes. +# openshift-ansible requires an explicit `become`. +ansible_become: true + +# # Flannel networking +#osm_cluster_network_cidr: 10.128.0.0/14 +#openshift_use_openshift_sdn: false +#openshift_use_flannel: true +#flannel_interface: eth1 diff --git a/playbooks/openstack/sample-inventory/group_vars/all.yml b/playbooks/openstack/sample-inventory/group_vars/all.yml new file mode 100644 index 000000000..921edb867 --- /dev/null +++ b/playbooks/openstack/sample-inventory/group_vars/all.yml @@ -0,0 +1,147 @@ +--- +openshift_openstack_clusterid: "openshift" +openshift_openstack_public_dns_domain: "example.com" +openshift_openstack_dns_nameservers: [] + +# # Used Hostnames +# # - set custom hostnames for roles by uncommenting corresponding lines +#openshift_openstack_master_hostname: "master" +#openshift_openstack_infra_hostname: "infra-node" +#openshift_openstack_node_hostname: "app-node" +#openshift_openstack_lb_hostname: "lb" +#openshift_openstack_etcd_hostname: "etcd" +#openshift_openstack_dns_hostname: "dns" + +openshift_openstack_keypair_name: "openshift" +openshift_openstack_external_network_name: "public" +#openshift_openstack_private_network_name: "openshift-ansible-{{ openshift_openstack_stack_name }}-net" +# # A dedicated Neutron network name for containers data network +# # Configures the data network to be separated from openshift_openstack_private_network_name +# # NOTE: this is only supported with Flannel SDN yet +#openstack_private_data_network_name: "openshift-ansible-{{ openshift_openstack_stack_name }}-data-net" + +## If you want to use a provider network, set its name here. +## NOTE: the `openshift_openstack_external_network_name` and +## `openshift_openstack_private_network_name` options will be ignored when using a +## provider network. +#openshift_openstack_provider_network_name: "provider" + +# # Used Images +# # - set specific images for roles by uncommenting corresponding lines +# # - note: do not remove openshift_openstack_default_image_name definition +#openshift_openstack_master_image_name: "centos7" +#openshift_openstack_infra_image_name: "centos7" +#openshift_openstack_node_image_name: "centos7" +#openshift_openstack_lb_image_name: "centos7" +#openshift_openstack_etcd_image_name: "centos7" +#openshift_openstack_dns_image_name: "centos7" +openshift_openstack_default_image_name: "centos7" + +openshift_openstack_num_masters: 1 +openshift_openstack_num_infra: 1 +openshift_openstack_num_nodes: 2 + +# # Used Flavors +# # - set specific flavors for roles by uncommenting corresponding lines +# # - note: do note remove openshift_openstack_default_flavor definition +#openshift_openstack_master_flavor: "m1.medium" +#openshift_openstack_infra_flavor: "m1.medium" +#openshift_openstack_node_flavor: "m1.medium" +#openshift_openstack_lb_flavor: "m1.medium" +#openshift_openstack_etcd_flavor: "m1.medium" +#openshift_openstack_dns_flavor: "m1.medium" +openshift_openstack_default_flavor: "m1.medium" + +# # Numerical index of nodes to remove +# openshift_openstack_nodes_to_remove: [] + +# # Docker volume size +# # - set specific volume size for roles by uncommenting corresponding lines +# # - note: do not remove docker_default_volume_size definition +#openshift_openstack_docker_master_volume_size: "15" +#openshift_openstack_docker_infra_volume_size: "15" +#openshift_openstack_docker_node_volume_size: "15" +#openshift_openstack_docker_etcd_volume_size: "2" +#openshift_openstack_docker_dns_volume_size: "1" +#openshift_openstack_docker_lb_volume_size: "5" +openshift_openstack_docker_volume_size: "15" + +## Specify server group policies for master and infra nodes. Nova must be configured to +## enable these policies. 'anti-affinity' will ensure that each VM is launched on a +## different physical host. +#openshift_openstack_master_server_group_policies: [anti-affinity] +#openshift_openstack_infra_server_group_policies: [anti-affinity] + +## Create a Cinder volume and use it for the OpenShift registry. +## NOTE: the openstack credentials and hosted registry options must be set in OSEv3.yml! +#openshift_openstack_cinder_hosted_registry_name: cinder-registry +#openshift_openstack_cinder_hosted_registry_size_gb: 10 + +## Set up a filesystem on the cinder volume specified in `OSEv3.yaml`. +## You need to specify the file system and volume ID in OSEv3 via +## `openshift_hosted_registry_storage_openstack_filesystem` and +## `openshift_hosted_registry_storage_openstack_volumeID`. +## WARNING: This will delete any data on the volume! +#openshift_openstack_prepare_and_format_registry_volume: False + +openshift_openstack_subnet_prefix: "192.168.99" + +## Red Hat subscription defaults to false which means we will not attempt to +## subscribe the nodes +#rhsm_register: False + +# # Using Red Hat Satellite: +#rhsm_register: True +#rhsm_satellite: 'sat-6.example.com' +#rhsm_org: 'OPENSHIFT_ORG' +#rhsm_activationkey: '<activation-key>' + +# # Or using RHN username, password and optionally pool: +#rhsm_register: True +#rhsm_username: '<username>' +#rhsm_password: '<password>' +#rhsm_pool: '<pool id>' + +#rhsm_repos: +# - "rhel-7-server-rpms" +# - "rhel-7-server-ose-3.5-rpms" +# - "rhel-7-server-extras-rpms" +# - "rhel-7-fast-datapath-rpms" + + +# # Roll-your-own DNS +#openshift_openstack_num_dns: 0 +#openshift_openstack_external_nsupdate_keys: +# public: +# key_secret: 'SKqKNdpfk7llKxZ57bbxUnUDobaaJp9t8CjXLJPl+fRI5mPcSBuxTAyvJPa6Y9R7vUg9DwCy/6WTpgLNqnV4Hg==' +# key_algorithm: 'hmac-md5' +# server: '192.168.1.1' +# private: +# key_secret: 'kVE2bVTgZjrdJipxPhID8BEZmbHD8cExlVPR+zbFpW6la8kL5wpXiwOh8q5AAosXQI5t95UXwq3Inx8QT58duw==' +# key_algorithm: 'hmac-md5' +# server: '192.168.1.2' + +# # Customize DNS server security options +#named_public_recursion: 'no' +#named_private_recursion: 'yes' + + +# NOTE(shadower): Do not change this value. The Ansible user is currently +# hardcoded to `openshift`. +ansible_user: openshift + +# # Use a single security group for a cluster (default: false) +#openshift_openstack_flat_secgrp: false + +# If you want to use the VM storage instead of Cinder volumes, set this to `true`. +# NOTE: this is for testing only! Your data will be gone once the VM disappears! +# openshift_openstack_ephemeral_volumes: false + +# # OpenShift node labels +# # - in order to customise node labels for app and/or infra group, set the +# # openshift_openstack_cluster_node_labels variable +#openshift_openstack_cluster_node_labels: +# app: +# region: primary +# infra: +# region: infra diff --git a/playbooks/openstack/sample-inventory/inventory.py b/playbooks/openstack/sample-inventory/inventory.py new file mode 100755 index 000000000..47c56d94d --- /dev/null +++ b/playbooks/openstack/sample-inventory/inventory.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python +""" +This is an Ansible dynamic inventory for OpenStack. + +It requires your OpenStack credentials to be set in clouds.yaml or your shell +environment. + +""" + +from __future__ import print_function + +import json + +import shade + + +def build_inventory(): + '''Build the dynamic inventory.''' + cloud = shade.openstack_cloud() + + inventory = {} + + # TODO(shadower): filter the servers based on the `OPENSHIFT_CLUSTER` + # environment variable. + cluster_hosts = [ + server for server in cloud.list_servers() + if 'metadata' in server and 'clusterid' in server.metadata] + + masters = [server.name for server in cluster_hosts + if server.metadata['host-type'] == 'master'] + + etcd = [server.name for server in cluster_hosts + if server.metadata['host-type'] == 'etcd'] + if not etcd: + etcd = masters + + infra_hosts = [server.name for server in cluster_hosts + if server.metadata['host-type'] == 'node' and + server.metadata['sub-host-type'] == 'infra'] + + app = [server.name for server in cluster_hosts + if server.metadata['host-type'] == 'node' and + server.metadata['sub-host-type'] == 'app'] + + nodes = list(set(masters + infra_hosts + app)) + + dns = [server.name for server in cluster_hosts + if server.metadata['host-type'] == 'dns'] + + load_balancers = [server.name for server in cluster_hosts + if server.metadata['host-type'] == 'lb'] + + osev3 = list(set(nodes + etcd + load_balancers)) + + inventory['cluster_hosts'] = {'hosts': [s.name for s in cluster_hosts]} + inventory['OSEv3'] = {'hosts': osev3} + inventory['masters'] = {'hosts': masters} + inventory['etcd'] = {'hosts': etcd} + inventory['nodes'] = {'hosts': nodes} + inventory['infra_hosts'] = {'hosts': infra_hosts} + inventory['app'] = {'hosts': app} + inventory['dns'] = {'hosts': dns} + inventory['lb'] = {'hosts': load_balancers} + + for server in cluster_hosts: + if 'group' in server.metadata: + group = server.metadata.group + if group not in inventory: + inventory[group] = {'hosts': []} + inventory[group]['hosts'].append(server.name) + + inventory['_meta'] = {'hostvars': {}} + + for server in cluster_hosts: + ssh_ip_address = server.public_v4 or server.private_v4 + hostvars = { + 'ansible_host': ssh_ip_address + } + + public_v4 = server.public_v4 or server.private_v4 + if public_v4: + hostvars['public_v4'] = public_v4 + # TODO(shadower): what about multiple networks? + if server.private_v4: + hostvars['private_v4'] = server.private_v4 + + node_labels = server.metadata.get('node_labels') + if node_labels: + hostvars['openshift_node_labels'] = node_labels + + inventory['_meta']['hostvars'][server.name] = hostvars + return inventory + + +if __name__ == '__main__': + print(json.dumps(build_inventory(), indent=4, sort_keys=True)) |