Ansible
This infrastructure section is both a information source of current ansible usage options and a guide on how to use ansible. Ansible is a simple it automation software. It uses ssh to connect to machines and deploy/configure/install software and more. More information can be found on their website.
It is basically an easy way to execute a command on many machines at the same time or many commands on one machine.
For detailed information about documentation visit ansible-doc
Start Ansible
There is one Controller Machine: the machine where Ansible is installed, responsible for running the provisioning on the servers you are managing. Most of the time it is installed on the administrators working station. From there ssh keys need to be set up for all machines that should be managed by ansible.
To download the newest version of ansible add the the repository as described:
sudo apt-add-repository ppa:ansible/ansible sudo apt-get update
Set up already configured ssh hosts in ..ansible-scripts/hosts. Already configured = Passwordless ssh login on the hosts with ssh public keys, configured in the ~/.ssh/config file.
Introduction
You can group hosts with [some-name]. Later you can specify commands only for some groups.
Now you can use different commands:
| Command | Effect |
|---|---|
| ansible all -m ping | Ping all nodes |
| ansible all -m ping -u paulth | Ping all nodes as user paulth |
| ansible new -m ping -u paulth -b | Ping all nodes in group new as user paulth with privilige escalation as sudo |
| ansible all -m ping -u paulth -b –become-user batman | Ping all nodes as paulth sudoing to batman |
| ansible all -m setup | Show all information of remote system |
To run a live command use the a flag:
ansible all -a "/usr/bin/whoami" #run with 10 parallel forks ansible new -a "/sbin/reboot" -f 10
The next step is to differentiate between ad-hoc commands and playbooks. Ad-hoc commands are for quick tasks and commands that you don't want to save for later. Playbooks are different steps and commands packed into one yml file. They are useful for basic setup.
In the table above there were already examples for ad-hoc commands. For more commands have a look at this site.
For Playbooks have a look at the next section:
Playbooks/Roles
Executing a playbook is done from the root of the ansible-scripts directory e.g.:
ansible-playbook plays/vault_test.yml
This is a list with available playbooks that can be executed. the playbook most likely contains roles that are documented below:
vault-test
Usage/Result:
Checks if the ansible vault key is correct and displays all variables defined for group named 'test'
Variables:
| Name | Note |
|---|---|
| test_variable | variable stored in usual text-file and showed in the output |
| test_secret_variable | variable stored encrypted in an ansible vault file and also shown in the playbook run output! |
Modules:
| Name | Parameters/Keys | |
|---|---|---|
| debug | msg | to display statements during execution we used the module debug.The key msg is used for string outputs |
Notes:
In the playbook we could have used the original name of the secret variable as they were defined in the vault file but for practical reasons we prefer to store all variables in one file and when needed define secret variables separately and redefine them additionally in the non-secret file.
snipe-it
Usage/Result
Deploy a full SnipeIT installation. With an apache and Mysql database. The Result should be a fresh SnipeIT Installation available under snipeit_env_url, ready to create admin user and further configurations.
Important Variables
| Name | Roles | Note |
|---|---|---|
| common_hostname_chair | common | was defined inside the playbook and committed to the role 'common' |
| apache2_sites | apache | contains fqhn and the webroot |
| mysql_root_pass | mysql | MYSQL root password stored under defaults/vault.yml |
| mysql_user_pass | mysql | MYSQL user password stored under defaults/vault.yml |
Basic structure
this part explains the first part of the playbook (all other parts contain almost the same basic elements and are therefore ignored).
name: Perform common Tasks # the name of the task that would be displayed if you run your playbook
host: snipeit # name of the remote-machine you want to perform you tasks on
remote_user: i11 # connect to the remote-machine as the named user
become: yes # execute all commands as superuser -> No password is required!
gather_facts: false # if false is set, no facts(variables) about the remote-system
# are discovered and saved
tasks:
- include_role: # list of directories of roles which are used
name: roles/common
vars: # list of different variables (one variable can have more than
common_hostname_chair: snipeit-dev # one attribute: -> var-name.attribute)
tags: common # if you want to exclude or specify the tasks you want to
# perform in your playbook use ad-hoc commands:
# --tags <tag-name>; or
# --skip-tags <tag-name>
Included Roles
Fore more Information about the content and modules of each role go to the next chapter
| Name | main function |
|---|---|
| common | set the hostname to snipeit-dev.cm.in.tum.de |
| apache | deploy a apache server |
| mysql | secure installation of mysql |
| snipeit | installation of snipeit and preparation of all important folders and files |
adduser
Usage/Result:
$ ansible-playbook plays/adduser.yml -e "username=test sudo= host=testhost" # example execution
Results in a new user test with passwordless sudo rights on every host from the [testhost] - block from the hosts file.
Variables:
| Name | Required | Note | Where to put it? |
|---|---|---|---|
| username | yes | name of the user to create | within -e quote |
| sudo | no | if set (a simple sudo= is enough), gives the created user sudo rights (adds him to adm group) | within -e quote |
| host | no | define another hostblock to execute the playbook on (default is “add_users”) | within -e quote |
| ssh-key | yes | has to exist as <username>.pub | pubkeys folder |
| remote_user | yes | has to have sudo rights and passwordless ssh login on hosts | edit in playbook |
Notes:
This playbook requires a ssh-key called <username>.pub in the pubkeys folder, because it automatically adds it to the authorized_keys file of the created user on the hosts. If you don't want the user to create to have sudo rights, then don't pass a sudo= as an extra variable. Same goes for the host. Only pass those variables if you want to make use of them!
ssh_keys
Usage/Result:
$ ansible-playbook plays/ssh_keys.yml -e "username=test replace= host=testhost key_to_replace=replace" # example execution replace
Replaces the key replace.key with test.key in authorized_keys file on every host in the [testhost] block from the hosts file.
$ ansible-playbook plays/ssh_keys.yml -e "username=test append=" # example execution append
Appends the key test.key to authorized_keys file of user test on every host in the [ssh_keys] block from the hosts file.
$ ansible-playbook plays/ssh_keys.yml -e "username=test delete= host=otherhost" # example execution delete
Deletes the key test.key from authorized_keys file of user test on every host in the [otherhost] block from the hosts file.
Variables:
| Name | Required | Note | Where to put it? |
|---|---|---|---|
| username | yes | name of the user to edit the ssh-keys | within -e quote |
| host | no | define another hostblock to execute the playbook on (default is “ssh_keys”) | within -e quote |
| ssh-key <username>.pub | yes | has to exist at all times for execution | pubkeys folder |
| key_to_replace | no | Optional: name of the key to replace without .pub extension (default: <username>_delete.key). It (either variant) has to exist in the pubkeys folder! | within -e quote |
| append | Option 1 | If defined, appends the <username>.key to <username> authorized_keys file | within -e quote |
| replace | Option 2 | If defined, replaces (if passed as variable) <key_to_replace>.pub or <username>_delete.pub with <username>.pub for <username> in authorized_keys file | within -e quote |
| delete | Option 3 | If defined, deletes <username>.key from authorized_keys file of <username> | within -e quote |
| remote_user | yes | has to have sudo rights and passwordless ssh login on hosts | edit in playbook |
Notes:
This playbook requires a ssh-key called <username>.pub in the pubkeys folder, because it is the key, that will be appended/replaced with/deleted to/in/from the authorized_keys file of the user on the hosts. The key_to_replace variable is optional, which means you don't need to pass it for successful execution! It will try to use <username>_delete.pub instead, which has to exist in this case!
You can only pass and use one option for execution!
get_vm_users
Functionality
Parses the output of getent passwd as well as /etc/login.defs to list all local users on an OpenNebula or RBG-VM. The users are then stored in the according database. An entry for the VM is created if not already present. Note that this only works for local users; LDAP-Users will NOT be listed!
update_db
Functionality
Reads the contents of /etc/login.user.allowed and /etc/security/group.conf and creates corresponding user entries in the VM-database
lastlog
Functionality
retrieves the last date every user logged into the virtual machine and stores the result in the correct database (For either ONE or RBG-VMs). The variable users should be defined in the hostvars.
update-vm-users
Functionalit
clears and recreates the files /etc/login.user.allowed and /etc/security/group.conf, which control which users are allowed to log in and which users are granted sudo priviledges via the adm group.
Variables
| Name | Type | Notes |
|---|---|---|
| users | List of String | defines which users are allowed to log in. Should be defined in hostvars |
| sudousers | List of String | These users are added to the adm group upon login. Should be defined in hostvars |
new-one-vm
Functionality
Instantiates a new OpenNebula VM and creates a corresponding database entry.
Variables
| Name | Type | Notes |
|---|---|---|
| vm_name | String | will also be the Hostname of the VM |
| purpose | String | What the VM will be used for |
| network | String | either mwn or chair |
| users | List of String | Which users are allowed to log in |
| sudousers | List of String | Which users are granted sudo access |
| survey_cpu | Float | CPU Power allocated to VM |
| survey_vcpu | Integer | Number of CPUs the VM sees |
| survey_disk | String | Disk size with suffix (KB, MB or GB) |
| survey_memory | String | RAM size with Suffix |
| services | List of String | list of services to be installed on the VM (see deploy_host) |
| custom_vars | JSON Dictionary | list of extra variables (can be used to set configurations for services, etc.) |
add_ldap_users
Functionality
Creates necessary entries in /etc/login.user.allowed and /etc/security/group.conf to allow new user to log in and/or give them sudo access. Also creates or updates the corresponding database entries.
Variables
| Name | Type | Notes |
|---|---|---|
| login_users | List of String | New users to grant ssh login access |
| sudo_users | List of String | Users to grant sudo access. They are automatically also allowed to log in via ssh |
Roles
common
Functionality
Changes the hostname in /etc/hosts accordingly to the common_hostname_chair variable and disables SSH root login in all instances (do NOT change this, unless you REALLY need to). It also disables password authentication for SSH by default.
Important Variables
| Variable | Type | Function |
|---|---|---|
| common_hostname_chair | String | The system identifier under which the machine should be known to DNS and other hosts |
| disable_ssh_pw_login | Boolean | Default: yes. Change to no, if password authentication is desired |
mysql
Functionality
Rolls out a whole MySQL setup with databases on the remote machine.
Important Variables
| Name | Type | Function |
|---|---|---|
| mysql_root_pass | Vault Variable | MySQL root password. Edit in the vault.yml file, and ensure re-encryption after editing |
| mysql_user_pass | Vault Variable | Default password for new MySQL-users. Edit in the vault.yml file |
| mysql_dbs | Array | Consists of database name and state. Those are the databases, MySQL will have available after installation |
| mysql_users | Array | Consists of user name, state, password and priviliges the user will have |
| mysql_secure_installation | Boolean | Default: yes. Deletes anonymous user “” for the database |
| mysql_service_enabled | Boolean | Default: yes. Whether MySQL should be enabled, after installation |
| mysql_service_state | started/stopped | Default: started. The state MySQL will be, after installation |
ldap_login
Functionality
Enables LDAP-Authentication on a machine. migrates local users to ldap users if possible
Note: A former configuration used the file /etc/login.user-sudo.allowed to manage which users are granted sudo access. Now every user gets an entry in /etc/security/group.conf. The task-file migrate-sudo-users.yml may be used to update the configuration and create the new entries.
Important Variables
| Variable Name | Type | Function |
|---|---|---|
| users | List of String | LDAP users allowed to log in (should be in hostvars) |
| sudousers | List of String | LDAP users grantedd sudo access (should be in hostvars) |
apache
Functionality
Rolls out a whole Apache Installation with a custom webpage (content is needed though) and SSL and Redirects, if necessary and configured. For SSL functionality, you need to insert the required files in the ssl_certs directory of the apache role directory and define the correct filenames in the playbook (see SSL variables).
Important Variables
| Variable | Type | Function |
|---|---|---|
| apache2_packages | Array | Default: apache2. Packages needed for the Apache Installation |
| apache2_sites | Array | id: identifier used in sites-available fqdn: Domain name used for the webpage webroot: Directory, where the webpage content will be stored and accessed from ip: IP adress, under which the webpage will explicitly be able to be accessed from port: Port number used for the webpage state: Whether the webpage will be available after installation |
| ssl | Array | port: Port number under which SSL will communicate key_name: Name of the key file in ssl_certs cert_name: Name of the certificate file in ssl_certs chain_name: Name of the chain file in ssl_certs (this one is optional for SSL functionality) |
| apache2_remove_default | Boolean | Default: no. Whether the Apache Default pages should be disabled during installation |
| apache2_service_enabled | Boolean | Default: yes. Whether Apache should be enabled after installation |
| apache2_service_state | started/stopped | Default: started. The state the Apache service should be in after installation |
| apache2_modules | Array | id: Name of the module, which should be enabled |
snipe-it
| Name | Parameter | Function |
|---|
TODO
onevm
Functionality
Provides tasks to instantiate virtual machines on OpenNebula and manage LDAP-users who have access to them.
Task Files
| File | Functionality | Required Variables | Notes |
|---|---|---|---|
| instantiate.yml | instantiates a new virtual machine | vm_name, network, users, sudousers | includes tasks from addusers.yml |
| addusers.yml | grants login/sudo access on a vm to new users by updating database | VM_ID, users, sudousers | sudousers are automatically also granted login permission |
| removeusers.yml | revokes permissions on vm for specified users | VM_ID, users, sudousers | users specified in sudousers will still be able to log in, those specified in users will be revoked all permissions |
| updateusers.yml | updates /etc/login.users.allowed and /etc/security/group.conf. | users, sudousers | |
Variables
| Variable Name | Type | Function |
|---|---|---|
| vm_name | String | Name of the virtual machine to instantiate |
| network | String | Network to attach to VM. Either “chair” or “mwn” |
| users | List of String | Users who are allowed to log in |
| sudousers | List of String | Users who are allowed sudoaccess |
Write Playbooks
In general you are going to execute one Playbook. This Playbook consists of tasks, Roles or Modules which will be performed on the remote-machines you have listed on top of each task. If you wish to work with Variables you can define them either directly inside a playbook, or in a different file (see: directory structure). The concept Ansible Scripts are managed in this directory structure:
group_vars/ # variables for specific host groups
all.yml # variables which can be used by all roles/playbooks
dbservers/ # variables for e.g. dbservers host group
vars.yml # non secret variables
vault.yml # secret variables protected by ansible vault
... # other group folders as dbserver
play/ # folder with playbooks as one yml files
roles/ # contains roles used in the playbooks
common/ # folder with the role 'common' used for all servers, vms, etc.
defaults/ # contains default variables for role named 'common'
main.yml # non-secret variables
vault.yml # secret variables which must be encryptd by ansible vault!
tasks/ # tasks for common role
main.yml # task file
files/ #contains files which can be deployed via this role
handlers/ #contains handlers, which may be used by this role or even anywhere outside this role
mysql/ # mysql role folder
ansible.cfg # scripts specific ansible variables, e.g. point to vault password
hosts # text file with host groups, should contain all servers
vpass # ansible vault file containing the vault password, ignored by git!
- Add a new group_vars folder for the group your playbook is written for, create one
varsand onevaultyml file - Add group variables to the vars file, copy secret variables to vaul file and prepend with vault_, link values to vars file, e.g.
mysql_root_pass: '{{ vault_mysql_root_pass }}' - This can also be done for specific roles under
..role/defaults/vault.yml, it must be separately included in the role tasks file..role/tasks/main.yml- name: Include Vault Variables include_vars: ../defaults/vault.yml
- Encrypt vault file with ansible-vault
ansible-vault encrypt group_vars/dbservers/vault.yml # use different command for decryption in order to change passwords, etc. ansible-vault decrypt group_vars/dbservers/vault.yml
During the execution of an ansible-playbook passwords may be to be set for services. We save those passwords in the vault-file and encrypt those. In order to let ansible properly decrypt the files, ask your supervisor for the vault-password, safe it as vpass in the root directory of the repository and execute the Makefile.
It is very important to execute the Makefile at least once after cloning the repository and before the first commit. The Makefile sets up git hooks, that check whether every vault file in the repository is encrypted before every commit.
schlenga