Turbo-charge Your Ansible

Techy-Brain Cells Needed (1-50): 30

Introduction

Ansible is a very popular orchestration, deployment, and provisioning software tool. In most cases, Ansible will automate your workload in an efficient and rapid manner. However, for certain processes and sizeable workloads, you may need to tweak your Ansible settings in various ways to achieve faster throughput. Here are several methods to make your coffee break shorter while waiting for your playbook to finish executing.

1. Forks

Forks are the default number of parallel processes to spawn when communicating with remote hosts. If you are orchestrating your playbook across many servers, then you may have noticed that the number of servers where the playbook executes simultaneously is 5. You can increase this number inside the ansible.cfg file:

# ansible.cfg 
forks = 10

or with a command line argument to ansible-playbook with the -f or –forks options. Note that this setting is dependent upon your hardware and software capabilities, such as network bandwidth and memory available on the control machine. Limit your forks according to your available resources.

2. Async

If you are executing a playbook with tasks that are not required by later tasks, or are not required until much later, then you can potentially derive a very significant speedup from the use of async.

Tasks in playbooks block by default, which means the connections stay open until the task is done on each node. With async you can launch a task and then immediately move on to the next task in a playbook.

---  
- hosts: all 
  remote_user: root 

tasks: 
- name: sleep 
  command: /bin/sleep 50 
  async: 60 
  poll: 10 
- name: not waiting for the device to wake up 
  command: /bin/move_on

If you truly do not care if and when the task completes, then you can set the poll value to 0 to ”fire and forget.” On the other hand, you can also employ a bit more responsible ”fire and forget” usage if there is a task occurring much later that does require the first task to eventually succeed.

--- 
- hosts: all 
  remote_user: root 
  tasks: 
  - name: sleep 
    command: /bin/sleep 50 
    async: 60 
    poll: 0 
    register: sleeper 
... 
  - name: waiting for the device to wake up 
    command: /bin/waiting_first 
    async_status: jid={{ sleeper.ansible_job_id }} 
    register: job_result 
    until: job_result.finished 
    retries: 30

This clearly can yield significant execution time improvements, but also requires a certain level of caution and responsibility correlated to the level of ambition implementing this feature.

3. Free Strategy

If you are executing a playbook without inter-node dependencies, such as provisioning and configuring a set of web servers, then you probably want each host to execute the playbook as quickly as possible and not wait for the other hosts to finish a task. By default, Ansible uses the linear strategy wherein each task waits for every host to finish before moving on to the next task. However, for many situations we just want each host to finish the playbook as quickly as possible. Ansible 2.0 introduced a new free strategy to achieve this.

---  
- name: setup webserver 
  hosts: webservers 
  strategy: free 
  tasks:

For example, if you are installing Apache on ten different servers and one of them has a yum lock, do you really want the other nine servers to wait around on the one server? With this strategy, those other nine servers can finish their tasks without waiting for the one server. For playbooks with inter-node dependencies, this could be dangerous, but for most use cases this is ideal.

4. Disable Fact Gathering

When executing a playbook, the facts are gathered and stored in a local memory cache on the control machine. In most instances this is fine, but for large numbers of servers with large sets of facts, this can quickly cause playbook slowdown on the local machine, or even crash the entire playbook due to lack of available memory. If you are not planning on using any facts for a specific playbook task set, then it is highly advisable to disable the fact gathering for that set of tasks.

--- 
- name: setup postgres 
  hosts: dbservers 
  gather_facts: no 
  tasks:

You can potentially gain massive speedups in playbooks when disabling fact gathering for large sets of servers with large sets of facts. Many playbooks will require facts, but disabling fact gathering for playbooks that do not require them yields significant benefits.

5. Pipelining

Pipelining is the modern Ansible method of speeding up your ssh connections across the network to the managed hosts. It replaces the former Accelerated Mode. It reduces the number of ssh operations required to execute a module by executing many Ansible modules without an actual file transfer.

# ansible.cfg 
pipelining = True

Although this can result in a very significant performance improvement when enabled, there is a substantial caveat to this feature. When using sudo operations in your playbook, you must first disable requiretty in /etc/sudoers on the managed hosts. Pipelining is disabled by default be- cause requiretty is enabled by default for many distros.

6. Development Environment

The previous topics entailed speeding up your Ansible playbook execution, but what about improving your ability to write Ansible playbooks and roles?

The preferred method for developing Ansible playbooks is within the Atom text editor developed by Github Engineering. The auto-indenting and auto- completion it provides are very helpful for developing within Ansible. With the language-ansible package, Atom has complete support for the Ansible language and recognizes all .yml files as Ansible.

Additionally, the linter-ansible-syntax package provides instant feedback on syntax errors, the linter-ansible-linting package provides instant feedback on style warnings, and the ansible-snippets package provides quick module templates within the editor, saving you time typing and looking up documentation. With this development environment, your playbooks and roles can be developed two or three times as fast compared to classic text editors like vim or emacs.

Summary

Ansible is a fantastic automation software tool, but automation is even better when it is running as fast as it can. These tips will help you significantly speed up your Ansible development, testing, and execution, so you can make better use of your time, including getting back to sleep for those always popular overnight pages.

If you would like to read additional articles on Ansible, try: