The perception of Ansible as a "slow" automation tool is a recurring theme among DevOps engineers and system administrators, ranging from those deploying simple one-liner commands to those orchestrating massive microservices architectures. While Ansible is lauded for its agentless architecture and simplicity, this very design introduces a series of architectural overheads that can lead to significant performance degradation if not properly understood and tuned. The "slowness" is rarely the result of a single bug but is instead a cumulative effect of SSH session management, Python interpreter discovery, temporary file orchestration, and configuration precedence. When a user executes a seemingly simple task, such as running the uptime command or looping through a small set of files, they are not triggering a single event but rather a complex sequence of remote executions. This article provides an exhaustive technical autopsy of why Ansible feels slow, the hidden mechanisms occurring behind the scenes, and the specific configuration levers available to mitigate these bottlenecks.
The Anatomy of a Single Ansible Task Execution
To understand why Ansible may feel sluggish, one must look past the abstraction of the playbook and examine the actual shell sessions initiated on the target host. A naive assumption is that running a command like ansible all -m command -a "uptime" results in a single SSH connection and a single command execution. In reality, the process is significantly more "chatty," often involving up to eight separate SSH sessions just to execute one simple command.
The following table breaks down the sequential operations Ansible performs during a standard module execution:
| Session Sequence | Action | Technical Command/Process | Purpose |
|---|---|---|---|
| Session 1 | Home Directory Discovery | /bin/sh -c 'echo ~ && sleep 0' |
Locates the user's home directory to determine where to place temporary files. |
| Session 2 | Temporary Directory Creation | /bin/sh -c '( umask 77 && mkdir -p ... )' |
Creates a secure, unique temporary working directory for the module payload. |
| Session 3 | Python Discovery | /bin/sh -c 'echo PLATFORM; ... command -v python3.12 ...' |
Probes the system for available Python versions to find a compatible interpreter. |
| Session 4 | Interpreter Validation | /bin/sh -c '/usr/bin/python3.12 && sleep 0' |
Executes a test run of the chosen Python interpreter to ensure it is functional. |
| Session 5 | Payload Transfer | SFTP/SCP Transfer | Uploads the bundled module (e.g., AnsiballZ_command.py) to the temporary directory. |
| Session 6 | Execution | Execution of the module | The actual Python script is executed on the remote host. |
| Session 7 | Cleanup | Removal of temp files | Deletes the temporary directory and the uploaded module script. |
| Session 8 | Session Closure | SSH Disconnect | Finalizes the connection to the remote host. |
This multi-step process explains why a simple command that takes milliseconds in a native Bash shell can take seconds in Ansible. The overhead is not the command itself, but the administrative scaffolding required to ensure the module runs in a consistent, isolated, and predictable environment.
The High Cost of Looping and Module Overhead
A critical performance bottleneck occurs when utilizing loops (loop or with_items) in conjunction with modules that require remote execution, such as the command or shell modules.
When a playbook loops over a list of items using the command module, Ansible does not send a single script containing a loop to the remote host. Instead, it executes the entire module lifecycle—discovery, upload, execution, and cleanup—for every single iteration of the loop. In a benchmark scenario where a task loops ten times to execute echo test, the execution time can reach 1.74 seconds on a localhost connection. This is approximately 87 times slower than an equivalent Bash script, which can complete the same operation in 0.02 seconds.
The technical reason for this disparity is that the debug module, by contrast, is executed locally on the control node. A loop involving the debug module returns immediately because it does not incur the cost of SSH sessions or remote Python discovery. However, the command module requires a remote shell, making it unviable for tasks involving very large lists unless optimized. When using ANSIBLE_DEBUG=1, it becomes evident that a substantial amount of time is spent within the communicate() function, which handles the input and output streams between the control node and the remote target.
Configuration Precedence and the Hidden Impact of ansible.cfg
Many users experience intermittent slowness because they are unaware of which configuration file Ansible is actually using. The ansible.cfg file controls critical performance settings, such as pipelining, but its application is governed by a strict order of precedence. If a user has a global configuration in /etc/ansible/ansible.cfg but also possesses a local ansible.cfg in the playbook directory, the local file takes precedence.
The order of precedence is as follows:
ANSIBLE_CONFIG(Environment Variable)ansible.cfg(Current working directory).ansible.cfg(User's home directory)/etc/ansible/ansible.cfg(Global default)
A common failure scenario occurs when a developer enables pipelining = True in the global configuration to speed up playbooks but then creates a local ansible.cfg to override the roles_path. Because Ansible stops searching for configuration files as soon as it finds the first one, the local file (which lacks the pipelining setting) disables the optimization, leading to a sudden and unexplained drop in performance.
Network and OS-Level Bottlenecks: The Solaris Case Study
Performance issues are not always rooted in Ansible's internal logic; they can be caused by the interaction between the SSH client and the target operating system's daemon. In environments using Solaris 10 or 11, extreme slowness has been reported during file copies or simple tasks.
One primary cause of this latency is the reverse DNS lookup performed by the Solaris SSH daemon. When a connection is initiated, the SSH daemon attempts to resolve the IP address of the connecting machine to a hostname. If the DNS configuration is missing or incorrect, the daemon will wait for the lookup to time out before allowing the connection to proceed. This adds several seconds of latency to every single SSH session. Given that a single Ansible task can trigger multiple sessions, this delay is multiplied exponentially.
Furthermore, legacy systems may lack support for modern SSH optimizations like ControlPersist. Users on such systems may find that they must clear ssh_args in their configuration, which removes the ability to reuse existing SSH connections, forcing Ansible to perform a full handshake for every operation.
Optimization Strategies for High-Performance Automation
To combat the inherent overhead of agentless automation, several technical optimizations must be implemented.
Pipelining
Pipelining reduces the number of SSH operations required to execute a module. Normally, Ansible transfers a module file to the remote host and then executes it via a separate SSH call. With pipelining enabled, Ansible executes the module by piping the Python code directly into the remote Python interpreter's stdin. This eliminates the need to upload the file to a temporary directory and then call it, significantly reducing the number of SSH sessions.
ControlPersist
ControlPersist is an SSH feature that allows the SSH client to reuse a single established connection for multiple sessions. Instead of performing a full TCP handshake and authentication for every task, Ansible can reuse a "socket" connection. This is particularly effective when playbooks contain many small tasks, as it bypasses the most time-consuming part of the SSH process.
Callback Plugins for Bottleneck Identification
To avoid guessing which task is causing slowness, users should employ callback plugins. These plugins provide detailed timing information for every task in a playbook, allowing engineers to identify "slow" tasks that may appear simple but are actually consuming the bulk of the execution time.
Strategic Use of Local Execution and Shell Scripts
When dealing with massive lists that require looping, the overhead of the command module becomes prohibitive. In these cases, the most effective optimization is to move the logic into a single shell script that is uploaded once and executed once, or to use a specialized module that handles the list internally rather than relying on the loop keyword.
Conclusion
The "slowness" associated with Ansible is a byproduct of its commitment to being agentless and its requirement for a consistent execution environment across diverse target systems. The process of discovering the home directory, verifying Python interpreters, and managing temporary files creates a "chatty" communication pattern that manifests as latency. This latency is amplified by configuration errors, such as ignoring the precedence of ansible.cfg or neglecting the impact of DNS timeouts on the target OS. By implementing pipelining, leveraging ControlPersist, and understanding the underlying session lifecycle, administrators can transform Ansible from a perceived bottleneck into a high-performance automation engine. The key to optimization lies in reducing the number of SSH round-trips and minimizing the work performed during the module's setup phase.