Automate? Always!

16 Sep 2022 - Giulio Vian - ~6 Minutes

Some years ago I gave a talk, titled “Automate? Always!", to stimulate a discussion about automation in software engineering. Today, I am putting the argument in writing to reach a bigger audience.

Please note: in this post I refer to any automation in software development lifecycle: spanning from editor macros to deployment scripts (I confess my preference for the latter as you will perceive).

Please bear with me because this will be a bit longer than other posts, although there is more to say that can fit a single article.

Common thinking: automation is cost-saving

Let’s start with this quote «our goal should be to automate all the activities that are repeated while creating and maintaining the system» ( Priti Biyani, ThoughtWorks , emphasis mine). ThoughtWorks suggests that repetion triggers automation, and, I note, the implicit background is of a greenfield project and its evolution.

In an less solemn, more graphical, way xkcd #1205 conveys the idea of time-saving.

If you do not spare time, Randall suggests, it is senseless implement some automation. You can find more examples in literature, blogs, and journals. The assumption is the same: automation is justified by time-saving, that is cost-saving.

Although this is plain common sense, I felt that reality is more nuanced and found a good list of reasons by Mattias Geniar . He enumerates six justifications:

to save time,
to gain consistency,
to gain speed, momentum and velocity,
to schedule tasks,
to reduce boring or less fun tasks,
to keep sysadmins and developers happy.

Here we start to see two new perspectives: psychology and operation. We can easily add that automation in software helps:

reducing human mistakes and manual errors,
improving process quality, through the design and analysis effort,
achieving more, for example by allowing people to do things without detailed knowledge of a system internals.

The rest of this article tries investigate these additional reasons for automating so you can better judge some less direct and intangible elements that may suggest that investing in automation is the right thing to do in a specific context.

Automation creates opportunities, cost-saving is a side-effect

I articulate seven dimension that adds value to the automation effort: exploring, process improvement, the ripple effect, human satisfaction, learning, inspection & validation, and tracking.

Exploring

You face a new application, a new system: people who worked on it has left, documentation is poor. What can you do?

Add automating is a way to understand an existing system. You may see the system as black box, and try to automate configuration changes, or write some deployment scripts. Maybe you add a CI pipeline replacing ad-hoc build scripts that assume a specific developer configuration.

All these activities requires some level of reverse engineering, and experimenting with the system in a semi-scientific way: you experiment and validate your hypothesis how the system interact with the environment. Configuration, dependencies, relations will emerge as the result of your trial-and-error attempts to automate it.

Improvement

Any automation removes a manual process. Isn’t self-evident? Not always. Removing human intervention is not simple and obvious: you may face resistance, power play, fear. Your automation must be transparent, and valuable. Transparency has two sides: one is the visibility of automation inner workings, the other is tracking automation steps for auditing and debugging. Value can have multiple facets:

an administrator reduces the time for a task,
a user has to wait less for a result,
more tasks completed without errors,
costly mistakes prevented by early checks.

In general we should consider both direct positive effects and negative, preventive effects; gains as well as reduction of costs.

Documenting

All of us have met a system poorly documented. Trying to automate is a way to describe its behavior. This approach is similar to the effect that many bloggers describe: to learn something a great technique is forcing ourselves to write about it. It is similar to implementing automated tests for an application: each test defines exactly a little bit of system’s behaviour. The value of automation is in what we may call “actionable documentation”: not a passive document, that requires interpretation, but something you can use to act on the system. Automating to document implies that the automation code is published in an easy-to-access location, where anybody interested in the target system will look for information.

Ripple Effect

Consider now these unintended positive effects:

success in automating a piece may induce or permit automating other pieces,
it may start a virtuous emulation trend and more people automates their chores with great gains for the overall organisation,
shows technical feasibility lowering barriers for other implementations,
demonstrates value to management.

Most of the items listed are psychological and reflects onto the organisation performance.

Satisfaction

Subtler psychological effects, less quantifiable though real, are:

happier people, delegating boring chores to machines and focusing on more interesting topics,
simplicity and ease of use, compared to following playbooks or ticking checklists,
that sense of accomplishment that any developer knows when a program is complete and works as expected,
when the automated task was quite complex, we reduce the mental effort for executing the task.

Learning

I mentioned before the idea of automate to learn, let’s talk now what one can learn. The most obvious things to pick up are new technologies and learning scripting languages. Less prominent are infrastructure concepts, experience with real practical constraints. What a magnificent way to appreciate network latency is copying files across the pond! Master how parallellism really works trying to minimize deployment times across a bunch of machines.

Inspect & Validate

Legacy, and a thrill goes down the spine of any engineer. Legacy system, legacy code, how can we trust it, how can we touch and change it? Well, automation is a way to inspect and review an existing component. Writing a build script, a deploy script, a script to change configuration, forces the author to review existing manual configurations, question their settings, and add automated validation checks along the process. I have often encountered configuration files that are just a chaos of unrelated properties, with no organisation nor standard names. Like coral or stalactites it grew unnoticed in the darkness. Time to question, refactor, and clean it!

Track

One last element to address is the gain in visibility. Automation makes easy to identify usage, dependencies, and configuration. Instead of searching for a Word document which states that you need to install version X of a component, this dependency is stated in plain view in an automation script. Automation allows collecting information and store it in a central place. An example is building an organisation-wide dependency database, crucial to manage security issues (Software Bill of Materials, SBOM).

Conclusions

automation.pngautomation.png ( Niall Murphy et al., Site Reliability Engineering, O′Reilly 2016 ) is a fundamental idea of modern software engineering. It’s about delegating more to machines and moving to a declarative style. Consider what to automate first, plan for adding incrementally more automation to your daily work, prioritise considering all effects, both direct and indirect.

Don’t fall into the trap of automating the wrong thing, at the wrong time: xkcd #1319

Capture the state before and after, better with at least one hard, objective metric. Communicate what you achieved, showing the benefits. Tell broadly displaying the measures: it will inspire others. Finally, do not forget that DevOps is not about automation. Automation supports the DevOps perspective.

References

What do you think? Am I wrong?

Architecture
DevOps Automation