There are few things in programming as imprecise as DevOps. The name itself is even in dispute with SysOps and SRE- also popular ways to describe the field. Not only that, there is no standard package of DevOps skills.
The role sits somewhere between that of a software developer and an operations engineer so the screening is highly specific to the exact skills the employer is looking for. So how do you screen for DevOps skills? To get a better idea, it is important to start at the beginning.
What is DevOps?
Fonte: Irma Kornilova
Created to support agile software development, a DevOps engineer sits somewhere in between a traditional software developer and an operations engineer. As Ernest Mueller defines it,
Fonte: the agile admin Fonte da imagem: the agile admin |
In the old days, developers would build a product and give it to operations to deploy it. If something went wrong, operations would make a support ticket and send it back to the developers who would create a fix and then send it back to operations. This was a very siloed approach that was as slow as it was rigid.
DevOps challenges this approach by bringing the dev team and operations team together to be continuously creating, deploying, maintaining, and improving applications. And it brings big rewards. The 2019 State of DevOps Report published by DevOps Research and Assessment found that the high performers (who instituted DevOps) have:-
- 208 times more frequent code deployments
- 106 times faster lead time from commit to deploy
- 2604 times faster mean time to recover from downtime
- 7 times lower change failure rate (changes are 1/7 as likely to fail)
This is all compared to organizations that don’t use DevOps.
What DevOps skills should a recruiter screen for?
Unlike other software engineering positions which have a defined skill set and focus, DevOps is a bit harder to pin down. As Mike Dilworth points out:
Fonte: Irma Kornilova Fonte da imagem: AngelList |
In other words, you need to hire for both domain-specific technical expertise as well as the ability to support and thrive in the agile culture that DevOps operates in. As a result, a DevOps engineer needs to have some combination of these skills.
Collaboration/soft skills
A DevOps engineer is first and foremost a member of a team. It is not a role that encourages standing out as an individual. It instead supports working together to keep iterating and improving a product. As Alan Zucker, principal of Project Management Essentials puts it,
Fonte: TechRepublic Fonte da imagem: LinkedIn |
Knowledge of tools and technologies
There are a lot of tools and technologies that could be used by a DevOps engineer. And they are often essential to maintaining the core concepts of DevOps. It is important to see if your candidate knows the technologies you use and is able to learn new technologies as needed.
Configuration management
This means being able to configure many systems using automation tools.
A DevOps engineer needs to be able to use tools and scripts to be able to deploy and maintain the app while they solve any issues that crop up along the way. Depending on the specifics of the position, security management testing skills can also be important to look for as well.
While configuration management tools are only a part of the toolbox that a DevOps engineer draws on, this is an area that can be easily tested, revealing a lot about the candidate along the way.
DevOps skills screening methods
Since DevOps skills vary so much the combination of a few different screen methods will get you the best results.
Check your candidates’ CVs for DevOps skills
A CV or resume screen is probably the first thing that most recruiters will do to narrow down the candidate pool. We have prepared a small glossary to help recruiters understand what they are looking for.
DevOps skills glossary for tech recruiters:
| Methodologies | Agile, DevOps, Continuous delivery, Continuous integration, Microservices, Test-driven development (TDD), Infrastructure as Code |
| Scripting Languages | Bash, Python, Ruby, Perl, PHP |
| Infrastructure Automation | Puppet, Chef, Ansible, Terraform |
| Source code management | Git, Mercurial, Subversion |
| Cloud Services | AWS, Azure, Google Cloud, OpenStack |
| Orchestration | Kubernetes, Nomad |
| Containers | Docker |
| Project Management | Jira, Trello, Asana |
| Continuous Integration and Continuous Delivery | Jenkins, Jenkins-X, Tekton, Travis CI, CircleCI, Azure, DevOps |
| Ensaios | TestComplete, TestingWhiz, Serverspec, Testinfra, InSpec, Consumer Driven Contracts |
| Monitoring | Prometheus, Nagios, Icinga, Zabbix, Splunk, ELK Stack, CloudWatch, Open Telemetry |
Once the CV screen is finished, there are two main skills that can be tested using an automated testing platform.
Whether you are an engineer deploying code or an operator keeping it going, being able to configure your server is an essential skill. But this skill can be split into two parts. The first is configuration management while the second is putting that to use in the configuration of a machine. And as you will come to see, both of these skills require different tools to effectively test them.
Configuration management DevOps skills require coding tests
One of the major pain points of a DevOps engineer is manually managing different server configurations. This can be a major pain as it turns the engineers into something more of an operation firefighter rather than someone who builds and deploys applications. Therefore, a key skill of a great DevOps engineer is the ability to multiply their effectiveness by writing configuration management code.
Put another way by Adam Kadzban, lead DevOps engineer at kCura,
Fonte: TechRepublic Fonte da imagem: LinkedIn |
The root of this attitude is the fact that a lot of manual tasks can be automated using dedicated tools like Puppet and Chef or even with scripting languages like Python, Ruby, and Bash. So the question becomes what is the best way to test these skills?
A work sample test is the best way to check scripting skills
A Puppet code isn’t simply an algorithm. Instead, it is an app that regulates the system. Because of this, a work sample test is probably the best way to test the right skills. We have explored the efficacy of work sample tests elsewhere on this blog and how to use them for common coding problems but how can they be applied to writing Puppet code for configuration management?
A good place to start is by looking at the process you need to go through to write an automation code. It is really pretty simple,
- Describe the configuration in code
- Run tests against it
The first thing your candidate needs to do is describe the configuration in code. So say you want them wanted to create a Puppet module for a file server. This is a pretty straightforward DevOps task which can be turned into a work sample test pretty straightforwardly, like in this task:
It is OK to have a premade module skeleton to point your candidate in the right direction and then define how you want the file server to be configured. When done with a platform like DevSkiller, the automation code that is produced can have unit tests run against it so that you can verify the robustness of the code and find any flaws.
And it is not just Puppet that can be tested this way. With DevSkiller’s test builder, your tech lead can build scripting tasks in multiple technologies including Chef, Bash, Python, Ruby, and Go. This test will give you a strong indication how well a DevOps Engineer can maximize their efficiency.
If a DevOps engineer can effectively automate tasks, they will then be able to move onto higher level tasks such as implementing new features. Not only that but once you start testing coding skills, you can move onto other DevOps functions such as testing git controls and writing code as infrastructure.
But writing automation code is only part of the puzzle. Sooner or later, a DevOps engineer is going to have to actually configure a machine. This is a problem as most companies don’t feel comfortable with a candidate getting control of one of their servers. After all, think about the havoc they could wreak on your system. But what would be even worse is if you hired somebody without knowing if they can actually configure a server without the worst happening.
Test system configuration DevOps skills with a virtual machine
A coding challenge can test how a script runs but you need an actual machine to see if a DevOps Engineer can configure a system. This is because servers are not like a program.
So how can you test whether a DevOps engineer can configure a server without using one of your own? Use a virtual machine. These are self-contained systems, completely separated from any of your critical systems, which allow you to do just about anything you want. And they are perfect environments in which to see if your DevOps engineer can manage all of the different inputs, changes, and issues they face configuring a system.
Rootskiller gives you a window onto your candidate’s DevOps skills
At its heart, Rootskiller is a virtual machine. Like any computer, you can give it commands, configure programs on it, and even run it into the ground if you want to. At the same time, it responds like any other system, processing your configurations while reacting to poorly executed commands and outside data. So how does this help you test DevOps?
Take a task like this one:
All of the technologies involved are common ones that a DevOps engineer will use for continuous deployment. GNU/Linux is one of the most widely used server-side operating systems and Docker is a common tool used for containerizations. These are an essential part of a continuous deployment, one of the main pillars of DevOps.
So after starting the virtual machine, the candidate will configure the server according to the instructions presented but that is not all. After the task is resolved, tests are run on the configuration that the candidate has created, checking to see if it works, what bugs are present, if it resolves the relevant edge cases, and how stable it is.
In addition to analyzing the actual solution, the entire session is recorded for a screen capture video. This way, a technical interviewer can see the choices that the candidate makes and see how they react to new challenges.
While the question here is about configuring Docker on a GNU/Linux machine, the test building tool means that you can set this up for many server-side technologies and deployment techniques like LAMP stack.
Terraform tests for testing Infrastructure as Code skills
Terraform is the tool of choice for managing cloud services across multiple platforms. It uses its own domain-specific language called HCL which closely resembles JSON. It is widely used to manage cloud infrastructure on various cloud platforms. It has a huge number of supported providers and it’s actually much more than a simple cloud-only tool. Because it has gained so much popularity over the years, it’s now a standard tool for managing everything from a declarative code. It’s been used by many organizations and is considered an essential tool in the DevOps community.
With Terraform, we are able to provide tasks that will test practical knowledge about various cloud providers. We believe that these tasks are better suited to the current DevOps landscape where writing Terraform code is so ubiquitous and requires a deeper understanding of the cloud APIs.
How we test Terraform code
We designed the Terraform tasks to make it easy for developers to create their own and to make the experience as smooth and fast as possible for candidates. We don’t require candidates to have an account on the cloud provider used by a task. Actually, we don’t even require the installation of Terraform – all a candidate needs is knowledge on how to write Terraform code that meets the requirements specified in a task. TalentScore will perform a dry-run (i.e. run “terraform plan” command) and will evaluate the results returned by the command. The cloud resources defined in the candidates’ submitted code will not be created so the feedback will be returned immediately to the candidate. It saves a lot of time and avoids any potential issues that could arise if the platform would actually create these resources.
To get a better experience of Terraform tasks we recommend candidates install Terraform on their workstations alongside some syntax checking software. We recommend VS Code with the official Terraform extension. It supports language server that provides autocompletion and built-in documentation. With this setup, working with Terraform code is fast and less error-prone.
What to do next
Once you have done a CV screen, tested your candidate’s scripting skills, and seen how well they can configure a system, bring the best candidates in for an interview. This can involve a combination of code pairing e behavior questions.
But before you do, check out DevSkiller to see how easy it is to automatically screen DevOps skills and for Infrastructure as Code skills please also check out what Terraform testing should look like.
