DevOps means Test Automation (too)

You have started down the road to DevOps. You have re-structured your teams and you are experimenting with DevOps tools and processes. You now understand that DevOps is a continuum that starts with planning and development and ends with deployment into operations. So where do you start your DevOps initiative?

DevOps projects tend to start in only one part of the continuum. Often, DevOps teams start with application build automation (development) or they start with automating the deployment of apps into operations. When the driver for DevOps is in the test organization, it is called Continuous Integration. However, it is also important to work to continuously expand automation across the continuum because for DevOps to pay off, there needs to be continuous automation from development all the way through to operations.

Test is like the Rodney Dangerfield of DevOps -- it gets no respect. But, that is all about to change. In a recent Gartner survey, over 50 percent of survey respondents listed test automation as a top enabler to DevOps success. This is really illustrative of the recognition that DevOps cannot succeed if only one part of the DevOps continuum is automated. In fact, the problem of test automation is particularly difficult and this means that the team that implements test automation is in a great position to lead DevOps initiatives beyond test.

Continuous Integration Challenges

Why is automating the various testing steps (regression, performance, stress, GUI, QA, Interoperability and security testing) so difficult? There are five key components that must be automated successfully:

1. The infrastructure -- It starts with one very important characteristic that is critical to testing which is not automated -- the ability to automatically mimic the infrastructure in which the software (or hardware) being tested is expected to run in when it gets to production. Having the ability to give each tester (or set of automated tests) their own personal replica of the target production infrastructure -- aka a "sandbox" is critical to ensuring that the software coming out of test can be automatically and continuously deployed into production. Automating tests is not nearly as difficult as automating the underlying infrastructure in which those tests run.
2. The production environment workload -- Often not considered in automating testing is the requirement to mimic the workloads that would be common in production. This includes the network traffic, other application workloads, and network security profiles. These activities need to be automated within the Sandbox to create a more realistic environment for testing.
3. The Tests themselves -- Of course, the tests themselves also need to be automated and there are a variety of tools to assist in this process. As software gets more complex, this continues to be a challenge, with mobile testing replacing GUI testing as the most challenging type of testing to automate.
4. Reporting and result analysis -- In order to provide continuous automation from development through testing to delivery, it is important to see and analyze test results automatically.
5. Tool Integration -- Finally, automation into and out of testing needs to be enabled. This means that Sandboxes and test automation suites need to be API driven so that they can be started by a previous tool in the DevOps toolchain and can also initiate a tool that follows testing in the toolchain.

Sandbox is a key enabler

Sandboxes are self contained infrastructure environments that can be configured to look exactly like the final target deployment environment, but can be created and run anywhere. For example, developers can create a sandbox that looks like the production environment -- from network and hardware to OS versions and software to cloud APIs. They do their development in that sandbox for a short period of time and when they are done they tear down the sandbox. Testers can do the same thing, and in addition, they can run a bunch of tests with the sandbox configured to look like their internal IT environment and then automatically re-configure the sandbox on the fly to look like the external cloud environment and run more tests. This allows them to test all of the possible environments that the application could run in without disrupting the actual production infrastructure.

Technically, what is a Sandbox? Intuitively we know that a sandbox is a protected space where you have complete control and others are allowed in only if you invite them. You can bring in your own toys to the sandbox and make anything you want in the sand. Just stomp it out if you don’t like it and start over. Technically, sandboxes follow these same rules. A number of vendors are now providing Sandbox solutions (some are called "Environment as a Service") that have a simple interface for creating any target infrastructure environment and configuring it with as much control as you want. They allow you to bring applications, tools, tests and automated processes into that sandbox. They provide protections so that others cannot mess with any infrastructure that you are currently using in your sandbox. They provide reservation and scheduling for many people so that whole teams of developers and/or testers can share physical and virtual infrastructure on-the-fly for hours, days or weeks at a time. Finally, a good Sandbox solution can be triggered from the outside (for example, from a Devops tool).

Continuous integration is a great place to start DevOps initiatives because test teams already recognize the value of automation and they are already thinking about how to replicate the underlying production environment early in the DevOps cycle. The methodologies and tools used for test automation can often be the building blocks for automating the end to end DevOps cycle.

Image Credit: Shutterstock/Sergey Nivens

Joan Wrabetz is the Chief Technology Officer for Quali. Prior to her current role, she was the Vice President and Chief Technology Officer for the Emerging Product Division of EMC. Joan has over 20 years of executive management experience at public and privately held technology companies. She has been an executive at a number of startup technology companies, has been a Venture Partner with BlueStream Ventures, and has been on the board of directors or advisory board of many early stage technology companies.