Automation: The key to continuous testing
One way continuous testing can help pick up the pace of testing is with automation. Agile and continuous testing have a tremendous amount of potential value to add to organizations, but if they aren’t fully automated it can be very taxing on the organization and create a lot of overhead, according to Parasoft’s Brown.
Tricentis’ Platz explains automation allows teams to test at the speed of agile and DevOps; reduce the time and effort it takes to obtain realistic and accurate test results; minimize test maintenance; and reduce false positives.
In order to successfully deploy test automation, teams need to have a good understanding of what test automation is, how it can be leveraged, and where it is going to be most successful based on the type of product, according to TechExcel’s Hammon.
This requires connecting a number of tools together within the overall software development chain, Brown explained. “Automation tools need to be very easily integrated with the continuous testing tools to really power the continuous testing process,” he said. Brown adds organizations need to adopt tools that enable service virtualization to support automation. “It is the easiest way to create test environments on demand without the effort. The process and vision of continuous testing needs to be completely coupled with the technologies and automation to realize the value, otherwise it is going to create a lot of overhead for teams,” Brown added.
In addition, test teams should be comfortable with developer assets to write test cases in the form of automated scripts, according to Kelly Emo, director of life-cycle and quality product marketing at Hewlett Packard Enterprise (HPE). “They need to be comfortable with IDEs, programming languages and application models. Whether Java, C#, Java script or other, they have to be comfortable working at the code level to create the most effective and reusable automation assets,” she said.
However, businesses should note that continuous testing is not just another buzzword for test automation. According to Dan McFall, president of Mobile Labs, automation is just a piece of the continuous testing puzzle, and it doesn’t mean that every test is going to be automated or even continuously tested. “You have to design the process and figure out where things are going to go, and then you figure out which technologies are going to come fit in and be in support of that,” he said.
In addition, automation doesn’t just apply to testing. Automation can be applied to data transfer, data flows, deployment, builds, and build management, according to McFall.
According to Rogue Wave’s Cope, in an ideal world everything would be fully automated, but realistically that is difficult unless you invest early on in the codebase, design and architecture. Cope adds at least on the UI side teams are always going to have to have some manual check and review in place if not heavy testing with every release. This is because things as simple as fonts and colors are hard to automatically test.
In addition, it requires a complex infrastructure to set up application servers, web servers, databases and configure all the different moving parts to be deployed in an isolated environment and run all the tests. However, more teams have been moving to containerization to make it easier to get environments up and running quickly, automate the process, and automate all the testing around it, Cope explained.
Other pieces of the puzzle
On top of automation, there are a multitude of other key strategies and tools necessary for a continuous testing workflow. “The continuous testing journey is a change. You can’t just jump into things, you need to take care of several other things first,” said Tricentis’ Platz.
For instance, Tricentis provides a continuous testing maturity model where customers can assess where they are today, and come up with a plan on how to get to the most mature stage.
“While we’ve all heard about organizations that were ‘built to last,’ the truth is, the only way an organization can survive and thrive today is to make sure they are ‘built to change,’” said CA’s Ravichandran.
Parasoft’s Brown says organizations need to invest in their developers, testers and release managers, and train them on what it means to be continuously testing. Then, you must train them on the environment and on the tools so they realize the value of what they invest in versus buying a product, not seeing its full potential, and becoming shelfware. “Training is probably the biggest element outside of the necessary tools and technologies needed to really power this, and make sure customers are successful at their continuous testing endeavors,” he said.
That requires a lot of communication, according to TechExcel’s Hammon. Teams need to understand why they are moving in a certain direction and choosing specific tools.
Communication is also a good skill QA engineers should have. That way, if an issue comes up, they can easily explain to the developer what happened and why it needs to be fixed. “They could also communicate with people that may be on the marketing or product management side to fully understand the initial goal of the feature to make sure they are really able to provide test cases that verify and validate the feature is actually implemented correctly,” Hammon said.
In addition, communication means taking into account customer feedback so teams can integrate that into the functionality of their system and make sure they are testing the right things. Having a good feedback loop in place gives teams the ability to see the overall health of the solution and provide early warning into issues, according to Mobile Labs’ McFall. “Without metrics or a good strategy, teams will be lost on whether or not they are actually delivering what the customer is expecting. If you don’t have a clear destination, just because you are going fast doesn’t mean you are going to the right place.”
HPE’s Emo explains there will always be some defects that leak out into production, but if you have that closed feedback loop and communication and an incident is found in production, it can get back to the agile team fast and they can quickly fix the problem.
CA’s Ravichandran adds to have a successful continuous testing strategy, teams need integrated solutions that enable “1) test environment simulation, 2) automatic test case creation, even from requirements, 3) on demand test data management, 4) always available SaaS-based performance testing, 5) open source integrations with tools like JMeter, Jenkins, Selenium, Appium, and 6) automated application performance monitoring and post-production performance testing to achieve continuous feedback.”
