Is the QA lab dead?

Quality assurance is changing. Internal teams too often don’t have the resources necessary to get releases to market on time, on budget, and without bugs. And offshoring comes with its own share of issues like irrelevant results and inefficient testing. In its current form -- with traditional lab and outsourced testing -- traditional QA is dying.

There are more than 45,000 unique mobile device profiles on the market today. And that’s just the mobile devices. Forget about desktop computers, connected devices, or any of the other places apps can live and be used on a day-to-day basis. Then add in all the different operating system and browser variations and it becomes clear internal teams need help to test on a massive breadth of devices.

But device coverage is only the tip of the iceberg. The biggest problem facing internal teams is a lack of code coverage. In a 2018 market survey by Applause, 60 percent of respondents said they are not confident in their level of code coverage. The 1,000 respondents kept coming back to the common themes of efficiency, coverage, and unproductive work because of a high noise-to-signal ratio. In fact, 80 percent of functional buyers rated themselves a "D" or "F" when surveyed about the quality of their test results.

These findings are not due to a lack of expertise by internal QA teams. They have more to do with the amount of time available to test and the speed at which results need to be gathered to keep up with release schedules. Internal teams want to test 100 percent of their code, but they simply don’t have the time or resources to do so. This isn’t a challenge that is solvable with traditional testing methods. Unfortunately for QA departments, they can’t exactly add more hours to the day.

Outsourced models, too, have proven time and again to limited be in their effectiveness. While they can be a cheaper alternative to traditional labs, outsourced models can’t account for real-world use cases and provide only a sliver of the test coverage enterprises need.

So, yes, testing is definitely changing. But let’s be clear: there will always be a place for internal QA embedded within Agile teams. However, they will have to evolve and combine their efforts with automation, on-demand testing, and real-world feedback.

Many companies have already realized the need for this evolution. According to KPMG, use of test automation grew 85 percent in the past few years, which is a great start. But many of these same companies are still struggling because automation is not a silver bullet solution. At least not on its own.

Augmenting internal and automated testing with on-demand manual and exploratory testing done in real-world scenarios can help brands get a complete picture of how their digital properties will be utilized in the daily lives of customers.

On-demand testers expand the coverage of internal teams. They can follow a set script of actions to ensure the right results are reached, or they can do exploratory testing to try and 'break' an app -- finding any use cases that were not planned for, but are relevant to real-world users. Access to on-demand resources means QA teams can kick off test cycles on a Friday, then come back into the office on Monday to find they have complete bug reports with screenshots, bug logs, and video captures so they can easily reproduce all issues discovered while they were out of the office.

The ability to test in real-world scenarios and locations is as essential as having an on-demand team standing by. Apps operate differently in different situations; what works on an iPhone X in California won’t necessarily work the same on an iPhone 6s in Rhode Island. Real-world testing gives internal QA teams the breadth of test coverage they need by providing real people to test in real locations on real devices. That means companies aren’t just testing with employees at the corporate headquarters. They are seeing how an app operates on Android phones running the Marshmallow operating system while switching from mobile data to Wi-Fi on the subway ride into the city.

There is no replacement for the QA lab and internal testing will always be necessary for most organizations. But automation, on-demand testing, and real-world feedback can augment traditional lab testing to help QA teams find bugs faster, more efficiently, and with more confidence. Organizations need to investigate their options and find what balance of these strategies works best for them. The companies that evolve quicker will surpass their competition as release cadences speed up and customer demands grow.

Image Credit: BackgroundStore/Shutterstock

Rob Mason is SVP, Engineering. at Applause and has more than 20 years of operational, management, and software development experience across many different companies, languages, platforms, and technologies. A meticulous builder and obsessive tester with an eye for talented engineers, Rob’s teams produce innovative, robust software. And, through his own example of hard work and ingenuity, he inspires his teams to reach new heights.