Software Testing – Is It Time To Fire Your QA Team?

Nick HarleyResource, Web Development7 Comments

Raygun's latest feature released: machine affected

Traditionally, for software development and software testing teams that are part of larger organizations, they go through a more rigid process when developing and ultimately deploying their code into production, versus ones that follow a more Agile approach. As organizations become larger, software applications also grow in size too, and stricter and stricter processes need to be in place to stop outages, bugs, errors, crashes and bad deployments affecting end users negatively. There can however be better ways to approach the deployment of code and issue management rather than big bang release cycles. Developers may also be able to have better visibility on how they are contributing to services and products with the code they write every day.

software testing

Throwing code over the wall

Some developers can spend months, or even years working on specific features or products and never see them ship. Or if they do ship, have no insights into how their work was received by customers.

Due to externalities. developers in large organisations can be asked to throw their hard work over the wall so to speak to the Quality Assurance (QA) department and software testing teams. Robust software testing finds issues that could have affected customers should the code be placed into the hands of end users, never to be seen again unless it needs further tweaking.

QAs and testers act as the final guard between stakeholders and customers, and they can be petrified that users could be exposed to major bugs or outages should bad code be released into the wild, especially for mission critical software. However once released, visibility to production issues rely on customers to report the issues they experience and for front line support staff to be the go-between of customer and developer.

software testing

The purpose of having a QA team with complex processes is to do as much as possible to prevent large bugs from hitting the user. Organizations are deathly afraid of those large, show stopping exceptions being present for users, and so they should be. QA teams should not go away entirely for enterprise level companies and the QA process is also essential at the startup level to ensure that you don’t lose traction due to a bug that was discovered too late in production. 

A bug identified and fixed in the conception or early design stages costs next to nothing, but bugs needing attention in production can cost considerably more.

The challenge is to give the developer a view of errors and crashes caused by in their own code. Though QA teams are effective, they’re often not best placed to find hidden issues. Often the people who built it are the most qualified to debug it.

Giving developer teams better visibility of their code

Here’s how things might work in traditional software development teams. This tends to be prevalent in larger organizations and though inefficient, works for a lot of companies where costs can be consumed and overall shipping of features can be done in regimented cycles. It does however limit the front line developer’s ability to see how their code is being used by end users, and if things are getting better or worse for customers.

Developers can create beautifully coded and complex features for users, but only ever get to see the results of their own colleagues internally, using what they built. There is a wall between them and the end user. This is how the process might look, with the Analysis, Design and Implementation phases of the Software Development Lifecycle compressed into ‘Developers’ below: 

software development lifec ycle

The developers can instead have full visibility of their code whether it is in QA, software testing, production or end user’s hands. How? Well, automated error monitoring and end user performance monitoring keeps a watchful eye on their software and when issues are found the entire team can be alerted immediately.

Rather than a wall in-front of them, developer teams now have a window into how users are interacting with their software and the features they build.

error monitoring development lifecycle

Deploying code safely without complex QA and software testing

For many teams that use automatic error monitoring tools,  they’ve completely eliminated the need for thorough QA and software testing of the developer’s code before it goes into production. Now this doesn’t mean they do not continue to do thorough unit, integration and acceptance tests of their code at the development stage and have the wider developer team perform code and peer reviews, but they no longer need to have a strict, inflexible QA process in place, or if they do implement one, it is pushed back till the organization is much, much larger, and only for critical processes.

This is due to an automated error monitoring tool being the guardian from bad code causing issues for users, rather than relying on humans to run test after test to ensure code is safe to deploy and for customers to use.

quality assurance

You’ll also see here that there is now a clear line of sight between developers and application errors and crashes. If we were to roll out a bug into production that caused an unhandled exception that affects and is seen by many users, the entire team can be notified of the issue within a few seconds, with diagnostic details providing enough information to fix it quickly.

Again, without the need for long release cycles, software testing and QA processes, code can be shipped on smaller timescales (hours or days) and the issue fixed before any other users encounter the same problem. There is a huge safety net in place should we push bad code into production.

Resolving errors and crashes for users the easy way

Here’s how this also breaks down for support issues specifically. In the traditional model when a customer reports an issue the support team will have to take this largely non-technical information to a specific individual on the developer team who is assigned to fix the problem.

Back and forth information ensues, which is often unhelpful for the developer, as they want information such as the user’s browser, operating system, error messages seen, page it happened on, time of day etc so they can check the logs. Why are you putting the biggest effort upon your customers here? Taking time out of their day to explain problems you caused them?

customer support for developer teams

How often has your support team struggled to replicate and/or fix a software issue that a user encountered, but spent hours trying to debug it, digging through log files and going on pasted screenshots in Word documents from a now disgruntled and angry user? Front line support staff are usually the ones dealing with the customer, and information is passed to the developer team inefficiently and often third hand, as the customer tries to explain which page, step or action they encountered the issue.

Our modern software development model with error monitoring solutions in production environments, have issues reported automatically, without the need for users to report them. The development team has full visibility on issues. Should support teams have a customer who has reported an issue, a simple search of their email address can bring up the diagnostic information about the exact error they encountered, including the stack trace and environment data that can be the key to a fast bug fix. It’s so, so easy.

fixing bugs

Developers who implement automatic error monitoring with their applications are also empowered to care more about how their code affects end users, because they get the insight into the issues they are encountering.

How the savings can add up

You may not think that it is worth radically changing your organizational structure, but the status quo of traditional development teams can end up hitting companies hard in the pocket. Error monitoring tools like Raygun offer reasonable monthly plans that can be seen as a business cost, however if you think about how much money an error monitoring tool can save you, the cost per year can be far outweighed by what you already pay in salaries.

Time is money and digging through log files isn’t a good way to spend it.

It takes time to overcome organizational inertia to implement Agile processes, and bring in a culture of frequent, small, safe and decoupled deployments using workflows and tooling such as Continuous Integration and container deployments, for example. Adding defect monitoring to deployables in production is a low-impact way to greatly reduce the cost and risks of bugs in live systems, without the need to get many stakeholders on board with a re-org.

QA teams are still an important part of enterprise IT teams. Rather than be replaced by error monitoring tools that can detect issues automatically, QA teams should embrace this technology shift to make their jobs easier. If an organization can save themselves from an extra hire or two in the software testing department, customer support department and software development department because their whole team is more streamlined and productive, the cost savings are substantial for any company that ships software.

Next level software intelligence across your entire stack.

7 Comments on “Software Testing – Is It Time To Fire Your QA Team?”

  1. Software Testing

    Software testing is changing. There are numerous tools that are making life much easier for developers and testers. I agree that Software Testers need to embrace these tools and adapt to the changing landscape. While manual testing will not go away completely, many of the traditional software testing jobs that existed earlier will be replaced by automation tools. Testers who adapt and these tools will be the ones who will move to the next level.

  2. Oliver E

    Hi,
    Sad to see such a headline and such utter misunderstanding of what testing is. Although I agree that you describe every project employing what I’d call “bad” testing. It is pervasive in the industry due to the “commoditisation of testing” by vendors. It is basically just some checking done by people. And yes, most of that is automatable. But that is NOT testing. Big projects become bogged down due to a lack of trust and extensive micromanagement (exactly the areas that Agile targets btw.) and testing suffers just as much as anyone else on the SDLC.

    “teams that use automatic error monitoring tools, they’ve completely eliminated the need for thorough QA and software testing of the developer’s code before it goes into production.” is just naiive. A very management compatible sentence though if you want an excuse to cut headcount. We’ve heard all that from test tool vendors since the mid 80ies. It has never ever eventuated. The need for testing has increased every year since then. Do your tools help developers and testers become better and more efficient? Undoubtedly but saying you can do away with testers makes about as much sense as saying you’ll do away with developers.

    Also, QA is a project management and overarching SDLC assurance task and has very little to do with testing. Testing informs QA but QA != Testing.

    The issues you describe are not due to any lack within testing. What you describe are management failures (creating of silos and walls) and problems adopting current and modern development processes (e.g. Scrum, Agile, fast iterative development,…). Believe me, every tester would be happy to work closer to development (that is one reason for implementing CI & CD by the way).

    Anyway, I have used Raygun in a production and testing environment and it is an excellent tool and actually does help a lot with closing the communiation gaps and bridging those walls. So yes I wholeheartedly support anyone using such aides. There are also other new tools that have sprung up in the last 5y, that will change the way we work. And yes, testers need to up their game in this world (but we’ve always been doing that….at least those that treat testing as a profession). But what we definitely don’t need is articles like these knocking testing.

    Cheers Oliver

  3. Charles R. Vavra

    The quality landscape has certainly changed over the last 35 years and the current trend seems to be to replace testers with automated testing tools. What has that got to do with quality assurance? There is a huge difference between quality assurance and quality control and both are completely different disciplines.

    Back in the day :), quality control was someone (usually the best coder we had) who took on the role of mentor and final approval of code changes based on testing feature-set functionality. Nowadays, we get a newbie who knows how to jockey a test tool and may be familiar with the latest coding tool en-vogue.

    Quality Assurance, on the other hand, was interleaved with all phases of the software development process and started way before any code was generated. It started with QA assuring that what the end users of the feature(s) wanted was properly documented and that these requirements accurately communicated what was going to be done to developers. They, in turn, would generate technical requirements in response to the business requirements and again, QA would assure nothing was left out and that the proposed implementation would be testable. QA was tenacious in getting a continuous stream of input/feedback from stakeholders which made change control a whole lot easier because if something got added to the requirements, it could be evaluated for impact to resources and schedules. Now, because we’re so bent on making sure we’re “agile”, it’s easier to just include the extra work and document it later. There’s again, a huge difference between being ‘development agile’ and ‘business nimble’. Besides, how will you get paid for the scope creep if you don’t control and manage changes? None-the-less, throughout the implementation process, QA would assure that QC had done it’s job and when all requirements were met and everything functioned as expected, QA would get the stakeholders to sign off. Then, and only then, would QA release the change for deployment into production.

    Now, it may go into production because we’ve reached our ‘monthly deployment date’. Woa, not so thoughty! If you were selling hot new sports cars, you wouldn’t deliver without all of the features your customer ordered just to get it off the lot on time. You’d keep it until everything ordered was deliverable and acceptable to your customer. Mostly, because having them come back for installation of the missing parts is very expensive and disruptive to the build-out of other new sales. You wouldn’t serve muffins at 6:30 if they were still doughy in the middle; you’d put them back into the oven until they were done!

    Oh! I could go on and on but have taken enough of your time. Seems like QA has fallen away, being replaced by QC which is simply becoming a test tool technician. Hey, if that person can manipulate a test tool, they can surely be more useful as adeveloper Besides, they’ll get paid more and have a clear career path… just sayin.

    CRV – CQA, CQE, CSTE, PMI

  4. Nina Wurst

    Thanks for good point, I totaly agree that automatized testing is still underdeveloped, so manual testing is here to stay. And I would also like to add that manual testing often deeply relies on proper testing software, according to my own experience. It is OK to use something like Google sheets or smth else, but really good thought is to use a specialized QA soft. I, myself, was using EasyQA and was pretty satisfied with the result, hope it`ll come in handy to you too.

    1. Freyja Spaven

      Yes, definitely agree here Harish. Code review can raise all ships when it comes to shipping quality code! Getting others to check work is an important part of the workflow.

Leave a Reply

Your email address will not be published. Required fields are marked *