What is the real cost of having errors in your software?
You may accept that your software has a few issues here and there, but you can’t measure the impact of errors, and they become an acceptable lost cost. Something you work around, not on.
It’s time to pay attention to how much software errors cost your company and start taking steps to recoup those costs.
Using a few industry averages, you can put a number to the real cost of software errors in your company, and start to plug cash leaks like wasted developer time and lost customers.
The cost of software errors to the economy
In a report on software fails, tricentis.com found that in 2017, software failures cost the economy US$1.7 trillion in financial losses. (Up from US$1.1 trillion in 2016.) In total, software failures at 314 companies affected 3.6 billion people and caused more than 268 years in downtime.
The report found that the retail and consumer technology industries were the worst affected, mostly due to smartphone updates and security and hacking risks. Although the public service sector saw a 30% lower failure rate than 2016, the Healthcare sector is responsible for some of the most shocking. In one example, 709,000 pieces of medical mail went undelivered due to a software glitch. 500,000 of the letters were found to have contained confidential patient data, and 1,700 cases were identified where the missing information.
It’s never been a better time to protect your company assets and move the needle on taking a proactive approach to solving software errors. By doing so, you’ll reduce the associated costs of software errors in your company.
Where does the money go? How you can measure the cost of software errors in your company
Software errors expose your end users to slow, buggy software. Or worse, compromise the security and safety of your products. However, many businesses don’t have visibility on their software errors, so measuring them and their impact can be hard.
We know that software errors were responsible for a majority of the software failures in Government, entertainment, finance, retail, services, and transportation in 2017:
What’s worse is software errors have multiple consequences varying on impact, so you can’t always pinpoint the cause and effect. The effects trickle down ultimately to:
Developer costs spent on finding and fixing errors
Ongoing lost revenue from unsatisfied customers
Using a few industry averages, we can help you calculate the cost in lost development time for your company.
How to calculate the cost of developer labor caused by software errors
Taking the industry averages of development wages in the USA from 2017, we can estimate the financial costs to your company and investigate where the money is going.
- During 2017, Raygun’s crash and error reporting software ingested 7.7 billion errors in total, which averages ~21 million per day. Some of our larger customers can even send 50 million crashes per month!
- The median developer wage for the USA in 2018 was $95,841
The average developer wages broken down by state in 2018 are as follows:
The errors in your applications need to be fixed, or they will affect end users and cause lost revenue. This is where support costs start to mount.
And, let’s be honest. Fixing software errors is low cost, reactive work.
You should be aiming for around 20% reactive work (finding and fixing errors, support costs), and 80% proactive work (building features and improving product) rather than vice versa. This is where you’ll add the real value to the business and your users.
In our experience, around 90% of that time is spent triaging and diagnosing issues. (This is why we built our software, Raygun Crash Reporting, which automates this process, freeing up time to solve more issues for your end users.)
Based on a 40 hour work week, the average software developer spends 32 hours each month fixing errors and replicating issues. Taking the average wage of US$95,841, this equates to US$1472 worth of development time per month spent on reactive work.
If you have 50 developers, the combined 1,600 hours of reactive work could potentially ramp up the cost of software errors to US$73,600 in lost time per month.
This is time spent away building regular features for customers. Reducing 1,600 hours by 50% to 800 hours would save your company US$36,800 every month in time while increasing overall product and software quality.
Calculating the cost of lost customers due to software errors
We know how website performance affects user behaviour:
Consider a large national pizza delivery company.
Assume the company’s e-commerce site generates five million errors per month (not unusual in a large company) as a result of engineers releasing new features and services.
If only one percent of those errors affect the order or checkout process, at $25 per pizza, the potential impact in lost revenue is $1.25 million per month.
How can you start to turn this around?
The first thing is to realize no software team is without errors, and a zero bug goal isn’t realistic — it’s harmful. It’s about tipping the balance in your favor.
One example is Microsoft Office. We all know how buggy it can be, and if a huge company like Microsoft with an unlimited budget still has software errors, perfection isn’t expected.
Prevention is better than cure
Errors become more costly as they move through the software development lifecycle, and it’s more cost effective to fix bugs earlier than later.
In development, you often have less data, use one browser and use the software exactly as intended. Plus you probably already have a debugger on the machine. The major problem with bugs in production is in the absence of supporting tools. Developers have no visibility and no way of debugging to figure out where the problem is coming from until a tool like Raygun helps to shine a light on issues.
Production is a much more hostile environment. Once issues are found in front-end, in particular, they can be even more complex to diagnose and resolve.
So find and fix errors early and often. Jacek Materna, CTO at Assembla agrees:
One of the best ways we deal with bugs is empowering all of our individuals developers to avoid bugs in the first place. We continuously deploy and use test-driven development, but one of our greatest assets is one of individuals and interactions: a rigorous code review process.”
Jacek Materna, CTO at Assembla
Which leads us to better testing.
The truth is that automated testing is still under-developed, so manual testing is here to stay. A poor testing methodology costs a lot of money in wasted time and can result in a diminishing scale of return.
For example, if your testing process is too thorough, your developers will be producing new features faster than you can test them, and you end up with a backlog of features waiting to be deployed to production.
Many senior engineers write unit tests to check critical parts of the code. If you haven’t written tests previously and are working on a legacy codebase, then add unit tests whenever you find software errors. This approach builds up a unit test set over time and proves that the fixes are being made.
Finding software errors is where most of the labor time is wasted (that reactive work we talked about earlier.) A tool like Raygun does the heavy lifting for you, alerting your team to errors in real time. There’s no need to dedicate 20% of your resources to finding software errors when your team can know instantly what’s wrong and where. If you get the stack trace, you can solve the problem faster: