What is the true cost of having errors in your software?
You may accept that your software has a few issues here and there, but you can’t measure the impact of errors, and they become an accepted ‘lost cost.’
Something you work around, not on.
It’s time to pay attention to how much software errors cost your company, and start taking steps to recoup those costs.
Using a few industry averages, you can put a number to the true cost of software errors in your company, and start to plug cash leaks like wasted developer time and lost customers.
The cost of software errors to the economy
Research by tricentis.com found that in 2016, software failures cost the economy USD$1.1 trillion in assets. In total, software failures at 363 companies affected 4.4 billion customers and caused more than 315 ½ years of lost time.
Worryingly, Government was one of the worst hit industries, followed by transportation and retail. Why could this be?
Private industries like governments and enterprises lean on customized software more and more to handle massive amounts of data. But development teams need to make sure they can handle the build with a sound infrastructure.
It’s never been a better time to protect your companies assets and move the needle on taking a proactive approach to solving software errors in your company. By doing so, you’ll reduce the associated costs of software errors in your company.
Where does the money go? How you can measure the cost of software errors in your company
Software errors expose your end users to slow, buggy software. Or worse, compromise the security and safety of your products. You can’t fix what you can’t see. Many businesses don’t have visibility on their software errors, so measuring them and their impact can be hard.
Software errors were responsible for a majority of the software failures in Government, entertainment, finance, retail, services and transportation in 2016 thanks to research conducted by tricentis.com:
What’s worse is software errors have multiple consequences varying on impact, so you can’t always pinpoint the cause and effect. The effects trickle down ultimately to:
Using a few industry averages, we can help you calculate the cost in lost development time for your company.
How to calculate the cost of developer labour caused by software errors
Taking the industry averages from 2016, we can estimate the financial costs to your company and investigate where the money is going.
- During 2016, Raygun’s crash and error reporting software ingested 7.7 billion errors in total, which averages ~21 million per day. Some of our larger customers can even send 50 million crashes per month!
- The median developer wage for the USA Jan. 2017 $69,565
The errors in your applications need to be fixed or they will affect end users and cause lost revenue. This is where support costs start to mount.
And let’s face it. Fixing software errors is low cost, reactive work.
You should be aiming for around 20% reactive work (finding and fixing errors, support costs), 80% proactive work (building features and improving product) rather than vice versa. This is where you’ll add the true value to the business and your users.
In our experience, around 90% of that time is spent triaging and diagnosing issues. Raygun automates this process, so this time could be eliminated freeing up time to solve more issues for your end users.
Based on a 40 hour work week at the average wage of USD$69,565, the average software developer could spend 32 hours each month ($1159.36 in time) fixing errors and replicating issues.
If you have 50 developers, the combined 1,600 hours of reactive work could potentially ramp up the cost of software errors to USD$57,968 in lost time per month.
This is time spent away building cool features for your customers. Reducing 1,600 hours by 50% would save your company USD$28,984 each month in time, while increasing overall product and software quality.
Calculating the cost of lost customers due to software errors
This one is a little harder to calculate, but it’s important to understand. We do know how website performance affects user behaviour:
Consider a large national pizza delivery company.
Assume the company’s ecommerce site generates five million errors per month (not unusual in a large company) as a result of engineers releasing new features and services.
If only one percent of those errors affect the order or checkout process, at $25 per pizza, the potential impact in lost revenue is $1.25 million per month.
How can you start to turn this around?
The first thing is to realize no software team is without errors, and a zero bug goal isn’t realistic – it’s harmful. It’s about tipping the balance in your favour by solving your software errors quickly to reduce the cost of software errors.
One example is Microsoft Office. We all know how buggy it can be, and if a huge company like Microsoft with an unlimited budget still has software errors, perfection isn’t expected from mere mortals like us.
Prevention is better than cure
Errors become more costly as they move through the software development lifecycle, and it’s more cost effective to fix bugs earlier than later.
In development, you often have less data, use one browser and use the software exactly as intended. Plus you probably already have a debugger on the machine. The major problem with bugs in production is in the absence of supporting tools. Developers have no visibility and no way of ‘debugging’ to figure out where the problem is coming from until a tool like Raygun helps to shine a light on issues.
Production is a much more hostile environment. Once issues are found in front-end in particular, they can be even more complex to diagnose and resolve.
So find and fix errors early and often. Jacek Materna, CTO at Assembla agrees:
“One of the best ways we deal with bugs is empowering all of our individuals developers to avoid bugs in the first place. We continuously deploy and use test-driven development, but one of our greatest assets is one of individuals and interactions: a rigorous code review process.”
Jacek Materna – CTO at Assembla
Which leads us to better testing.
The truth is that automated testing is still under-developed, so manual testing is here to stay. A poor testing methodology costs a lot of money in wasted time and can result in a diminishing scale of return.
For example, if your testing process is too thorough, your developers will be producing new features faster than you can test them, and you end up with a backlog of features waiting to be deployed to production.
Many senior engineers would write unit tests to automatically test critical parts of the code at a minimum. If you haven’t written tests previously and are working on a legacy code base, then add unit tests when software errors are found. This approach builds up a unit test set over time and proves that the fixes are being made.
Finding software errors is where most of the labour time is wasted (that reactive work we talked about earlier.) A tool like Raygun does the heavy lifting for you, alerting your team to errors in real time. There’s no need to dedicate 20% of your resources to finding software errors when your team can know instantly what’s wrong and where:
Once you have a fix ready, shipping it can be half the battle if you don’t have automatic deployments. It’s easy for your developers to lose a whole day deploying a fix.
If you use a software intelligence tool like Raygun, you can also track your deployments to see which deployments and versions are creating the most errors. (We talk about this in more detail here.)
Tools like deployment tracking allow your developers to discover defects quickly in a deployment. Roll back the deployment, or rapidly patch the defect and push out a new build as a solution before end users (and sales) are affected.
This can shave hours off your development time. Invest time in building new features and delighting your end users, not fixing bugs.
Prioritize your bug fixes
What’s more important – a bug that occurs 500 times?
Or 10,000 times?
Well, it depends.
Which one is affecting your end users more?
Amazon is one company that is famous for measuring customer impact of any outage or error. Prioritizing software errors this way will allow you to allocate your resources accordingly, saving valuable costs throughout the SDLC.
Having the correct tools in place protects your business right form the first line of code written. We designed our software intelligence platform for this reason – to be able to give full visibility into software errors affecting your end users:
This allows for better allocation of resources and to protect end users from having a poor experience. If the goal is to reduce the cost of software errors, prioritize your bug fixes with the end user in mind first.
Encouraging a security first approach
Security breaches are a large contributor to companies can lose money. We can measure the losses of the worst security breaches in billions of dollars. Software errors can cause security breaches so huge they affect companies for years to come.
Are companies just becoming lazy? Could breaches like Yahoo’s (which was cited as history’s largest) have been prevented? In 2016, millions of passwords and private information was publicly leaked thanks to a security breach. Yahoo recorded a 6.5% share price fall and a potential US$1 billion in lost funds.
Avoid Yahoo’s fate by thinking about about security first, while the project is still in ideation phase. Security is not an afterthought in your development process. By prioritizing security, you’ll help to reduce the cost of software errors nationally. (And avoid the endless list posts highlighting companies that lost billions due to poor software security.) When your developers understand your errors more, it’s easier for them to spot discrepancies and keep your code safe.
For example, Raygun gives a detailed list of all your errors in one dashboard, so all it takes is a glance to see how many errors are thrown by your applications.
Providing a healthy work environment where employees can speak up can work wonders for preventing security breaches, especially developers who are in the code base all day long. Unfortunately, software security is not on the curriculum in most universities. Therefore, it falls on you as the employer to provide best practices around security.
The more your development team knows about the health of your software, the more likely someone will notice when something is ‘off’. Encourage and cultivate that in your company culture- it might just save you billions.
The true cost can be hard to measure, but you can rescue lost revenue with better workflows and tools
The true cost of software errors can be hard to measure, but visibility on errors is the first step to reducing the impact on your business. Focus on the mission critical issues causing damage to your revenue in the form of buggy apps and lost customers.
By being aware of the small things, you’ll shine a light on the small issues that have big consequences. You may just save your company billions.