How much could software errors be costing your company?

What is the real cost of having errors in your software?

You may accept that your software has a few issues here and there, but you can’t measure the impact of errors, and they become an acceptable lost cost. Something you work around, not on.

It’s time to pay attention to how much software errors cost your company and start taking steps to recoup those costs.

Using a few industry averages, you can put a number to the real cost of software errors in your company, and start to plug cash leaks like wasted developer time and lost customers.

The cost of software errors to the economy

In a report on software fails, tricentis.com found that in 2017, software failures cost the economy US$1.7 trillion in financial losses. (Up from US$1.1 trillion in 2016.) In total, software failures at 314 companies affected 3.6 billion people and caused more than 268 years in downtime.

cost of software errors

The report found that the retail and consumer technology industries were the worst affected, mostly due to smartphone updates and security and hacking risks. Although the public service sector saw a 30% lower failure rate than 2016, the Healthcare sector is responsible for some of the most shocking. In one example, 709,000 pieces of medical mail went undelivered due to a software glitch. 500,000 of the letters were found to have contained confidential patient data, and 1,700 cases were identified where the missing information.

It’s never been a better time to protect your company assets and move the needle on taking a proactive approach to solving software errors. By doing so, you’ll reduce the associated costs of software errors in your company.

Where does the money go? How you can measure the cost of software errors in your company

Software errors expose your end users to slow, buggy software. Or worse, compromise the security and safety of your products. However, many businesses don’t have visibility on their software errors, so measuring them and their impact can be hard.

We know that software errors were responsible for a majority of the software failures in Government, entertainment, finance, retail, services, and transportation in 2017:

software fail type

What’s worse is software errors have multiple consequences varying on impact, so you can’t always pinpoint the cause and effect. The effects trickle down ultimately to:

  • Developer costs spent on finding and fixing errors

  • Ongoing lost revenue from unsatisfied customers

Using a few industry averages, we can help you calculate the cost in lost development time for your company.

How to calculate the cost of developer labor caused by software errors

Taking the industry averages of development wages in the USA from 2017, we can estimate the financial costs to your company and investigate where the money is going.

  • During 2017, Raygun’s crash and error reporting software ingested 7.7 billion errors in total, which averages ~21 million per day. Some of our larger customers can even send 50 million crashes per month!
  • The median developer wage for the USA in 2018 was $95,841

The average developer wages broken down by state in 2018 are as follows:

developer wages

The errors in your applications need to be fixed, or they will affect end users and cause lost revenue. This is where support costs start to mount.

And, let’s be honest. Fixing software errors is low cost, reactive work.

You should be aiming for around 20% reactive work (finding and fixing errors, support costs), and 80% proactive work (building features and improving product) rather than vice versa. This is where you’ll add the real value to the business and your users.

In our experience, around 90% of that time is spent triaging and diagnosing issues. (This is why we built our software, Raygun Crash Reporting, which automates this process, freeing up time to solve more issues for your end users.)

Reactive work

Based on a 40 hour work week, the average software developer spends 32 hours each month fixing errors and replicating issues. Taking the average wage of US$95,841, this equates to US$1472 worth of development time per month spent on reactive work.

If you have 50 developers, the combined 1,600 hours of reactive work could potentially ramp up the cost of software errors to US$73,600 in lost time per month.

This is time spent away building regular features for customers. Reducing 1,600 hours by 50% to 800 hours would save your company US$36,800 every month in time while increasing overall product and software quality.

Calculating the cost of lost customers due to software errors

We know how website performance affects user behaviour:

Performance

Consider a large national pizza delivery company.

Assume the company’s e-commerce site generates five million errors per month (not unusual in a large company) as a result of engineers releasing new features and services.

If only one percent of those errors affect the order or checkout process, at $25 per pizza, the potential impact in lost revenue is $1.25 million per month.

How can you start to turn this around?

The first thing is to realize no software team is without errors, and a zero bug goal isn’t realistic — it’s harmful. It’s about tipping the balance in your favor.

Read how one Raygun customer reduced his company’s errors by 99%

One example is Microsoft Office. We all know how buggy it can be, and if a huge company like Microsoft with an unlimited budget still has software errors, perfection isn’t expected.

Prevention is better than cure

Errors become more costly as they move through the software development lifecycle, and it’s more cost effective to fix bugs earlier than later.

Fire your QA team blog post.

In development, you often have less data, use one browser and use the software exactly as intended. Plus you probably already have a debugger on the machine. The major problem with bugs in production is in the absence of supporting tools. Developers have no visibility and no way of debugging to figure out where the problem is coming from until a tool like Raygun helps to shine a light on issues.

Production is a much more hostile environment. Once issues are found in front-end, in particular, they can be even more complex to diagnose and resolve.

So find and fix errors early and often. Jacek Materna, CTO at Assembla agrees:

One of the best ways we deal with bugs is empowering all of our individuals developers to avoid bugs in the first place. We continuously deploy and use test-driven development, but one of our greatest assets is one of individuals and interactions: a rigorous code review process.”

Jacek Materna, CTO at Assembla

jacek assembla

Which leads us to better testing. 

Better testing

The truth is that automated testing is still under-developed, so manual testing is here to stay. A poor testing methodology costs a lot of money in wasted time and can result in a diminishing scale of return.

For example, if your testing process is too thorough, your developers will be producing new features faster than you can test them, and you end up with a backlog of features waiting to be deployed to production.

Many senior engineers write unit tests to check critical parts of the code. If you haven’t written tests previously and are working on a legacy codebase, then add unit tests whenever you find software errors. This approach builds up a unit test set over time and proves that the fixes are being made.

Finding software errors is where most of the labor time is wasted (that reactive work we talked about earlier.) A tool like Raygun does the heavy lifting for you, alerting your team to errors in real time. There’s no need to dedicate 20% of your resources to finding software errors when your team can know instantly what’s wrong and where. If you get the stack trace, you can solve the problem faster:

Stacktrace

Automating deployments

Once you have a fix ready, shipping it can be half the battle if you don’t have automatic deployments. It’s easy for your developers to lose a whole day deploying a fix.

If you use a software intelligence tool like Raygun, you can also track your deployments to see which deployments and versions are creating the most errors. We talk about this in more detail here.

Deployment lines

Tools like deployment tracking allow your developers to discover defects quickly in a deployment. Roll back the deployment, or rapidly patch the defect and push out a new build as a solution before end users (and sales) are affected. 

This can shave hours off your development time. Invest time in building new features and delighting your end users, not fixing bugs.

Prioritize your bug fixes

What’s more important — a bug that occurs 500 times?

Whats more detrimental to the cost of software errors ?

Or 10,000 times?

Whats more detrimental to the cost of software errors ?

Well, it depends.

Which one is affecting your end users more?

Amazon is one company famous for measuring customer impact of any outage or error. Prioritizing software errors this way will allow you to allocate your resources accordingly, saving valuable costs throughout the Software Development Lifecycle (SDLC)

Having the correct tools in place protects your business right from the first line of code written. We designed our software intelligence platform for this reason — to be able to give full visibility into software errors affecting your end users:

Affected user

This allows for better allocation of resources and to protect end users from having a poor experience. If the goal is to reduce the cost of software errors, prioritize your bug fixes with the end user in mind first.

Encouraging a security-first approach

Security breaches are also a large contributor to lost revenue, and the effects are often measured in billions of dollars.

Are companies just becoming lazy? Could breaches like Yahoo’s (which was cited as history’s largest) have been prevented? In 2016, millions of passwords and private information were publicly leaked thanks to a security breach. Yahoo recorded a 6.5% share price fall and a potential US$1 billion in lost funds.

Avoid Yahoo’s fate by thinking about security first, while the project is still in ideation phase. Security is not an afterthought in your development process. By prioritizing security, you’ll help to reduce the cost of software errors nationally. (And avoid the endless list posts highlighting companies that lost billions due to inadequate software security.) When your developers understand your errors more, it’s easier for them to spot discrepancies and keep your code safe.

For example, Raygun Crash Reporting gives a detailed list of all your errors in one dashboard, so all it takes is a glance to see how many errors are thrown by your applications.

Dashboard Raygun Crash Reporting

Providing a healthy work environment where employees can speak up can work wonders for preventing security breaches, especially developers who are in the code base all day long. Unfortunately, software security is not on the curriculum in most universities. Therefore, it falls on you as the employer to provide best practices around security.

The more your development team knows about the health of your software, the more likely someone will notice when something is ‘off.’ Encourage and cultivate that in your company culture - it might just save you billions.

The real cost can be hard to measure, but you can rescue lost revenue with better workflows and tools

The true cost of software errors can be hard to measure, but visibility on errors is the first step to reducing the impact on your business. Focus on the mission-critical issues causing damage to your revenue in the form of buggy apps and lost customers.

By being aware of the minor things, you’ll shine a light on the small issues that have significant consequences. You may just save your company billions.

Further reading: