Enter your details to view and download the full PDF version
.png)
While we’ve all been urging each other to “measure more” for years now, there’s still not a solid consensus on exactly how or what to measure. Measuring the wrong indicators or measuring too much can easily become misleading, overwhelming, or even contradictory. Sure, data is valuable, but only if it’s relevant, accurate, and actionable. If you’re not sure how to measure experience, you’re not alone: only 38% of execs say their organization is proficient at measuring the customer experience, and 27% say they struggle. When it comes to measuring experience, we’ve often relied on self-report and surveys, forms of data capture that are prone to bias and misinterpretation. As the customer journey moves more and more into digital spaces, we’re able to capture more and more information about real actions and interactions, quantifying and revealing how functional, fast and flawless the experiences we deliver really are. By combining this objective, quantitative data with traditional data about the customer’s thoughts and feelings, we can start to see — for the first time — the full picture.
Digital experience, as a category, can measure the efforts of design, customer experience, development, and customer support. The lines can start to blur between what’s attributable to multiple activities and departments. So it’s a complex challenge: you have to select the right metrics, capture them, correlate them to their causes, and then take action to improve them. If you get it right, though, the rewards speak for themselves. A report from McKinsey finds that top-performing businesses are much more focused on measurement than their lower-achieving counterparts. Top-quartile companies are more likely to track their technology organizations’ performance, using business-oriented metrics such as user satisfaction, time to market, and financial impact. A recent survey found that in the past 18 months, 32% of organizations had begun using new metrics to measure the customer experience. So, there’s no doubt that monitoring and measuring more is a good thing. The harder questions, then, are what do I measure? How do I measure it? And, once I have these metrics, how do I improve them? We’ve pulled together the answers below. These are your top 10 digital experience metrics.
%20(2)%20(1).png)
Google’s holy trinity of customer-oriented digital experience. These scores of a site’s speed (Largest Contentful Paint), stability (Cumulative Layout Shift), and responsiveness (First Input Delay) are designed to measure the quality of your website through the eyes of the customer. All three metrics combine to give you an overall pass or fail, which then factors into your search rankings. Maybe even more importantly, Core Web Vitals are also proven to have a significant impact on critical metrics like page abandonment and conversion rates. If your site is fast, stable, and responsive enough to give 75% of your customers a “Good” experience, you get a passing score on your Core Web Vitals. Anything below this fails the assessment, compromises search ranking, and loses out on potential revenue from website visitors. Yikes.
To get you started, Google provides a few free tools to help assess, diagnose and test your Core Web Vitals. Once you’ve established a baseline and you’re ready to take action, Google highly recommends installing a Real User Monitoring (RUM) tool to understand how these metrics impact your real users. First off, just drop your URL into PageSpeed Insights to get your initial score and an overview of your three Core Web Vitals. You can view your scores for your entire site at origin-level, or key pages at URL-level. Then, use the accurate real-world diagnostics of Real User Monitoring to keep track of and maintain your scores across your entire live user base, long-term.
Google’s PageSpeed Insights will also provide a list of “opportunities” you can use to improve your Core Web Vitals. Reviewing this in combination with Real User Monitoring will help you work out which pages and which scores you can focus on to get the best and fastest results. Test optimizations as you go to assess their effectiveness, and review in PageSpeed Insights every 28 days (Google’s data update cycle) to confirm that you’re getting real results from your efforts. To maintain Core Web Vitals scores and all their associated benefits over time, you need to make optimizing and upholding your scores an ongoing part of your development cadence.
Core Web Vitals are a big deal in web performance, and they benefit different teams in different ways, from increased user engagement to maintaining or improving your organic search ranking.
Ready to get the benefits of Core Web Vitals, but not sure where to begin? Luckily, we’ve put together in-depth guides to Core Web Vitals for both tech leaders and development teams. It's also important to note that while Core Web Vitals is here to stay, it’s also an evolving program, and that Google will continue to adjust the three current metrics in the future. It’s a good idea to subscribe to Google’s developer blog to stay informed, but also note that a decent RUM tool will roll out changes as they’re announced to keep you ahead of the curve.
%20(1).png)
Your Page Experience report is based on a larger set of Page Experience Signals (including Core Web Vitals). This larger group, including mobile friendliness, the absence of intrusive interstitials, and HTTPS security, forms the overall page experience signal. These “signals”, combined with Core Web Vitals, help to determine your Google Search Engine Ranking.
This one is a walk in the park. Just enter your URL in Search Console, and Google will return an overview and breakdown (your Core Web Vitals are also available here). It’ll also provide details of where your site is falling short on mobile usability and Core Web Vitals, so you can zero in on your “problem” areas.
This part is less straightforward. First off, get HTTPS-secured if you’re not already (unless you have a really good reason not to). Your security certificate will likely set you back about $50-70USD annually. The other signals require ongoing changes to your processes and approaches, like taking a mobile-first approach to development, and implementing an ongoing strategy to center and improve Core Web Vitals (see above). Yes, it’s a commitment, but these changes are worth making, as they not only give your customers a safer, easier, and mobile-optimized experience, but they also bump up your search ranking and make you that much easier to find. And remember, mobile is steadily outstripping every other device in terms of time spent, traffic, and revenue, so you can't afford to neglect it.
Your Crash-Free Users score, as the name suggests, measures the percentage of users that didn’t encounter a crash or error while using your software (over the last two months maximum). This is a very simple but accurate way to take the pulse of your software and directly link quality with what’s actually happening in your customer’s real-world interactions, without the potential misdirection of averages. It's often reported to leaders at the executive or director level, as a rapid-fire, self-explanatory number that serves as an overarching indication of development progress. You should aim to keep this at or above 95% to achieve a “good” standard of digital experience, with ambitious teams aiming to get above 99%.
You’ll need a Real User Monitoring tool running in your production environment which integrates with an Error and Crash Reporting tool. This will gather and report accurate data about your customer’s experiences, pick up errors and crashes and give you a rolling score. Ideally, you’ll add Crash-Free Users to a dashboard of key performance metrics, so that stakeholders throughout the business have a user-friendly way to check how you’re tracking.
Marshal your efforts for maximum impact by targeting the errors that matter most. Don't fall into the trap of prioritizing by error count, but focus on the number of customers affected. An intelligent monitoring tool will show you which errors are impacting the largest number of customers to help you instantly prioritize your most important fixes. You may find that fixing even your top three most widespread errors can significantly increase your proportion of Crash-free Users.
Sometimes also defined as “Mean Time to Repair/Recover”, this is the average duration from the beginning of an incident to the deployment of a fix. This metric is owned by the engineering team, and is one of the more “inward-facing” measures of digital experience. The higher your MTTR, the greater the impact on the organization, whether in cost, risk, or diminished trust from customers or partners. Most importantly for our purposes, the higher your MTTR, the longer bugs and issues are in the wild, and the greater the impact on your standard of experience.
In the simplest terms, MTTR requires hitting start on a timer when things go sideways, and calling time when you ship a solution. To track your average, you’ll need a process — ideally automated — to add this data to your records. You can automate the logging of incidents with a ticketing tool like Jira, which should integrate with your monitoring solution. Your alerting system can also double as initial timestamps. Our best advice is to avoid manual processes, because when an outage is in progress, you want your team’s skills and attention focused on resolving the incident, not worrying about whether they properly updated a spreadsheet.
Find out where slowdowns are occurring: are the right people being instantly alerted by your systems as soon as an issue or crash occurs? Does your tech team have the data to replicate and diagnose issues quickly? Does your support team have enough technical insight to effectively triage and assign tickets? Are diagnostic processes clear and frictionless? When a fix is created, do reviews and deployments happen without any unnecessary delays? Keep detailed records and pull data from your tools to see where things are being held up. If you’re serious about resolving incidents as effectively as possible, you need the tools to monitor your customer's experience and diagnose the root cause of issues. If you’re not already, use a monitoring tool like Raygun to trigger alerts based on specific conditions, custom thresholds, and filters, and automatically assign specific team members. You can use tagging to group errors that fit certain key criteria. For example, an online retailer might group all known shopping cart errors under the same tag, and create an alert that automatically notifies the relevant frontend team about the issue. When an error is immediately and automatically shared with the right team and flagged as urgent, MTTR drops.
It’s a good idea to use MTTR in combination with other metrics to get a fuller understanding of reliability. You may also want to track a metric like Mean Time Between Failures (MTBF) to understand how often incidents occur.
Development, DevOps, and Site Reliability Engineering teams can collectively reduce MTTR by creating a single “source of truth” for how shared systems and platforms should operate which documentation methods like runbooks, collaborative documents on the established process of remedying a known issue. When teams understand what led to a particular incident, they can learn from what happened to help deal with future errors and outages. You can also start performing a blameless postmortem for every significant incident. This results in clearer discussion and documentation of the incident, why it happened, and the steps used to resolve it. Done right, blameless postmortems encourage a culture of continuous improvement, and the more you can use an incident as a learning opportunity, the better your chances of reducing MTTR.
For more on MTTR, check out Site Reliability Engineering, a Google-developed operational playbook for improving DevOps, which includes processes like practical alerting, effective troubleshooting, and valuable postmortems.
This one needs no introduction. Load speed is the original performance measurement, simply assessing how fast a page is from initial request to fully loaded. In the last few years, it’s become particularly critical for mobile pages, as mobile now sees the majority of traffic but continues to underperform on speed. This is a big deal for e-commerce sites or anybody that offers digital transactions, as slower pages overwhelmingly result in lower conversions: a one-second delay in mobile load times can impact conversion rates by up to 20%, and the highest conversion rates occur on pages with load times between 0-2 seconds.
Because this metric is so widespread and well-established, you have a range of tools for this, including lots of free ones. Try PageSpeed Insights or WebPage Test to check your current load times, so that you know where you’re starting from. These will give you load times in combination with overall assessments of performance, a mix of user-centric performance data like Core Web Vitals and other raw speed scores like Time to First Byte. Then, once you’re ready to take action to improve your performance, add a Real User Monitoring solution to understand the exact causes of any lags. Raygun Real User Monitoring has the granularity to pinpoint the actions you need to take, whether it’s slow XHR calls, images that need optimization, or JavaScript files that could be minified.
Improving load speed is all about diagnosis and targeted optimization. The key is to accurately identify which aspects of your tech stack and design choices are slowing things down, and the tools above will help you pinpoint which pages and assets are really dragging the chain. It’s a great idea to set a speed budget so that bloat doesn’t creep back in over time. Get a monitoring tool that allows custom Alerting, so that you can set up a threshold for page load time, so that as soon as your standards start to slip, you’ll be notified to take action. If you’re unsure where that threshold should be, we recommend 4 seconds as the gold standard for customer-facing sites. You could even apply this to just a few key pages, e.g. homepage, hero product pages, and checkout.
One disclaimer about load speed; while this metric remains important, the performance engineering community has been seeking more nuanced indicators, with the growing recognition that a site that looks and feels fast, for example by loading the most crucial elements at the top of the page, will likely beat out a page that technically loads every component “faster”. This is why load speed tends to appear alongside newer metrics like Core Web Vitals.
To gauge the experience of a large cohort, you usually use an average or a median to find the “middle” data point. But when it comes to performance, your upper outliers are also part of the picture. It only makes sense to prioritize the actions that will have the biggest impact on the largest segment of customers, but it’s important to keep even your slowest sessions within reasonable parameters. Every value represents the real experience of a real person interacting with your product. Averaging works well when your data set is clustered within a relatively small range. But when it comes to measuring digital experience, an average can actually work to conceal the reality of large portions of your customer base. Because experience signals like page load time tend to fall across a broad range, it’s easy for a few outliers to skew your average and lead you astray or for the outer values (which need the most attention) to be hidden. At Raygun, we use the median as a better indication of where the ‘typical’ customer actually sits because your median, by definition, will show where most of your customers are. And when it comes to controlling the acceptable upper limits, we choose to track the P99 – the time experienced by the 99th percentile of users. Expect this to be slow, but decide what your allowable definition of “slow” is; you still want to keep this closer to 5 seconds than 25 seconds. Tracking your P99 ensures that even the laggiest, buggiest, most frustrating experiences you deliver are still within reasonable parameters. Keeping your P99 in range acknowledges that we can't guarantee perfection every time, but we can still keep ourselves honest.
%20(4)%20(1).png)
It's best to avoid tracking P100 as it’s the domain of total timeouts and bots that hold connections open, and it’s generally misleading about real users.
If you’re working on a large website with a lot of traffic, you’ll want to maintain P99 load times even during periods of significant fluctuation (say if you work in e-commerce and you need to ensure site stability over Black Friday weekend).
For this one, you’re going to need a monitoring solution in place. Sampling a subset of users or using a synthetic data set to approximate response times doesn't really cut it here; the safest bet is a Real User Monitoring tool, which will capture full session data on the actual experiences of live users. Once you’ve set the upper limits of what your team terms an acceptable customer experience, your P99 shouldn’t exceed this. It will be a lot higher than your other targets, but you still want to keep this within a reasonable range.
To improve your P99, find the errors and performance inhibitors that are hurting your numbers the most and impacting the largest portion of users. As a general rule, take your 2-3 most hard-hitting errors and add these to your next sprint. Again, a monitoring solution that can order errors by user impact is the fastest and most accurate way to do this. However, just remember that your P99 is always going to look high, just by nature. Aim for progress, not perfection.
Apdex (Application Performance Index), like Core Web Vitals, is a score that aims to center the user experience and get different teams on the same page. Apdex categorizes back-end response time data according to the type of experience produced, measuring the portion of users who are satisfied, tolerating, or frustrated. This turns impersonal numerical data, which can be difficult to interpret, into human-friendly insights about real user satisfaction, which helps disparate (and often non-technical) teams align on new features or improvements that could improve scores.
You define a unique target response time of one second or less, depending on the nature of your software. Users who fall within your target response time are “satisfied”. To calculate your “tolerating” response time, multiply the target response time by 4: if your target “satisfied” time is 1 second, your “tolerating” point is 4 seconds. And a user experiencing anything above 4 seconds is in the “frustrated” category. To get your overall Apdex score, you take the total of “satisfied” response times, add that to half the tolerable response times, and divide by the total number of samples. You’ll always get a number between 0 and 1.
%20(3)%20(1).png)
This is simple enough, but requires a bit of effort to measure and track on a rolling basis. There are a range of monitoring tools that will automatically measure and report your Apdex score for you.
To improve Apdex, you need to establish a cause-and-effect relationship between your scores and other activities. For example, you might use your scores as a metric for success with each deployment to detect if shipping a new feature improves or lowers them. You can also use Apdex scores in testing on staging servers to catch user experience issues before they go to production.
While there’s definitely value in the simplicity of calculating and communicating Apdex across the business, Apdex isn’t really a standalone metric. Used correctly, Apdex scores are a strong insight into platform-wide trends, but should be used as a part of a wider monitoring stack and applied to the right contexts, not across the board as a measurement of success. They’re also not particularly useful for identifying specific issues that cause a degraded user experience. Discovering that, say, the backend inadvertently blocks additional requests while processing new users would take a huge amount of troubleshooting and testing—and even dumb luck—if you rely on Apdex in isolation.
Ultimately, Apdex is all about improving response times, a central component of performance. To consistently improve Apdex scores, you need to make performance a habit. Integrate regular, focused work on performance issues into your processes, like devoting a weeklong sprint every quarter.
Usability is an entire field of its own, and is measured in a range of ways. However, there is one consistent leading metric: completion rate. Can your user perform the action they need to perform? For pretty obvious reasons, this is vital to the quality of customer experience and has a huge bearing on loyalty. It can also drive down your support costs and churn/abandonment rates, and bolster your word of mouth.
Completion rate is measured by real testers interacting with your product or website to try to accomplish a specific action or task. If they’re able to do it quickly and easily, you’ll pass. This is displayed as a binary measure of pass or fail (coded as 1 or 0). You can then determine whether you want to sample the completion rate of current customers, a specific subset of users, or simply whatever test group you have access to.
There’s a spectrum of factors that go into usability, and UX and design are among the most critical. Development work also plays a major role, as well as the in-app copy, layout, and software quality. Conduct A/B tests on these elements individually to pinpoint the ones that have the greatest bearing on completion. If your users are completing in-app purchases, you might want to focus usability testing on your checkout process or on your key purchase flows. Better completion rates not only reduce customer frustration, they often boost general customer engagement and revenue.
Now, full disclosure: we’ve included NPS here because it remains ingrained in so many businesses, processes, and industries, and shouldn’t be ignored. But this entry is also about the risks and blind spots NPS can introduce. NPS tends to amplify the voices of the most extreme user experiences, so isn’t representative of the majority. It’s also designed to reflect only the most recent interaction the user has had with your brand. According to Gartner, by 2025, more than 75% of organizations will have abandoned Net Promoter Score as a measure of customer service and support. That may or may not come true — there are good reasons why NPS became an industry standard, and a strong focus on customer feedback has raised standards and reminded businesses why they exist (hint: it’s the customer). However, many of us over-rely on the famous score, and in doing so, overlook the risks and misrepresentations NPS can introduce.
You can pony up for purpose-built software like Hotjar or AskNicely, rig up a manual workaround via a Google form, or build your own. Once you have a means for gathering responses, you combine your Detractors, Passives, and Promoters and use the NPS formula to calculate your score. NPS is a number, but it also gathers qualitative data, and this can be one of its most productive uses, whether it’s relaying service feedback and giving upsell opportunities to your customer-facing teams, sharing feature requests with your product team, or passing promoters to your marketing team as potential testimonials.
It can be challenging to accurately attribute or improve NPS scores, because this could be a matter of improving your product, service, price, or even changing the market you’re selling to. NPS is incredibly broad. Even if we narrow our focus to the digital component of customer satisfaction, the subjective nature of the NPS question format means it’s difficult to use it as an indication of the digital experience your software delivers. You could deliver a slow and buggy website experience followed by an excellent support interaction, and still score a nine based on the last contact the customer had. If you’re only looking at NPS in that scenario, you’ll never flag or address the bug, and other customers who encounter the same issue might just quietly leave. In short, NPS is still relevant as a large-scale, subjective indication of how your customers are feeling, giving them the chance to tell you what’s on their minds. But it doesn’t stand alone as an objective measure of experience, particularly for technical teams.
Not strictly a measure of experience, but a metric that correlates with experience. Higher deployment frequency tends to reflect a high-velocity, customer-oriented development culture. This also gives you the ability to rapidly and constantly detect and fix flaws and bugs as they’re introduced, upholding high customer experience standards as a matter of course. While longer release cycles allow teams to release large, noteworthy updates that can make an impression on customers, shorter release cycles enable teams to move rapidly and react to change, and respond to the customer. The faster feedback cycle that comes with constant deployment means you can ship solutions rapidly, get new ideas to market quickly, and respond fluidly to changes in market conditions or customer needs.
At the risk of stating the obvious, you need to keep records of how often your team ships code. This could be a good opportunity to create a public-facing changelog, with brief explanatory notes on what each deployment contains and is intended to achieve.
To accelerate deployment frequency, you need lean and efficient processes. Luckily, the growing popularity of DevOps has yielded thorough best practice guidelines on CI/CD (Continuous Integration/Continuous Delivery) methods. To sustainably ship code at a rapid rate, development, test, and operations teams need to work together to create a cohesive release process. Incorporating smart automation wherever possible also helps achieve speed — high-velocity teams often automate build management, deployment, and post-deployment tracking and alerting. Incorporating automated code reviews and testing throughout the CI/CD pipeline can also reduce delivery times by catching issues earlier, so you can nip them in the bud.
While everyone in the organization may be committed to your customers, it’s another challenge entirely to develop a consistent, unified approach to tracking and improving customer experience as a team.
In a recent report, analysts found that digital experience monitoring is shifting away from a focus on performance and availability (traditional technical metrics like uptime and speed), and towards correlating infrastructure and operations (I&O) with business outcomes.
While 61% of CEOs say their CX capability is aligned to their business strategy, only 47% of operational and CX teams agree. Executives, focused on the bigger picture, don’t have time to scrutinize or decipher technical data, while technical teams lack the language to easily convey progress or connect their work to the broader concerns of the team. Marketers and customer-facing teams don’t have an accessible way to understand how software experience is tracking or how this might affect their ongoing efforts to attract and retain business. Even while the overall goal is the same, everybody is pursuing and measuring it differently.
When different functions are working separately without a collective reference point, issues can develop around accountability, clarity, and cohesion. Friction can develop around the management and ownership of metrics and reporting, especially when digital experience as a category can cut across teams who look after data, infrastructure, product, engineering, marketing, design, and more. A monitoring tool gives your team one “source of truth” and establishes clear metrics that everybody can see, understand, and work to improve, supporting a shared performance culture. This can start with, and be led by, anybody and will be adopted differently in every business, depending on your particular structure and culture. With unlimited seats, you can grant access to anyone in the organization who might benefit from visibility.
Here’s how that works for the team at Skimmer, one of Raygun’s most culture-forward customers:
"Application performance and visibility on the user experience is something every customer-obsessed organization should care about. And this is an accessible window for a non-technical user to learn about it, understand it, and be in a position to ask questions of our technical teams. We don't have to have the answers, but we want to be able to ask questions and hold them accountable. This creates a shared company awareness. A really clear feedback loop and visibility for non-technical people like myself."
Flavia de la Fuente, Director of Operations, Skimmer
“Having these metrics allows us to be able to communicate more effectively across organizations because we can be certain we're talking about the same things, and always in the context of our customers.”
David Peden, VP of Engineering, Skimmer
Your cross-functional digital experience team could include experts in web development, marketing, support, and design. This group will define the standards of experience that you’re aiming for, and ensure that performance and customer experience are key considerations at each stage of new projects, from planning and discovery, evaluating the performance cost of elements during content mapping and wireframing, and weighed against any and all integrations, forms, and plugins.
While leadership and engineering are the key players when it comes to monitoring, it also pays to add customer and market-facing team members to speak to key growth targets, keep an eye on the competition and measure campaign-based optimizations, plus technical experts, planners, and any additional implementation experts. Heightened awareness of customer-centric metrics like Crash-Free Users or Core Web Vitals means that everybody can support and participate in monitoring, discussing, and improving key performance indicators.
Your digital experience team might include any or all of these roles:
So, you’ve got some ideas about what to measure, and who should help to gather, understand, and act on the data. That still sounds like a big task — so here’s a final recommendation. There’s a range of powerful tools available to make that measurement more effortless and effective. Implementing digital experience monitoring is the first step to gathering usable insights and improving code quality in a way that has the greatest impact for the customer. Here are two of the tools we use to improve many of the metrics in this guide:
It’s increasingly impossible to compete without effective visibility and measurement, and successful data strategies are selective and tactical. Pick your metrics well to avoid being left behind, or getting distracted by the sheer volume of numbers that aren’t directly relevant to your customer experience. View your software in terms of the human beings who are interacting with it. This means that even metrics that begin with code quality or business outcomes should be viewed in terms of their impact on the customer, to establish causal relationships and improve accordingly. Digital experience monitoring tools will give your team a new understanding of the experiences they’re delivering and the connection between code and customer. Equipping your team to measure and monitor how development work is impacting people is the fastest way to improve your customer’s digital experience.
Ready to put metrics into action? Want the best monitoring tools in the market? Try Raygun Digital Experience Monitoring, including Crash Reporting and Real User Monitoring, free for 14 days.
