How to decide between cloud and on-premise monitoring

Application performance monitoring systems tend to be available in two modes: on-premise and cloud-based SaaS. Which is the “right” choice? Well, it depends on your situation, but overall cloud-based SaaS offerings have significant benefits when compared to on-premise.

However, it’s not always so simple. The right selection depends on the facts on the ground.

Using my experience working for a large-scale cloud solutions department, I’ve put together some key things you’ll want to consider before you make a decision, starting with some benefits and challenges.

The main benefits of cloud-based solutions

All things being equal, SaaS offerings are generally superior to on-premise systems in several ways. Here are the most important reasons why that’s the case.

More scalable

When application performance monitoring (or any other supporting service, for that matter) is installed in your data center, you “own” it in more than one sense. You own all of it: the servers/VMs, the networking, the storage, and everything else the system requires. The support infrastructure will have to scale along with your application infrastructure. The TCO (total cost of ownership) grows as a constant factor of your overall application infrastructure footprint.

This is one of the key value propositions for cloud-based SaaS solutions: you can disconnect the growth of your support costs from your core application infrastructure costs. SaaS-based systems typically bill in very predictable increments, based on your usage. The servers, memory, and storage needed to monitor your systems become the SaaS-provider’s problem, rather than yours.

Smoother deployments

Supporting systems that run in your data center must be upgraded regularly. Your internal user base will want critical fixes and new features immediately after they become available. Your Operations team, in this scenario, has yet another deployment to worry about. Deployments are labor-intensive, disruptive (just ask your developers,) and time-consuming. They require up-front preparation, testing, and validation to perform properly.

If there are problems with a deployment, it can lead to a series of problems. Your Operations team may be dealing with rework, time on the phone with a vendor, and an emergency rollback or re-deployment. Developers may experience a loss of the service entirely during the outage. All these potential problems distract IT resources from working on application development.

SaaS-based systems eliminate these complications because the provider handles the deployment of new versions.

Features and fixes are managed by the vendor

While your Operations team is planning the next deployment of an on-premise system, your developers are waiting for the features and fixes in the next version.

Cloud-based systems shift those burdens to the provider. Fixes and features are made available to your developers, immediately after release, without your Operations team having to lift a finger. Your developers get the value you’re paying the SaaS provider for, as soon as it’s available, without having to wait for them to be deployed.

Lower total cost of ownership

On-premise solution pricing can be complicated, and the licensing costs can be opaque, to say the least. Compute-based pricing schemes make it difficult to predict the actual capital cost of the licensing. When the quarterly “true-up” comes, the actual bill can be quite a surprise.

Capital costs are just one of the concerns with on-premise systems. Operational expenses, in the form of labor costs and infrastructure, are significantly higher in on-premise systems when compared to SaaS.

Cloud-SaaS services operate on a relatively simple “pay for what you use” model. Your total solution costs for the service are much more predictable. The pricing does typically increase with usage, but even the most expensive tiers of a SaaS service can fix the cost at a fraction of the total cost of operating an on-premise system.

Raygun, for example, operates on a usage-based pricing model.

Security

Data security is a major concern for any organization. The data that is gathered by an application performance monitoring system can be sensitive and must be protected. The data collected must be stored and secured.

Most cloud-SaaS services operate on one of the major cloud providers (AWS, Azure, GCP.) All of those providers have industry-leading security features. SaaS providers obtain the benefits of the security inherent in those provider platforms. They make it their business to be experts in securing the data they store on their customer’s behalf.

The main challenges of cloud-based solutions

SaaS-based services have advantages over on-premise, but that’s not the whole story. There are challenges to be faced and questions to be answered when considering the tradeoffs. Some of the following points may make a SaaS-based solution impossible until something changes internally.

Company regulations and government policy

Externally-hosted, cloud-SaaS services are sometimes excluded from consideration due to policy limitations. These include private company policy, government institution policy, and legal regulations.

Older policies (drafted before the advent of SaaS offerings) are a particular challenge. These policies and regulations don’t consider cloud architecture, so they may be vague on the question of whether a cloud solution is feasible. Leadership teams may be too risk-averse to interpret existing policy with a favorable view toward cloud solutions.

The risk of data loss

There is always risk involved in the development and operations of any system that stores sensitive data. Changes to those systems introduce additional risks. It’s the job of the system architects to assess these risks and make the right choices for their organization.

This begs the question, “are cloud-based SaaS offerings inherently more risky?”

Security in the cloud is certainly different, but by no means worse. The shared responsibility model requires users of cloud services to adjust to a new security landscape. At the same time, it’s incumbent on cloud-SaaS providers to become experts in cloud security, for the benefit of their customers.

Security

SaaS offerings benefit from the security “built-in” to their underlying cloud host (e.g., AWS.) However, the software itself is only as secure as the SaaS provider makes it. Makers of SaaS software are responsible for producing a securely-written platform and implementing it on their cloud host of choice in a secure way.

Read Raygun’s security policies

Now we have the benefits and challenges out the way, you’re probably wondering how you can apply them to your own organization so you can make a more informed decision. The next step will be to weigh up your options.

Weighing the options of cloud vs. on-premise against policy and security

Systems architects and business stakeholders must account for all the tradeoffs when designing solutions. Cloud SaaS services have become far more prevalent, especially in digital-first organizations. Their benefits should be weighed against the challenges on a case-by-case basis.

Where policy and regulation appear to rule out cloud-SaaS, architects and business stakeholders should review the rules carefully and ask, “does this policy/regulation explicitly forbid cloud-SaaS?” The older the policy, the less likely that is the case.

It may be that your engineers and architects know how to comply with a regulation in an on-premise context, but not in the cloud. New deployment models are always difficult, especially in a highly-regulated environment. Once a precedent is set for cloud-SaaS, additional new solutions based on that model become easier to pursue.

The security implications of moving to the cloud are very important and should not be underestimated. The risk of data loss should be well understood. Cloud-SaaS providers should be able to tell their customers a complete story about how they address these risks.

Expand the options for the net benefit of the company

Cloud-SaaS is a compelling option when compared to on-premise monitoring solutions, especially for supporting functions that are not part of your revenue-generating products. TCO and operational burdens are lower. Developers can focus on using the product, instead of waiting for new deployments.

The challenges faced by potential first-time cloud-SaaS users are real, and Cloud-SaaS providers must be able to speak to these challenges. Cloud service adoption is no silver bullet. However, the benefits in both cost and scale are well-documented.

SaaS-based services can serve as an introduction to cloud adoption, before going “all in” on cloud infrastructure. Whatever the motivation may be, expanding architectural options into the cloud is more likely than not to be a net benefit for any organization.