Raygun Feature Request

Feature Request

Environment aware applications

Current Status:

Planned

Votes:

107

+1

Avatar

wiifm

When you create an application in Raygun, it seems like you are expected to create one application per environment - eg one for development, one for staging, and one for UAT and another for production. You would then have 4 different API keys.

When sending the error to Raygun, could an (optional) environment specific attribute be added? This would allow you to drill down into just the environment you care about at the time. Coupled with the version specific filter, this would be a nice addition.


Avatar

michael

Posted on
Apr 08 2014

I'm ok with setting up multiple applications but I'd like to be able to designate some of them as "non-production". these "non-production" applications would have very low retention rates, very low error storage limits and not count against the application limit of the pricing plan (ie. free-ish).

Avatar

Mathew Hartley

Posted on
Aug 05 2014

I'm an ex-airbrake user, and find the lack of separation of environments frustrating. Here's the user experience for the two platforms, the same error in two different environments. Airbrake: airbrake errors

And Raygun: ray ray raygun

I don't particularly want to check four dashboards (for four environments).

Avatar

Caleb

Posted on
Aug 05 2014

+1. Creating separate apps is not ideal.

Avatar

tobycox

Posted on
Sep 11 2014

+1 too

Avatar

zip

Posted on
Sep 17 2014

+1 this is needed for standard dev practices

Avatar

PrototypeAlex

Posted on
Oct 20 2014

+1 : becomes super useful when you have multiple WIP enviroments + staging + production.

Avatar

Madman

Posted on
Dec 08 2014

I'm fine with creating multiple applications, and I think I prefer this for ease of use for other workflows (splitting errors in production vs. errors on staging vs. errors on UAT etc.)

This is also easier to setup for people integrating Raygun into their app - e.g. you don't want another mandatory parameter, unless you make it part of the overall setup process ("Insert your API key here" and "Insert your environment name here")

However, perhaps an alternative would be to group applications together - so I create a "Website" group, with "Website | Production", "Website | UAT", "Website | Staging", "Website | Integration Testing" etc. applications inside it.

This could also mean you can create a special "group" dashboard that aggregates all errors from all applications as an added feature.

What are people's thoughts on that?

Avatar

Callum

Posted on
Dec 08 2014

Madman,

Thanks for the suggestion about the app grouping - we'll be taking it into consideration. Conveniently, as of right now we're rolling out the beta of a global dashboard feature to our users, which does include an aggregate view of all applications, as well as a view which presents all errors grouped by application. This may go some way towards enhancing the ability to manage errors across environments. I have enabled this for your account now, and if any other Raygun users see this and want it enabled immediately please contact us and we'll switch it on.

Avatar

mltsy

Posted on
Dec 09 2014

To respond to MadMan's concern, I don't thing a mandatory parameter is necessary. If you don't want to include an environment, don't include one. It can be as simple as an optional parameter or even a tag that you can filter by. If the tag exists, you can filter by it, if not, no problem.

Avatar

JamesC

Posted on
Jan 31 2015

Having just started evaluating RayGun myself, immediately hit this issue. Yes please ++++1.

Avatar

benmccallum

Posted on
May 15 2015

+1

Avatar

Stefan Kip

Posted on
Jul 02 2015

+111111111

Avatar

Jon Crowell

Posted on
Jul 08 2015

The screen shot Matthew posted of how Airbrake handles this issue is very close to what I want. I'd also like to be able to color-code environments (e.g. QA = yelow, Prod = red).

Avatar

liorh

Posted on
Jul 10 2015

+1

Avatar

jrummell

Posted on
Jul 30 2015

While this would be great, I just add a tag for the environment. E.g. dev, qa, prod.

Avatar

Tom Hundley

Posted on
Aug 25 2015

+1 This is absurd that this is not in place already

Avatar

Raygun

John-Daniel Trask

Posted on
Aug 25 2015

Hi everyone,

The reason we're not doing this is because it creates a really lousy user experience in getting setup. I appreciate that some other vendors have done this and set a defacto standard, but I'm still not convinced it's the Right Thing To Do.

Here's how it would work if we just bolted on environments like others:

  1. You create an app, and we wait to see an environment field in an error, see if it exists or not. If it doesn't, make a new environment to filter on.
  2. This works great the first time you use Raygun - you get one environment, you go through an configure everything. Your notifications, your integrations to source control, messaging apps, configure team access.
  3. We see a second environment value come through. Do we default to notifying you on all those existing settings you configured for the first environment? Yes? No? Usual feedback is 'make another option that lets me choose the default behavior'. That sounds pretty janky in a user experience sense, options piled on top of options.
  4. The UI around the integrations page, notifications page, team pages, which is already quite complicated and off putting, now needs the options for setting the behavior per-environment.
  5. Every time we see another new unique value in the environment field, those configuration pages get even more complicated.

So, we decided to keep it simple - just make a new app and send the associated API key per environment. Simple, works with most platforms for changing out an API key per environment. No overloaded and overly-complicated UI on many of the existing screens.

What about being able to see all environments at once?

That's partly why we built the Global Dashboard. If you wanted to stack applications together, you absolutely could. You can then see a unified view with the graph, and the error groups below.

What about linking errors between environments?

This is something that could be improved and has been in discussion within the team. But we could do that without ramming complicating environments UI into the app pages.

But I don't want to upgrade my account to get more apps just for my stage environment

Get in touch if being able to support a dev environment or stage environment would result in needing to upgrade. Totally appreciate that you shouldn't need to pay for a dev or staging environment (assuming the data volumes are appropriately low!).

So are you saying you won't do this ever?

No, I just have yet to hear of how to do it that doesn't mean Raygun ends up with the same poor experience as other vendors in the space. We don't blindly follow what others do because we think we do things better.

For example, at present we think the best approach would be some form of grouping - where you can say there is a relationship between apps. This keeps the user experience around notifications & integrations clean and simple, but could introduce a way of more easily seeing environments on a per-app basis.

I hope that this update helps everyone understand what we're thinking. We're extremely busy at present with Project Neutron (can't wait to show it to you) but we'll be reviewing this and other top requested capabilities once that has shipped.

Thanks for the feedback,

John-Daniel Trask
Co-founder
Raygun

Avatar

Madman

Posted on
Aug 25 2015

Hey John-Daniel,

With the global dashboard now, this makes a lot more sense. I'd still like app grouping (I look after multiple envs of multiple apps, so my dashboard is a bit all over the place, ideally I could have a group of apps per website that I develop on).

One query - the first comment in this thread mentions a way of denoting certain apps as being 'non-production' - with suitably low retention rates, and no cost for these applications. I think this could be a good idea - it means you don't have to worry about people sending you messages as you mention, and you don't have to worry about checking these accounts to see if they're actually abusing the system. This could be as simple as a checkbox on the app creation screen.

That should be a bit easier to control - retention of 30days maximum, with a 1,000 error count limit per month, or something like that? Either drop old errors, or refuse to store new ones (with appropriate messaging, and email notifications).

Anyway, just my 2c :)

Cheers, Matt.

Avatar

RyanR

Posted on
Jan 12 2016

So I sent an email requesting additional seats for our dev versions of our apps, but no one has responded to the request for over a week. I assume the email just got lost and not purposely ignored. If someone would like to contact me that would be great.

Also I need "naive message-based grouping" enabled, but I have just made that request in another thread. Hopefully that or this will be seen soon. If I can't resolve the erroneous grouping then none of this will matter. At this point I'd rather risk having too many groups than not enough as right now too many errors are being grouped and essentially hidden from view since it is not correct for our environment.

Avatar

sirkirby

Posted on
Jul 12 2016

Is this still not being considered?

We are an Enterprise customer and have been using the recommended, "create apps for all environments" approach for almost a year now. At this point, I can definitively tell you that this is not manageable at scale. Sure, there is the global dashboard, but you can only have a single personal filter applied at a time. So If I want to see all of my dev env apps, I need to uncheck all others and check the ones I want. I then have to repeat that for each platform and environment...the same goes for every member of my 30+ person team. This really can't be the best way to go about this.

Environments with separate settings and notifications per app would really solve some of this pain. Especially if we could just quickly switch between environments on the global dashboard. Even better would be the ability to save, create, and share views on the global dashboard to manage this even further.

Am I way off here? If so, what are me and my team missing on how to manage this type of workflow?

Avatar

Stefan Kip

Posted on
Jul 12 2016

@sirkirby
IMO you're not off at all, I really think this is a must-have.

Avatar

Raygun

John-Daniel Trask

Posted on
Jul 13 2016

Hi,

We have been doing work internally about handling this better (especially for folks with hundreds of apps for this reason!).

As mentioned, just doing things based on an environment key in the inbound data is not a great solution at all. So we're looking at the concept of grouping together related apps so they're:

  1. More manageable in the app switcher
  2. Can be filtered between within a single crash reporting dashboard
  3. Provide aggregate data on error groups (so in the right hand side bar of an error group you can see what environments we have seen the errors in already)

This has the benefits that:

  1. You can manage multiple environments
  2. The app switcher experience is simplified, especially with many apps
  3. You can still manage notifications and integrations per-environment without a cumbersome UI*

We think that it will work a lot better this way. It's in the early stages and we're doing more planning on this than we typically do on a new capability just because it will have significant feedback (we don't want to move everyone's cheese and have a lot of unhappy & confused customers!).

  • We are also working on making it so that replicating integration settings and notification options across apps is a lot nicer, but that's being done as a different body of work.

I hope that helps with understanding where we are.

John-Daniel Trask

Avatar

megamozg

Posted on
May 12 2017

+1

Avatar

NoelAdy

Posted on
Jan 18 2018

I appreciate you want to keep the solution usable. And adding environment tag seems to be something you dont want. If you are suggesting you will support this feature with multiple applications and you have suggested you dont want to make peple pay for pre prod apps . (Which i agree with). Could you possibly provide a simple way to add a pre production app which doesnt take an app license when set up. Appreciate it would have limited erorr logginglimits etc . But that would help , i think. Its a convenience thing. This is a sticking point for us . We are a software deve;lopment consultancy and making a reccomendation for raygun tends to get pushed out when this issue / question comes up. And it always comes up.

Avatar

Raygun

John-Daniel Trask

Posted on
Jan 18 2018

Hi Noel,

We have removed all app limits on our new plan types as of early last year. We also have launched custom dashboards that allow folks to bring data in together more easily (and you can create multiple dashboards, they're live too -- if you haven't checked them out yet then you're missing out!).

Here's the current state of play on environment support in Raygun:

  1. You can still use different apps (and you might want to if you want to send notifications to different slack end points, control email notifications, manage visibility rights etc).
  2. You can use a custom tag, which is what some other providers do. You could even prefix them to make life easier (e.g. env:environmentname). Raygun's powerful in-app filtering will even apply those filters when you go into groups so you're only looking at a given environment.
  3. Regardless of approach, you can now create custom dashboards that suit both models (we're also iterating a lot here so you will also be apply to apply filters on individual tiles).
  4. Raygun Plans since early 2017 include unlimited apps so you don't need to worry about choosing the app-approach and having it use up your application counts (if you're not on these plan types, please use the 'contact Raygun' link in the side bar of the app to talk to our account managers about moving to this - you get a lot of other great stuff too).
  5. We still have some small improvements that will make life even easier (e.g. saved filter sets so you can more quickly recall filtering by environments if you want the tag approach).

I'm contemplating locking this thread, since there's so many folks on it now and we don't want to be hitting everyone on each reply (we do appreciate the passion of this area though!)

I feel we've ended up with a flexible approach that solves for the different preferences our customers have for environments handling.

I hope that helps!

John-Daniel Trask