Merging and Resolving Super Slow
Josh
Posted on
May 09 2020
Merging and resolving have been super slow all day today for me. I tweeted about this earlier to @raygunio. It's still happening now. Please advise.
John-Daniel Trask
Raygun
Posted on
May 09 2020
Hi Josh,
Sorry to hear this - nothing is registering as being overly delayed right now so we'll have a team member investigate.
As a note, merge jobs can be slow if the groups are huge, but generally is managed as an offline task. They do queue, so if you have one really massive merge job, and a bunch of small ones, it may look slower than it is.
Changing the status to resolved though shouldn't be slow since that's just updating a status field on a database row, so we'll look into that too.
Usually in these situations it can just be a queue of merge jobs, and there's no action to be taken (other than continuing to think of ways to make it faster for everyone), but I'll update here on if something significant is found.
Kind regards,
John-Daniel Trask
John-Daniel Trask
Raygun
Posted on
May 09 2020
Hi Josh,
I've just tested resolving issues and it's instant on our account. You're just doing this from the main Crash Reporting view?
Kind regards,
John-Daniel Trask
John-Daniel Trask
Raygun
Posted on
May 09 2020
Hi Josh,
FInal update - the merge jobs all just completed as the team were investigating. There were more than 300 queued up, accounting for ~1.9m errors being re-assigned into new groups.
I suspect his one will fall into the 'how can we make this faster', or at least look completed while the heavy lifting grinds through our data stores.
Apologies for the inconvienance. Do let me know about the 'resolve' being slow and how you were doing that as it should be nearly instant.
Kind regards,
John-Daniel Trask
John-Daniel Trask
Raygun
Posted on
May 09 2020
Hi Josh,
Sorry - slip of the tongue, the 1.9m figure was your number over the lifetime of your account. The actual number was tiny - I'll have the team do a deeper dive during the work week to find out why only a few thousand errors being merged took so long and hasn't caused any monitor trips or notice to us on it being this noticably slow for you.
Sorry about that,
John-Daniel Trask
Josh
Posted on
May 09 2020
Thanks for looking in to this! You had me going with the 1.9m for a split second. Yeah, this was mostly just onesy and twosy merges where the id was different with a few slightly bigger ones that may be a few hundred to a few thousand items. It's never taken this long to register in the UI for me before. However, I have noticed that the merged item often times does NOT immediately update with all of the merged info. I immediately suspected that was because it was handled in a background task as you said. However, this delay has been weird. On one of the merges earlier, it updated the UI display and looked like it completed but when checking back later for another reason it looked like the merge had never really happened.
Josh
Posted on
May 09 2020
It does appear to be working normally now.
Josh
Posted on
Jun 02 2020
FYI... Things are merging very slowly again today.
Deleted User
Raygun
Posted on
Jun 02 2020
Hi Josh,
Thank you for getting in touch with us again. We have not detected anything that may be directly impacting the merging of error groups. We will continue to monitor the situation and let you know if we encounter a specific issue within our system that is causing the slow performance.
While performance improvements can be a large undertaking, we may be able to make some improvements to our grouping logic in the short term. Is there particular error groups you are commonly seeing that should be grouped together automatically? If you share with us some examples, we can see what can be done to release to a new grouping version to reduce the need for merging.
Thank you, Mitchell
Aun Ali
Posted on
Aug 28 2020
.