We don't have this feature at present unfortunately.
However (and this still needs a bunch of work sorry), you could look at setting up inbound filters for your pen-test. You could choose to have Raygun ignore errors from an IP, or if the software uses a specific user agent, ignore that. The reason it would still need a bunch of work is due to needing to do it on a per-app basis sadly.
We are looking at a range of features to apply things across multiple apps at once but they won't be out in time to help here unfortunately.