Sentry investigation #1689
Description
Investigation on Sentry profiling not working
In this investigation we'll attempt to explain why the Profile section in Sentry is empty when this was working and showing data before.
1st fact:
In april 2025 Sentry changed their existing Profile system to two different Profile systems (UI & Continuous Profile Hours) which have a new PAYG (Pay-as-you-go) billing model. Some info about the transition can be found on this links:
- Official blog entry
- Support article
- In the Stats & Usage section in Sentry, we can see a notification about the new Profile methods but also the usage graph which shows all corresponding data being "Rate Limited":
2nd fact:
Quintel is currently on the "Team plan" subscription from Sentry. This plan includes profiling on a PAYG (Pay-as-you-go) basis, in fact the new profiling systems are only available as PAYG. So this is billed per usage, that said a PAYG quota can be enabled by the organization owner to have control over it. All this information can be found in the following sections:
- Docs pricing and more specifically Profiling pricing
- In the Subscription overview we can see the current plan details and Continuous Profile Hours (which will be the closest to what we had before) shows nothing:
3rd fact:
Migration to the new system seems to not be yet available for all SDKs as explained on the first link below so as I understand it we can't totally migrate to the new system in rails yet. Nonetheless, from some other articles like in the highlighted text in the second link below it is understood that the legacy Transaction-based profiling (which is what the sentry-ruby gem uses) is automatically converted to the new systems, so no real action is needed to keep getting the old profiling data:
Conclusions
Given the last point of the 3rd fact we should still be capable of seeing profile results. Until all other SDK have been made compatible with the new profiling systems, this automatic conversion will surely remain in place. I could not find any timelines so I believe it could still be a while.
So why aren't we seeing any profiling data at all? Given the last point of the 1st fact the most likely reason is because we do not have the PAYG quota properly set and everything gets "Rate limited".
Proposal
I would suggest that the first logical step to take here is to try and set a small PAYG quota and see if then we recover the profiling data. Then later if it gives results we can evaluate a reasonable quota to spend on this monthly. This change should be possible with the owner role user of the Quintel organization which is @noracato.