Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(metrics): Map statsd to metric_bucket #11505

Merged
merged 2 commits into from
Apr 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions packages/core/src/transports/base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ export function createTransport(

// Drop rate limited items from envelope
forEachEnvelopeItem(envelope, (item, type) => {
const envelopeItemDataCategory = envelopeItemTypeToDataCategory(type);
if (isRateLimited(rateLimits, envelopeItemDataCategory)) {
const dataCategory = envelopeItemTypeToDataCategory(type);
if (isRateLimited(rateLimits, dataCategory)) {
const event: Event | undefined = getEventForEnvelopeItem(item, type);
options.recordDroppedEvent('ratelimit_backoff', envelopeItemDataCategory, event);
options.recordDroppedEvent('ratelimit_backoff', dataCategory, event);
} else {
filteredEnvelopeItems.push(item);
}
Expand Down
6 changes: 3 additions & 3 deletions packages/types/src/datacategory.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// This type is used in various places like Client Reports and Rate Limit Categories
// See:
// - https://develop.sentry.dev/sdk/rate-limiting/#definitions
// - https://github.com/getsentry/relay/blob/c3b339e151c1e548ede489a01c65db82472c8751/relay-common/src/constants.rs#L139-L152
// - https://github.com/getsentry/relay/blob/ec791fed9c2260688f25ea6a6d53ab913927e9a5/relay-base-schema/src/data_category.rs#L91
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: Maybe we can just reference master branch here? 🤔 but not sure, feel free to disregard.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't add it from master, as the file was moved already (why I changed the link). If the link would have been from master, it would've been harder to actually find the correct file as you would just get a 404. But as I could still see the old implementation, it was easier for me to find the new place of the code :)

// - https://develop.sentry.dev/sdk/client-reports/#envelope-item-payload under `discarded_events`
export type DataCategory =
// Reserved and only used in edgecases, unlikely to be ever actually used
Expand All @@ -26,8 +26,8 @@ export type DataCategory =
| 'monitor'
// Feedback type event (v2)
| 'feedback'
// Statsd type event for metrics
| 'statsd'
// Metrics sent via the statsd or metrics envelope items
| 'metric_bucket'
// Span
| 'span'
// Unknown data category
Expand Down
2 changes: 2 additions & 0 deletions packages/types/src/envelope.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ export type DynamicSamplingContext = {
sampled?: string;
};

// https://github.com/getsentry/relay/blob/311b237cd4471042352fa45e7a0824b8995f216f/relay-server/src/envelope.rs#L154
// https://develop.sentry.dev/sdk/envelopes/#data-model
export type EnvelopeItemType =
| 'client_report'
| 'user_report'
Expand Down
4 changes: 2 additions & 2 deletions packages/utils/src/envelope.ts
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ const ITEM_TYPE_TO_DATA_CATEGORY_MAP: Record<EnvelopeItemType, DataCategory> = {
check_in: 'monitor',
feedback: 'feedback',
span: 'span',
statsd: 'statsd',
statsd: 'metric_bucket',
};

/**
Expand All @@ -220,7 +220,7 @@ export function envelopeItemTypeToDataCategory(type: EnvelopeItemType): DataCate
return ITEM_TYPE_TO_DATA_CATEGORY_MAP[type];
}

/** Extracts the minimal SDK info from from the metadata or an events */
/** Extracts the minimal SDK info from the metadata or an events */
export function getSdkMetadataForEnvelopeHeader(metadataOrEvent?: SdkMetadata | Event): SdkInfo | undefined {
if (!metadataOrEvent || !metadataOrEvent.sdk) {
return;
Expand Down
12 changes: 7 additions & 5 deletions packages/utils/src/ratelimit.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { TransportMakeRequestResponse } from '@sentry/types';
import type { DataCategory, TransportMakeRequestResponse } from '@sentry/types';

// Intentionally keeping the key broad, as we don't know for sure what rate limit headers get returned from backend
export type RateLimits = Record<string, number>;
Expand Down Expand Up @@ -32,15 +32,15 @@ export function parseRetryAfterHeader(header: string, now: number = Date.now()):
*
* @return the time in ms that the category is disabled until or 0 if there's no active rate limit.
*/
export function disabledUntil(limits: RateLimits, category: string): number {
return limits[category] || limits.all || 0;
export function disabledUntil(limits: RateLimits, dataCategory: DataCategory): number {
return limits[dataCategory] || limits.all || 0;
}

/**
* Checks if a category is rate limited
*/
export function isRateLimited(limits: RateLimits, category: string, now: number = Date.now()): boolean {
return disabledUntil(limits, category) > now;
export function isRateLimited(limits: RateLimits, dataCategory: DataCategory, now: number = Date.now()): boolean {
return disabledUntil(limits, dataCategory) > now;
}

/**
Expand Down Expand Up @@ -74,6 +74,8 @@ export function updateRateLimits(
* <category>;<category>;...
* <scope> is what's being limited (org, project, or key) - ignored by SDK
* <reason_code> is an arbitrary string like "org_quota" - ignored by SDK
* <namespaces> Semicolon-separated list of metric namespace identifiers. Defines which namespace(s) will be affected.
* Only present if rate limit applies to the metric_bucket data category.
*/
for (const limit of rateLimitHeader.trim().split(',')) {
const [retryAfter, categories] = limit.split(':', 2);
Expand Down