Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(metrics): Map statsd to metric_bucket #11505

Merged
merged 2 commits into from
Apr 9, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions packages/core/src/transports/base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ export function createTransport(

// Drop rate limited items from envelope
forEachEnvelopeItem(envelope, (item, type) => {
const envelopeItemDataCategory = envelopeItemTypeToDataCategory(type);
if (isRateLimited(rateLimits, envelopeItemDataCategory)) {
const dataCategory = envelopeItemTypeToDataCategory(type);
if (isRateLimited(rateLimits, dataCategory)) {
const event: Event | undefined = getEventForEnvelopeItem(item, type);
options.recordDroppedEvent('ratelimit_backoff', envelopeItemDataCategory, event);
options.recordDroppedEvent('ratelimit_backoff', dataCategory, event);
} else {
filteredEnvelopeItems.push(item);
}
Expand Down
6 changes: 3 additions & 3 deletions packages/types/src/datacategory.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// This type is used in various places like Client Reports and Rate Limit Categories
// See:
// - https://develop.sentry.dev/sdk/rate-limiting/#definitions
// - https://github.com/getsentry/relay/blob/c3b339e151c1e548ede489a01c65db82472c8751/relay-common/src/constants.rs#L139-L152
// - https://github.com/getsentry/relay/blob/ec791fed9c2260688f25ea6a6d53ab913927e9a5/relay-base-schema/src/data_category.rs#L91
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: Maybe we can just reference master branch here? 🤔 but not sure, feel free to disregard.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't add it from master, as the file was moved already (why I changed the link). If the link would have been from master, it would've been harder to actually find the correct file as you would just get a 404. But as I could still see the old implementation, it was easier for me to find the new place of the code :)

// - https://develop.sentry.dev/sdk/client-reports/#envelope-item-payload under `discarded_events`
export type DataCategory =
// Reserved and only used in edgecases, unlikely to be ever actually used
Expand All @@ -26,8 +26,8 @@ export type DataCategory =
| 'monitor'
// Feedback type event (v2)
| 'feedback'
// Statsd type event for metrics
| 'statsd'
// Metrics sent via the statsd or metrics envelope items. `namespace` defines which namespace(s) will be affected
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does namespace relate to in this comment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That relates to the rate limiting: https://develop.sentry.dev/sdk/rate-limiting/
But it is about the quota_limit responses and might be confusing in this context. Should I remove it? The first sentence should be enough I think.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I was just confused about the comment in this context as "namespace" is not mentioned in this entry - IMHO OK to remove this or move this comment closer to where namespace is actually used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I moved the comment to where namespaces is defined

| 'metric_bucket'
// Span
| 'span'
// Unknown data category
Expand Down
2 changes: 2 additions & 0 deletions packages/types/src/envelope.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ export type DynamicSamplingContext = {
sampled?: string;
};

// https://github.com/getsentry/relay/blob/311b237cd4471042352fa45e7a0824b8995f216f/relay-server/src/envelope.rs#L154
// https://develop.sentry.dev/sdk/envelopes/#data-model
export type EnvelopeItemType =
| 'client_report'
| 'user_report'
Expand Down
4 changes: 2 additions & 2 deletions packages/utils/src/envelope.ts
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ const ITEM_TYPE_TO_DATA_CATEGORY_MAP: Record<EnvelopeItemType, DataCategory> = {
check_in: 'monitor',
feedback: 'feedback',
span: 'span',
statsd: 'statsd',
statsd: 'metric_bucket',
};

/**
Expand All @@ -220,7 +220,7 @@ export function envelopeItemTypeToDataCategory(type: EnvelopeItemType): DataCate
return ITEM_TYPE_TO_DATA_CATEGORY_MAP[type];
}

/** Extracts the minimal SDK info from from the metadata or an events */
/** Extracts the minimal SDK info from the metadata or an events */
export function getSdkMetadataForEnvelopeHeader(metadataOrEvent?: SdkMetadata | Event): SdkInfo | undefined {
if (!metadataOrEvent || !metadataOrEvent.sdk) {
return;
Expand Down
11 changes: 6 additions & 5 deletions packages/utils/src/ratelimit.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { TransportMakeRequestResponse } from '@sentry/types';
import type { DataCategory, TransportMakeRequestResponse } from '@sentry/types';

// Intentionally keeping the key broad, as we don't know for sure what rate limit headers get returned from backend
export type RateLimits = Record<string, number>;
Expand Down Expand Up @@ -32,15 +32,15 @@ export function parseRetryAfterHeader(header: string, now: number = Date.now()):
*
* @return the time in ms that the category is disabled until or 0 if there's no active rate limit.
*/
export function disabledUntil(limits: RateLimits, category: string): number {
return limits[category] || limits.all || 0;
export function disabledUntil(limits: RateLimits, dataCategory: DataCategory): number {
return limits[dataCategory] || limits.all || 0;
}

/**
* Checks if a category is rate limited
*/
export function isRateLimited(limits: RateLimits, category: string, now: number = Date.now()): boolean {
return disabledUntil(limits, category) > now;
export function isRateLimited(limits: RateLimits, dataCategory: DataCategory, now: number = Date.now()): boolean {
return disabledUntil(limits, dataCategory) > now;
}

/**
Expand Down Expand Up @@ -74,6 +74,7 @@ export function updateRateLimits(
* <category>;<category>;...
* <scope> is what's being limited (org, project, or key) - ignored by SDK
* <reason_code> is an arbitrary string like "org_quota" - ignored by SDK
* <namespaces> Semicolon-separated list of metric namespace identifiers. Only present if rate limit applies to the metric_bucket data category
*/
for (const limit of rateLimitHeader.trim().split(',')) {
const [retryAfter, categories] = limit.split(':', 2);
Expand Down