Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure Sentry ingestion does not block PR merges #11910

Open
mydea opened this issue May 6, 2024 · 0 comments
Open

Ensure Sentry ingestion does not block PR merges #11910

mydea opened this issue May 6, 2024 · 0 comments
Assignees
@mydea
Labels
Type: Tests

Comments

@mydea
Copy link
Member

mydea commented May 6, 2024

Today, most of our E2E tests ensure we send payloads to Sentry. This means we wait for stuff to be ingested by Sentry, which also means that if that is delayed or broken the tests failed, meaning we cannot merge PRs.

We do not want to be blocked like this. So to fix this, I propose that we:

  1. Create 1 or 2 new E2E tests that literally only test that we can send stuff to Sentry. We make this tests non-blocking, like the flaky test detector - meaning even if that fails you can merge a PR. There is no need to test this for every framework/SDK, it should be enough to maybe have one browser, one node, maybe next.js there.
  2. All other E2E tests stop testing that we send to sentry, and instead only check payloads etc. via the event proxy.
This was referenced May 28, 2024
Merged
Merged
mydea added a commit that referenced this issue May 28, 2024
@mydea
test: Improve E2E node tests (#12258)
d5f1caf 
This does two main things:

1. Updates node E2E tests to use `fetch` instead of axios
2. Update node E2E tests to avoid sending to Sentry.

Instead, we check everywhere via the proxy that the data sent is OK.


Part of #11910
mydea added a commit that referenced this issue May 29, 2024
@mydea
ci: Add optional E2E tests for sending to Sentry (#12259)
a1725a1 
This adds a new E2E test `react-send-to-sentry` that is run optionally.

For now this is the same as the old `standard-frontend-react` test -
eventually we can/should update that test (and others) to stop sending
to sentry.

This test will run in a separate group that we do not block merging on
when it fails.

For now, there is one browser and one node test that checks that they
send events successfully to Sentry - IMHO that should cover stuff
decently for now. I also made the source maps debug ID test optional, as
that inherently sends to Sentry.

We can in follow ups get rid of all the event sending stuff from the
remaining E2E tests.

Part of #11910
@mydea mydea self-assigned this May 29, 2024
This was referenced May 31, 2024
Merged
Merged
mydea added a commit that referenced this issue May 31, 2024
@mydea
test: Refactor webpack E2E tests to avoid sending to Sentry (#12312)
c68f1ad 
Part of #11910
@mydea mydea mentioned this issue May 31, 2024
Merged
mydea added a commit that referenced this issue May 31, 2024
@mydea
test(e2e): Streamline react-router-6-use-routes test (#12315)
31cb741 
Part of #11910 - we
already have error & transaction tests there, so we can just delete the
sending tests.
mydea added a commit that referenced this issue May 31, 2024
@mydea
test(e2e): Update react-router-6 test to avoid sending to Sentry (#…
70f9c14 
…12313)

Part of #11910

I renamed `standard-frontend-react` to `react-router-6` as this is more
descriptive (what does standard frontend even mean?).

This test now also uses the event proxy.
c298lee pushed a commit that referenced this issue Jun 4, 2024
@mydea @c298lee
test: Refactor webpack E2E tests to avoid sending to Sentry (#12312)
4772147 
Part of #11910
c298lee pushed a commit that referenced this issue Jun 4, 2024
@mydea @c298lee
test(e2e): Streamline react-router-6-use-routes test (#12315)
66a8cf9 
Part of #11910 - we
already have error & transaction tests there, so we can just delete the
sending tests.
c298lee pushed a commit that referenced this issue Jun 4, 2024
@mydea @c298lee
test(e2e): Update react-router-6 test to avoid sending to Sentry (#…
1880316 
…12313)

Part of #11910

I renamed `standard-frontend-react` to `react-router-6` as this is more
descriptive (what does standard frontend even mean?).

This test now also uses the event proxy.
@mydea mydea mentioned this issue Jun 5, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mydea mydea

Labels
Type: Tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mydea @AbhiPrasad