Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to spawn child Celery tasks that do NOT inherit the parent's trace id #2332

Closed
RaduW opened this issue Aug 31, 2023 · 1 comment
Labels

Comments

@RaduW
Copy link
Contributor

RaduW commented Aug 31, 2023

Problem Statement

We would like to be able to spawn Celery tasks that do not have the parent's task trace_id .

The reason for this is that sometimes it makes sense to sample the child tasks independently from the parent task ( on the server, i.e. dynamic sampling).

A typical scenario is of a task that runs seldom (once a week) and that spawns a lot of independent tasks ( e.g. one task per organisation). Having the child tasks linked by trace_id would mean that we either keep all traces from all organisations (tens of thousands) or none when we would actually need to always keep the traces for a few orgs.

Currently we can disable passing trace_id but only globally for all tasks and all errors.
This is not a usable solution since a lot of times passing the trace_id makes sense.

Solution Brainstorm

There is already a proof of concept PR opened by @jan-auer that implements the feature ( no tests though)
2331

In this proof of concept, we are passing a header in apply_async() that only disables passing the trace_id for the current call.

@RaduW RaduW added the enhancement New feature or request label Aug 31, 2023
@antonpirker
Copy link
Member

Fixed here: #2331

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants