otelhttptrace is not concurrent-safe

[pellared]

From https://pkg.go.dev/net/http/httptrace#ClientTrace:

Functions may be called concurrently from different goroutines and some may be called after the request has completed or failed.

If I understand this correctly it may even mean that the current implementation may contain logical race conditions as I do not see how activeHooks synchronizes multiple concurrent HTTP requests.

Originally posted by @pellared in #3892 (review)

[pellared]

It would be good to first write a test that would confirm the issue.

[Alkorin]

Except if I missed something, each http request allocate its own httptrace.ClientTrace

Either with

	ctx = httptrace.WithClientTrace(ctx, otelhttptrace.NewClientTrace(ctx))
	req, err := http.NewRequestWithContext(ctx, ...)

Or with otelhttp.WithClientTrace option of otelhttp.NewTransport which call httptrace.WithClientTrace(ctx, t.clientTrace(ctx)) inside Transport.RoundTrip

• https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/instrumentation/net/http/otelhttp/config.go#L198
• https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/instrumentation/net/http/otelhttp/transport.go#L76
• https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/instrumentation/net/http/otelhttp/transport.go#L109

otelhttptrace.NewClientTrace must not be used on several http requests

[pellared]

You are right. It is me who missed something 😉 Thanks ❤️