Memory leak caused by otelhttp when no Meter Provider is configured.

[strideynet]

Description

When calling otelhttp.NewTransport() multiple times, memory is leaked.

Supposedly this was resolved by open-telemetry/opentelemetry-go#4820 but I do not think this fixes the issue when no meter provider has been configured and the default global delegating meter provider is in place.

Related: #4226
Introduced by: #4707

Environment

go version go1.22.0 darwin/arm64

	go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws v0.49.0
	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0
	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0
	go.opentelemetry.io/otel v1.24.0
	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.24.0
	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.24.0
	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.24.0
	go.opentelemetry.io/otel/metric v1.24.0
	go.opentelemetry.io/otel/sdk v1.24.0
	go.opentelemetry.io/otel/trace v1.24.0
	go.opentelemetry.io/proto/otlp v1.1.0

Steps To Reproduce

package bench

import (
	"testing"

	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/metric/noop"
)

func BenchmarkWithNoMeterProviderSet(b *testing.B) {
	b.ReportAllocs()
	for i := 0; i < b.N; i++ {
		otelhttp.NewTransport(nil)
	}
}

func BenchmarkWithNoopMeterProviderSet(b *testing.B) {
	b.ReportAllocs()
	otel.SetMeterProvider(noop.MeterProvider{})
	for i := 0; i < b.N; i++ {
		otelhttp.NewTransport(nil)
	}
}

Run each of these with test.memprofile and analyze the inuse heap objects/space difference between them.

Alternatively, write an application which calls otelhttp.NewTransport and analyze heap usage.

Expected behavior

Calling otelhttp.NewTransport() repeatedly doesn't leak memory

Workaround

Call otel.SetMeterProvider(noop.MeterProvider{}) at the start of your application.

[tsuna]

We also ran into this issue at Arista.