Migrate from OpenCensus to OpenTelemetry

[anniefu]

OpenCensus & OpenTracing have merged to form OpenTelemetry (see https://github.com/census-instrumentation/opencensus-go & https://medium.com/opentracing/a-roadmap-to-convergence-b074e5815289).

Knative should to move to instrumenting metrics and traces with OpenTelemetry instead of OpenCensus ahead of OpenCensus getting deprecated.

[grantr]

Looks like OpenTelemetry Go SDK is in beta now: https://github.com/open-telemetry/opentelemetry-go

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[vagababov]

/lifecycle frozen
/assign @evankanderson @jjzeng-seattle

[skonto]

I have done an evaluation on this topic here are the takeaways. From my POV otel is not ready yet, I would wait for GA. There is also a spike for this in wip status. We discussed this on a community meeting as well.

[evankanderson]

/unassign

[evankanderson]

/unassign @jjzeng-seattle

[dprotaso]

FYI - there's a bridge for OpenCensus => OpenTelemetry

So we can use the OTel exporters and slowly transition the API usage over

https://github.com/open-telemetry/opentelemetry-go/tree/main/bridge/opencensus#user-journey

I think tracing would be the best POC to test this out with

[dprotaso]

FYI - Bug in metrics/otelhttp handler causes high cardinality
open-telemetry/opentelemetry-go-contrib#3765 - fixed in open-telemetry/opentelemetry-go-contrib#4277

[dprotaso]

When performing the migration we should consider how we name the service -

The suggestion here is that hostname is not a good one because it changes as pods come up and down - knative/serving#14075

[skonto]

Otel go sdk i snow stable wrt metrics. See knative/serving#14486 for pointers. We should define a plan on how to work on this one.