Promote Realtime Compositions to Beta

[sttts]

What problem are you facing?

This is the tracking issue for promoting realtime compositions to Beta.
With the feature recently merged into the master under the --enable-realtime-compositions feature gate, we don't have any issues yet, but we will list them here as they appear.

How could Crossplane help solve your problem?

Promote realtime compositions to Beta once we feel confident about it.

Tasks

Beta Give feedback

1. Realtime compositions E2E tests flaky #5228
   bug composition e2e flaky-test realtime-composition +5
   
2. Crossplane fails to synchronize claims with XRs #5400
   backport release-1.14 backport release-1.15 bug realtime-composition +4
   
3. cannot get composite resource #5151
   bug composition realtime-composition +3
   
4. Realtime Compositions: cannot list in CompositionRevision handler #5533
   bug composition realtime-composition +3

[negz]

I just closed #4316, as a "duplicate" of this tracking issue. Part of what that issue was tracking was removing polling support altogether. Perhaps we could bring that into scope for beta. I know the main constraint will be figuring out how/when to invoke the Function pipeline, assuming folks might want it to be invoked even when the XR/composed resources haven't changed.

[sttts]

@negz compare #4822 for a quick attempt to see what explodes in CI. Not much actually.

[negz]

I noticed while working on #5651 that there might be a data race in the realtime compositions implementation:

crossplane/internal/controller/apiextensions/definition/composed.go

Lines 198 to 201 in 432e163

	i.cdCaches[gvk] = cdCache{
	cache: ca,
	cancelFn: cancelFn,
	}

crossplane/internal/controller/apiextensions/definition/composed.go

Line 269 in 432e163

delete(i.cdCaches, gvk)

These both write to a map, but they don't have a write lock on the mutex that guards that map. In the first case there's a read lock on an RWMutex. In the second case there's no lock at all.

Perhaps this is contributing to some of the flakiness we've seen?

[negz]

Something to be aware of: crossplane-contrib/provider-kubernetes#246 (comment)

It seems like in some cases SSA will bump the resource version of the applied object even if nothing changed. In theory this could cause a tight loop of reconciles if functions and realtime compositions are used together:

1. Crossplane uses SSA to apply function desired composed resource state
2. API server updates composed resource resource version
3. Realtime compositions notices updated resource version and queues a reconcile
4. Back to step 1

No-one has reported this yet, so it's possibly only a hypothetical issue.