Support Kraft post Confluent Platform 7.4.0

[danielpetisme]

As part of Confluent Platform 7.4.0, Docker images have been updated to simplify Kraft support.

confluentinc/kafka-images#209

This PR aims to take these changes into consideration while still supporting the older Docker images (CP 7.0.x-7.3.x).

[eddumelendez]

Thank you @danielpetisme ! This is something we noticed last week. I'll take a look later 🙂

[eddumelendez]

I think we can use it as

Suggested change

	withEnv("CLUSTER_ID", getEnvMap().computeIfAbsent("CLUSTER_ID", key -> clusterId));
	getEnvMap().computeIfAbsent("CLUSTER_ID", key -> clusterId);

[danielpetisme]

Looks like it works...
Should I change the syntax of the following lines too?

withEnv(
            "KAFKA_NODE_ID",
            getEnvMap().computeIfAbsent("KAFKA_NODE_ID", key -> getEnvMap().get("KAFKA_BROKER_ID"))
        );

and

 withEnv(
            "KAFKA_CONTROLLER_QUORUM_VOTERS",
            getEnvMap()
                .computeIfAbsent(
                    "KAFKA_CONTROLLER_QUORUM_VOTERS",
                    key -> {
                        return String.format(
                            "%s@%s:9094",
                            getEnvMap().get("KAFKA_NODE_ID"),
                            getNetwork() != null ? getNetworkAliases().get(0) : "localhost"
                        );
                    }
                )
        );

[eddumelendez]

yes, please

[eddumelendez]

@danielpetisme LMK when you feel comfortable about reviewing this PR. I noticed that somehow I marked as a ready for review

[stejani-cflt]

Is this only called for < CP 7.4? If yes, it may be good to add a defense in depth and check that it is indeed isLessThanCP740 ()

[danielpetisme]

Yes, it supposed to be invoked in Zk mode or when Kraft AND CP <7.4
https://github.com/testcontainers/testcontainers-java/pull/7014/files#diff-7c5a407b71c96d4816697ed454df5cb084987573025af294ffa6c182dbd8879eR199-R209

Would like to add a test?

TBH, I'm questioning the optimization gains of removing the checks /etc/confluent/docker/ensure contains... For now, I suggested this solution to be fully backward compatible.

@eddumelendez here are the checks the script is performing... IMHO, the gain does not worth the "complexity" of the code... but that's my 2cts...
https://github.com/confluentinc/kafka-images/blob/master/kafka/include/etc/confluent/docker/ensure

[eddumelendez]

agree, the module is already complex enough 😅 Let's rollback that change, please

[danielpetisme]

Done
#7030

[danielpetisme]

@eddumelendez I'm done with everything I wanted to update.
Ready for a full review

[eddumelendez]

any reason to move it? I guess there is preferences but I think it doesn't hurt 😄

[danielpetisme]

Yep, definitively Kraft is the preferred option from now on.

[danielpetisme]

Sounds like this change did not get merged. 😢
@eddumelendez as said, Kraft would be the preferred option, would you mind to invert the order and put Kraft first?

[eddumelendez]

I think that's up to the user. That's why I said it doesn't hurt.

[eddumelendez]

Thanks for your contribution, @danielpetisme ! This is now in main branch and it will part of the next release.

[danielpetisme]

Thanks @eddumelendez.

• Do you know when the next release would occur?
• I created/closed an issue to make the Kraft+CP.4 scenario explicit and help users to find a clear status on this

[eddumelendez]

I'm planning to do the release as soon as GitHub is stable.