Invalid memory configuration for netty maxDirectMemory in native image

[antonwiens]

I thinks this only concerns native image builds.

Setting either
-XX:MaxDirectMemorySize
or
-Dio.netty.maxDirectMemory
does not work at runtime and runtime memory is incorrectly reflected in io.netty.util.internal.PlatformDependent.DIRECT_MEMORY_LIMIT
at runtime.

i think the problem is that io.netty.buffer.PooledByteBufAllocator uses io.netty.util.internal.PlatformDependent
which is not initialized during runtime but build time.

I tested this with Quarkus 1.10.5.Final, but looking at the current code of NettyProcessor shows that PlatformDependent is still not initialized at runtime, only PooledByteBufAllocator is.

[gsmet]

/cc @cescoffier @stuartwdouglas

[gsmet]

I'm having a look at this one.

[gsmet]

Outch, this is a Pandora box :). I'm very surprised we didn't have ton of small issues because of that.

[cescoffier]

Hum, yeah... I've tried a few things, but I'm blocked on AsciiString (which needs a substitution for the constructor, but at the same time have static methods accessing the constructor (and this is not supported by GraalVM)).

[cescoffier]

The runtime initialization is not going to work or will be very expensive. Substituting the PlatformDependent class sounds a lot better.

[antonwiens]

Any updates on this one?

[antonwiens]

Is this still relevant?

[antonwiens]

@gsmet Sorry, to bump this issue again, but i think it is quite relevant. Cause this makes runtime memory settings dependent on the build machine.

[cescoffier]

Would be great to have an update reproducer, as I never encountered the issue.

[geoand]

@antonwiens can you attach a sample application using one of the latest Quarkus versions that behaves like you mention?

[antonwiens]

@antonwiens can you attach a sample application using one of the latest Quarkus versions that behaves like you mention?

I looked at the PlatformDependent and PooledByteBufAllocator class code of the current version of netty, which still initializes the maxDirectMemory in static variables and gets defined during native image build time. thats why it will use max direct memory from the native image build time environment and not the runtime environment, this creating buffers and pools using the wrong maxdirectmemory from build time and not runtime.
The natve-image properties of netty don't include a platformdependent and there are no substitutes there, so it should still exist.

I will create a reproducer, but i have little time and it may take a while till i come to it.
I hope the reproducer will clarify this.

[geoand]

@antonwiens did you have any luck with the reproducer?

[antonwiens]

@antonwiens did you have any luck with the reproducer?

Sorry @geoand, had no time to get on this yet.

[geoand]

No worries

[geoand]

Closing for now. If you do have more information, feel free to comment and / or reopen

[gsmet]

I think we need to keep this open as it's still very much an issue: PlatformDependent is initialized at build time and initializes a ton of host specific things at build time, which could lead to incorrect behaviors on some systems.

@zakkak I wonder if you would have an idea on how we could reinitialize the values at runtime? I'm thinking about MAX_DIRECT_MEMORY, NORMALIZED_ARCH and a few others.

[geoand]

I personally I don't see the reason for keeping something open when we crearly need more info we don't have.
But I won't fight it :)

[gsmet]

We don't need more info. We know what the problem is (see my comment above). We just have not figured out how to fix it yet.

[geoand]

Reading the thread, we seem to want a reproducer

[zakkak]

A reproducer would indeed help a lot, but even without one it looks like MAX_DIRECT_MEMORY, NORMALIZED_ARCH and similar still need to be runtime (re)initialized. Working on a fix.

[gsmet]

even without one it looks like MAX_DIRECT_MEMORY, NORMALIZED_ARCH and similar still need to be runtime (re)initialized. Working on a fix.

Yes, that was my point.

[cescoffier]

Sure, but having Platform0 initialized at runtime is not simple.... It breaks a lot of stuff, and the blast radius will touch dozen of extensions.

Also, I never had a problem, even when building on large machines and running on tiny containers. That's why I wanted to see a reproducer highlighting a real problem.

[zakkak]

Sure, but having Platform0 initialized at runtime is not simple....

That's what I am hitting as well :/
I haven't given up yet though, I will try some more things tomorrow.

[gsmet]

Unfortunately, we had to revert the fix because of #32289 .

[zakkak]

Resurrecting this and planning to work on it again.

Kind of Reproducer

cd /tmp
git clone --branch 2023-11-23-quarkus-17839-reproducer-netty-platform-dependent https://github.com/zakkak/issue-reproducers reproducer
cd reproducer
mvn package
java -jar ./target/reproducer-1.0-SNAPSHOT.jar

Depending on the memory of the system we are running on this prints something like:

maxDirectMemory= 15 GB

Building a native image from it:

native-image --initialize-at-build-time=. \
  --no-fallback \
  -jar target/reproducer-1.0-SNAPSHOT.jar

during the build native-image informs us about the resources being used, e.g.:

Build resources:
 - 7.70GB of memory (12.4% of 61.93GB system memory, determined at start)
 - 32 thread(s) (100.0% of 32 available processor(s), determined at start)

and the resulting binary prints:

maxDirectMemory= 7 GB

If we set -Xmx we can control the amount of memory used at build time which also affects the maxDirectMemory reported by netty:

native-image --initialize-at-build-time=. \
  -J-Xmx32g \
  --no-fallback \
  -jar target/reproducer-1.0-SNAPSHOT.jar

during build this will print:

Build resources:
 - 28.44GB of memory (45.9% of 61.93GB system memory, set via '-Xmx32g')
 - 32 thread(s) (100.0% of 32 available processor(s), determined at start)

and running the binary prints:

maxDirectMemory= 28 GB

Even if we move the binary to a different machine, e.g. one printing:

maxDirectMemory= 3 GB

when run in HotSpot, the native executable still prints:

maxDirectMemory= 28 GB

which is not desirable, the same happens with other fields as well.

Workaround

As a workaround if someone is having issues with invalid maxDirectMemory, they could possibly use -J-Xmx when building the native image to control its size.

[zakkak]

I created a better reproducer using Quarkus this time, see https://github.com/zakkak/issue-reproducers/blob/2023-11-28-quarkus-17839-reproducer-netty-platform-dependent-in-quarkus/Readme.md and opened #37347 which resolves the issue.

[antonwiens]

I created a better reproducer using Quarkus this time, see https://github.com/zakkak/issue-reproducers/blob/2023-11-28-quarkus-17839-reproducer-netty-platform-dependent-in-quarkus/Readme.md and opened #37347 which resolves the issue.

Hi Zakkak, it looks like your previous PR in #31016, just without the unsafe thing. The problem was performance, you think without unsafe it won`t affect performance anymore?

[zakkak]

@antonwiens, correct. Disabling unsafe was the main performance issue. Fortunately we no longer need to do this (not sure why but I suspect netty/netty#13643).

FWIW, if anyone wants to limit/control the max direct memory seen by netty they need to use -Xmx (like in JVM mode), e.g.:

./reproducer-1.0-SNAPSHOT -Xmx5g 
maxDirectMemory= 5 GB
./reproducer-1.0-SNAPSHOT -Xmx12g
maxDirectMemory= 12 GB

[cescoffier]

Yes, it's something we discussed a long time ago with @franz1981, and he found a way to workaround Unsafe.