Error: "argument list too long" (because of large BULIDKITE_PLUGINS env var)

[andpol]

Is your feature request related to a problem? Please describe.
We use this monorepo plugin which seems to pass it's configurations via encoded JSON through the BUILDKITE_PLUGINS environment variable. We recently found the limits of that, and have the following error when running the step:

fork/exec /buildkite-agent: argument list too long

This seems closely related to the existing (resolved) issues regarding long Git commit messages and the BUILDKITE_MESSAGE env var: #1302 and #1307.

Describe the solution you'd like
It would be nice if the BUILDKITE_PLUGINS env var could accept longer strings (and maybe generalized to other env vars?) Not being super familiar to Buildkite and the source code, I'm not sure if this is actually possible, but hoping someone here can advise if this is one potential solution. I would argue this is not an unreasonable feature to add, since it seems not unreasonable that plugins could have more extensive configurations.

Describe alternatives you've considered
As a short-term workaround, we've split up the Buildkite Steps that run the Monorepo plugin. This works for now, but scales based on the number of applications/files we've configured with the plugin, and will likely break for us in the future once again.

Another alternative is changing the Monorepo plugin, but I'm unfamiliar with how Buildkite plugins work, so maybe there isn't a great path forwards from that perspective?

Additional context
As an attempted fix, we increased the stack size limit (ulimit -s) on our Buildkite agents, with no effect.

[pda]

Interesting!

I've done some digging in the past re. environment variable limits on different operating systems, and recently tested/updated it for macOS. Unfortunately the citations are lost to the sands of time. Here's the notes I made at the time:

macOS: 1 MiB (previously 256 KiB?) shared across all env and argv
Linux: 128 KiB per env key=value, (2 MiB total? depends on… things)
Windows: 16 MiB shared by all env (And argv? Does Windows have argv?)
POSIX bare minimum: 4 KiB shared by all env and argv

[andpol]

We're operating our Agents on Linux (in a K8s cluster). The BUILDKITE_PLUGINS environment variable for the build in question is 168KiB in size, so hitting the error makes sense in the context of the limits you've found.

Do you think there is a reasonable way to pass along the plugin data to the agent in a way that doesn't use environment variables?