`jetty.sh` cannot handle complex Jetty properties from `start.d/*.ini` #9309

amjaliks · 2023-02-03T14:26:37Z

Jetty version(s)
Jetty 11.0.13

Java version/vendor (use: java -version)
openjdk version "17.0.5" 2022-10-18
OpenJDK Runtime Environment (build 17.0.5+8-Ubuntu-2ubuntu122.04)
OpenJDK 64-Bit Server VM (build 17.0.5+8-Ubuntu-2ubuntu122.04, mixed mode, sharing)

OS type/version
Distributor ID: Ubuntu
Description: Ubuntu 22.04 LTS
Release: 22.04
Codename: jammy

Description
Can't set custom request log format with spaces. Even example from comment.

If I add this line to requestlog.ini:

jetty.requestlog.formatString=%{client}a - %u %{dd/MMM/yyyy:HH:mm:ss ZZZ|GMT}t "%r" %s %O "%{Referer}i" "%{User-Agent}i"

Jetty fails to run with error message (for a full stacktrace see below):

2023-02-03 14:22:39.491:WARN :oejx.XmlConfiguration:main: Unable to execute XmlConfiguration
java.security.PrivilegedActionException: java.nio.file.NoSuchFileException: -\
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1818)

It is possible to add format without spaces and start Jetty successfully, for example:

jetty.requestlog.formatString="%{client}a-%u-%t-"%r"-%s-%O-%{ms}T"

Full stacktrace:

2023-02-03 14:22:39.491:WARN :oejx.XmlConfiguration:main: Unable to execute XmlConfiguration
java.security.PrivilegedActionException: java.nio.file.NoSuchFileException: -\
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1818)
Caused by: 
java.nio.file.NoSuchFileException: -\
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
	at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422)
	at java.base/java.nio.file.Files.newInputStream(Files.java:160)
	at org.eclipse.jetty.util.resource.PathResource.getInputStream(PathResource.java:456)
	at org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:220)
	at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$4(XmlConfiguration.java:1849)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1818)
Exception in thread "main" java.security.PrivilegedActionException: java.nio.file.NoSuchFileException: -\
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1818)
Caused by: java.nio.file.NoSuchFileException: -\
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
	at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422)
	at java.base/java.nio.file.Files.newInputStream(Files.java:160)
	at org.eclipse.jetty.util.resource.PathResource.getInputStream(PathResource.java:456)
	at org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:220)
	at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$4(XmlConfiguration.java:1849)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
	... 1 more

The text was updated successfully, but these errors were encountered:

Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>

joakime · 2023-02-03T14:47:06Z

I'm unable to replicate on the command line with Jetty 11.0.13.
I went ahead and added a testcase to Jetty 10.0.x as well (we develop in jetty-10.0.x and then merge to jetty-11.0.x) that also doesn't fail.

joakime · 2023-02-03T14:48:51Z

The error you are seeing is not from the format string, as the exception has something to do with resolving of an XML file.

Can you run the following and give use the output?

$ cd /path/to/my-jetty-base/
$ java -jar /path/to/jetty-home-11.0.13/start.jar --list-config

Note: these should be different directories.

amjaliks · 2023-02-03T15:01:31Z

Yes, I see it doesn't appear to be directly related to requestlog. However, I run into this issue, if try add this property to config.

Here is list-config output: jetty-list-modules.txt.

This is without formatString. If I add, it appears in properties list:

jetty.requestlog.formatString = %{client}a - %u %{dd/MMM/yyyy:HH:mm:ss ZZZ|GMT}t "%r" %s %O "%{Referer}i" "%{User-Agent}i"

joakime · 2023-02-03T15:25:55Z

Interesting filename ...

 1:                    5.0.1 | ${jetty.base}/lib/ext/remotecontent?filepath=com%2Fzaxxer%2FHikariCP%2F5.0.1%2FHikariCP-5.0.1.jar

It should have no impact on your issue, just thought it was interesting to see.

Can you check that you have access to the files listed in ...

Jetty Active XMLs:
------------------
 ${jetty.home}/etc/jetty-bytebufferpool.xml
 ${jetty.home}/etc/jetty-threadpool.xml
 ${jetty.home}/etc/jetty.xml
 ${jetty.home}/etc/jetty-webapp.xml
 ${jetty.home}/etc/jetty-deploy.xml
 ${jetty.home}/etc/jetty-ssl.xml
 ${jetty.home}/etc/jetty-ssl-context.xml
 ${jetty.home}/etc/jetty-https.xml
 ${jetty.home}/etc/jetty-requestlog.xml
 ${jetty.home}/etc/jetty-ssl-context-reload.xml
 ${jetty.base}/etc/jndi.xml

Also check your start.d/*.ini files for a rogue line that might say just - or -\ or \ (those lines would be attempted to be loaded as XML files)

The output of --dry-run might reveal something useful.

$ $ java -jar /path/to/jetty-home/start.jar --dry-run

Finally, try running with start.jar debug turned on, and see what the debug before the failure indicates?

$ java -jar /path/to/jetty-home/start.jar --debug

The command line for XmlConfiguration would be interesting to see.

amjaliks · 2023-02-03T15:33:22Z

Just found this issue arises only when I try run as a service.

If run it directly (java -jar start.jar) it starts normally.

Here is output from service jetty check:

JAVA                  =  /usr/bin/java
JAVA_OPTIONS          =  -Xmx4g -server -Djava.net.preferIPv4Stack1=true
JETTY_HOME            =  <HOME>
JETTY_BASE            =  <BASE>
START_D               =  <BASE>/start.d
START_INI             =  <BASE>/start.ini
JETTY_START           =  <HOME>/start.jar
JETTY_CONF            =  <HOME>/etc/jetty.conf
JETTY_ARGS            =  jetty.state=<BASE>/jetty-11.state jetty-started.xml
JETTY_RUN             =  <BASE>/jetty
JETTY_PID             =  <BASE>/jetty/jetty-11.pid
JETTY_START_LOG       =  <BASE>/logs/start.log
JETTY_STATE           =  <BASE>/jetty-11.state
JETTY_START_TIMEOUT   =  60
RUN_CMD               =  /usr/bin/java -Djetty.home=<HOME> -Djava.io.tmpdir=/tmp -Djetty.home=<HOME> -Djetty.base=<BASE> -Xmx4g -server -Djava.net.preferIPv4Stack1=true --class-path <BASE>/lib/ext/postgresql-42.3.6.jar:<BASE>/lib/ext/remotecontent?filepath=com%2Fzaxxer%2FHikariCP%2F5.0.1%2FHikariCP-5.0.1.jar:<BASE>/resources:<HOME>/lib/logging/slf4j-api-2.0.5.jar:<HOME>/lib/logging/jetty-slf4j-impl-11.0.13.jar:<HOME>/lib/jetty-jakarta-servlet-api-5.0.2.jar:<HOME>/lib/jetty-http-11.0.13.jar:<HOME>/lib/jetty-server-11.0.13.jar:<HOME>/lib/jetty-xml-11.0.13.jar:<HOME>/lib/jetty-util-11.0.13.jar:<HOME>/lib/jetty-io-11.0.13.jar:<HOME>/lib/jetty-jndi-11.0.13.jar:<HOME>/lib/jetty-security-11.0.13.jar:<HOME>/lib/jetty-servlet-11.0.13.jar:<HOME>/lib/jetty-webapp-11.0.13.jar:<HOME>/lib/jetty-plus-11.0.13.jar:<HOME>/lib/jakarta.transaction-api-2.0.0.jar:<HOME>/lib/jetty-annotations-11.0.13.jar:<HOME>/lib/annotations/asm-9.4.jar:<HOME>/lib/annotations/asm-analysis-9.4.jar:<HOME>/lib/annotations/asm-commons-9.4.jar:<HOME>/lib/annotations/asm-tree-9.4.jar:<HOME>/lib/annotations/jakarta.annotation-api-2.1.1.jar:<HOME>/lib/jetty-deploy-11.0.13.jar org.eclipse.jetty.xml.XmlConfiguration java.net.preferIPv4Stack1=true java.version=17.0.5 java.version.major=17 java.version.micro=5 java.version.minor=0 java.version.platform=17 jetty.base=<BASE> jetty.base.uri=file://<BASE> jetty.home=<HOME> jetty.home.uri=file://<HOME> jetty.requestlog.append=true jetty.requestlog.formatString=%{client}a\ -\ %u\ %{dd/MMM/yyyy:HH:mm:ss\ ZZZ|GMT}t\ \"%r\"\ %s\ %O\ \"%{Referer}i\"\ \"%{User-Agent}i\" jetty.ssl.port=<PORT> jetty.sslContext.keyStorePassword=antons jetty.state=<BASE>/jetty-11.state jetty.webapp.addServerClasses=org.eclipse.jetty.logging.,file://<HOME>/lib/logging/,org.slf4j. runtime.feature.alpn=true slf4j.version=2.0.5 <HOME>/etc/jetty-bytebufferpool.xml <HOME>/etc/jetty-threadpool.xml <HOME>/etc/jetty.xml <HOME>/etc/jetty-webapp.xml <HOME>/etc/jetty-deploy.xml <HOME>/etc/jetty-ssl.xml <HOME>/etc/jetty-ssl-context.xml <HOME>/etc/jetty-https.xml <HOME>/etc/jetty-requestlog.xml <HOME>/etc/jetty-ssl-context-reload.xml <BASE>/etc/jndi.xml <HOME>/etc/jetty-started.xml

amjaliks · 2023-02-03T15:41:24Z

Here is output for dry-run:

/usr/lib/jvm/java-17-openjdk-amd64/bin/java -Djava.io.tmpdir=/tmp -Djetty.home=<HOME> -Djetty.base=<BASE> --class-path <BASE>/lib/ext/postgresql-42.3.6.jar:<BASE>/lib/ext/remotecontent?filepath=com%2Fzaxxer%2FHikariCP%2F5.0.1%2FHikariCP-5.0.1.jar:<BASE>/resources:<HOME>/lib/logging/slf4j-api-2.0.5.jar:<HOME>/lib/logging/jetty-slf4j-impl-11.0.13.jar:<HOME>/lib/jetty-jakarta-servlet-api-5.0.2.jar:<HOME>/lib/jetty-http-11.0.13.jar:<HOME>/lib/jetty-server-11.0.13.jar:<HOME>/lib/jetty-xml-11.0.13.jar:<HOME>/lib/jetty-util-11.0.13.jar:<HOME>/lib/jetty-io-11.0.13.jar:<HOME>/lib/jetty-jndi-11.0.13.jar:<HOME>/lib/jetty-security-11.0.13.jar:<HOME>/lib/jetty-servlet-11.0.13.jar:<HOME>/lib/jetty-webapp-11.0.13.jar:<HOME>/lib/jetty-plus-11.0.13.jar:<HOME>/lib/jakarta.transaction-api-2.0.0.jar:<HOME>/lib/jetty-annotations-11.0.13.jar:<HOME>/lib/annotations/asm-9.4.jar:<HOME>/lib/annotations/asm-analysis-9.4.jar:<HOME>/lib/annotations/asm-commons-9.4.jar:<HOME>/lib/annotations/asm-tree-9.4.jar:<HOME>/lib/annotations/jakarta.annotation-api-2.1.1.jar:<HOME>/lib/jetty-deploy-11.0.13.jar org.eclipse.jetty.xml.XmlConfiguration java.version=17.0.5 java.version.major=17 java.version.micro=5 java.version.minor=0 java.version.platform=17 jetty.base=<BASE> jetty.base.uri=file://<BASE> jetty.home=<HOME> jetty.home.uri=file://<HOME> jetty.requestlog.append=true jetty.requestlog.formatString=%{client}a\ -\ %u\ %{dd/MMM/yyyy:HH:mm:ss\ ZZZ|GMT}t\ \"%r\"\ %s\ %O\ \"%{Referer}i\"\ \"%{User-Agent}i\" jetty.ssl.port=<PORT> jetty.sslContext.keyStorePassword=antons jetty.webapp.addServerClasses=org.eclipse.jetty.logging.,file://<HOME>/lib/logging/,org.slf4j. runtime.feature.alpn=true slf4j.version=2.0.5 <HOME>/etc/jetty-bytebufferpool.xml <HOME>/etc/jetty-threadpool.xml <HOME>/etc/jetty.xml <HOME>/etc/jetty-webapp.xml <HOME>/etc/jetty-deploy.xml <HOME>/etc/jetty-ssl.xml <HOME>/etc/jetty-ssl-context.xml <HOME>/etc/jetty-https.xml <HOME>/etc/jetty-requestlog.xml <HOME>/etc/jetty-ssl-context-reload.xml <BASE>/etc/jndi.xml

Interesting filename ...
5.0.1 | ${jetty.base}/lib/ext/remotecontent?filepath=com%2Fzaxxer%2FHikariCP%2F5.0.1%2FHikariCP-5.0.1.jar

I assume I forgot to change filename created by wget.

joakime · 2023-02-03T15:41:36Z

Your service is attempting to use --dry-run output in a raw RUN_CMD.
That's not going to work reliably with complex command line entries like that format string.

Easiest option:

If you can run without --dry-run then do so.
Just use start.jar directly.

Next option:

Take all of the properties defined after org.eclipse.jetty.xml.XmlConfiguration and put them in a *.properties file.
Remove them from RUN_CMD and simply have a reference to the properties file as the way to reference properties after the XmlConfiguration portion (you'll still need the XML arguments)

amjaliks · 2023-02-03T15:43:44Z

I am using default service script jetty.sh from Jetty distribution.

joakime · 2023-02-03T15:47:50Z

I am using default service script jetty.sh from Jetty distribution.

sigh. That script is a super sensitive file to maintain (seems every time we address something there we break someone's workflow).

We have some niceties for --dry-run=<section>, i wonder if we can have a --properties-export=<filename> that the jetty.sh can use with --dry-run without properties as a possible solution.

I really don't want to get into the business of shell escaping those properties in --dry-run or within jetty.sh

joakime · 2023-02-03T15:51:07Z

Should probably use /var/run/<jetty-service-id>/start.properties as a temporary filename for this --properties-export=<filename> option.

Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>

joakime · 2023-02-03T16:24:20Z

Opened PR #9313 to make the escaping of command line arguments more comprehensive.

Change is in branch fix/jetty-10.0.x/jetty-sh-start-properties

@amjaliks I know you are using Jetty 11, and this PR is currently only in Jetty 10, but is there any way you can test it? (even if your environment starts, but fails to deploy the webapp, it's a great sign we are on the right path)

amjaliks · 2023-02-03T16:27:50Z

I have an instance with Jetty 10 too.
How can I test this? Do I need to checkout this branch and build it?

joakime · 2023-02-03T16:33:26Z

You can checkout the branch and build it ...

$ git checkout fix/jetty-10.0.x/jetty-sh-start-properties
$ mvn clean install -DskipTests
# The as-built jetty-home can be found in jetty-home/target/jetty-home-10.0.14-SNAPSHOT.tar.gz

amjaliks · 2023-02-03T19:10:16Z

No, this doesn't help. The same result, when trying to run as a service.

dry-run outputs: jetty.requestlog.formatString=\%\{client\}a\ -\ \%u\ \%t\ \"\%r\"\ \%s\ \%O\ \%\{ms\}T.

joakime · 2023-02-03T19:40:07Z

I'll keep working on the escaping side (it's needed for all things command line related, not just properties)

I also need to introduce the --property-export=<filename> option to allow jetty.sh to work with even more complex command lines.

…mat-with-spaces Issue #9309 - Introducing test for requestlog format with spaces

joakime · 2023-02-06T21:50:33Z

@amjaliks an alternate approach was just pushed to that branch.
Can you give it a go? (it works for me now)

Also, can you please post your results in PR #9313 ?

…x-documentation-operations-logging * upstream/jetty-12.0.x: (35 commits) Fixes jetty#9326 - Rename DecryptedEndPoint to SslEndPoint. Jetty 10 Upgrade to Hazelcast 5 and totally disable auto join multicast etc.. (fix build on CI) (jetty#9331) jetty#9328 - changes from review jetty#9287 - catch error in ee9 maxRequestSize MultiPart test Jetty 12.0.x 9301 fix ee10 jstl jpms (jetty#9321) Issue jetty#9301 Fix dependencies for ee10-glassfish-jstl module (jetty#9303) Jetty 12 Hazelcast 5.x and disable auto detection/multicast" (jetty#9332) jetty#9287 - fix further test failures Fixed imports. Issue jetty#7650 - Fix race condition when stopping QueuedThreadPool (jetty#9325) jetty#9287 - remove unpaired release of Content.Chunk Issue jetty#8991 - rename websocket isDemanding() method to isAutoDemanding() Issue jetty#9287 - fix failing tests changes f rom review add todo to revert to normal pool after fix for jetty#9311 Issue jetty#9309 - Introducing test for requestlog format with spaces use non-pooling RetainableByteBufferPool to work around performance bug consumeAvailable should use number of reads instead of bytes fix for retainable merge changes from review ...

amjaliks · 2023-02-22T13:36:32Z

@joakime Sorry! I was away for a while.

Should I post my results here and in #9313? Or just move our discussion there?

Unfortunately, with the latest changes in this branch my 10.0.x instance doesn't start at all as a service. Even before I start experimenting with a custom log format.

joakime · 2023-02-22T14:04:03Z

@amjaliks keep the discussion on #9313 if possible.

…-properties Issue #9309 - Better jetty.sh integration for start.jar with eye on supporting odd properties

joakime · 2023-03-24T18:46:58Z

This fix has been merged into jetty-10.0.x and then merged up to jetty-11.0.x and jetty-12.0.x

joakime · 2023-06-06T13:23:50Z

jetty.sh is broken, reopening.

joakime · 2023-06-06T13:47:44Z

Closing, as new issues are covered in #9880

amjaliks added the Bug For general bugs on Jetty side label Feb 3, 2023

joakime added a commit that referenced this issue Feb 3, 2023

Issue #9309 - Introducing test for requestlog format with spaces

46a316d

Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>

joakime self-assigned this Feb 3, 2023

joakime added this to the 10.0.x milestone Feb 3, 2023

joakime changed the title ~~Can't set custom request log format with spaces~~ jetty.sh cannot handle complex Jetty properties from start.d/*.ini Feb 3, 2023

joakime added a commit that referenced this issue Feb 3, 2023

Issue #9309 - More comprehensive escaping of command line options

45fdebc

Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>

joakime added a commit that referenced this issue Feb 6, 2023

Merge pull request #9310 from eclipse/fix/jetty-10.0.x/requestlog-for…

0a4a077

…mat-with-spaces Issue #9309 - Introducing test for requestlog format with spaces

joakime linked a pull request Feb 6, 2023 that will close this issue

Issue #9309 - Better jetty.sh integration for start.jar with eye on supporting odd properties #9313

Merged

amjaliks mentioned this issue Feb 22, 2023

Issue #9309 - Better jetty.sh integration for start.jar with eye on supporting odd properties #9313

Merged

joakime mentioned this issue Mar 13, 2023

Bad directory permissions reported from Jetty 10.0.14 when using Docker 23.0.1 #9491

Closed

joakime closed this as completed in #9313 Mar 24, 2023

joakime added a commit that referenced this issue Mar 24, 2023

Merge pull request #9313 from eclipse/fix/jetty-10.0.x/jetty-sh-start…

a9bae8e

…-properties Issue #9309 - Better jetty.sh integration for start.jar with eye on supporting odd properties

joakime mentioned this issue Apr 19, 2023

Jetty 10.0.15: Env variable in console-capture.ini not expanded anymore #9663

Open

joakime reopened this Jun 6, 2023

joakime closed this as completed Jun 6, 2023

This was referenced Jul 5, 2023

do not automatically quote system properties containing spaces #9999

Closed

Improved start.jar dry-run command line quoting #10090

Closed

Quoting for start arguments #10099

Merged

l-cdc mentioned this issue Aug 8, 2023

jetty.sh does not stop jetty anymore #10271

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`jetty.sh` cannot handle complex Jetty properties from `start.d/*.ini` #9309

`jetty.sh` cannot handle complex Jetty properties from `start.d/*.ini` #9309

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 3, 2023

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023 •

edited

amjaliks commented Feb 3, 2023 •

edited

amjaliks commented Feb 3, 2023 •

edited

joakime commented Feb 3, 2023 •

edited

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 3, 2023

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 6, 2023

amjaliks commented Feb 22, 2023

joakime commented Feb 22, 2023

joakime commented Mar 24, 2023

joakime commented Jun 6, 2023

joakime commented Jun 6, 2023

jetty.sh cannot handle complex Jetty properties from start.d/*.ini #9309

jetty.sh cannot handle complex Jetty properties from start.d/*.ini #9309

Comments

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 3, 2023

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023 • edited

amjaliks commented Feb 3, 2023 • edited

amjaliks commented Feb 3, 2023 • edited

joakime commented Feb 3, 2023 • edited

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 3, 2023

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

amjaliks commented Feb 3, 2023

joakime commented Feb 3, 2023

joakime commented Feb 6, 2023

amjaliks commented Feb 22, 2023

joakime commented Feb 22, 2023

joakime commented Mar 24, 2023

joakime commented Jun 6, 2023

joakime commented Jun 6, 2023

`jetty.sh` cannot handle complex Jetty properties from `start.d/*.ini` #9309

`jetty.sh` cannot handle complex Jetty properties from `start.d/*.ini` #9309

joakime commented Feb 3, 2023 •

edited

amjaliks commented Feb 3, 2023 •

edited

amjaliks commented Feb 3, 2023 •

edited

joakime commented Feb 3, 2023 •

edited