[MBUILDCACHE-64] Exclusion mechanism bugfix #91

kbuntrock · 2023-08-01T20:36:49Z

Description of this pull request

Fix a bug regarding the exclusion mechanism not working properly on folders.
It also adds an integration test on this functionality (global + per project) + some documentation on these aspects.

Here a screenshot of the new "exclude" element in the documentation to understand how this fix intend to work.

I have not implemented the following sentence in the Jira issue since I don't think we need to overthink the feature without some use case examples :

"That function could be enhanced with an additional criterion like in MavenProjectInput.java#L510"

Nevertheless, I think this MR is already a huge improvement on the exclusion topic since it's an important feature and it was not possible to use it on folders before (actually inconvenient to use the extension on a project with a "node_modules" folder for example).

MR checklist

Following this checklist to help us incorporate your
contribution quickly and easily:

Make sure there is a MBUILDCACHE JIRA issue filed
for the change (usually before you start working on it). Trivial changes like typos do not
require a JIRA issue. Your pull request should address just this issue, without
pulling in other changes.
Each commit in the pull request should have a meaningful subject line and body.
Format the pull request title like [MBUILDCACHE-XXX] - Fixes bug in ApproximateQuantiles,
where you replace MBUILDCACHE-XXX with the appropriate JIRA issue. Best practice
is to use the JIRA issue title in the pull request title and in the first line of the
commit message.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Run mvn clean verify to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
You have run the Core IT successfully.

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004
In any other case, please file an Apache Individual Contributor License Agreement.

…th folders)

…nctionnality, from global and project configuration

kbuntrock · 2023-08-03T20:52:11Z

Miss-clicked on the sync button on my repo for this branch (I only wanted to sync master ...).
So I finished to rebase it on master to keep the tree clean. But it reseted the workflow, sorry. :s

olamy · 2023-08-04T01:48:09Z

LGTM

gnodet

I'm a bit skeptical about the way which has been chosen to filter files. I think it would be more intuitive to use a glob PathMatcher rather than a startsWith() call on the path.
This seems inconsistent with what is usually done in the maven land, for example https://maven.apache.org/plugins/maven-jar-plugin/examples/include-exclude.html

kbuntrock · 2023-08-04T17:58:32Z

I'm a bit skeptical about the way which has been chosen to filter files. I think it would be more intuitive to use a glob PathMatcher rather than a startsWith() call on the path. This seems inconsistent with what is usually done in the maven land, for example https://maven.apache.org/plugins/maven-jar-plugin/examples/include-exclude.html

Sure, I can do this. It will also bring a bit more flexibility. I suggest that I stay focused on the exclude section and I don't touch the input section.

PS : Since I'm new there, hello everyone and great job! This project is a really good idea. I'm dying to be able to use it on my projects. :)

…ude functionality

kbuntrock · 2023-08-07T17:02:30Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

-    /**
-     * default glob, bbsdk/abfx specific
-     */
-    public static final String DEFAULT_GLOB = "{*.java,*.groovy,*.yaml,*.svcd,*.proto,*assembly.xml,assembly"


Was not used except in a UT.

kbuntrock · 2023-08-07T17:04:09Z

src/test/java/org/apache/maven/buildcache/checksum/MavenProjectInputTest.java

-    @Test
-    public void testGetDirectoryFiles() {
-        List<Path> directoryFiles = new ArrayList<>();
+    public void testCollectFilteredFiles() {


Cleaned dead code and updated a test with a wrong assumption. Not the most useful test but better than before.

kbuntrock · 2023-08-07T17:05:34Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

+            inputExcludeDirectoryPathMatchers.add(new TreeWalkerPathMatcher(pathMatcherGlob, false));
+        }
+
+        // If the pattern ends with "/**", the directory exclude list gets an extra version without this suffix.


Important for a folder containing a lot of files, as "node_modules" for example.

kbuntrock · 2023-08-07T17:12:43Z

Changed the exclude mechanism to use path matchers. A bit more than a bugfix now, feedbacks appreciated if you see any lack, implementation detail you disagree with or extra test case to add.

maximilian-novikov-db · 2023-08-21T07:25:48Z

Changed the exclude mechanism to use path matchers. A bit more than a bugfix now, feedbacks appreciated if you see any lack, implementation detail you disagree with or extra test case to add.

@kbuntrock nicely done, it would be useful to update the sample config as well https://github.com/apache/maven-build-cache-extension/blob/master/src/site/resources/maven-build-cache-config.xml

AlexanderAshitkin

Using paths when working with directories feels more natural compared to analyzing globs. Separate abstractions for normalized paths and(or) globs might make the code more robust.

AlexanderAshitkin · 2023-08-18T18:30:12Z

src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java

+    /**
+     * Wrapped regular path matcher
+     */
+    private final PathMatcher pathMatcher;


Could it lead to OS-specific errors? For example, if we replace the slashes, could that break the path matching on Windows? Ideally, we need to encapsulate Paths and work on the normalized paths only to ensure portability.

It should not, the underlying conversion from glob to regex pattern (sun.nio.fs.Globs#toRegexPattern) used by different FS pathmatcher implementations (ex : sun.nio.fs.WindowsFileSystem#getPathMatcher) seems to handle FS specific syntax conversion since java 8. (https://github.com/adoptium/jdk8u)

But PathMatching is not something I've dealt with a lot. Do you think we should take a belt and braces approach ? (french expression, but it looks like existing in English too ^^)

PS : On a side note, I'm always testing on Windows (but not all jvm ...) since I primarily work on Windows those days.

There might be room for error here. For example, the same file with back/forward slashes is not matched on my workstation:

String linux = "dir/file.txt"; String win = "dir\\file.txt"; Path linuxPath = Paths.get(linux); Path winPath = Paths.get(win); System.out.println("linuxPath: " + linuxPath); System.out.println("winPath: " + winPath); PathMatcher linuxMatcher = FileSystems.getDefault().getPathMatcher("glob:" + linuxPath); System.out.println(linuxMatcher.matches(winPath)); System.out.println(linuxMatcher.matches(linuxPath)); ------OUT---- linuxPath: dir/file.txt winPath: dir\file.txt false true

I have no objections against the matcher as such. But I wouldn't rely on it without a test demonstrating the same results on Windows and Mac in the presence of exclusions.

I could use some intel about the current IT tests on different platforms and how to launch them / develop some for not yet supported platforms.
And I get that it could be useful to normalize it before going through the implementation of "getPathMatcher" in order to reduce bug investigation in case of an exotic "getPathMatcher" implementation, I will think about adding this extra step.

AlexanderAshitkin · 2023-08-18T18:58:05Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

+    /**
+     * Glob suffix meaning "all files in this directory or any sub-directory"
+     */
+    private static final String GLOB_SX_ALL_SUB_FILES = "/**";


I would expect /* for all sub-files. GLOB_SX_FULL_SUB_TREE could be a more accurate naming for a full depth subtree.

Good call, fixed

AlexanderAshitkin · 2023-08-18T19:31:02Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

@@ -165,45 +184,94 @@ public MavenProjectInput(
        this.repoSystem = repoSystem;
        this.remoteCache = remoteCache;
        Properties properties = project.getProperties();
-        this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
+        this.defaultFilenameGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());


The glob is not limited to filenames, and the exact format is not guaranteed, etc. There is no base to incorporate the content assumption in the variable name. Better naming would be projectGlob, which properly reflects the intention - a project-specific glob, overriding default if specified.

Good point, changed by the suggested name.

AlexanderAshitkin · 2023-08-18T19:32:10Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

        this.processPlugins =
                Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS, config.isProcessPlugins()));
        this.tmpDir = System.getProperty("java.io.tmpdir");

+        this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") + "/";


For consistency, this should probably be normalized to a <path>/** format, or the variable should be renamed *Path.

This constant is used as a prefix and concatenated with other values. Renamed it to baseDirectoryGlobPrefix to improve the readability.

AlexanderAshitkin · 2023-08-18T19:37:38Z

src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java

+    }
+
+    @Override
+    public boolean matches(Path path) {


We may need a wrapper type here to ensure consistency in the path matching. The second, less preferable option is to normalize paths within the matcher. But in the latter case, the normalized paths cannot be reused in the other parts of the code.

Do you mean in case we decide to do some extra work about OS specific filesystems? Or anyway?

Paths could be expressed differently, and before matching, it should be normalized - placeholders resolved, etc.

PathMatcher pathMatcher = FileSystems.getDefault().getPathMatcher("glob:./file.txt"); System.out.println("./file.txt -> "+ Paths.get("./file.txt").normalize().toAbsolutePath()); System.out.println("./dir/../file.txt -> "+ Paths.get("dir/../file.txt").normalize().toAbsolutePath()); System.out.println(pathMatcher.matches(Paths.get("dir/../file.txt"))); ----Out--- ./file.txt -> <dirpath>/sandbox/file.txt ./dir/../file.txt -> <dirpath>/sandbox/file.txt false

The matcher might be required to normalize paths to match reliably. Just taking any path as an input might not work. It's better to have clear semantics here.

Ok, got it. As I do not want to go too far with this MR, I suggest to do like the WalkKey class for the include section -> naming the path in the "matches" function : "normalized". To indicate to a developer that a provided path should be normalized.

Added to the list of topics to be discussed on Thursday.

AlexanderAshitkin · 2023-08-18T20:00:30Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

+        inputExcludePathMatchers.add(new TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+        // Globs ending with "**" should end any sub-directory inspection
+        if (pathMatcherGlob.endsWith("**")) {


This is a very thin ice. Glob syntax is rich. For example, it could be [*][*] or /my/fav/dir/{**} or /my/*/dir/**. It is a fragile approach to interpreting the globs based on the content. excludedValue could be anything, and the logic might not be applied properly. That's why the original logic operated with normalized paths.

I totally agree, even if the flexibility offered is pretty nice, I wasn't so pleased with the complexity of this demand.
As a trade-off, my strategy was :

if the syntax is not written in an exotic manner : exclusion is fast as it can stop tree walking

if not : exclusion is slower but works as expected

AlexanderAshitkin · 2023-08-18T20:18:14Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

@@ -526,6 +603,10 @@ public FileVisitResult visitFileFailed(Path path, IOException exc) throws IOExce
        });
    }

+    private boolean entryMustBeSkipped(Path path) {
+        return inputExcludePathMatchers.stream().anyMatch(pathMatcher -> pathMatcher.matches(path));


neat picking: use method reference instead of the lambda - PathMatcher::matches

I can't, "matches" takes a parameter (or you refer to a syntax I don't know).

Right, iteration is over the matchers, not paths 🤦

AlexanderAshitkin · 2023-08-18T20:21:51Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

-            }
-        }
-        return false;
+        return inputExcludeDirectoryPathMatchers.stream().anyMatch(matcher -> matcher.stopTreeWalking(path));


The logic for skipping dirs should be paths based. Globs should be used as globs without any content assumptions. To get the benefits of both approaches, creating a wrapper class for the globs might be better. And if the glob is path-based (eg derived by adding /**, some optimizations could be applied). Which will keep consistent values of the original literal, derived path, and the derived glob.

I'm not following. Could you give me some code snippets to illustrate your reasoning?

Well, there are different semantics now and before for project-level properties.
Before: declaring <exclude.me>mydir/mysubdir</exclude.me> exclusion was filtering out the whole subtree (through the Path#startsWith)
Now: declaring mydir/mysubdir doesn't exclude anything.

FileSystems.getDefault().getPathMatcher("glob:dir"); System.out.println(pathMatcher.matches(Paths.get("dir/file.txt")));` -----OUT----- false

This is what I meant - properties might assume the exclusion of directories. For consistency, we must update docs, keep the path semantics, or migrate properties to globs/regex semantics, delegating filtering expressions to the users. But the main point is that it should be consistent with inclusion semantics (and maven in general), transparent to an end user, and deterministic.

EDIT : My response was wrong. Removing its content to improve readability.

I agree with the point, but I did not want to go too far with this MR. :P

AlexanderAshitkin · 2023-08-18T20:23:10Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

-                normalizedPath(build.getDirectory()), // target by default
-                normalizedPath(build.getOutputDirectory()),
-                normalizedPath(build.getTestOutputDirectory())));
+        addToExcludedSection(


There is room for a small optimization here - not adding sub-dirs of the target will reduce the length of the filter.

Good call. Added it. I did a naive implementation to keep the code simple.

AlexanderAshitkin · 2023-08-18T20:25:59Z

src/test/java/org/apache/maven/buildcache/its/IncludeExcludeTest.java

@@ -0,0 +1,107 @@
+/*


Thank you for implementing this test. It is long past due to implement 👍

kbuntrock · 2023-08-23T21:55:54Z

@kbuntrock nicely done, it would be useful to update the sample config as well https://github.com/apache/maven-build-cache-extension/blob/master/src/site/resources/maven-build-cache-config.xml

Done it. Wasn't comfortable with this file since I found it confusing when I discovered this extension + it was still valid with my changes. Added some comments on some related other parameters around. Is it ok for you @maximilian-novikov-db ?

gnodet · 2023-08-25T06:05:09Z

I'm starting to wonder if the use of the JDK PathMatcher is sufficient here. If we want to optimize the file system access and not traverse all files / subdirectories, we'll have to use Files.walkFileTree as it has built-in support for skipping subtrees. However, I'm not sure there is any good implementation of glob syntax that would skip subtree when known to not match.

kbuntrock · 2023-08-28T12:25:17Z

I'm starting to wonder if the use of the JDK PathMatcher is sufficient here. If we want to optimize the file system access and not traverse all files / subdirectories, we'll have to use Files.walkFileTree as it has built-in support for skipping subtrees. However, I'm not sure there is any good implementation of glob syntax that would skip subtree when known to not match.

Well, good news, we are already using Files#walkFileTree. One of the discussion is precisely how I managed to keep as well as possible the capacity to skip subtree scanning (it is a key point for the projects I want to use this extension on).

@ALL : Seems this PR draw attention to details about this functionality but I would suggest to not go too far with it.
I already agreed to change it from a bugfix to an evolution. Maybe the current state is good enough for now. And if good ideas emerge they can be implemented in another PR. I have the feeling that in term of functionality / performance / test and documentation, this PR brings already a lot to the table. And people will be easily capable after merging to change some details they would have seen differently.

AlexanderAshitkin · 2023-08-31T02:43:17Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

+     * Add a value from the excluded section list to the directories and/or the filenames ban list.
+     * @param excludedValue a value from the exclude list
+     */
+    private void addToExcludedSection(String excludedValue, boolean addProjectBaseDir) {


Why need the addProjectBaseDir parameter? Intuitively, the project dir should always be prepended. Could that lead to an exclude applied to an unexpected subtree? Like dir1/file.txt applied to <root>/dir2/dir1/file.txt?

it's because of the default exclude sections. Their values have already the project dir prepended. But I'm open to suggestions.

AlexanderAshitkin · 2023-08-31T02:46:55Z

src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java

+                (addProjectBaseDir ? baseDirectoryGlobPrefix : "")
+                +
+                // If the glob start with "/", we remove it since it's already added in the added basedir glob
+                (excludedValue.startsWith("/") ? excludedValue.substring(1) : excludedValue);


Stripping the leading / from user-supplied parameters might be confusing. a common convention is that / denotes an absolute path.

Got it. What about forbidding a leading "/" ? I don't see any use of an absolute glob exclusion.

AlexanderAshitkin · 2023-08-31T19:52:10Z

src/test/projects/include-exclude/pom.xml

+        <!-- Exclude directory in an extra included folder  -->
+        <maven.build.cache.exclude.subfolder>**/excluded_subfolder/**</maven.build.cache.exclude.subfolder>
+        <!-- Exclude by filepath + filename -->
+        <maven.build.cache.exclude.filepath>folder_outside_src/this_one_should_NOT_be_scanned.txt</maven.build.cache.exclude.filepath>


It will be good to cover ambiguous cases here:

just <maven.build.cache.exclude.subfolder>excluded_subfolder</maven.build.cache.exclude.subfolder> - previously, I expected to have the whole dir subtree excluded consistently with second_folder_outside_src. Now - it excludes only a file with this name.

just <maven.build.cache.exclude.subfolder>excluded_subfolder/**</maven.build.cache.exclude.subfolder> - it should exclude <dirRoot>/excluded_subfolder/**, but not <dirRoot>/subfolder/excluded_subfolder/**

<maven.build.cache.exclude.subfolder>/tmp/**</maven.build.cache.exclude.subfolder> - I believe it should exclude the absolute path, e.g. all tmp files

AlexanderAshitkin · 2023-08-31T20:08:10Z

src/test/projects/include-exclude/pom.xml

+        <maven.build.cache.input.extraFile>extraFile.txt</maven.build.cache.input.extraFile>
+        <maven.build.cache.input.extraFolder>second_folder_outside_src</maven.build.cache.input.extraFolder>
+        <!-- Exclude directory in an extra included folder  -->
+        <maven.build.cache.exclude.subfolder>**/excluded_subfolder/**</maven.build.cache.exclude.subfolder>


Please notice that the documentation describes the parameters as follows:

Paths to exclude from source code search. Relative paths calculated from the current project/module root

Please notice the documentation doesn't say this is a pattern, a glob, or an ant expression. We need to update the documentation to reflect the support for globs and how they are applied. The key issue is whether the base dir is prepended, with or without leading slashes. To make it deterministic, we might follow this approach:

If the pattern starts with /, it is considered an absolute path expression

If not, any path expression is prepended by the absolute project dir path

The content of the properties will be interpreted as a glob expression.

kbuntrock · 2023-09-28T12:15:07Z

About this PR, I'm still working on it. Even if I couldn't find the time the last past weeks.
I turn it in draft @maximilian-novikov-db et @AlexanderAshitkin, I think I'll be able to submit a more consensual version in the next days.

kbuntrock · 2023-10-03T06:43:54Z

Re-opening the PR for merge.

It's now a mix between the very first implementation and the one with globs, and it's combining better with the input section mechanism.

I feel much more at ease with this version. We can use only path declaration, only glob declaration, or combine both if we want the best trade-off between performances and functionalities.

Here a small list of improvements:

the glob interpretation can be configured to better handle any specific case (= less code to handle the specific cases ... :p )
easier to test the exclusion mechanism with a unit test (added one)
unit + IT test about unix/dos path style (and tested on Windows and Debian)
handling absolute path like "inputs" (even if I suppose it will only used to test a configuration)

Here is the documentation presenting it :

And the updated "project property" page.

Looking forward to your feedbacks. :)

Especially in the project property section, I'm undecided about the naming of the exclusion value : "maven.build.cache.exclude.value.my-exclusion" versus "maven.build.cache.exclude.my-exclusion". (currently the first option, simpler syntax / readability with the additional properties like "maven.build.cache.exclude.glob.my-exclusion")

AlexanderAshitkin

LGTM

kbuntrock changed the title ~~[MBUILDCACHE-64] Exclusion mechanism bugfix + improvement~~ [MBUILDCACHE-64] Exclusion mechanism bugfix Aug 2, 2023

olamy added the bug Something isn't working label Aug 3, 2023

kbuntrock added 3 commits August 3, 2023 22:48

[MBUILDCACHE-64] Fix exclusions not working properly (specifically wi…

556965c

…th folders)

[MBUILDCACHE-64] Added an integration test for the include/exclude fu…

95c483a

…nctionnality, from global and project configuration

[MBUILDCACHE-64] Add documentation about the include / exclude mechanism

70b1992

kbuntrock force-pushed the MBUILDCACHE-64-exclusions-improvement branch from 6984d4e to 70b1992 Compare August 3, 2023 20:49

olamy approved these changes Aug 4, 2023

View reviewed changes

olamy requested a review from gnodet August 4, 2023 01:48

gnodet requested changes Aug 4, 2023

View reviewed changes

kbuntrock added 2 commits August 7, 2023 18:56

[MBUILDCACHE-64] Exclude mecanism based on glob patterns

a0af3df

[MBUILDCACHE-64] Update IT test and documentation of the include/excl…

f1c3c6a

…ude functionality

kbuntrock commented Aug 7, 2023

View reviewed changes

kbuntrock requested review from gnodet and olamy August 7, 2023 17:08

AlexanderAshitkin reviewed Aug 21, 2023

View reviewed changes

[MBUILDCACHE-64] Code review comments

45f2e1a

AlexanderAshitkin reviewed Aug 31, 2023

View reviewed changes

AlexanderAshitkin approved these changes Sep 17, 2023

View reviewed changes

maximilian-novikov-db approved these changes Sep 28, 2023

View reviewed changes

kbuntrock marked this pull request as draft September 28, 2023 12:20

kbuntrock force-pushed the MBUILDCACHE-64-exclusions-improvement branch from 67faf5a to 0801c66 Compare October 2, 2023 21:20

kbuntrock marked this pull request as ready for review October 2, 2023 21:24

kbuntrock force-pushed the MBUILDCACHE-64-exclusions-improvement branch from 0801c66 to fb8e211 Compare October 2, 2023 21:28

[MBUILDCACHE-64] Rework path + pattern exclusion

eeaa49f

kbuntrock force-pushed the MBUILDCACHE-64-exclusions-improvement branch from fb8e211 to eeaa49f Compare October 2, 2023 21:31

kbuntrock requested review from AlexanderAshitkin and maximilian-novikov-db October 3, 2023 06:44

AlexanderAshitkin approved these changes Oct 4, 2023

View reviewed changes

maximilian-novikov-db approved these changes Oct 6, 2023

View reviewed changes

olamy approved these changes Oct 12, 2023

View reviewed changes

olamy merged commit 6eb05e6 into apache:master Oct 12, 2023
14 checks passed

[MBUILDCACHE-64] Exclusion mechanism bugfix #91

[MBUILDCACHE-64] Exclusion mechanism bugfix #91

Conversation

kbuntrock commented Aug 1, 2023 • edited

Description of this pull request

MR checklist

kbuntrock commented Aug 3, 2023

olamy commented Aug 4, 2023

gnodet left a comment

Choose a reason for hiding this comment

kbuntrock commented Aug 4, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbuntrock commented Aug 7, 2023

maximilian-novikov-db commented Aug 21, 2023

AlexanderAshitkin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexanderAshitkin Aug 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbuntrock Aug 23, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexanderAshitkin Aug 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbuntrock Sep 4, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbuntrock commented Aug 23, 2023

gnodet commented Aug 25, 2023

kbuntrock commented Aug 28, 2023

AlexanderAshitkin Aug 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexanderAshitkin Aug 31, 2023 • edited

Choose a reason for hiding this comment

AlexanderAshitkin Aug 31, 2023 • edited

Choose a reason for hiding this comment

kbuntrock commented Sep 28, 2023 • edited

kbuntrock commented Oct 3, 2023 • edited

AlexanderAshitkin left a comment

Choose a reason for hiding this comment

kbuntrock commented Aug 1, 2023 •

edited

AlexanderAshitkin Aug 31, 2023 •

edited

kbuntrock Aug 23, 2023 •

edited

AlexanderAshitkin Aug 31, 2023 •

edited

kbuntrock Sep 4, 2023 •

edited

AlexanderAshitkin Aug 31, 2023 •

edited

AlexanderAshitkin Aug 31, 2023 •

edited

AlexanderAshitkin Aug 31, 2023 •

edited

kbuntrock commented Sep 28, 2023 •

edited

kbuntrock commented Oct 3, 2023 •

edited