CodeQL build times increased to over an hour

[codeboten]

The CodeQL builds jumped from under 15 minutes to over an hour somewhere between https://github.com/open-telemetry/opentelemetry-collector/actions/runs/8255636805 and https://github.com/open-telemetry/opentelemetry-collector/actions/runs/8255818248

The version of CodeQL increased from 2.16.3 to 2.16.4 between those versions.

[lavishpal]

@codeboten
Could you assign this issue to me and it will be helpful if you provide guidance on how to get started?

[codeboten]

@lavishpal happy to assign, unfortunately everything i know about this problem is in the description of the issue. For some reason the duration of the codeql has risen pretty severly when the tool's version was upgraded from 2.16.3 to 2.16.4. I have no idea if that's what caused the problem or not, maybe the place to start is to pin the version of the tool and see if that causes the duration to go back down?

[asreehari-splunk]

@lavishpal did not notice this was assigned to you. I've filed a question on the CodeQL project github/codeql#16448. Hopefully, that helps in narrowing this down

[asreehari-splunk]

@codeboten ... the CodeQL folks got back with the following

For mainly historical reasons, we run make if there is a Makefile in the repository before we begin extraction of the source code. Your Makefile in particular seems to build and test all of your code.

When we implemented the changes to the Go autobuilder in 2.16.4, we kept the part that invokes make before extraction to ensure that CodeQL would not suddenly break for repositories which relied on this behaviour. However, it seems that this now gets erroneously invoked for every go.mod file in your repository. I will look into getting this fixed ASAP.

In the meantime, to avoid this issue until it is fixed, you can either revert to 2.16.3 (but fewer Go sources files will get extracted) or switch to a custom build. The latter would involve replacing the autobuild step in your workflow with a step that invokes the right build commands for your repository (possibly just make

The entire thread has more info if you are interested github/codeql#16448

We could tie it to 2.16.3 until they get back with an update. let us know what you think.