[builder] Support for --skip-new-go-module

[kristinapathak]

A continuation of #9253 and #9631

Description: Adds a --skip-new-go-module flag to the OTC builder. This enables users working in an existing go module environment (say, a "monorepo") to update the module they have, vs forcing the use of a new module.

With the new support inside an existing Go module, a collector main package can be generated using a go:generate directive. For example, in the directory where I want my collector built, the file generate.go has this line:

//go:generate builder --skip-new-go-module --skip-compilation --strict-versioning --config=./build-config.yaml
In the same directory, the build-config.yaml describes the collector to build. The builder generates the other files in the same directory. At this point, normal Go workflows can be used to update indirect dependencies.

Link to tracking Issue: #9252

Testing: Will add unit tests in the next few days.

Documentation: Yes.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: kristinapathak / name: Kristina Pathak (1b5f8ae, b901f03, 6fa9678, d26542f, a9558ea)

[codecov]

Codecov Report

Attention: Patch coverage is 40.00000% with 33 lines in your changes are missing coverage. Please review.

Project coverage is 91.49%. Comparing base (d294537) to head (3b91ace).
Report is 12 commits behind head on main.

❗ Current head 3b91ace differs from pull request most recent head b901f03

Please upload reports for the commit b901f03 to get more accurate results.

Files	Patch %	Lines
cmd/builder/internal/builder/main.go	18.42%	30 Missing and 1 partial ⚠️
cmd/builder/internal/command.go	33.33%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10098      +/-   ##
==========================================
- Coverage   91.90%   91.49%   -0.41%     
==========================================
  Files         361      362       +1     
  Lines       16970    16799     -171     
==========================================
- Hits        15596    15371     -225     
- Misses       1032     1087      +55     
+ Partials      342      341       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mx-psi]

I am not sure which one would be better, but we also have the option of doing go mod edit -require=example.org@v1.2.0. Why is go get preferred here?

[kristinapathak]

From what I can tell, the difference is that go get installs and builds the packages needed whereas go mod edit doesn't. Given that we run go mod tidy after this in GetModules(), I'll change this to use go mod edit.

[kristinapathak]

One thing to note about using go mod edit -require:

These flags are mainly for tools that understand the module graph. Users should prefer go get path@version or go get path@none, which make other go.mod adjustments as needed to satisfy constraints imposed by other modules.

I'm not sure what these other adjustments are or if we need them here? I would expect if we change the go.mod into some incompatible graph that go mod tidy will fail after this.

[mx-psi]

Thinking more about this, I think I would prefer using go mod edit -require at first. The section you quoted sounds to me as "go get also does go mod tidy, which you probably want", and since, as you point out, we already go mod tidy after this, I would prefer to keep things simple

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[mx-psi]

Is there any chance to increase testing coverage here? Some error cases would be harder to reach, but there are other parts that should be doable

[kristinapathak]

@mx-psi, sorry for the delay. I hit this issue when adding back tests and then got pulled into other things:

failed to update go.mod: go subcommand failed with args '[mod tidy -compat=1.21]': exit status 1, error message: go: finding module for package go.opentelemetry.io/collector/internal/testdata
        	            	go: finding module for package go.opentelemetry.io/collector/confmap/converter/expandconverter
        	            	go: found go.opentelemetry.io/collector/confmap/converter/expandconverter in go.opentelemetry.io/collector/confmap/converter/expandconverter v0.101.0
        	            	go: finding module for package go.opentelemetry.io/collector/internal/testdata
        	            	go: go.opentelemetry.io/collector/cmd/builder/internal/tester/output imports
        	            		go.opentelemetry.io/collector/connector tested by
        	            		go.opentelemetry.io/collector/connector.test imports
        	            		go.opentelemetry.io/collector/internal/testdata: module go.opentelemetry.io/collector@latest found (v0.101.0), but does not contain package go.opentelemetry.io/collector/internal/testdata

I'll try to fix this and increase coverage more this week.