Desync between vault/sdk@v0.7.0 and vault/vault@v1.12.3?

[kuanyong-wong-partior]

Describe the bug
I tried writing a unit test that creates an inmem vault. The unit test fails to run with the following errors:

$ go test ./...
# github.com/hashicorp/vault/vault
/home/user/go/pkg/mod/github.com/hashicorp/vault@v1.12.3/vault/logical_system.go:4311:9: not enough arguments in call to framework.NewOASDocument
        have ()
        want (string)
/home/user/go/pkg/mod/github.com/hashicorp/vault@v1.12.3/vault/mount_util.go:50:9: cannot use extendedSystemViewImpl{…} (value of type extendedSystemViewImpl) as type extendedSystemView in return statement:
        extendedSystemViewImpl does not implement extendedSystemView (missing VaultVersion method)
/home/user/go/pkg/mod/github.com/hashicorp/vault@v1.12.3/vault/policy_store.go:257:67: cannot use sysView (variable of type *dynamicSystemView) as type logical.SystemView in argument to NewPolicyStore:
        *dynamicSystemView does not implement logical.SystemView (missing VaultVersion method)
/home/user/go/pkg/mod/github.com/hashicorp/vault@v1.12.3/vault/testing.go:619:21: cannot use dsv (variable of type *dynamicSystemView) as type pluginutil.RunnerUtil in struct literal:
        *dynamicSystemView does not implement pluginutil.RunnerUtil (missing VaultVersion method)
FAIL    test/internal/api [build failed]
FAIL    test/internal/test [build failed]
FAIL

If I try to exclude sdk v0.7.0, it fails with the following:

$ go mod edit -exclude github.com/hashicorp/vault/sdk@v0.7.0
$ go mod tidy
go: dropping requirement on excluded version github.com/hashicorp/vault/sdk v0.7.0
go: finding module for package github.com/hashicorp/vault/sdk/helper/testhelpers/schema
test/internal/test tested by
        test/internal/test.test imports
        github.com/hashicorp/vault-plugin-secrets-kv tested by
        github.com/hashicorp/vault-plugin-secrets-kv.test imports
        github.com/hashicorp/vault/sdk/helper/testhelpers/schema: module github.com/hashicorp/vault/sdk@latest found (v0.6.2), but does not contain package github.com/hashicorp/vault/sdk/helper/testhelpers/schema

To Reproduce
Steps to reproduce the behavior:
checkout repo and run go unit test
https://github.com/wildsheepz/tryingoutvault.git

Expected behavior
No errors.

Environment:
Debian GNU/Linux 11

Additional context
Nil

[kuanyong-wong-partior]

I managed to get past the error by excluding vault-plugin-secrets-kv@v0.14.0 and vault/sdk@v0.7.0

go mod edit -exclude github.com/hashicorp/vault/sdk@v0.7.0
go mod edit -exclude github.com/hashicorp/vault-plugin-secrets-kv@v0.14.0

[hsimon-hashicorp]

Hello! Unfortunately we don't support importing Vault as a module, as that can often run into conflicts when importing both Vault, along with the api and sdk modules. Because of the large number of potential conflicts and variables, we simply can't make any guarantees that it will work the way you would like. I'm going to go ahead and close this issue now, but please feel free to open a new issue if you run into other bugs. Thanks!

[tstraley]

I'm facing similar import conflicts with these vault packages

go mod tidy -v
go: finding module for package github.com/hashicorp/vault/vault
go: finding module for package github.com/hashicorp/vault-plugin-secrets-kv
go: finding module for package github.com/hashicorp/vault/http
go: found github.com/hashicorp/vault-plugin-secrets-kv in github.com/hashicorp/vault-plugin-secrets-kv v0.14.2
go: found github.com/hashicorp/vault/http in github.com/hashicorp/vault v1.12.3
go: found github.com/hashicorp/vault/vault in github.com/hashicorp/vault v1.12.3
go: finding module for package github.com/hashicorp/vault/sdk/version
<my module>/vault tested by
	<my module>/vault.test imports
	github.com/hashicorp/vault/http imports
	github.com/hashicorp/vault/sdk/version: module github.com/hashicorp/vault/sdk@latest found (v0.8.1), but does not contain package github.com/hashicorp/vault/sdk/version

@hsimon-hashicorp do you have any alternative suggestions / recommendations for how consumers of the vault kvv2 api can / should write tests against that code? This solution of using the vault.NewTestCluster used to work until this weeks version changes, and is really the only recommendation I've been able to see documented anywhere.

Maybe this issue needs to actually get addressed now #8440
Or maybe some other test server exists and is exported for use (see un-exported option https://github.com/hashicorp/vault/blob/0.11.5/command/command_test.go#L96)

[marhar]

I'm also dealing with this issue, as I would like to unit test my vault code as per the accepted answer on

https://stackoverflow.com/questions/57771228/mocking-hashicorp-vault-in-go

Any guidance appreciated!

[hsimon-hashicorp]

https://stackoverflow.com/questions/57771228/mocking-hashicorp-vault-in-go

That answer is several years old, unfortunately. Previously it was "best effort" support, but the team made the decision last year to no longer support importing Vault, the API, or the SDK as modules in Go. I'm sorry for the inconvenience.

[maxb]

but the team made the decision last year to no longer support importing Vault, the API, or the SDK as modules in Go. I'm sorry for the inconvenience.

Um, don't you mean "the Vault root module"? Considering the entire purpose of the API and SDK being separate is so they can be.

[tstraley]

@hsimon-hashicorp - understood on the supportability aspect of the various vault modules. But given this information, how do you recommend consumers of the API module unit test / int test things in Go? What is the use-case for TestCoreUnsealed and related functions if they cannot be used with the Go API client?

How do you test the API module itself?

FYI to others, I was able to get this working again with the current rc of the vault root module (go get github.com/hashicorp/vault@v1.13.0-rc1). My import issues were tied to the following changes:

• Remove the last vestiges of sdk/version. #19068
• Make API not depend on SDK #18962 - this is in the rc, but not in latest stable, and gets my code working again

[raskchanky]

@maxb Yes, you are correct. We don't support importing the Vault root module, but importing API and SDK are supported.