go.mod replace directives are breaking out of tree builds

[kris-watts-gravwell]

What are the steps to reproduce this issue?

1. Create a super simple ingester outside the main tree (example attached)

2. build it

What happens?

it can't build

What were you expecting to happen?

that it would build

Any logs, error output, etc?

/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1264:34: undefined: jsonparser.FuzzDelete
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1265:35: undefined: jsonparser.FuzzEachKey
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1266:38: undefined: jsonparser.FuzzGetBoolean
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1267:36: undefined: jsonparser.FuzzGetFloat
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1268:34: undefined: jsonparser.FuzzGetInt
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1269:37: undefined: jsonparser.FuzzGetString
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1270:43: undefined: jsonparser.FuzzGetUnsafeString
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1271:38: undefined: jsonparser.FuzzObjectEach
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1272:37: undefined: jsonparser.FuzzParseBool
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1273:38: undefined: jsonparser.FuzzParseFloat
/home/kris/mygo/pkg/mod/github.com/gravwell/gravwell/v3@v3.8.15/ingest/processors/plugin/packages.go:1273:38: too many errors

Any other comments?

I think THIS is the problem:

replace github.com/buger/jsonparser => github.com/floren/jsonparser v0.0.0-20210727191945-e5063027fceb

What versions of software are you using?

5.1.3

Replace directives we have:

• buger/jsonparser: PR is in, Array strings buger/jsonparser#180
• open-networks/go-msgraph: my PR has been merged, so we should be able to get rid of it
• fsnotify/fsnotify: @kris-watts-gravwell needs to do a PR, we'll see if we can get it upstreamed.

[john-floren-gravwell]

I think we can get rid of the msgraph replace, because they merged my PR: open-networks/go-msgraph#3

I will test this.

Note that go-msgraph is now deprecated because Microsoft added an official Go library... unfortunately that means I'll probably have to rewrite stuff to make it work: https://github.com/microsoftgraph/msgraph-sdk-go

[john-floren-gravwell]

I updated my PR for the jsonparser library, which is very minimal.

[john-floren-gravwell]

@kris-watts-gravwell it looks like your fsnotify fork is very close to the upstream; have you considered doing a PR? Or have they indicated they're not interested?

[john-floren-gravwell]

buger/jsonparser#180 I feel like this is a pretty straightforward fix but it's been pocketed for years now.

[kris-watts-gravwell]

i will push a PR up to fsnotify, some of the others will likely need full up forks on some of these.

Lets just get this moving and punt as needed.

[kris-watts-gravwell]

i suggest that this be a longer term project, lets keep walking this issue forward so we don't lose it, but we can just track which replace directives have upstream PRs, which are accepted, etc...

[john-floren-gravwell]

buger/jsonparser#180 I feel like this is a pretty straightforward fix but it's been pocketed for years now.

Executive decision: if we don't manage to get it merged by the end of January, we'll just give up and switch to my fork.

[kris-watts-gravwell]

PR for fsnotify is in: fsnotify/fsnotify#550

It is going into main, so even if its accepted we won't be able to use it until a new fsnotify release is pushed out.

[kris-watts-gravwell]

I am making the call, we fork jsonparser and use our fork. Check the other PRs that are also idling, there might be some more goodness/stability fixes in them too.

I am working with a dev on fsnotify to get our shit fixed there, then we can update the fsnotify lib