Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

setup-go action should allow for existing cache files #403

Open
2 of 5 tasks
shaunco opened this issue Jul 27, 2023 · 10 comments · May be fixed by #412
Open
2 of 5 tasks

setup-go action should allow for existing cache files #403

shaunco opened this issue Jul 27, 2023 · 10 comments · May be fixed by #412
Assignees
@dsame
Labels
feature request New feature or request to improve the current logic

Comments

@shaunco
Copy link

shaunco commented Jul 27, 2023

Description:
On a self-hosted runner that is already caching tools, the v4 setup-go action attempts to call tar to extract its mod cache and ends up with a bunch of "Cannot open: File exists" errors, and eventually

Action version:
v4

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:
^1.20

Repro steps:

Run actions/setup-go@v4
  with:
    go-version: ^1.[2](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:2)0
    check-latest: false
    token: ***
    cache: true
  env:
    GO111MODULE: on
    GOPRIVATE: github.com/xyz/*
    SSH_AUTH_SOCK: /tmp/ssh-XXXXXX/agent.1965
    SSH_AGENT_PID: 1966
Setup go version spec ^1.20
Found in cache @ /opt/hostedtoolcache/go/1.20.6/x64
Added go to the path
Successfully set up Go version ^1.20
/opt/hostedtoolcache/go/1.20.6/x64/bin/go env GOMODCACHE
/opt/hostedtoolcache/go/1.20.6/x64/bin/go env GOCACHE
/home/runner/go/pkg/mod
/home/runner/.cache/go-build
Received 4194[3](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:3)0[4](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:4) of 9246941[5](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:5)4 (0.5%), 4.0 MBs/sec
Received 11324[6](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:6)208 of 924694154 (12.2%), 54.0 MBs/sec
Received 205520896 of 924694154 (22.2%), 65.3 MBs/sec
Received 3103[7](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:7)[8](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:8)4[9](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:9)6 of 924694154 (33.6%), 74.0 MBs/sec
Received 427819008 of 924694154 (46.3%), 81.6 MBs/sec
Received 536870912 of 924694154 (58.1%), 85.3 MBs/sec
Received 666894336 of 924694154 (72.1%), 90.8 MBs/sec
Received 771751936 of 924694154 (83.5%), 91.9 MBs/sec
Received 872415232 of 924694154 (94.3%), 92.3 MBs/sec
Received 924694154 of 924694154 ([10](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:10)0.0%), 93.3 MBs/sec
Cache Size: ~882 MB (924694[15](https://github.com/xyz/abc/actions/runs/123/job/123#step:5:16)4 B)
/usr/bin/tar -xf /home/runner/work/_temp/c44c4bfb-8a98-491b-b4a7-2f38e4fd677c/cache.tzst -P -C /home/runner/work/abc/abc--use-compress-program unzstd
/usr/bin/tar: ../../../go/pkg/mod/go.temporal.io/sdk@v1.15.0/interceptor/tracing_interceptor.go: Cannot open: File exists
/usr/bin/tar: ../../../go/pkg/mod/go.temporal.io/sdk@v1.15.0/interceptor/tracing_interceptor_test.go: Cannot open: File exists
.
.
.
/usr/bin/tar: Exiting with failure status due to previous errors
Warning: Failed to restore: "/usr/bin/tar" failed with error: The process '/usr/bin/tar' failed with exit code 2
Cache is not found

Expected behavior:
Existing files should be ignored via the -k or --skip-old-files option on tar

Actual behavior:
Existing files are treated as errors.
@shaunco shaunco added bug Something isn't working needs triage labels Jul 27, 2023
@dusan-trickovic
Copy link
Contributor

Hello @shaunco ! Thank you for reporting the issue, we will investigate it and get back to you as soon as we have more info :)

@shaunco
Copy link
Author

shaunco commented Jul 28, 2023

Thanks @dusan-trickovic ! Not sure if it is related, but we also seem to end up with quite a few duplicate setup-go action caches that all have the same sha256 and need to be cleared out every few days:
image

@dsame dsame self-assigned this Jul 31, 2023
@dsame
Copy link
Contributor

dsame commented Jul 31, 2023 •
edited

Hello @shaunco,
can you please try to disable caching for the action:

  - uses: actions/setup-go@v4
    with:
      go-version: '...'
      cache: false

this should solve the problem.

@bc-lee
Copy link

bc-lee commented Aug 1, 2023

Based on my understanding, go get downloads source codes with write permissions removed. As a result, extracting (tar -xf) cache files to GOPATH may cause errors. It would be helpful for setup-go to add write permissions to the cache directory before extracting and remove them after the process is complete.

@dsame
Copy link
Contributor

dsame commented Aug 1, 2023 •
edited

@bc-lee Self-hosted runner preserves the cache and builds results so caching does not have sense for them, we should disable caching for them by default. Currently it can be turned off manually

@bc-lee
Copy link

bc-lee commented Aug 1, 2023

@dsame Aha, that makes sense. I'm not sure if changing the default based on where the runner is running (i.e., provided by GitHub or self-hosted) is a good idea. I think it would be much better for the user to explicitly specify that option.

@dsame dsame linked a pull request Aug 1, 2023 that will close this issue
Add JSON output of go env and some env as strings #334
Open
2 tasks
@dsame dsame removed a link to a pull request Aug 2, 2023
Add JSON output of go env and some env as strings #334
Open
2 tasks
@dsame
Copy link
Contributor

dsame commented Aug 15, 2023

Hello @bc-lee my motivation to have the different value of cache is to have the same omitted cache input for both github-hosted and self-hosted builds in the other case users have to have to diffrenet workflows

@dsame dsame added feature request New feature or request to improve the current logic and removed bug Something isn't working labels Aug 15, 2023
@dsame
Copy link
Contributor

dsame commented Aug 15, 2023

I changed this bug to feature request basing on

@dsame dsame linked a pull request Aug 17, 2023 that will close this issue
Open
2 tasks
@dsame dsame linked a pull request Aug 18, 2023 that will close this issue
Set default cache input to false for self-hosted runners #412
Open
2 tasks
@matthewhughes-uw matthewhughes-uw mentioned this issue Sep 15, 2023
Open
5 tasks
@athoscouto athoscouto mentioned this issue Oct 10, 2023
Merged
@sebrandon1 sebrandon1 mentioned this issue Nov 15, 2023
Merged
@jizi
Copy link

jizi commented Dec 1, 2023

Hi @dsame

Self-hosted runner preserves the cache and builds results so caching does not have sense for them

This is not always true. We use the ephemeral self-hosted runners so it would be great to have the cache available.
With cache: false I can see that the dependencies are downloaded always.

@HaraldNordgren
Copy link

HaraldNordgren commented Jan 17, 2024 •
edited

@shaunco Any updates on this? After updating to Go version:

--- a/go.mod
+++ b/go.mod
@@ -1,6 +1,8 @@
 module github.com/company/repo

-go 1.20
+go 1.21.1
+
+toolchain go1.21.6

 require (

We start having these warnings in the build:

/usr/bin/tar: ../../../go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.6.linux-amd64/lib/time/mkzip.go: Cannot open: File exists
/usr/bin/tar: ../../../go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.6.linux-amd64/lib/time/zoneinfo.zip: Cannot open: File exists
/usr/bin/tar: ../../../go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.6.linux-amd64/lib/time/README: Cannot open: File exists
...

@jchappelow jchappelow mentioned this issue Jan 22, 2024
Merged
@deltics deltics mentioned this issue Mar 18, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dsame dsame

Labels
feature request New feature or request to improve the current logic
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Set default cache input to false for self-hosted runners akv-platform/setup-go
7 participants
@dsame @shaunco @bc-lee @HaraldNordgren @jizi @dusan-trickovic and others