Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

checkout overwrites tags with lightweight tags #882

Open
Adam- opened this issue Aug 7, 2022 · 6 comments
Open

checkout overwrites tags with lightweight tags #882

Adam- opened this issue Aug 7, 2022 · 6 comments

Comments

@Adam-
Copy link

Adam- commented Aug 7, 2022

When running checkout with fetch-depth: 0 on a tag ref, checkout clobbers the fetched tag by replacing it with its own lightweight tag.

The cause of this is testRef is comparing a tag-object with a commit-object. git rev-parse on a non-lightweight tag ref returns the tag object and not the commit object:

  // refs/tags/
  else if (upperRef.startsWith('REFS/TAGS/')) {
    const tagName = ref.substring('refs/tags/'.length)
    return (
      (await git.tagExists(tagName)) && commit === (await git.revParse(ref))
    )
  }

After this test fails, checkout then performs a 2nd fetch of the commit which results in a lightweight tag being created, overwriting the original tag.

This breaks git describe and other things.

Build log:

$ git cat-file -t 64c28bc66b5420d8bfb158088b830fc6cdb02815
tag
$ git cat-file -p 64c28bc66b5420d8bfb158088b830fc6cdb02815
object 4583104281ae74cb64a04aa28615c062752f1d64
type commit
tag R2
tagger James Carroll <eloquism@gmail.com> 1659879755 +0100

Test
$ git cat-file -t 4583104281ae74cb64a04aa28615c062752f1d64
commit
$ git cat-file -p 4583104281ae74cb64a04aa28615c062752f1d64
tree 0f020f42f0099856c28e138234f6cb22130bde36
parent 483c83b4723c31e32a6722985f16a840742c99b3
author James Carroll <eloquism@gmail.com> 1659879639 +0100
committer James Carroll <eloquism@gmail.com> 1659879727 +0100

Test fetch without force
 /usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
...
   * [new branch]      main       -> origin/main
   * [new tag]         R1         -> R1
   * [new tag]         R2         -> R2
  /usr/bin/git tag --list R2
  R2
  /usr/bin/git rev-parse refs/tags/R2
  64c28bc66b5420d8bfb158088b830fc6cdb02815
  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules origin +4583104281ae74cb64a04aa28615c062752f1d64:refs/tags/R2
  From https://github.com/MrCarroll/runelite-snap
   t [tag update]      4583104281ae74cb64a04aa28615c062752f1d64 -> R2
@ihostage
Copy link

ihostage commented Aug 30, 2022

It's really very unexpected behaviour 🤔
Moreover, this issue initiates other problem with using some tools that depends on type of tags (JGitVer for example).

And for cutting a release I must use workaround and re-fetch all tags manually 😞

- name: Checkout
  uses: actions/checkout@v3
  with:
    # we don't know what commit the last tag was
    fetch-depth: 0
- run: git fetch --tags --force origin # WA: https://github.com/actions/checkout/issues/882

@Simonl9l
Copy link

Simonl9l commented Nov 5, 2022

not sure if anyone is monitoring this...any and all help appreciated!

I added the work around but if I do a repo.Tags.CountI() I still get zero!

the is what I see in the log:

Run git fetch --tags --force origin
  git fetch --tags --force origin
  shell: /usr/bin/bash -e {0}

I note when the workflow is retrieve the repo it runs:

 /usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*

It seem I'm running checkout@2 as that is what Nuke.build is currently using.

ijjk added a commit to vercel/next.js that referenced this issue Dec 2, 2022
Follow-up to #32337 this removes
the un-necessary step where we fetch all of the tags which requires
pulling a lot of un-necessary git history inflating cache size and
publish times.

The only reason these tags were needing to be fetched is due to an issue
in how the `actions/checkout` step works
(actions/checkout#882).

This reduces the publish times by at least 4 minutes by removing the
tags fetching step
https://github.com/vercel/next.js/actions/runs/3598569786/jobs/6061449995#step:16:14

As a further optimization this adds concurrency to the `npm publish`
calls themselves to hopefully reduce time spent there as well.
ijjk added a commit to vercel/next.js that referenced this issue Dec 2, 2022
Follow-up to #32337 this removes
the un-necessary step where we fetch all of the tags which requires
pulling a lot of un-necessary git history inflating cache size and
publish times.

The only reason these tags were needing to be fetched is due to an issue
in how the `actions/checkout` step works
(actions/checkout#882).

This reduces the publish times by at least 4 minutes by removing the
tags fetching step
https://github.com/vercel/next.js/actions/runs/3598569786/jobs/6061449995#step:16:14

As a further optimization this adds concurrency to the `npm publish`
calls themselves to hopefully reduce time spent there as well.
ijjk added a commit to vercel/next.js that referenced this issue Dec 2, 2022
Follow-up to #32337 this removes
the un-necessary step where we fetch all of the tags which requires
pulling a lot of un-necessary git history inflating cache size and
publish times.

The only reason these tags were needing to be fetched is due to an issue
in how the `actions/checkout` step works
(actions/checkout#882).

This reduces the publish times by at least 4 minutes by removing the
tags fetching step
https://github.com/vercel/next.js/actions/runs/3598569786/jobs/6061449995#step:16:14

As a further optimization this adds concurrency to the `npm publish`
calls themselves to hopefully reduce time spent there as well.
sethmlarson added a commit to sethmlarson/secure-python-package-template that referenced this issue Dec 11, 2022
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this issue Dec 30, 2022
In commit b7fa3a5 of PR git-lfs#5236 we
replaced the deprecated actions/setup-ruby@v1 GitHub Action in our
CI and release workflows with the current ruby/setup-ruby@v1 Action.

We now update our other Actions to their respective latest versions
as well:

  actions/checkout:           v1 -> v3
  actions/download-artifact:  v1 -> v3
  actions/setup-go:           v2 -> v3
  actions/upload-artifact:    v1 -> v3
  docker/setup-qemu-action:   v1 -> v2

As of v2 of the actions/download-artifact Action, downloaded assets are
not placed in a new subdirectory created with the name from the step's
"name" argument, but in the current working directory instead.  We want
to retain the previous behaviour so we add a "path" argument with the
same name as each of the macos-assets and windows-assets download steps.

By default, the actions/checkout Action (as of v2) performs a Git fetch
with a --depth=1 option, so a shallow clone is made.  As a result, when
our Makefile calls "git describe HEAD" to set its VERSION variable, no
tags are available and Git responds with an error message.

Many of our workflow jobs succeed despite logging that error, including
the build-docker and build-docker-cross jobs in both our CI and Release
workflows.  (The Docker builds create upload artifacts with the correct
filenames despite the lack of any tags because they rely on the Git LFS
version strings in our debian/changelog file and in our binary; the
rpm/build_rpms.bsh script builds a binary just to run "git-lfs version"
and determine the version string from its output.)

However, our workflow jobs which run the "make release" command fail
outright in the absence of any Git tags, as they search for build
artifacts using filenames constructed with the empty VERSION variable,
such as "git-lfs-windows-amd64-.zip".  When no files are found, the
tar command fails, halting the job.  This affects both the build-default
job in our CI workflow (for Linux and macOS), and all of build-main,
build-macos, and build-windows jobs in our Release workflow.

To resolve this in the case of a PR or other push to a branch, we set
a fetch-depth value of 0 for our actions/checkout@v3 steps, which
downloads the full Git history and all tags.  This is somewhat more
expensive than a shallow clone, but our project's history is not
excessively large.

Due to the GitHub Actions bug documented in actions/checkout#882,
though, this resolution is insufficient in the case of a push to a
tag.  At present, the actions/checkout@v3 incorrectly determines the
SHA of an annotated tag to be the SHA of its associated commit, and
then proceeds as if the tag had been updated on the server since the
Action was started, and so rewrites the tag locally to refer to the
commit SHA.  This has the effect of making the local tag into a
lightweight tag, which "git describe" then ignores (since we don't
pass the --tags option to it).

As a temporary fix for this problem, we add a step after the
actions/checkout@v3 step which updates the local tag again to match
the remote one.  We only run this step when the pushed reference
was a tag, because on a branch push it would fail as Git would refuse
to update the currently checked-out branch.  In our Release workflow,
since it only runs on pushes to tags, we can run this step
unconditionally.  (We could also continue to use the default fetch-depth
of 1 for the actions/checkout@v3 step, since we always subsequently
fetch the relevant tag, but to be consistent and to avoid future
issues once actions/checkout#882 is fixed upstream, we do not do so.)
@chrisd8088
Copy link

We've run into this as well when upgrading our workflows for the Git LFS project. Our proposed workaround at present is to run the following:

    - uses: actions/checkout@v3
      with:
        fetch-depth: 0
    - run: git fetch origin "+${GITHUB_REF}:${GITHUB_REF}"
      if: ${{ github.ref_type == 'tag' }}
      shell: bash

That seems to be enough to reverse the tag update made by mistake which overwrites the annotated tag and turns it into a lightweight one.

chrisd8088 added a commit to chrisd8088/git-lfs that referenced this issue Jan 4, 2023
In commit b7fa3a5 of PR git-lfs#5236 we
replaced the deprecated actions/setup-ruby@v1 GitHub Action in our
CI and release workflows with the current ruby/setup-ruby@v1 Action.

We now update our other Actions to their respective latest versions
as well:

  actions/checkout:           v1 -> v3
  actions/download-artifact:  v1 -> v3
  actions/setup-go:           v2 -> v3
  actions/upload-artifact:    v1 -> v3
  docker/setup-qemu-action:   v1 -> v2

As of v2 of the actions/download-artifact Action, downloaded assets are
not placed in a new subdirectory created with the name from the step's
"name" argument, but in the current working directory instead.  We want
to retain the previous behaviour so we add a "path" argument with the
same name as each of the macos-assets and windows-assets download steps.

By default, the actions/checkout Action (as of v2) performs a Git fetch
with a --depth=1 option, so a shallow clone is made.  As a result, when
our Makefile calls "git describe HEAD" to set its VERSION variable, no
tags are available and Git responds with an error message.

Many of our workflow jobs succeed despite logging that error, including
the build-docker and build-docker-cross jobs in both our CI and Release
workflows.  (The Docker builds create upload artifacts with the correct
filenames despite the lack of any tags because they rely on the Git LFS
version strings in our debian/changelog file and in our binary; the
rpm/build_rpms.bsh script builds a binary just to run "git-lfs version"
and determine the version string from its output.)

However, our workflow jobs which run the "make release" command fail
outright in the absence of any Git tags, as they search for build
artifacts using filenames constructed with the empty VERSION variable,
such as "git-lfs-windows-amd64-.zip".  When no files are found, the
tar command fails, halting the job.  This affects both the build-default
job in our CI workflow (for Linux and macOS), and all of build-main,
build-macos, and build-windows jobs in our Release workflow.

To resolve this in the case of a PR or other push to a branch, we set
a fetch-depth value of 0 for our actions/checkout@v3 steps, which
downloads the full Git history and all tags.  This is somewhat more
expensive than a shallow clone, but our project's history is not
excessively large.

Due to the GitHub Actions bug documented in actions/checkout#882,
though, this resolution is insufficient in the case of a push to a
tag.  At present, the actions/checkout@v3 incorrectly determines the
SHA of an annotated tag to be the SHA of its associated commit, and
then proceeds as if the tag had been updated on the server since the
Action was started, and so rewrites the tag locally to refer to the
commit SHA.  This has the effect of making the local tag into a
lightweight tag, which "git describe" then ignores (since we don't
pass the --tags option to it).

As a temporary fix for this problem, we add a step after the
actions/checkout@v3 step which updates the local tag again to match
the remote one.  We only run this step when the pushed reference
was a tag, because on a branch push it would fail as Git would refuse
to update the currently checked-out branch.  In our Release workflow,
since it only runs on pushes to tags, we can run this step
unconditionally.  (We could also continue to use the default fetch-depth
of 1 for the actions/checkout@v3 step, since we always subsequently
fetch the relevant tag, but to be consistent and to avoid future
issues once actions/checkout#882 is fixed upstream, we do not do so.)
Pesa added a commit to named-data/actions that referenced this issue Feb 7, 2023
rawlins added a commit to rawlins/crawl that referenced this issue Jun 1, 2023
a9a6e9e and ea43df2 removed a step that was probably intended to
work around actions/checkout#290 and
actions/checkout#882, but broke the CI build
if anyone used a lightweight tag (because these fetch lines actually
shallow the tag depth to 1). This commit attempts to thread the needle
by making this workaround conditional only to releases, where (if the
person doing the release has followed the steps correctly), there is
guaranteed to be an annotated tag at depth 1 in the tag history. (If I'm
parsing the checkout bugs correctly, they only trigger on a tag action
anyways.)
rawlins added a commit to rawlins/crawl that referenced this issue Jun 1, 2023
a9a6e9e and ea43df2 removed a step that was probably intended to
work around actions/checkout#290 and
actions/checkout#882, but broke the CI build
if anyone used a lightweight tag (because these fetch lines actually
shallow the tag depth to 1). This commit attempts to thread the needle
by making this workaround conditional only to releases, where (if the
person doing the release has followed the steps correctly), there is
guaranteed to be an annotated tag at depth 1 in the tag history. (If I'm
parsing the checkout bugs correctly, they only trigger on a tag action
anyways.)
rawlins added a commit to rawlins/crawl that referenced this issue Jun 1, 2023
a9a6e9e and ea43df2 removed a step that was probably intended to
work around actions/checkout#290 and
actions/checkout#882, but broke the CI build
if anyone used a lightweight tag (because these fetch lines actually
shallow the tag depth to 1). This commit attempts to thread the needle
by making this workaround conditional only to releases, where (if the
person doing the release has followed the steps correctly), there is
guaranteed to be an annotated tag at depth 1 in the tag history. (If I'm
parsing the checkout bugs correctly, they only trigger on a tag action
anyways.)
rawlins added a commit to crawl/crawl that referenced this issue Jun 2, 2023
a9a6e9e and ea43df2 removed a step that was probably intended to
work around actions/checkout#290 and
actions/checkout#882, but broke the CI build
if anyone used a lightweight tag (because these fetch lines actually
shallow the tag depth to 1). This commit attempts to thread the needle
by making this workaround conditional only to releases, where (if the
person doing the release has followed the steps correctly), there is
guaranteed to be an annotated tag at depth 1 in the tag history. (If I'm
parsing the checkout bugs correctly, they only trigger on a tag action
anyways.)
rawlins added a commit to crawl/crawl that referenced this issue Jun 2, 2023
a9a6e9e and ea43df2 removed a step that was probably intended to
work around actions/checkout#290 and
actions/checkout#882, but broke the CI build
if anyone used a lightweight tag (because these fetch lines actually
shallow the tag depth to 1). This commit attempts to thread the needle
by making this workaround conditional only to releases, where (if the
person doing the release has followed the steps correctly), there is
guaranteed to be an annotated tag at depth 1 in the tag history. (If I'm
parsing the checkout bugs correctly, they only trigger on a tag action
anyways.)

(cherry picked from commit 8713efd)
hdonnay added a commit to hdonnay/clair that referenced this issue Jul 27, 2023
This also adds a workaround for actions/checkout#882 where the new tag
get blasted away with a new, lightweight tag for the duration of the
run.

Signed-off-by: Hank Donnay <hdonnay@redhat.com>
rebornplusplus added a commit to rebornplusplus/chisel that referenced this issue Jul 31, 2023
Create binaries on push, tag creation and when a release
has been published. Create an archive with the binary, and
attach it to the release if a release is published. In all
cases, attach the archive in the workflow run as an artifact.

The actions/checkout@v3 action overwrites annonated tag with
lightweight tags breaking ``git describe``. See [1], [2].
This commit adds a workaround to restore the latest tag to
it's original state.

References:
- [1] actions/checkout#882
- [2] actions/checkout#290
@rinnnn
Copy link

rinnnn commented Aug 7, 2023

For me the annotated tag appears as lightweight even when I checkout without the fetch-depth: 0 restriction. Workarounds suggested above work fine.

rebornplusplus added a commit to rebornplusplus/chisel that referenced this issue Aug 10, 2023
Create binaries on push, tag creation and when a release
has been published. Create an archive with the binary, and
attach it to the release if a release is published. In all
cases, attach the archive in the workflow run as an artifact.

The actions/checkout@v3 action overwrites annonated tag with
lightweight tags breaking ``git describe``. See [1], [2].
This commit adds a workaround to restore the latest tag to
it's original state.

References:
- [1] actions/checkout#882
- [2] actions/checkout#290
jnsgruk pushed a commit to canonical/chisel that referenced this issue Aug 10, 2023
Create binaries on push, tag creation and when a release
has been published. Create an archive with the binary, and
attach it to the release if a release is published. In all
cases, attach the archive in the workflow run as an artifact.

The actions/checkout@v3 action overwrites annonated tag with
lightweight tags breaking ``git describe``. See [1], [2].
This commit adds a workaround to restore the latest tag to
it's original state.

References:
- [1] actions/checkout#882
- [2] actions/checkout#290
@DJViking
Copy link

DJViking commented Nov 27, 2023

A full workday wasted to investigate why Jgitver did not work in GitHub Actions.
Then I found this issue. The problem was filed here over a year ago. Why has it not yet been fixed?

Thankfully a workaround exist.
I am currently using this in my workflow

    run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*

But this is a hack, and it clutters the workflow.

kpushkaryov added a commit to plesk/ubuntu18to20 that referenced this issue Dec 20, 2023
walles added a commit to walles/px that referenced this issue Feb 29, 2024
@walles
Copy link

walles commented Feb 29, 2024

Related to #290, possibly a duplicate?

juanep97 added a commit to juanep97/iop4 that referenced this issue Apr 15, 2024
Add workaround for #95, caused by a bug in the checkout action (github.com/actions/checkout/issues/882).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants