Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to fetch git blame info on specific files #155

Closed
mwarkentin opened this issue Jun 21, 2021 · 30 comments
Closed

Unable to fetch git blame info on specific files #155

mwarkentin opened this issue Jun 21, 2021 · 30 comments
Labels
wontfix This will not be worked on

Comments

@mwarkentin
Copy link

mwarkentin commented Jun 21, 2021

@nimrodkor Most of my git tagging issues have been resolved after #147, however it seems like one of the files in the sub-dir I tested this on was still unable to fetch git blame info:

2021/06/18 09:01:16 [WARNING] Unable to get git blame for file envs/cetus/receipts/receipts.tf: failed to get blame for latest commit of file envs/cetus/receipts/receipts.tf because of error contents and commits have different length
2021/06/18 09:01:29 [WARNING] Failed to tag receipts_bucket with git tags, err: failed to get blame for latest commit of file envs/cetus/receipts/receipts.tf because of error failed to get blame for latest commit of file envs/cetus/receipts/receipts.tf because of error contents and commits have different length

You can see that one of the resources was only tagged with yor_trace:

New Resources Traced (4):
+------------------------------------------+--------------------------------+----------------------+------------------------------------------+--------------------------------------+
|                   FILE                   |            RESOURCE            |       TAG KEY        |                TAG VALUE                 |                YOR ID                |
+------------------------------------------+--------------------------------+----------------------+------------------------------------------+--------------------------------------+
| envs/cetus/receipts/receipts.tf          | receipts_bucket                | yor_trace            | 0a4a88ef-f713-4f9f-8e50-effadb1422b4     | 0a4a88ef-f713-4f9f-8e50-effadb1422b4 |
+                                          +                                +----------------------+------------------------------------------+                                      +
|                                          |                                | yor_trace            | 0a4a88ef-f713-4f9f-8e50-effadb1422b4     |                                      |
+------------------------------------------+--------------------------------+----------------------+------------------------------------------+--------------------------------------+
| envs/cetus/receipts/secretsmgr.tf        | aws_iam_policy.secrets_manager | yor_trace            | 0b7c9745-0ded-438b-8b59-7876b1b4e712     | 0b7c9745-0ded-438b-8b59-7876b1b4e712 |
+                                          +                                +----------------------+------------------------------------------+                                      +
|                                          |                                | git_org              | waveaccounting                           |                                      |
+                                          +                                +----------------------+------------------------------------------+                                      +
|                                          |                                | git_repo             | keller                                   |                                      |
+                                          +                                +----------------------+------------------------------------------+                                      +

Not sure what error contents and commits have different length means. From slack:

It seems like we're not handling git correctly for merge commits

Originally posted by @mwarkentin in #147 (comment)

@mwarkentin mwarkentin changed the title @nimrodkor Not sure if this is related to your change or a different issue (happy to split off another issue if you'd like). It seems like one of the files in the sub-dir I tested this on was still unable to fetch git blame info: Unable to fetch git blame info on specific files Jun 21, 2021
@mwarkentin
Copy link
Author

mwarkentin commented Jun 21, 2021

Confirmed the last commit on that file was a merge commit:

commit 971fc554229e64317774cf8d4532d748b7d8460a
Merge: ff444b53e 824f5106b
Author: **** <****@users.noreply.github.com>
Date:   Fri Jun 4 13:36:12 2021 -0400

    Merge branch 'master' into JNO-2750-iam-authentication

@gustavoortega
Copy link

Hi, i have the same issue.
Our pipeline job fails with the same error. The job is executed after the MR was MERGED.
Any update on this? Thanks!

@justinas-b
Copy link

Seems to be also related/duplicated by below issues:

For me it also fails as described above in both cases: a) when running on merge request and b) when running after the merge request on master. I am using GitLab.

Below is job configuration I am using

yor:
  image:
    name: bridgecrew/yor:0.1.146
    entrypoint: [""]
  stage: test
  variables:
    AUTO_COMMITTER_EMAIL: "release@organization.com"
    AUTO_COMMITTER_NAME: "GitOps"
    GITLAB_USER_NAME: "oauth2"
    GIT_PUSH_TOKEN: $GITLAB_TOKEN
  before_script:
    - git remote set-url origin "https://oauth2:$GITLAB_TOKEN@$CI_SERVER_HOST/$CI_PROJECT_PATH.git"
    - git fetch
  script:
    - git checkout ${CI_COMMIT_REF_NAME}
    - git status
    - git pull
    - git blame main.tf
    - yor tag --directory . --parsers Terraform --output cli
    - *git-script
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      exists:
        - '**/*.tf'  

Here is my CI output:

[24]$ git remote set-url origin "https://oauth2:$GITLAB_TOKEN@$ci_server_host/$CI_PROJECT_PATH.git"
[25](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L25)$ git fetch
[26](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L26)From https://gitlab.com/<...>/terraform_module1
[27](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L27) * [new branch]      justinas.balinskas-master-patch-27581 -> origin/justinas.balinskas-master-patch-27581
[28](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L28) * [new branch]      master     -> origin/master
[29](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L29)$ git checkout ${CI_COMMIT_REF_NAME}
[30](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L30)branch 'justinas.balinskas-master-patch-27581' set up to track 'origin/justinas.balinskas-master-patch-27581'.
[31](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L31)Switched to a new branch 'justinas.balinskas-master-patch-27581'
[32](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L32)$ git status
[33](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L33)On branch justinas.balinskas-master-patch-27581
[34](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L34)Your branch is up to date with 'origin/justinas.balinskas-master-patch-27581'.
[35](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L35)nothing to commit, working tree clean
[36](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L36)$ git pull
[37](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L37)Already up to date.
[38](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L38)$ git blame main.tf
[39](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L39)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   1) provider "aws" {
[40](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L40)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   2)   region                      = "us-east-1" # <<<<< Try changing this to eu-west-1 to compare the costs
[41](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L41)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   3)   skip_credentials_validation = true
[42](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L42)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   4)   skip_requesting_account_id  = true
[43](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L43)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   5)   access_key                  = "mock_access_key"
[44](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L44)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   6)   secret_key                  = "mock_secret_key"
[45](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L45)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   7) }
[46](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L46)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   8) 
[47](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L47)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000   9) resource "aws_instance" "web_app" {
[48](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L48)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  10)   ami           = "ami-674cbc1e"
[49](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L49)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  11)   instance_type = "m5.4xlarge" # <<<<< Try changing this to m5.8xlarge to compare the costs
[50](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L50)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  12) 
[51](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L51)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  13)   root_block_device {
[52](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L52)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  14)     volume_size = 50
[53](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L53)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  15)   }
[54](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L54)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  16) 
[55](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L55)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  17)   ebs_block_device {
[56](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L56)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  18)     device_name = "my_data"
[57](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L57)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  19)     volume_type = "io1" # <<<<< Try changing this to gp2 to compare costs
[58](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L58)12226f2f (Justinas Balinskas 2022-06-22 05:26:59 +0000  20)     volume_size = 1015
[59](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L59)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  21)     iops        = 800
[60](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L60)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  22)   }
[61](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L61)e3077845 (GitOps             2022-06-21 21:19:21 +0000  23)   tags = {
[62](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L62)e3077845 (GitOps             2022-06-21 21:19:21 +0000  24)     yor_trace = "84af908a-cd39-4e99-8bd3-d1fddec2e4a0"
[63](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L63)e3077845 (GitOps             2022-06-21 21:19:21 +0000  25)   }
[64](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L64)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  26) }
[65](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L65)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  27) 
[66](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L66)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  28) 
[67](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L67)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  29) resource "aws_instance" "web_app2" {
[68](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L68)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  30)   ami           = "ami-674cbc1e"
[69](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L69)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  31)   instance_type = "m5.8xlarge" # <<<<< Try changing this to m5.8xlarge to compare the costs
[70](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L70)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  32) 
[71](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L71)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  33)   root_block_device {
[72](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L72)3b34cab3 (Justinas Balinskas 2022-06-21 22:07:35 +0000  34)     volume_size = 80
[73](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L73)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  35)   }
[74](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L74)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  36) 
[75](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L75)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  37)   ebs_block_device {
[76](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L76)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  38)     device_name = "my_data"
[77](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L77)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  39)     volume_type = "io1" # <<<<< Try changing this to gp2 to compare costs
[78](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L78)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  40)     volume_size = 1000
[79](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L79)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  41)     iops        = 800
[80](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L80)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  42)   }
[81](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L81)e3077845 (GitOps             2022-06-21 21:19:21 +0000  43)   tags = {
[82](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L82)e3077845 (GitOps             2022-06-21 21:19:21 +0000  44)     yor_trace = "9946760c-981d-4099-aca8-1e5627ee51f1"
[83](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L83)e3077845 (GitOps             2022-06-21 21:19:21 +0000  45)   }
[84](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L84)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  46) }
[85](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L85)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  47) 
[86](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L86)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  48) resource "aws_lambda_function" "hello_world" {
[87](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L87)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  49)   function_name = "hello_world"
[88](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L88)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  50)   role          = "arn:aws:lambda:us-east-1:account-id:resource-id"
[89](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L89)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  51)   handler       = "exports.test"
[90](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L90)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  52)   runtime       = "nodejs12.x"
[91](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L91)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  53)   memory_size   = 1024 # <<<<< Try changing this to 512 to compare costs
[92](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L92)e3077845 (GitOps             2022-06-21 21:19:21 +0000  54)   tags = {
[93](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L93)e3077845 (GitOps             2022-06-21 21:19:21 +0000  55)     yor_trace = "f414a8f5-3c97-47f2-a329-159e2d0038a6"
[94](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L94)e3077845 (GitOps             2022-06-21 21:19:21 +0000  56)   }
[95](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L95)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  57) }
[96](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L96)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  58) 
[97](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L97)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  59) output "aws_instance_type" {
[98](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L98)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  60)   value = aws_instance.web_app.instance_type
[99](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L99)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  61) }
[100](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L100)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  62) 
[101](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L101)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  63) provider "google" {
[102](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L102)e3077845 (GitOps             2022-06-21 21:19:21 +0000  64)   credentials = "{\"type\":\"service_account\"}"
[103](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L103)e3077845 (GitOps             2022-06-21 21:19:21 +0000  65)   region      = "us-central1"
[104](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L104)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  66) }
[105](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L105)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  67) 
[106](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L106)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  68) resource "google_compute_instance" "instance1" {
[107](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L107)e3077845 (GitOps             2022-06-21 21:19:21 +0000  69)   name         = "instance1"
[108](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L108)e3077845 (GitOps             2022-06-21 21:19:21 +0000  70)   machine_type = "n1-standard-32" # <<<<< Try changing this to n1-standard-8 to compare the costs
[109](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L109)e3077845 (GitOps             2022-06-21 21:19:21 +0000  71)   zone         = "us-central1-a"
[110](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L110)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  72) 
[111](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L111)e3077845 (GitOps             2022-06-21 21:19:21 +0000  73)   boot_disk {
[112](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L112)e3077845 (GitOps             2022-06-21 21:19:21 +0000  74)     initialize_params {
[113](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L113)e3077845 (GitOps             2022-06-21 21:19:21 +0000  75)       image = "debian-cloud/debian-9"
[114](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L114)e3077845 (GitOps             2022-06-21 21:19:21 +0000  76)     }
[115](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L115)e3077845 (GitOps             2022-06-21 21:19:21 +0000  77)   }
[116](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L116)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  78) 
[117](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L117)e3077845 (GitOps             2022-06-21 21:19:21 +0000  79)   scheduling {
[118](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L118)e3077845 (GitOps             2022-06-21 21:19:21 +0000  80)     preemptible = true
[119](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L119)e3077845 (GitOps             2022-06-21 21:19:21 +0000  81)   }
[120](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L120)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  82) 
[121](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L121)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  83)   guest_accelerator {
[122](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L122)e3077845 (GitOps             2022-06-21 21:19:21 +0000  84)     type  = "nvidia-tesla-t4" # <<<<< Try changing this to nvidia-tesla-p4 to compare the costs
[123](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L123)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  85)     count = 4
[124](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L124)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  86)   }
[125](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L125)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  87) 
[126](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L126)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  88)   network_interface {
[127](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L127)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  89)     network = "default"
[128](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L128)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  90) 
[129](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L129)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  91)     access_config {
[130](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L130)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  92)     }
[131](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L131)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  93)   }
[132](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L132)e3077845 (GitOps             2022-06-21 21:19:21 +0000  94)   labels = {
[133](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L133)e3077845 (GitOps             2022-06-21 21:19:21 +0000  95)     yor_trace = "816d2ed4-97e2-4a2e-8380-a223a70aa759"
[134](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L134)e3077845 (GitOps             2022-06-21 21:19:21 +0000  96)   }
[135](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L135)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  97) }
[136](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L136)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  98) 
[137](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L137)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000  99) resource "google_dns_record_set" "frontend" {
[138](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L138)e3077845 (GitOps             2022-06-21 21:19:21 +0000 100)   name         = "frontend.123"
[139](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L139)e3077845 (GitOps             2022-06-21 21:19:21 +0000 101)   type         = "A"
[140](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L140)e3077845 (GitOps             2022-06-21 21:19:21 +0000 102)   ttl          = 300
[141](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L141)e3077845 (GitOps             2022-06-21 21:19:21 +0000 103)   rrdatas      = ["123.123.123.123]"]
[142](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L142)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000 104)   managed_zone = "zone"
[143](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L143)^a567270 (Justinas Balinskas 2022-06-21 21:02:44 +0000 105) }
[144](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L144)$ yor tag --directory . --parsers Terraform --output cli
[145](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L145)2022/06/22 05:27:52 [WARNING] Unable to get git blame for file main.tf: failed to get blame for latest commit of file main.tf because of error object not found
[146](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L146)2022/06/22 05:27:52 [WARNING] Failed to tag aws_instance.web_app with git tags, err: failed to get blame for latest commit of file main.tf because of error failed to get blame for latest commit of file main.tf because of error object not found
[147](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L147)2022/06/22 05:27:52 [WARNING] Unable to get git blame for file main.tf: failed to get blame for latest commit of file main.tf because of error object not found
[148](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L148)2022/06/22 05:27:52 [WARNING] Failed to tag aws_instance.web_app2 with git tags, err: failed to get blame for latest commit of file main.tf because of error failed to get blame for latest commit of file main.tf because of error object not found
[149](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L149)2022/06/22 05:27:52 [WARNING] Unable to get git blame for file main.tf: failed to get blame for latest commit of file main.tf because of error object not found
[150](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L150)2022/06/22 05:27:52 [WARNING] Failed to tag aws_lambda_function.hello_world with git tags, err: failed to get blame for latest commit of file main.tf because of error failed to get blame for latest commit of file main.tf because of error object not found
[151](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L151)2022/06/22 05:27:52 [WARNING] Unable to get git blame for file main.tf: failed to get blame for latest commit of file main.tf because of error object not found
[152](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L152)2022/06/22 05:27:52 [WARNING] Failed to tag google_compute_instance.instance1 with git tags, err: failed to get blame for latest commit of file main.tf because of error failed to get blame for latest commit of file main.tf because of error object not found
[153](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L153)  __    __
[154](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L154)  \ \  / /
[155](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L155)   \ \/ /___  _  ____
[156](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L156)    \  /  _ \| |/  __|
[157](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L157)    | |  |_| |   /
[158](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L158)    |_|\____/|__|v0.1.146
[159](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L159) Yor Findings Summary
[160](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L160) Scanned Resources:	  23
[161](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L161) New Resources Traced: 	  0
[162](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L162) Updated Resources:	  0
[163](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L163)$ cd $CI_PROJECT_DIR # collapsed multi-line command

[164](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L164)
Cleaning up project directory and file based variables
00:00
[165](https://gitlab.com/<...>/terraform_module1/-/jobs/2622977266#L165)Job succeeded

@kayman-mk
Copy link
Contributor

I have a slightly different problem. The error message reads [WARNING] Unable to get git blame for file short.tf: failed to get blame for latest commit of file short.tf because of error object not found

@stale
Copy link

stale bot commented Feb 5, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Feb 5, 2023
@kayman-mk
Copy link
Contributor

The problem is still there ...

@stale stale bot removed the wontfix This will not be worked on label Feb 9, 2023
@managedkaos
Copy link

Just adding that I am running into this problem as well in GitHub Actions:

yor version 0.1.169
/home/runner/work/_temp/7bbdf4eb-26a1-4d52-be56-26048870b868/yor tag -d . --tag-groups git,code2cloud --tags git*,yor_trace --tag-local-modules false
Warning: 3 05:28:27 [WARNING] Unable to get git blame for file main.tf: failed to get blame for latest commit of file main.tf because of error object not found
Warning: 3 05:28:27 [WARNING] Failed to tag aws_iam_user.users with git tags, err: failed to get blame for latest commit of file main.tf because of error failed to get blame for latest commit of file main.tf because of error object not found
Warning: 3 05:28:27 [WARNING] Unable to get git blame for file main.tf: failed to get blame for latest commit of file main.tf because of error object not found
Warning: 3 05:28:27 [WARNING] Failed to tag aws_s3_bucket.buckets with git tags, err: failed to get blame for latest commit of file main.tf because of error failed to get blame for latest commit of file main.tf because of error object not found
Warning: 3 05:28:27 [WARNING] Unable to get git blame for file main.tf: failed to get blame for latest commit of file main.tf because of error object not found
Warning: 3 05:28:27 [WARNING] Failed to tag aws_iam_policy.policy with git tags, err: failed to get blame for latest commit of file main.tf because of error failed to get blame for latest commit of file main.tf because of error object not found

@stale
Copy link

stale bot commented Mar 25, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Mar 25, 2023
@kayman-mk
Copy link
Contributor

The problem is still there. ...

@stale stale bot removed the wontfix This will not be worked on label Mar 25, 2023
@stale
Copy link

stale bot commented Apr 25, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Apr 25, 2023
@kayman-mk
Copy link
Contributor

The problem is still there. ...

@stale stale bot removed the wontfix This will not be worked on label Apr 25, 2023
@lonegunmanb
Copy link
Contributor

lonegunmanb commented Apr 27, 2023

I'm debugging the very same issue. It looks like a go-git bug.

The contents and commits have different length error was caused by the file has different line count than the blame informations.

In go-git:

func Blame(c *object.Commit, path string) (*BlameResult, error) {

	b := new(blame)
	b.fRev = c
	b.path = path

	// get all the file revisions
	if err := b.fillRevs(); err != nil {
		return nil, err
	}

The b.fillRevs() loaded the commits that contain the file, and sort these commits by time:

// calculate the history of a file "path", starting from commit "from", sorted by commit date.
func (b *blame) fillRevs() error {
	var err error

	b.revs, err = references(b.fRev, b.path)
	return err
}

In fillRevs() it called references() to read the commits:

func references(c *object.Commit, path string) ([]*object.Commit, error) {
	var result []*object.Commit
	seen := make(map[plumbing.Hash]struct{})
	if err := walkGraph(&result, &seen, c, path); err != nil {
		return nil, err
	}

	// TODO result should be returned without ordering
	sortCommits(result)

	// for merges of identical cherry-picks
	return removeComp(path, result, equivalent)
}

You would find that result wouldn't contain the merge commits. Let's say main.tf in your branch new-feature contains 100 lines, and this branch has been merged with the main branch, and on the merged branch main.tf has 120 lines, but the merged commit was discarded so go-git would has a main.tf contains 120 line and a blame record which belongs to branch new-feature.

Let's drill down into walkGraph:

func walkGraph(result *[]*object.Commit, seen *map[plumbing.Hash]struct{}, current *object.Commit, path string) error {
	// check and update seen
	if _, ok := (*seen)[current.Hash]; ok {
		return nil
	}
	(*seen)[current.Hash] = struct{}{}

	// if the path is not in the current commit, stop searching.
	if _, err := current.File(path); err != nil {
		return nil
	}

	// optimization: don't traverse branches that does not
	// contain the path.
	parents, err := parentsContainingPath(path, current)
	if err != nil {
		return err
	}
	switch len(parents) {
	// if the path is not found in any of its parents, the path was
	// created by this commit; we must add it to the revisions list and
	// stop searching. This includes the case when current is the
	// initial commit.
	case 0:
		*result = append(*result, current)
		return nil
	case 1: // only one parent contains the path
		// if the file contents has change, add the current commit
		different, err := differentContents(path, current, parents)
		if err != nil {
			return err
		}
		if len(different) == 1 {
			*result = append(*result, current)
		}
		// in any case, walk the parent
		return walkGraph(result, seen, parents[0], path)
	default: // more than one parent contains the path
		// TODO: detect merges that had a conflict, because they must be
		// included in the result here.
		for _, p := range parents {
			err := walkGraph(result, seen, p, path)
			if err != nil {
				return err
			}
		}
	}
	return nil
}

It traversed the whole git commits tree, filter out commits those are related to the current file, uses different, err := differentContents(path, current, parents) to verify whether the commit has changed the file, if the answer is yes, then put the commit into results.

The problem is:

default: // more than one parent contains the path
		// TODO: detect merges that had a conflict, because they must be
		// included in the result here.
		for _, p := range parents {
			err := walkGraph(result, seen, p, path)
			if err != nil {
				return err
			}
		}
	}

For a merged commit, it has two parent commits, but in this block it forgot to check the different and store the releated commits into the result. That's why we've met this issue.

I'll try to commit a pr to go-git repo.

@stale
Copy link

stale bot commented Jun 8, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Jun 8, 2023
@gustavoortega
Copy link

Do you plan to take a look at this? If not, I will stop trying to keep this problem in open status.
Thank you,

@stale stale bot removed the wontfix This will not be worked on label Jun 8, 2023
@lonegunmanb
Copy link
Contributor

Hi @gustavoortega, go-git has a working pr which could solve this issue once and for all, please stay tuned.

@stale
Copy link

stale bot commented Jul 9, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Jul 9, 2023
@kayman-mk
Copy link
Contributor

Should stay open. The mentioned PR has been merged a couple of days ago.

@stale stale bot removed the wontfix This will not be worked on label Jul 10, 2023
@lonegunmanb
Copy link
Contributor

@kayman-mk Thanks for the update! Really good news, once go-git release a new tag we can fix this issue.

@flaviostutz
Copy link

By using the latest version of the lib from "master" this PR fixed my issue. Thank you very much @onegunmanb!! 🎉🎉

@lonegunmanb
Copy link
Contributor

Glad to hear that @flaviostutz , now we need to wait for a new go-git version so we can upgrade yor.

@kayman-mk
Copy link
Contributor

@lonegunmanb v5.8.1 is out as I just noticed.

@oponomarov-tu
Copy link

Any update on this? 🙏

@lonegunmanb
Copy link
Contributor

@oponomarov-tu I think once #375 has been merged or go-git version has been upgraded to the latest this issue would be solved.

@stale
Copy link

stale bot commented Sep 9, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Sep 9, 2023
@kayman-mk
Copy link
Contributor

Still waiting for #375

@lonegunmanb
Copy link
Contributor

I found that #375 failed at the "security scan" step, I'll take a look but since I cannot access the CI log details, I'm not sure whether I can fix the issue.

@stale stale bot removed the wontfix This will not be worked on label Sep 11, 2023
@kayman-mk
Copy link
Contributor

I tried something else today as I saw a warning from Sonar that I used an shallow clone. Adding a git fetch --unshallow before running Yor added all missing tags.

@stale
Copy link

stale bot commented Oct 21, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Oct 21, 2023
@oponomarov-tu
Copy link

Should stay open.

@stale stale bot closed this as completed Nov 26, 2023
@lonegunmanb
Copy link
Contributor

Should stay open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

8 participants