Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add parallelism to major chunk of hooks. Check Parallelism section in README #620

Merged
merged 55 commits into from
Feb 17, 2024

Conversation

MaxymVlasov
Copy link
Collaborator

@MaxymVlasov MaxymVlasov commented Feb 9, 2024

Put an x into the box if that apply:

  • This PR introduces breaking change.
  • This PR fixes a bug.
  • This PR adds new functionality.
  • This PR enhances existing functionality.

Reasoning

We have a GH workflow that runs lockflies updates every week (implementation and reasoning here).
It usually takes from 2h 30min to 3h 15min. That was fine for us, till we found that our GH runners, based on AWS EC2s, started silently failing after 30min "without recent logs", and that was fixed by crutch which sends a dummy log every 10min.

However, during the debugging, I spent some time describing why hooks were not utilizing all the provided resources.
And that means a waste of time and money, not only for that corner case but for every huge commit, which can cause opting out by git commit -n of using hooks locally for changes that affect many directories.

Description of your changes

  • Add per-hook --parallelism-limit setting to --hook-config. Defaults to number of logical CPUs - 1
  • Main implementation uses flock, which not pre-installed in macOS and some ancient BSD/Linux systems.
    • To maintain backward compatibility, for cases where no flock - fallback to "bash-native locking mechanism" was added.
  • As quick tests show, ~5% of stacks face race condition problem, no matter if any locking mechanism exists or dirs try to init in parallel. I suppose the lock failed as it uses disk when hooks run in memory, so the creation of the lock can take some time as there bunch of caches between Mem and Disk. These milliseconds are enough to allow running a few t init in parallel.
  • Final implementation uses a retry mechanism for cases when race condition failed to t init directory.

In quick tests, I can say that on big changes:

  • With flock, there 300-500% speed increase for most of the hooks.
  • Without flock, there 250-450% speed increase for most of the hooks.
  • Up to 2000% speed increase for terraform_validate, and up to 500% - for other affected hooks.
  • When --parallelism-limit=1 I observed an insignificant increase in time (about 5-10%) compared to v1.86.0 which has no parallelism at all. This may be the cost of maintaining parallelism or the result of external factors since the tests were not conducted in a vacuum.

For small changes, improvements are less significant.

How can we test changes

There is no easy way to test it, but that could be used as a testing template
repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: f8d26f85b054e2842ef1b0c5bcef82d8c8cf1832
  hooks:
    # - id: infracost_breakdown
    #   args:
    #     # - --args=--config-file=test/infracost.yml
    #     - --args=--path=./environment/qa
    #     - --args=--terraform-plan-flags="-var-file=../../test.tfvars"t
    #     - --args=--terraform-plan-flags="-var-file=../../test2.tfvars"
    #     - --hook-config='.totalHourlyCost >= 0.1'
    #     - --hook-config=".totalHourlyCost|tonumber > 1"
    #     - -h ".projects[].diff.totalMonthlyCost|tonumber!=10000"
    #     - -h [.projects[].diff.totalMonthlyCost | select (.!=null) | tonumber] | add > 1000
    #   verbose: true # Always show costs

    # - id: terraform_checkov
    #   args:
    #   #   - --args=--quiet
    #     - --args=--skip-check CKV_SECRET_14
    #     # - --env-vars=ANSI_COLORS_DISABLED=true
    #     # - --hook-config=--parallelism-limit=1


    # - id: terraform_docs
    #   args:
    #   # - --hook-config=--path-to-file=README.md        # Valid UNIX path. I.e. ../TFDOC.md or docs/README.md etc.
    #   # - --hook-config=--add-to-existing-file=true     # Boolean. true or false
    #   - --hook-config=--create-file-if-not-exist=true # Boolean. true or false
    #   # - --args=--config=__GIT_WORKING_DIR__/.terraform-docs.yml
    #   - --args=--config=.terraform-docs.yml
    #   # - --args=--lockfile=false
    #   # - --args=--max-depth=2


    # - id: terraform_fmt
    #   exclude: environment\/.*$


    # - id: terraform_providers_lock
    #   args:
    #   # - --args=-platform=windows_amd64
    #   # - --args=-platform=darwin_amd64
    #   - --args=-platform=linux_amd64

    #   - --hook-config=--mode=always-regenerate-lockfile
    #   # - --hook-config=--parallelism-limit=CPU_NUM*2
    #   verbose: true

    # - id: terraform_tflint
    #   args:
    #     - --args=--config=__GIT_WORKING_DIR__/.config/tflint.hcl
    #     # # - '--args=--config=__GIT_WORKING_DIR__/${CONFIG_FILE}' # export CONFIG_FILE=.tflint.hcl
    #     # - --args=--config=__GIT_WORKING_DIR__/${CONFIG_NAME}.${CONFIG_EXT} # export CONFIG_NAME=.tflint; export CONFIG_EXT=hcl
    #     - --args=--module

    # - id: terraform_tfsec
    #   args:
    #       - --args=--config-file=.tfsec.json
    #       - --args=--force-all-dirs
    #       - --args=--exclude-downloaded-modules
    #       - --args=--concise-output
    #       - --args=--config-file=__GIT_WORKING_DIR__/.tfsec.json

    # - id: terraform_trivy
    #   args:
    #     - --hook-config=--parallelism-limit=CPU_NUM*4


    # - id: terraform_validate
    #   args:
    #     - --tf-init-args=-upgrade
    #     - --tf-init-args=-get=true
    #     - --env-vars=AWS_DEFAULT_REGION="us-west-2"
    #     - --env-vars=AWS_ACCESS_KEY_ID="anaccesskey"
    #     - --env-vars=AWS_SECRET_ACCESS_KEY="asecretkey"
    #     - --hook-config=--retry-once-with-cleanup=true     # Boolean. true or false
    # #    - --hook-config=--parallelism-limit=CPU_NUM*4
    #   verbose: true



    # - id: terraform_validate
    #   args:
    #     - --hook-config=--retry-once-with-cleanup=true


    # - id: terragrunt_fmt


    # - id: terragrunt_validate


    # - id: terrascan
    #   args:
    #     - --args=--non-recursive # avoids scan errors on subdirectories without Terraform config files
    #     - --args=--policy-type aws
    #     # - --hook-config=--parallelism-limit=CPU_NUM*2


    - id: tfupdate
      name: Autoupdate AWS provider versions
      args:

        - --args=provider aws
        - --args=--version '~> 4.2.0'
        - --hook-config=--parallelism-limit=1


    # - id: tfupdate
    #   name: Autoupdate Helm provider versions
    #   args:
    #     - --args=provider helm
    #     - --args=-v 2.5.0
    #     - --args=--ignore-path sdfs/sfdfs

    # - id: tfupdate
    #   name: Update Google provider to specified version
    #   args:
    #     - --args=provider google
    #     - --args=--ignore-path sdfs/sfdfs
    #     - --args=--version ">= 4.18.0, < 5"

    # - id: tfupdate
    #   name: Update Terraform provider to specified version
    #   args:
    #     - --args=terraform
    #     - --args=--version "> 1.1.8"
    #     - --env-vars=AWS_DEFAULT_REGION="us-west-2"
    #     - --env-vars=AWS_ACCESS_KEY_ID="anaccesskey"
    #     - --env-vars=AWS_SECRET_ACCESS_KEY="asecretkey"

Sorry, something went wrong.

Verified

This commit was signed with the committer’s verified signature.
rueian Rueian

Verified

This commit was signed with the committer’s verified signature.
rueian Rueian
As non-zero codes now can popup only in subshells, there is no more
reason to keep that option in main shell

Verified

This commit was signed with the committer’s verified signature.
rueian Rueian

Verified

This commit was signed with the committer’s verified signature.
rueian Rueian

Verified

This commit was signed with the committer’s verified signature.
rueian Rueian
Co-authored-by: George L. Yermulnik <yz@yz.kiev.ua>
README.md Outdated Show resolved Hide resolved
f
@MaxymVlasov MaxymVlasov requested review from antonbabenko and robinbowes and removed request for robinbowes February 13, 2024 13:33
README.md Show resolved Hide resolved
hooks/_common.sh Outdated Show resolved Hide resolved
…623)

So, I found that `nproc` always shows how many CPUs available is. K8s "limits" and docker `--cpus` are throttling mechanisms, which do not hide the visibility of all cores.
There are a few workarounds, but  IMO, it is better to implement checks for that than do them

>Workaround for docker - set `--cpuset-cpus`
>Workaraund for K8s - somehow deal with [kubelet static CPU management policy](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#cpu-management-policies), as [recommend in Reddit](https://news.ycombinator.com/item?id=25224714)

* Send all "colorify" logs through stderr, as make able to add user-facing-logs in functions that also need to return same value to the function-caller. Needed for `common::get_cpu_num` err_msg show up

---------

Co-authored-by: George L. Yermulnik <yz@yz.kiev.ua>
hooks/_common.sh Outdated Show resolved Hide resolved
Details:
#620 (review)
README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
hooks/_common.sh Outdated Show resolved Hide resolved
hooks/_common.sh Show resolved Hide resolved
MaxymVlasov and others added 4 commits February 15, 2024 21:05
Co-authored-by: George L. Yermulnik <yz@yz.kiev.ua>
Copy link
Owner

@antonbabenko antonbabenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OMG! It seems to work as expected ‼️

@yermulnik
Copy link
Collaborator

OMG! It seems to work as expected ‼️

Yeah, that's a bit of magic by Max 😄

@MaxymVlasov MaxymVlasov merged commit 6c6eca4 into master Feb 17, 2024
5 checks passed
@MaxymVlasov MaxymVlasov deleted the feat/parallelizm branch February 17, 2024 20:41
antonbabenko pushed a commit that referenced this pull request Feb 17, 2024
# [1.87.0](v1.86.1...v1.87.0) (2024-02-17)

### Features

* Add parallelism to major chunk of hooks. Check `Parallelism` section in README ([#620](#620)) ([6c6eca4](6c6eca4))
@antonbabenko
Copy link
Owner

This PR is included in version 1.87.0 🎉

github-actions bot pushed a commit to MaxymVlasov/pre-commit-terraform-712 that referenced this pull request Jan 24, 2025
# 1.0.0 (2025-01-24)

### Bug Fixes

* `grep: warning: stray \ before /` which pop-up in `grep 3.8` ([antonbabenko#625](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/625)) ([e1a93b2](e1a93b2))
* **`terraform_docs`:** Fix issue and prioritize `output.file` setting from `.terraform-docs.yml` config over `--hook-config=--path-to-file=` ([antonbabenko#698](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/698)) ([9d6a22b](9d6a22b))
* **`terraform_docs`:** Fix issue with processing multiply files without `terraform-docs` markers. Issue introduced in v1.95.0 ([antonbabenko#720](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/720)) ([2b1aec8](2b1aec8)), closes [antonbabenko#717](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/717) [/github.com/antonbabenko/pre-commit-terraform/blob/869a106a4c8c48f34f58318a830436142e31e10a/hooks/terraform_docs.sh#L216](https://github.com//github.com/antonbabenko/pre-commit-terraform/blob/869a106a4c8c48f34f58318a830436142e31e10a/hooks/terraform_docs.sh/issues/L216)
* **`terraform_docs`:** Fix non-GNU `sed` issues, introduced in v1.93.0 ([antonbabenko#704](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/704)) ([3c8734d](3c8734d))
* **`terraform_docs`:** Fix non-GNU sed issues, introduced in v1.93.0, as previous fix doesn't work correctly ([antonbabenko#708](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/708)) ([c986c5e](c986c5e))
* **`terraform_docs`:** Restore `--hook-config=--add-to-existing-file` default behavior. Regression from 1.94.0. ([antonbabenko#716](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/716)) ([315342e](315342e))
* **`terraform_docs`:** Restore multiply `--hook-config` args support. Regression from v1.95.0 ([antonbabenko#731](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/731)) ([87143fb](87143fb))
* **`terraform_docs`:** Suppress "terraform command not found" error message in case binary does not exist ([antonbabenko#693](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/693)) ([6ff3572](6ff3572))
* **`terraform_docs`:** Suppress redundant warnings pop-ups introduced in v1.92.2 ([antonbabenko#700](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/700)) ([59b2454](59b2454))
* **`terraform_providers_lock`:** Require `terraform init` (and `terraform_validate` hook) run when only lockfile changed ([antonbabenko#649](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/649)) ([02c1935](02c1935))
* **`terraform_validate`:** Run `terraform init` on "Missing required provider" error ([antonbabenko#586](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/586)) ([6e2bb2e](6e2bb2e))
* Add `--env-vars`, deprecate `--envs` ([antonbabenko#410](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/410)) ([2b35cad](2b35cad))
* Add `--tf-init-args`, deprecate `--init-args` ([antonbabenko#407](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/407)) ([c4f8251](c4f8251))
* analyse all folders with tflint and don't stop on first execution ([antonbabenko#289](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/289)) ([7c6ad7c](7c6ad7c))
* Change terraform_validate hook functionality for subdirectories with terraform files ([antonbabenko#100](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/100)) ([7694fb9](7694fb9))
* Check all directories with changes and pass all args in terrascan hook ([antonbabenko#305](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/305)) ([66401d9](66401d9))
* command not found ([antonbabenko#251](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/251)) ([e33c654](e33c654))
* Correct deprecated parameter to terraform-docs ([antonbabenko#156](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/156)) ([3a07570](3a07570))
* Correctly handle arrays in terraform_docs.sh ([antonbabenko#141](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/141)) ([f2cab31](f2cab31))
* Describe migration instructions from `terraform_docs_replace` ([antonbabenko#451](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/451)) ([a8bcaa7](a8bcaa7))
* **docker:** Checkov installation silently fails on `docker build` in arm64. Workaround till issue will be fixed in `checkov` itself ([antonbabenko#635](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/635)) ([f255b05](f255b05))
* Dockerfile if INSTALL_ALL is not defined ([antonbabenko#233](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/233)) ([3bdcf51](3bdcf51))
* Dockerized pre-commit-terraform ([antonbabenko#219](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/219)) ([ce02f94](ce02f94))
* **docker:** Prevent all possible "silent errors" during `docker build` ([antonbabenko#644](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/644)) ([0340c8d](0340c8d))
* execute tflint once in no errors ([antonbabenko#250](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/250)) ([390a264](390a264))
* Extend `terraform_validate` `--retry-once-with-cleanup` errors list ([antonbabenko#566](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/566)) ([19188e5](19188e5))
* Fix `terraform_providers_lock` hook broken in v1.79.0 ([antonbabenko#521](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/521)) ([6bfc5bf](6bfc5bf))
* Fix and pin versions in Dockerfile ([antonbabenko#193](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/193)) ([d27074b](d27074b))
* Fix regex considering terraform-docs v0.10.0 old ([antonbabenko#151](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/151)) ([d773f4a](d773f4a))
* Fix terraform_wrapper_module_for_each for when resource name contains 'variable' ([antonbabenko#573](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/573)) ([941177e](941177e))
* Fix terraform_wrapper_module_for_each hook heredoc vars defaults ([antonbabenko#554](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/554)) ([6fd4263](6fd4263))
* Fix the terraform_wrapper_module_for_each hook for modules without outputs or variables ([antonbabenko#552](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/552)) ([f24b3fa](f24b3fa))
* Fixed `terraform_fmt` with `tfenv`, when `terraform` default version is not specified ([antonbabenko#389](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/389)) ([1b9476a](1b9476a))
* Fixed `tfupdate` to work in all cases, not only `pre-commit run --all` ([antonbabenko#375](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/375)) ([297cc75](297cc75))
* Fixed 1.54.0 where `terraform_docs` was broken ([antonbabenko#272](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/272)) ([4c50943](4c50943))
* Fixed args expand in terraform_docs ([antonbabenko#260](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/260)) ([01a6170](01a6170))
* Fixed docker build ([antonbabenko#288](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/288)) ([4543f10](4543f10))
* Fixed git fatal error in Dockerfile ([antonbabenko#372](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/372)) ([c3f8dd4](c3f8dd4))
* Fixed ordering issue in terraform_wrapper_module_for_each hook ([antonbabenko#565](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/565)) ([dc12be1](dc12be1))
* Fixed spacing in `terraform_wrapper_module_for_each` hook ([antonbabenko#503](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/503)) ([ddc0d81](ddc0d81))
* Fixed url for wrappers in generated README (terraform_wrapper_module_for_each) ([antonbabenko#429](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/429)) ([fe29c6c](fe29c6c))
* Improve `tflint --init` command execution ([antonbabenko#361](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/361)) ([d31cb69](d31cb69))
* Improve README and drop quotes from hook env vars ([antonbabenko#651](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/651)) ([daec682](daec682))
* label auto-adding after label rename ([antonbabenko#226](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/226)) ([4faee7b](4faee7b))
* Make hooks bash 3.2 compatible ([antonbabenko#339](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/339)) ([4ad825d](4ad825d))
* make terraform_docs Windows compatible ([antonbabenko#129](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/129)) ([81770aa](81770aa))
* make terraform_tfsec.sh executable ([antonbabenko#140](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/140)) ([077c423](077c423))
* **non-linux:** Bash environment variables in arguments not expanded + Add `trace` log level ([antonbabenko#645](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/645)) ([a2a2990](a2a2990))
* Pass args and env vars to terraform validate ([antonbabenko#125](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/125)) ([774c63e](774c63e))
* Pass command line arguments to tflint init ([antonbabenko#487](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/487)) ([29a8c00](29a8c00))
* Passed scenario in `terraform_docs` hook now works as expected ([7ac2736](7ac2736))
* pre-build docker image ([antonbabenko#292](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/292)) ([01d262c](01d262c))
* Pre-commit-terraform terraform_validate hook ([antonbabenko#401](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/401)) ([d9f482c](d9f482c))
* Properly exclude .terraform directory with checkov hook ([antonbabenko#306](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/306)) ([b431a43](b431a43))
* remove dead code from terraform-docs script ([antonbabenko#229](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/229)) ([ff54bb4](ff54bb4))
* remove sed postprocessing from the terraform_docs_replace hook to fix compatibility with terraform-docs 0.11.0+ ([antonbabenko#176](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/176)) ([90d4521](90d4521))
* Replace `mapfile` to support Bash 3.2.57 pre-installed in macOS ([antonbabenko#628](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/628)) ([01ab3f0](01ab3f0))
* Run `terraform_tfsec` only on terraform code changes ([antonbabenko#571](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/571)) ([4253162](4253162))
* Speed up x2 TFLint hook execution in dirs with violations ([antonbabenko#514](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/514)) ([49974ab](49974ab))
* Speedup `terrascan` hook up to x3 times in big repos ([antonbabenko#307](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/307)) ([2e8dcf9](2e8dcf9))
* Squash terraform_docs bug ([antonbabenko#138](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/138)) ([6c77a6c](6c77a6c))
* Support custom TF paths which contains spaces ([antonbabenko#714](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/714)) ([2bca410](2bca410))
* Suppress duplicate error messages in `terraform_validate` ([antonbabenko#577](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/577)) ([4ea6b14](4ea6b14))
* Terraform validate for submodules ([antonbabenko#172](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/172)) ([827af52](827af52))
* terraform_tflint hook executes in a serial way to run less often ([antonbabenko#211](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/211)) ([3404eed](3404eed))
* **terraform_tflint:** Restore current working directory behavior ([antonbabenko#302](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/302)) ([93029dc](93029dc))
* terraform-docs version 0.10 removed with-aggregate-type-defaults ([antonbabenko#150](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/150)) ([6f3b125](6f3b125))
* terrafrom_tflint ERROR output for files located in repo root ([antonbabenko#243](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/243)) ([3f66432](3f66432))
* TFSec outputs the same results multiple times ([antonbabenko#237](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/237)) ([71f7c34](71f7c34))
* trigger terraform-docs on changes in lock files ([antonbabenko#228](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/228)) ([b67dbd9](b67dbd9))
* typo in arg name for terraform-docs ([antonbabenko#283](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/283)) ([feabecc](feabecc))
* Updated formatting in README (closes [antonbabenko#113](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/113)) ([27e6369](27e6369))
* Updates all dependencies used in Dockerfile and fix Docker image ([antonbabenko#507](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/507)) ([dc177fe](dc177fe))
* **WSL:** Make parallelism work appropriately ([antonbabenko#728](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/728)) ([e87ee43](e87ee43))

### Features

* **`terraform_docs`:** Add `terraform-docs` default markers support and describe how to migrate to them ([antonbabenko#609](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/609)) ([4a0e1fe](4a0e1fe))
* **`terraform_docs`:** Add support for custom markers to better support other formats than Markdown ([antonbabenko#752](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/752)) ([cd090b6](cd090b6))
* **`terraform_docs`:** Drop support for `terraform-docs` <0.12.0 ([antonbabenko#717](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/717)) ([81e4572](81e4572))
* **`terraform_docs`:** Start seamless migration to `terraform-docs` markers ([antonbabenko#701](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/701)) ([d03f44f](d03f44f))
* **`terraform_providers_lock`:** Add `--mode` option and deprecate previous workflow ([antonbabenko#528](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/528)) ([2426b52](2426b52))
* **`terraform-docs`:** Add support for `replace` mode  for TF 0.12+; Use native saving to file for TF 0.12+. Both requires `terraform-docs` v0.12.0+ which released in 2021. ([antonbabenko#705](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/705)) ([1a1b4a3](1a1b4a3))
* Add __GIT_WORKING_DIR__ to terraform_checkov ([antonbabenko#399](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/399)) ([ae88ed7](ae88ed7))
* add __GIT_WORKING_DIR__ to tfsec ([antonbabenko#255](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/255)) ([2973f85](2973f85))
* Add `--retry-once-with-cleanup` to `terraform_validate` ([antonbabenko#441](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/441)) ([96fe3ef](96fe3ef))
* Add `terraform_docs` hook settings ([antonbabenko#245](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/245)) ([7b11401](7b11401))
* Add `terragrunt_providers_lock` hook ([antonbabenko#632](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/632)) ([77940fd](77940fd))
* Add `terragrunt_validate_inputs` hook to check unused and undefined inputs ([antonbabenko#677](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/677)) ([a139b71](a139b71))
* Add checkov support ([antonbabenko#143](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/143)) ([293b64c](293b64c))
* Add GH checks and templates ([antonbabenko#222](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/222)) ([53a866e](53a866e))
* Add infracost_breakdown hook ([antonbabenko#252](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/252)) ([cff42e6](cff42e6))
* Add mixed line ending check to prevent possible errors ([antonbabenko#221](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/221)) ([c920368](c920368))
* Add new hook for `terraform providers lock` operation ([antonbabenko#173](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/173)) ([d69e86d](d69e86d))
* Add parallelism to major chunk of hooks. Check `Parallelism` section in README ([antonbabenko#620](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/620)) ([6c6eca4](6c6eca4))
* Add PATH outputs when TFLint found any problem ([antonbabenko#234](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/234)) ([ce02cd1](ce02cd1))
* Add possibility to share tflint config file for subdirs ([antonbabenko#149](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/149)) ([cf07b5e](cf07b5e))
* Add support for `pre-commit/pre-commit-hooks` in Docker image ([antonbabenko#374](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/374)) ([017da74](017da74))
* Add support for quoted values in `infracost_breakdown` `--hook-config` ([antonbabenko#269](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/269)) ([e2604ea](e2604ea))
* Add support for set env vars inside hook runtime ([antonbabenko#408](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/408)) ([d490231](d490231))
* Add support for specify terraform-docs config file ([antonbabenko#244](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/244)) ([25cddd9](25cddd9))
* Add support for version constraints in `tfupdate` ([antonbabenko#437](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/437)) ([a446642](a446642))
* add terragrunt validate hook ([antonbabenko#134](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/134)) ([f6caf21](f6caf21))
* Added `terraform_checkov` (run per folder), deprecated `checkov` hook ([antonbabenko#290](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/290)) ([e3a9834](e3a9834))
* Added semantic release ([antonbabenko#296](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/296)) ([1bcca44](1bcca44))
* Added support for `tfupdate` to update version constraints in Terraform configurations ([antonbabenko#342](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/342)) ([ef7a0f2](ef7a0f2))
* Added terraform_wrapper_module_for_each hook ([antonbabenko#376](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/376)) ([e4e9a73](e4e9a73))
* Added Terramate as sponsor ([antonbabenko#676](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/676)) ([dae1a48](dae1a48))
* Adding init to terraform_tflint hook ([antonbabenko#352](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/352)) ([1aff30f](1aff30f))
* Adds support for Terrascan ([antonbabenko#195](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/195)) ([fee2387](fee2387))
* Allow `terraform_providers_lock` specify terraform init args ([antonbabenko#406](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/406)) ([32b232f](32b232f))
* Allow env vars expansion in `--args` section for all hooks ([antonbabenko#363](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/363)) ([caa01c3](caa01c3))
* Allow passing of args to terraform_fmt ([antonbabenko#147](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/147)) ([de2f624](de2f624))
* Allow running container as non-root UID/GID for ownership issues (docker) ([antonbabenko#433](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/433)) ([abc2570](abc2570))
* **ci:** Build multi-arch Docker images (`amd64`, `arm64`) ([antonbabenko#496](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/496)) ([923c2c6](923c2c6))
* **deps:** Bump Python version in docker image from 3.11.5 to v3.12.0 ([antonbabenko#597](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/597)) ([28e3cde](28e3cde))
* **docker:** Add ssh-client to Docker image to access private modules via ssh ([antonbabenko#553](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/553)) ([1d76157](1d76157))
* Expand environment variables in `--args=` which contains lowercase symbols, like `${TF_VAR_lowercase}` ([antonbabenko#719](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/719)) ([bf156b4](bf156b4))
* have option for terraform_tfsec hook to only run in relevant modified directories ([antonbabenko#135](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/135)) ([108c75f](108c75f))
* Hook terraform_wrapper_module_for_each should use versions.tf from the module if it exists ([antonbabenko#657](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/657)) ([b127601](b127601))
* Improve performance during `pre-commit --all (-a)` run ([antonbabenko#327](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/327)) ([7e7c916](7e7c916))
* Improved speed of `pre-commit run -a` for multiple hooks ([antonbabenko#338](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/338)) ([579dc45](579dc45))
* Make terraform_validate to run init if necessary ([antonbabenko#158](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/158)) ([d303bff](d303bff))
* Pass custom arguments to terraform init in `terraform_validate` hook ([antonbabenko#293](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/293)) ([45575c3](45575c3))
* Removed `coreutils` (realpath) from dependencies for MacOS ([antonbabenko#368](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/368)) ([944a2e5](944a2e5))
* Set up PR reviewers automatically ([antonbabenko#258](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/258)) ([cc59119](cc59119))
* Skip legacy modules (with provider block) in terraform_wrapper_module_for_each hook ([antonbabenko#560](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/560)) ([456cc76](456cc76))
* Speedup `terraform_validate` - firstly try run validate without checking is `.terraform/` is valid ([antonbabenko#524](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/524)) ([d0d08ac](d0d08ac))
* Support for TFSec ([antonbabenko#103](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/103)) ([2be8fe5](2be8fe5))
* Support set custom TF/OpenTofu binary. | If you use a custom Docker image build, please note that `TERRAFORM_VERSION` now must be provided ([antonbabenko#670](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/670)) ([c7011c0](c7011c0))
* Suppress color for all hooks if `PRE_COMMIT_COLOR=never` set ([antonbabenko#409](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/409)) ([b12f0c6](b12f0c6))
* TFLint: Add `--hook-config=--delegate-chdir` to use `tflint -chdir` ([antonbabenko#512](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/512)) ([1e9debc](1e9debc))
* **trivy:** Add `terraform_trivy` hook and deprecate `terraform_tfsec` ([antonbabenko#606](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/606)) ([f3c819a](f3c819a))
* Updated Docker image from Ubuntu to Alpine ([antonbabenko#278](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/278)) ([71302a9](71302a9))
* When a config file is given, do not specify formatter on cli (terraform_docs) ([antonbabenko#386](https://github.com/MaxymVlasov/pre-commit-terraform-712/issues/386)) ([962054b](962054b))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants