Replace some uses of the deprecated mktemp function #1770
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The tempfile.mktemp function is deprecated, because of a race condition where the file may be concurrently created between when its name is generated and when it is opened. Other faciliies in the tempfile module overcome this by generating a name, attempting to create the file or directory in a way that guarantees failure if it already existed, and, in the occasional case that it did already exist, generating another name and trying again (stopping after a predefined limit). For further information on mktemp deprecation: - https://docs.python.org/3/library/tempfile.html#tempfile.mktemp - gitpython-developers/smmap#41 The security risk of calls to mktemp in this project's test suite is low. However, it is still best to avoid using it, because it is deprecated, because it is (at least slightly) brittle, and because any use of mktemp looks like a potential security risk and thereby imposes a burden on working with the code (which could potentially be addressed with detailed comments analyzing why it is believed safe in particular cases, but this would typically be more verbose, and at least as challenging to add, as replacing mktemp with a better alternative). This commit replaces *some* uses of mktemp in the test suite: those where it is readily clear how to do so in a way that preserves the code's intent: - Where a name for a temporary directory is generated with mktemp and os.mkdir is called immediately, mkdtemp is now used. - Where a name for a temporary file that is not customized (such as with a prefix) is generated with mktemp, such that the code under test never uses the filename but only the already-open file-like object, TemporaryFile is now used. As the name isn't customized, the test code in these cases does not express an intent to allow the developer to inspect the file after a test failure, so even if the file wasn't guaranteed to be deleted with a finally block or context manager, it is fine to do so. TemporaryFile supports this use case well on all systems including Windows, and automatically deletes the file. - Where a name for a temporary file that *is* customized (such as with a prefix) to reflect the way the test uses it is generated with mktemp, and the test code does not attempt deterministic deletion of the file when an exception would make the test fail, NamedTemporaryFile with delete=False is now used. The original code to remove the file when the test succeeds is modified accordingly to do the same job, and also commented to explain that it is being handled this way to allow the file to be kept and examined when a test failure occurs. Other cases in the test suite should also be feasible to replace, but are left alone for now. Some of them are ambiguous in their intent, with respect to whether the file should be retained after a test failure. Others appear deliberately to avoid creating a file or directory where the code under test should do so, possibly to check that this is done properly. (One way to preserve that latter behavior, while avoiding the weakness of using mktemp and also avoiding inadverently reproducing that weakness by other means, could be to use a path in a temporary directory made for the test.) This commit also doesn't address the one use of mktemp in the code under test (i.e., outside the test suite, inside the git module).
This makes two related changes to git.index.util.TemporaryFileSwap: - Replace mktemp with mkstemp and then immediately closing the file. This avoids two possible name clashes: the usual name clash where the file may be created by another process between when mktemp generates the name and when the file is created, and the problem that mktemp does not check for files that contain the generated name as a part. The latter is specific to the use here, where a string generated by mktemp was manually concatenated to the base filename. This addresses that by passing the filename as the prefix for mkstemp to use. - Replace os.remove with os.replace and simplify. This is made necessary on Windows by the file already existing, due to mkstemp creating it. Deleting the file and allowing it to be recreated would reproduce the mktemp race condition in full (obscured and with extra steps). But os.replace supports an existing target file on all platforms. It is now also used in __exit__, where it allows the Windows check for the file to be removed, and (in any OS) better expresses the intent of the code to human readers. Furthermore, because one of the "look before you leap" checks in __exit__ is removed, the minor race condition in cleaning up the file is somewhat decreased. A small amount of related refactoring is included. The interface is not changed, even where it could be simplified (by letting __exit__ return None), and resource acquisition remains done on construction rather than in __enter__, because changing those aspects of the design could break some existing uses. Although the use of mktemp replaced here was in the git module and not the test suite, its use was to generate filenames for use in a directory that would ordinarily be under the user's control, such that the ability to carry out typical mktemp-related attacks would already require the ability to achieve the same goals more easily and reliably. Although TemporaryFileSwap is a public class that may be used directly by applications, it is only useful for making a temporary file in the same directory as an existing file. Thus the intended benefits of this change are mainly to code quality and a slight improvement in robustness.
This was referenced Dec 13, 2023
Merged
Byron
approved these changes
Dec 13, 2023
EliahKagan
added a commit
to EliahKagan/GitPython
that referenced
this pull request
Dec 21, 2023
This is a general test for TemporaryFileSwap, but by being parametrized by the type of file_path, it reveals a regression introduced in 9e86053 (gitpython-developers#1770). TemporaryFileSwap still works when file_path is a string, but is now broken when it is a Path. That worked before, and the type annotations document that it should be able to work. This is at least a bug because TemporaryFileSwap is public. (I am unsure whether, in practice, GitPython itself uses it in a way that sometimes passes a Path object as file_path. But code that uses GitPython may call it directly and pass Path.)
EliahKagan
added a commit
to EliahKagan/GitPython
that referenced
this pull request
Dec 21, 2023
This fixes the regression introduced in 9e86053 (gitpython-developers#1770) where the file_path argument to TemporaryFileSwap.__init__ could no longer be a Path object. The change also makes this truer to the code from before gitpython-developers#1770, still without the race condition fixed there, in that str was called on file_path then as well. However, it is not clear that this is a good thing, because this is not an idiomatic use of mkstemp. The reason the `prefix` cannot be a Path is that it is expected to be a filename prefix, with leading directories given in the `dir` argument.
This was referenced Dec 21, 2023
lettuce-bot bot
added a commit
to lettuce-financial/github-bot-signed-commit
that referenced
this pull request
Jan 10, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [GitPython](https://togithub.com/gitpython-developers/GitPython) | `==3.1.40` -> `==3.1.41` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>gitpython-developers/GitPython (GitPython)</summary> ### [`v3.1.41`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.41): - fix Windows security issue [Compare Source](https://togithub.com/gitpython-developers/GitPython/compare/3.1.40...3.1.41) The details about the Windows security issue [can be found in this advisory](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx). Special thanks go to [@​EliahKagan](https://togithub.com/EliahKagan) who reported the issue and fixed it in a single stroke, while being responsible for an incredible amount of improvements that he contributed over the last couple of months ❤️. #### What's Changed - Add `__all__` in git.exc by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1719 - Set submodule update cadence to weekly by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1721 - Never modify sys.path by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1720 - Bump git/ext/gitdb from `8ec2390` to `ec58b7e` by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1722 - Revise comments, docstrings, some messages, and a bit of code by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1725 - Use zero-argument super() by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1726 - Remove obsolete note in \_iter_packed_refs by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1727 - Reorganize test_util and make xfail marks precise by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1729 - Clarify license and make module top comments more consistent by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1730 - Deprecate compat.is\_<platform>, rewriting all uses by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1732 - Revise and restore some module docstrings by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1735 - Make the rmtree callback Windows-only by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1739 - List all non-passing tests in test summaries by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1740 - Document some minor subtleties in test_util.py by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1749 - Always read metadata files as UTF-8 in setup.py by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1748 - Test native Windows on CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1745 - Test macOS on CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1752 - Let close_fds be True on all platforms by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1753 - Fix IndexFile.from_tree on Windows by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1751 - Remove unused TASKKILL fallback in AutoInterrupt by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1754 - Don't return with operand when conceptually void by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1755 - Group .gitignore entries by purpose by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1758 - Adding dubious ownership handling by [@​marioaag](https://togithub.com/marioaag) in [gitpython-developers/GitPython#1746 - Avoid brittle assumptions about preexisting temporary files in tests by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1759 - Overhaul noqa directives by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1760 - Clarify some Git.execute kill_after_timeout limitations by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1761 - Bump actions/setup-python from 4 to 5 by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1763 - Don't install black on Cygwin by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1766 - Extract all "import gc" to module level by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1765 - Extract remaining local "import gc" to module level by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1768 - Replace xfail with gc.collect in TestSubmodule.test_rename by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1767 - Enable CodeQL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1769 - Replace some uses of the deprecated mktemp function by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1770 - Bump github/codeql-action from 2 to 3 by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1773 - Run some Windows environment variable tests only on Windows by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1774 - Fix TemporaryFileSwap regression where file_path could not be Path by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1776 - Improve hooks tests by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1777 - Fix if items of Index is of type PathLike by [@​stegm](https://togithub.com/stegm) in [gitpython-developers/GitPython#1778 - Better document IterableObj.iter_items and improve some subclasses by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1780 - Revert "Don't install black on Cygwin" by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1783 - Add missing pip in $PATH on Cygwin CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1784 - Shorten Iterable docstrings and put IterableObj first by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1785 - Fix incompletely revised Iterable/IterableObj docstrings by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1786 - Pre-deprecate setting Git.USE_SHELL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1782 - Deprecate Git.USE_SHELL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1787 - In handle_process_output don't forward finalizer result by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1788 - Fix mypy warning "Missing return statement" by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1789 - Fix two remaining Windows untrusted search path cases by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1792 #### New Contributors - [@​marioaag](https://togithub.com/marioaag) made their first contribution in [gitpython-developers/GitPython#1746 - [@​stegm](https://togithub.com/stegm) made their first contribution in [gitpython-developers/GitPython#1778 **Full Changelog**: gitpython-developers/GitPython@3.1.40...3.1.41 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/lettuce-financial/github-bot-signed-commit). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
renovate bot
added a commit
to allenporter/flux-local
that referenced
this pull request
Jan 11, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [GitPython](https://togithub.com/gitpython-developers/GitPython) | `==3.1.40` -> `==3.1.41` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>gitpython-developers/GitPython (GitPython)</summary> ### [`v3.1.41`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.41): - fix Windows security issue [Compare Source](https://togithub.com/gitpython-developers/GitPython/compare/3.1.40...3.1.41) The details about the Windows security issue [can be found in this advisory](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx). Special thanks go to [@​EliahKagan](https://togithub.com/EliahKagan) who reported the issue and fixed it in a single stroke, while being responsible for an incredible amount of improvements that he contributed over the last couple of months ❤️. #### What's Changed - Add `__all__` in git.exc by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1719 - Set submodule update cadence to weekly by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1721 - Never modify sys.path by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1720 - Bump git/ext/gitdb from `8ec2390` to `ec58b7e` by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1722 - Revise comments, docstrings, some messages, and a bit of code by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1725 - Use zero-argument super() by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1726 - Remove obsolete note in \_iter_packed_refs by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1727 - Reorganize test_util and make xfail marks precise by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1729 - Clarify license and make module top comments more consistent by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1730 - Deprecate compat.is\_<platform>, rewriting all uses by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1732 - Revise and restore some module docstrings by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1735 - Make the rmtree callback Windows-only by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1739 - List all non-passing tests in test summaries by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1740 - Document some minor subtleties in test_util.py by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1749 - Always read metadata files as UTF-8 in setup.py by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1748 - Test native Windows on CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1745 - Test macOS on CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1752 - Let close_fds be True on all platforms by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1753 - Fix IndexFile.from_tree on Windows by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1751 - Remove unused TASKKILL fallback in AutoInterrupt by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1754 - Don't return with operand when conceptually void by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1755 - Group .gitignore entries by purpose by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1758 - Adding dubious ownership handling by [@​marioaag](https://togithub.com/marioaag) in [gitpython-developers/GitPython#1746 - Avoid brittle assumptions about preexisting temporary files in tests by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1759 - Overhaul noqa directives by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1760 - Clarify some Git.execute kill_after_timeout limitations by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1761 - Bump actions/setup-python from 4 to 5 by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1763 - Don't install black on Cygwin by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1766 - Extract all "import gc" to module level by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1765 - Extract remaining local "import gc" to module level by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1768 - Replace xfail with gc.collect in TestSubmodule.test_rename by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1767 - Enable CodeQL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1769 - Replace some uses of the deprecated mktemp function by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1770 - Bump github/codeql-action from 2 to 3 by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1773 - Run some Windows environment variable tests only on Windows by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1774 - Fix TemporaryFileSwap regression where file_path could not be Path by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1776 - Improve hooks tests by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1777 - Fix if items of Index is of type PathLike by [@​stegm](https://togithub.com/stegm) in [gitpython-developers/GitPython#1778 - Better document IterableObj.iter_items and improve some subclasses by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1780 - Revert "Don't install black on Cygwin" by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1783 - Add missing pip in $PATH on Cygwin CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1784 - Shorten Iterable docstrings and put IterableObj first by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1785 - Fix incompletely revised Iterable/IterableObj docstrings by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1786 - Pre-deprecate setting Git.USE_SHELL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1782 - Deprecate Git.USE_SHELL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1787 - In handle_process_output don't forward finalizer result by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1788 - Fix mypy warning "Missing return statement" by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1789 - Fix two remaining Windows untrusted search path cases by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1792 #### New Contributors - [@​marioaag](https://togithub.com/marioaag) made their first contribution in [gitpython-developers/GitPython#1746 - [@​stegm](https://togithub.com/stegm) made their first contribution in [gitpython-developers/GitPython#1778 **Full Changelog**: gitpython-developers/GitPython@3.1.40...3.1.41 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/allenporter/flux-local). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
EliahKagan
added a commit
to EliahKagan/GitPython
that referenced
this pull request
Feb 29, 2024
This expands the comment added in 41fac85 (gitpython-developers#1770) to make more clear that this particular cleanup is deliberately done only when the operation was successful (and why).
otc-zuul bot
pushed a commit
to opentelekomcloud-infra/grafana-docs-monitoring
that referenced
this pull request
Mar 6, 2024
Bump gitpython from 3.1.37 to 3.1.41 Bumps gitpython from 3.1.37 to 3.1.41. Release notes Sourced from gitpython's releases. 3.1.41 - fix Windows security issue The details about the Windows security issue can be found in this advisory. Special thanks go to @EliahKagan who reported the issue and fixed it in a single stroke, while being responsible for an incredible amount of improvements that he contributed over the last couple of months ❤️. What's Changed Add __all__ in git.exc by @EliahKagan in gitpython-developers/GitPython#1719 Set submodule update cadence to weekly by @EliahKagan in gitpython-developers/GitPython#1721 Never modify sys.path by @EliahKagan in gitpython-developers/GitPython#1720 Bump git/ext/gitdb from 8ec2390 to ec58b7e by @dependabot in gitpython-developers/GitPython#1722 Revise comments, docstrings, some messages, and a bit of code by @EliahKagan in gitpython-developers/GitPython#1725 Use zero-argument super() by @EliahKagan in gitpython-developers/GitPython#1726 Remove obsolete note in _iter_packed_refs by @EliahKagan in gitpython-developers/GitPython#1727 Reorganize test_util and make xfail marks precise by @EliahKagan in gitpython-developers/GitPython#1729 Clarify license and make module top comments more consistent by @EliahKagan in gitpython-developers/GitPython#1730 Deprecate compat.is_, rewriting all uses by @EliahKagan in gitpython-developers/GitPython#1732 Revise and restore some module docstrings by @EliahKagan in gitpython-developers/GitPython#1735 Make the rmtree callback Windows-only by @EliahKagan in gitpython-developers/GitPython#1739 List all non-passing tests in test summaries by @EliahKagan in gitpython-developers/GitPython#1740 Document some minor subtleties in test_util.py by @EliahKagan in gitpython-developers/GitPython#1749 Always read metadata files as UTF-8 in setup.py by @EliahKagan in gitpython-developers/GitPython#1748 Test native Windows on CI by @EliahKagan in gitpython-developers/GitPython#1745 Test macOS on CI by @EliahKagan in gitpython-developers/GitPython#1752 Let close_fds be True on all platforms by @EliahKagan in gitpython-developers/GitPython#1753 Fix IndexFile.from_tree on Windows by @EliahKagan in gitpython-developers/GitPython#1751 Remove unused TASKKILL fallback in AutoInterrupt by @EliahKagan in gitpython-developers/GitPython#1754 Don't return with operand when conceptually void by @EliahKagan in gitpython-developers/GitPython#1755 Group .gitignore entries by purpose by @EliahKagan in gitpython-developers/GitPython#1758 Adding dubious ownership handling by @marioaag in gitpython-developers/GitPython#1746 Avoid brittle assumptions about preexisting temporary files in tests by @EliahKagan in gitpython-developers/GitPython#1759 Overhaul noqa directives by @EliahKagan in gitpython-developers/GitPython#1760 Clarify some Git.execute kill_after_timeout limitations by @EliahKagan in gitpython-developers/GitPython#1761 Bump actions/setup-python from 4 to 5 by @dependabot in gitpython-developers/GitPython#1763 Don't install black on Cygwin by @EliahKagan in gitpython-developers/GitPython#1766 Extract all "import gc" to module level by @EliahKagan in gitpython-developers/GitPython#1765 Extract remaining local "import gc" to module level by @EliahKagan in gitpython-developers/GitPython#1768 Replace xfail with gc.collect in TestSubmodule.test_rename by @EliahKagan in gitpython-developers/GitPython#1767 Enable CodeQL by @EliahKagan in gitpython-developers/GitPython#1769 Replace some uses of the deprecated mktemp function by @EliahKagan in gitpython-developers/GitPython#1770 Bump github/codeql-action from 2 to 3 by @dependabot in gitpython-developers/GitPython#1773 Run some Windows environment variable tests only on Windows by @EliahKagan in gitpython-developers/GitPython#1774 Fix TemporaryFileSwap regression where file_path could not be Path by @EliahKagan in gitpython-developers/GitPython#1776 Improve hooks tests by @EliahKagan in gitpython-developers/GitPython#1777 Fix if items of Index is of type PathLike by @stegm in gitpython-developers/GitPython#1778 Better document IterableObj.iter_items and improve some subclasses by @EliahKagan in gitpython-developers/GitPython#1780 Revert "Don't install black on Cygwin" by @EliahKagan in gitpython-developers/GitPython#1783 Add missing pip in $PATH on Cygwin CI by @EliahKagan in gitpython-developers/GitPython#1784 Shorten Iterable docstrings and put IterableObj first by @EliahKagan in gitpython-developers/GitPython#1785 Fix incompletely revised Iterable/IterableObj docstrings by @EliahKagan in gitpython-developers/GitPython#1786 Pre-deprecate setting Git.USE_SHELL by @EliahKagan in gitpython-developers/GitPython#1782 ... (truncated) Commits f288738 bump patch level ef3192c Merge pull request #1792 from EliahKagan/popen 1f3caa3 Further clarify comment in test_hook_uses_shell_not_from_cwd 3eb7c2a Move safer_popen from git.util to git.cmd c551e91 Extract shared logic for using Popen safely on Windows 15ebb25 Clarify comment in test_hook_uses_shell_not_from_cwd f44524a Avoid spurious "location may have moved" on Windows a42ea0a Cover absent/no-distro bash.exe in hooks "not from cwd" test 7751436 Extract venv management from test_installation 66ff4c1 Omit CWD in search for bash.exe to run hooks on Windows Additional commits viewable in compare view You can trigger a rebase of this PR by commenting @dependabot rebase. Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: @dependabot rebase will rebase this PR @dependabot recreate will recreate this PR, overwriting any edits that have been made to it @dependabot merge will merge this PR after your CI passes on it @dependabot squash and merge will squash and merge this PR after your CI passes on it @dependabot cancel merge will cancel a previously requested merge and block automerging @dependabot reopen will reopen this PR if it is closed @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the Security Alerts page. Note Automatic rebases have been disabled on this pull request as it has been open for over 30 days. Reviewed-by: Vladimir Vshivkov
JoeWang1127
pushed a commit
to googleapis/sdk-platform-java
that referenced
this pull request
Apr 6, 2024
](https://renovatebot.com){kind=link}
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [GitPython](https://togithub.com/gitpython-developers/GitPython) | `==3.1.40` -> `==3.1.41` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. ### GitHub Vulnerability Alerts #### [CVE-2024-22190](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx) ### Summary This issue exists because of an incomplete fix for CVE-2023-40590. On Windows, GitPython uses an untrusted search path if it uses a shell to run `git`, as well as when it runs `bash.exe` to interpret hooks. If either of those features are used on Windows, a malicious `git.exe` or `bash.exe` may be run from an untrusted repository. ### Details Although GitPython often avoids executing programs found in an untrusted search path since 3.1.33, two situations remain where this still occurs. Either can allow arbitrary code execution under some circumstances. #### When a shell is used GitPython can be told to run `git` commands through a shell rather than as direct subprocesses, by passing `shell=True` to any method that accepts it, or by both setting `Git.USE_SHELL = True` and not passing `shell=False`. Then the Windows `cmd.exe` shell process performs the path search, and GitPython does not prevent that shell from finding and running `git` in the current directory. When GitPython runs `git` directly rather than through a shell, the GitPython process performs the path search, and currently omits the current directory by setting `NoDefaultCurrentDirectoryInExePath` in its own environment during the `Popen` call. Although the `cmd.exe` shell will honor this environment variable when present, GitPython does not currently pass it into the shell subprocess's environment. Furthermore, because GitPython sets the subprocess CWD to the root of a repository's working tree, using a shell will run a malicious `git.exe` in an untrusted repository even if GitPython itself is run from a trusted location. This also applies if `Git.execute` is called directly with `shell=True` (or after `Git.USE_SHELL = True`) to run any command. #### When hook scripts are run On Windows, GitPython uses `bash.exe` to run hooks that appear to be scripts. However, unlike when running `git`, no steps are taken to avoid finding and running `bash.exe` in the current directory. This allows the author of an untrusted fork or branch to cause a malicious `bash.exe` to be run in some otherwise safe workflows. An example of such a scenario is if the user installs a trusted hook while on a trusted branch, then switches to an untrusted feature branch (possibly from a fork) to review proposed changes. If the untrusted feature branch contains a malicious `bash.exe` and the user's current working directory is the working tree, and the user performs an action that runs the hook, then although the hook itself is uncorrupted, it runs with the malicious `bash.exe`. Note that, while `bash.exe` is a shell, this is a separate scenario from when `git` is run using the unrelated Windows `cmd.exe` shell. ### PoC On Windows, create a `git.exe` file in a repository. Then create a `Repo` object, and call any method through it (directly or indirectly) that supports the `shell` keyword argument with `shell=True`: ```powershell mkdir testrepo git init testrepo cp ... testrepo git.exe # Replace "..." with any executable of choice. python -c "import git; print(git.Repo('testrepo').git.version(shell=True))" ``` The `git.exe` executable in the repository directory will be run. Or use no `Repo` object, but do it from the location with the `git.exe`: ```powershell cd testrepo python -c "import git; print(git.Git().version(shell=True))" ``` The `git.exe` executable in the current directory will be run. For the scenario with hooks, install a hook in a repository, create a `bash.exe` file in the current directory, and perform an operation that causes GitPython to attempt to run the hook: ```powershell mkdir testrepo cd testrepo git init mv .git/hooks/pre-commit.sample .git/hooks/pre-commit cp ... bash.exe # Replace "..." with any executable of choice. echo "Some text" >file.txt git add file.txt python -c "import git; git.Repo().index.commit('Some message')" ``` The `bash.exe` executable in the current directory will be run. ### Impact The greatest impact is probably in applications that set `Git.USE_SHELL = True` for historical reasons. (Undesired console windows had, in the past, been created in some kinds of applications, when it was not used.) Such an application may be vulnerable to arbitrary code execution from a malicious repository, even with no other exacerbating conditions. This is to say that, if a shell is used to run `git`, the full effect of CVE-2023-40590 is still present. Furthermore, as noted above, running the application itself from a trusted directory is not a sufficient mitigation. An application that does not direct GitPython to use a shell to run `git` subprocesses thus avoids most of the risk. However, there is no such straightforward way to prevent GitPython from running `bash.exe` to interpret hooks. So while the conditions needed for that to be exploited are more involved, it may be harder to mitigate decisively prior to patching. ### Possible solutions A straightforward approach would be to address each bug directly: - When a shell is used, pass `NoDefaultCurrentDirectoryInExePath` into the subprocess environment, because in that scenario the subprocess is the `cmd.exe` shell that itself performs the path search. - Set `NoDefaultCurrentDirectoryInExePath` in the GitPython process environment during the `Popen` call made to run hooks with a `bash.exe` subprocess. These need only be done on Windows. --- ### Release Notes <details> <summary>gitpython-developers/GitPython (GitPython)</summary> ### [`v3.1.41`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.41): - fix Windows security issue [Compare Source](https://togithub.com/gitpython-developers/GitPython/compare/3.1.40...3.1.41) The details about the Windows security issue [can be found in this advisory](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx). Special thanks go to [@​EliahKagan](https://togithub.com/EliahKagan) who reported the issue and fixed it in a single stroke, while being responsible for an incredible amount of improvements that he contributed over the last couple of months ❤️. #### What's Changed - Add `__all__` in git.exc by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1719 - Set submodule update cadence to weekly by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1721 - Never modify sys.path by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1720 - Bump git/ext/gitdb from `8ec2390` to `ec58b7e` by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1722 - Revise comments, docstrings, some messages, and a bit of code by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1725 - Use zero-argument super() by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1726 - Remove obsolete note in \_iter_packed_refs by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1727 - Reorganize test_util and make xfail marks precise by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1729 - Clarify license and make module top comments more consistent by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1730 - Deprecate compat.is\_<platform>, rewriting all uses by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1732 - Revise and restore some module docstrings by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1735 - Make the rmtree callback Windows-only by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1739 - List all non-passing tests in test summaries by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1740 - Document some minor subtleties in test_util.py by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1749 - Always read metadata files as UTF-8 in setup.py by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1748 - Test native Windows on CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1745 - Test macOS on CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1752 - Let close_fds be True on all platforms by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1753 - Fix IndexFile.from_tree on Windows by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1751 - Remove unused TASKKILL fallback in AutoInterrupt by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1754 - Don't return with operand when conceptually void by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1755 - Group .gitignore entries by purpose by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1758 - Adding dubious ownership handling by [@​marioaag](https://togithub.com/marioaag) in [gitpython-developers/GitPython#1746 - Avoid brittle assumptions about preexisting temporary files in tests by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1759 - Overhaul noqa directives by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1760 - Clarify some Git.execute kill_after_timeout limitations by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1761 - Bump actions/setup-python from 4 to 5 by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1763 - Don't install black on Cygwin by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1766 - Extract all "import gc" to module level by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1765 - Extract remaining local "import gc" to module level by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1768 - Replace xfail with gc.collect in TestSubmodule.test_rename by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1767 - Enable CodeQL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1769 - Replace some uses of the deprecated mktemp function by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1770 - Bump github/codeql-action from 2 to 3 by [@​dependabot](https://togithub.com/dependabot) in [gitpython-developers/GitPython#1773 - Run some Windows environment variable tests only on Windows by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1774 - Fix TemporaryFileSwap regression where file_path could not be Path by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1776 - Improve hooks tests by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1777 - Fix if items of Index is of type PathLike by [@​stegm](https://togithub.com/stegm) in [gitpython-developers/GitPython#1778 - Better document IterableObj.iter_items and improve some subclasses by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1780 - Revert "Don't install black on Cygwin" by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1783 - Add missing pip in $PATH on Cygwin CI by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1784 - Shorten Iterable docstrings and put IterableObj first by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1785 - Fix incompletely revised Iterable/IterableObj docstrings by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1786 - Pre-deprecate setting Git.USE_SHELL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1782 - Deprecate Git.USE_SHELL by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1787 - In handle_process_output don't forward finalizer result by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1788 - Fix mypy warning "Missing return statement" by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1789 - Fix two remaining Windows untrusted search path cases by [@​EliahKagan](https://togithub.com/EliahKagan) in [gitpython-developers/GitPython#1792 #### New Contributors - [@​marioaag](https://togithub.com/marioaag) made their first contribution in [gitpython-developers/GitPython#1746 - [@​stegm](https://togithub.com/stegm) made their first contribution in [gitpython-developers/GitPython#1778 **Full Changelog**: gitpython-developers/GitPython@3.1.40...3.1.41 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/googleapis/sdk-platform-java). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
tempfile.mktempis deprecated for security reasons and the race condition inherent to its use can have an impact on robustness even in situations where the security impact is small or nonexistent. I do not believe the changes made here are fixing security vulnerabilities; I was not able, at least so far, to think of likely ways they could be exploited. Nonetheless I believe these changes are justified, as detailed in the commit messages.Only one of the uses of
mktempwas in thegitmodule itself. Others are in the test suite. There are a few more in the test suite where I am unsure what the best approach is to replacing them, though I hope to propose a change to eliminate them in the future. All of these considerations, as well as the general issues surroundingmktempand its deprecation, are detailed in the two commit messages.gitmodule.See also:
mktemp. (I've proposed a fix.) That looks like the only use ofmktempin smmap. There are some in gitdb as well, and (as noted) some in GitPython that this PR does not cover.