Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating regex pattern to handle unicode whitespaces. #1853

Conversation

jcole-crowdstrike
Copy link

Replacing the \s whitespace characters with normal spaces (" ") to prevent breaking on unicode whitespace characters (e.g., NBSP). Without this, if a branch name contains a unicode whitespace character that falls under \s, then the branch name will be truncated.

Replacing the \s whitespace characters with normal spaces (" ") to prevent breaking on unicode whitespace characters (e.g., NBSP). Without this, if a branch name contains a unicode whitespace character that falls under \s, then the branch name will be truncated.
Copy link
Member

@Byron Byron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the fix! It's great to see it has a test as well which would fail without it.

I tried to restart the failing windows tests which seem unrelated. The lint, however, seems to be related to this PR in case this can be fixed easily.

Thanks again for your help.

@jcole-crowdstrike
Copy link
Author

Thanks Byron! I will update the PR to accommodate the lint-related fix today.

Adding a second blank line after the newly added test case.
@EliahKagan
Copy link
Contributor

EliahKagan commented Mar 1, 2024

The failing Windows test jobs are actually related: the test case that fails is TestRemote.test_fetch_unsafe_branch_name, which this pull request introduces.

For example, for the test in Python 3.12, the summary lists exactly one test with failing status, whose full details can be read higher in the report.

(I have checked that this is the case for all six versions of Python tested, and in both runs of the test on 5ca0fba. The same should be expected in the test runs on 827e986 since that change is just to fix the lint checks.)

@Byron I am wondering if the reason it had seemed unrelated is that pytest output now contains details of tests with xfail (expected failure) status, which are unrelated, and that may have distracted from the related test's failure. This changed in major version 8 of pytest (which GitPython is using automatically except on Python 3.7 which doesn't support it). Overall, for most projects, it seems to me that this is an improvement, since it's useful to be able to see the specific details of xfailing tests without extra effort (and check if they really match up with the messages written in the xfail decorators). However, it may be that for GitPython's tests it makes common tasks more cumbersome. If so, then the pytest configuration can be changed (this does not require downgrading pytest).

@Byron
Copy link
Member

Byron commented Mar 2, 2024

Thanks for double-checking, @EliahKagan! Indeed, when looking again I realized that the test introduced here is failing in the very last line of the output. All the XFail warnings are printed in red, which seem to exaggerate their importance in this case.

Now that I know where to look I can probably point to the exact line of test failure in future, also to help PR authors to find what matters to them. However, if the xfail information isn't always important or examined, maybe it can also be tuned down by default, but tuned up in PRs that need it, or benefit from it. If in doubt, everything could stay as is as I think it can be handled.

@jcole-crowdstrike
Copy link
Author

I did a lot of tracing and eventually found:
stdout <_io.TextIOWrapper name=7 encoding='cp1252'>
I think we have an encoding issue whereby stdout is being encoded as Windows-1252, instead of UTF-8, due to passing universal_newlines=True (thanks StackOverflow) to subprocess. I have a commit with a couple changes and the new test that I added now passes when I run it in Windows. I will push that and you can see what you think.

Stdout is being encoded as Windows-1251 instead of UTF-8 due to passing
universal_newlines as True to subprocess.
Copy link
Member

@Byron Byron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your help!

I think the lint failure is legitimate for sure and should be easy to fix. However, the otherone puzzles me but a fix might be possible by adjusting the expectation. Could you take another look?

git/remote.py Outdated Show resolved Hide resolved
jcole-crowdstrike added 2 commits March 8, 2024 12:41
This fix avoids encoding problems on Windows, as was done for fetch() in the previous commit. However, here it is being made more explicit and adds the same fix for the pull function.
…lures

Removing this block of code that strips away nonprintable ASCII characters, as it is causing some tests to fail inappropriately.

Successful tests are identified by the suffix ", done", which is being removed from some messages due to this aforementioned block of code. The codeblock probably intends to just strip away traililng non-ASCII characters from strings, but does not break, and so will instead continue to iterate through the string and then strip away both printable/nonprintable ASCII; this is causing the loss of the ", done" because anytime a nonprintable character is anywhere but the end of the string then the entire section of string after the character is truncated.

Looking back through commits, this codeblock was added in 882ebb1 as a workaround after the addition of universal_newlines. Seeing as we have removed universal_newlines in the previous commit, we can also remove this codeblock.

In future, if something likw this is needed, then maybe a join that concatenates all printable ASCII characters would serve better (e.g., "".join(b for b in line if b >= 32)), instead of trying to cut out the nonprintable chars.
@EliahKagan
Copy link
Contributor

The failure of the Cygwin test job in 8b8c76a (currently the tip of this PR) is unrelated to the changes here. It is instead due to a problem downloading and verifying the authenticity of Cygwin. See the Signature files missing thread on the Cygwin mailing list, and the comment discussion in egor-tensin/setup-cygwin#10.

This prevented that job from running any tests, so there are no results here for Cygwin tests. @Byron I think directly causing them to rerun in the PR is something only you can do. If the workflow is rerun, Cygwin should be able to install with no problems now and the unit tests should be able to run.

Copy link
Member

@Byron Byron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing that out, and sorry for not realizing this myself (I wouldn't want to bother you for petty things like this in future).

Indeed, CI passes now so this PR can be merged.

Thanks a lot to everyone involved, you contribution is much appreciated.

@Byron Byron merged commit d40e7d0 into gitpython-developers:main Mar 11, 2024
26 checks passed
@jcole-crowdstrike
Copy link
Author

My pleasure. Thanks so much everybody.

renovate bot added a commit to allenporter/flux-local that referenced this pull request Mar 31, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [GitPython](https://togithub.com/gitpython-developers/GitPython) |
`==3.1.42` -> `==3.1.43` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/GitPython/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/GitPython/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/GitPython/3.1.42/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/GitPython/3.1.42/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>gitpython-developers/GitPython (GitPython)</summary>

###
[`v3.1.43`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.43)

[Compare
Source](https://togithub.com/gitpython-developers/GitPython/compare/3.1.42...3.1.43)

#### Particularly Important Changes

These are likely to affect you, please do take a careful look.

- Issue and test deprecation warnings by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1886
- Fix version_info cache invalidation, typing, parsing, and
serialization by [@&#8203;EliahKagan](https://togithub.com/EliahKagan)
in
[gitpython-developers/GitPython#1838
- Document manual refresh path treatment by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1839
- Improve static typing and docstrings related to git object types by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1859

#### Other Changes

- Test in Docker with Alpine Linux on CI by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1826
- Build online docs (RTD) with -W and dependencies by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1843
- Suggest full-path refresh() in failure message by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1844
- `repo.blame` and `repo.blame_incremental` now accept `None` as the
`rev` parameter. by [@&#8203;Gaubbe](https://togithub.com/Gaubbe) in
[gitpython-developers/GitPython#1846
- Make sure diff always uses the default diff driver when
`create_patch=True` by
[@&#8203;can-taslicukur](https://togithub.com/can-taslicukur) in
[gitpython-developers/GitPython#1832
- Revise docstrings, comments, and a few messages by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1850
- Expand what is included in the API Reference by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1855
- Restore building of documentation downloads by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1856
- Revise type annotations slightly by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1860
- Updating regex pattern to handle unicode whitespaces. by
[@&#8203;jcole-crowdstrike](https://togithub.com/jcole-crowdstrike) in
[gitpython-developers/GitPython#1853
- Use upgraded pip in test fixture virtual environment by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1864
- lint: replace `flake8` with `ruff` check by
[@&#8203;Borda](https://togithub.com/Borda) in
[gitpython-developers/GitPython#1862
- lint: switch Black with `ruff-format` by
[@&#8203;Borda](https://togithub.com/Borda) in
[gitpython-developers/GitPython#1865
- Update readme and tox.ini for recent tooling changes by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1868
- Split tox lint env into three envs, all safe by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1870
- Slightly broaden Ruff, and update and clarify tool configuration by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1871
- Add a "doc" extra for documentation build dependencies by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1872
- Describe `Submodule.__init__` parent_commit parameter by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1877
- Include TagObject in git.types.Tree_ish by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1878
- Improve Sphinx role usage, including linking Git manpages by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1879
- Replace all wildcard imports with explicit imports by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1880
- Clarify how tag objects are usually tree-ish and commit-ish by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1881

#### New Contributors

- [@&#8203;Gaubbe](https://togithub.com/Gaubbe) made their first
contribution in
[gitpython-developers/GitPython#1846
- [@&#8203;can-taslicukur](https://togithub.com/can-taslicukur) made
their first contribution in
[gitpython-developers/GitPython#1832
- [@&#8203;jcole-crowdstrike](https://togithub.com/jcole-crowdstrike)
made their first contribution in
[gitpython-developers/GitPython#1853
- [@&#8203;Borda](https://togithub.com/Borda) made their first
contribution in
[gitpython-developers/GitPython#1862

**Full Changelog**:
gitpython-developers/GitPython@3.1.42...3.1.43

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/allenporter/flux-local).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
lettuce-bot bot added a commit to lettuce-financial/github-bot-signed-commit that referenced this pull request Apr 1, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [GitPython](https://togithub.com/gitpython-developers/GitPython) |
`==3.1.42` -> `==3.1.43` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/GitPython/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/GitPython/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/GitPython/3.1.42/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/GitPython/3.1.42/3.1.43?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>gitpython-developers/GitPython (GitPython)</summary>

###
[`v3.1.43`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.43)

[Compare
Source](https://togithub.com/gitpython-developers/GitPython/compare/3.1.42...3.1.43)

#### Particularly Important Changes

These are likely to affect you, please do take a careful look.

- Issue and test deprecation warnings by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1886
- Fix version_info cache invalidation, typing, parsing, and
serialization by [@&#8203;EliahKagan](https://togithub.com/EliahKagan)
in
[gitpython-developers/GitPython#1838
- Document manual refresh path treatment by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1839
- Improve static typing and docstrings related to git object types by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1859

#### Other Changes

- Test in Docker with Alpine Linux on CI by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1826
- Build online docs (RTD) with -W and dependencies by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1843
- Suggest full-path refresh() in failure message by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1844
- `repo.blame` and `repo.blame_incremental` now accept `None` as the
`rev` parameter. by [@&#8203;Gaubbe](https://togithub.com/Gaubbe) in
[gitpython-developers/GitPython#1846
- Make sure diff always uses the default diff driver when
`create_patch=True` by
[@&#8203;can-taslicukur](https://togithub.com/can-taslicukur) in
[gitpython-developers/GitPython#1832
- Revise docstrings, comments, and a few messages by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1850
- Expand what is included in the API Reference by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1855
- Restore building of documentation downloads by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1856
- Revise type annotations slightly by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1860
- Updating regex pattern to handle unicode whitespaces. by
[@&#8203;jcole-crowdstrike](https://togithub.com/jcole-crowdstrike) in
[gitpython-developers/GitPython#1853
- Use upgraded pip in test fixture virtual environment by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1864
- lint: replace `flake8` with `ruff` check by
[@&#8203;Borda](https://togithub.com/Borda) in
[gitpython-developers/GitPython#1862
- lint: switch Black with `ruff-format` by
[@&#8203;Borda](https://togithub.com/Borda) in
[gitpython-developers/GitPython#1865
- Update readme and tox.ini for recent tooling changes by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1868
- Split tox lint env into three envs, all safe by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1870
- Slightly broaden Ruff, and update and clarify tool configuration by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1871
- Add a "doc" extra for documentation build dependencies by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1872
- Describe `Submodule.__init__` parent_commit parameter by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1877
- Include TagObject in git.types.Tree_ish by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1878
- Improve Sphinx role usage, including linking Git manpages by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1879
- Replace all wildcard imports with explicit imports by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1880
- Clarify how tag objects are usually tree-ish and commit-ish by
[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in
[gitpython-developers/GitPython#1881

#### New Contributors

- [@&#8203;Gaubbe](https://togithub.com/Gaubbe) made their first
contribution in
[gitpython-developers/GitPython#1846
- [@&#8203;can-taslicukur](https://togithub.com/can-taslicukur) made
their first contribution in
[gitpython-developers/GitPython#1832
- [@&#8203;jcole-crowdstrike](https://togithub.com/jcole-crowdstrike)
made their first contribution in
[gitpython-developers/GitPython#1853
- [@&#8203;Borda](https://togithub.com/Borda) made their first
contribution in
[gitpython-developers/GitPython#1862

**Full Changelog**:
gitpython-developers/GitPython@3.1.42...3.1.43

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/lettuce-financial/github-bot-signed-commit).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants