Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Loosen dependency on typing-extensions #7747

Merged
merged 1 commit into from
Jun 1, 2023

Conversation

Fokko
Copy link
Contributor

@Fokko Fokko commented Jun 1, 2023

In the CI we pin the typing-extensions dependency on 4.5.0, but it looks like some bug has been introduced in 4.6.1. This PR bumps typing-extensions to 4.6.2.

By loosening this requirement, we install the same version as when someone would when they don't install the dev requirements.

This was broken in Pydantic 1.10.7, and fixed in 1.10.8: pydantic/pydantic#5826

Resolves #7746

In the CI we pin the `typing-extensions` dependency on `4.5.0`,
but it looks like some bug has been introduced in `4.6.1`.

By loosening this requirement, we install the same version as when
someone would when they don't install the dev requirements.

Resolves apache#7746
@Fokko Fokko merged commit ce23dbd into apache:master Jun 1, 2023
7 checks passed
@Fokko Fokko deleted the fd-loosen-pin branch June 1, 2023 10:23
@Fokko
Copy link
Contributor Author

Fokko commented Jun 1, 2023

Thanks for the quick review @nastra

@rajaie-sg
Copy link

Does a new version need to be released to PyPI? https://pypi.org/project/pyiceberg/#history

I tried 0.3.0 but it still suffers from the issue in #7746

@Fokko
Copy link
Contributor Author

Fokko commented Jun 19, 2023

@rajaie-sg A new version is incoming. The issue should only happen if you also install the dev dependency. Otherwise, you can run pip3 install --upgrade typing-extensions.

@rajaie-sg
Copy link

Ah I see, I was running pip3 install "pyiceberg[glue,s3fs,hive]"

@rajaie-sg
Copy link

rajaie-sg commented Jun 19, 2023

Actually, it looks like the issue here is with pydantic, which is fixed to version 1.10.4 here https://github.com/apache/iceberg/blob/pyiceberg-0.3.0/python/pyproject.toml#LL58C1-L58C9

Even when I run pip3 install --upgrade typing-extensions, I still run into the problem.

It's only when I run pip3 install --upgrade pydantic does the problem get resolved, but this isn't a solution since we are using Poetry and it prevents us from specifying a version of Pydantic other than 1.10.4 in the pyproject.toml file.

pydantic gets installed by default with pyiceberg

I was able to work around this by setting this in my pyproject.toml

typing-extensions ="==4.5.0"

typing-extensions gets installed automatically because it's a dep of pydantic https://github.com/pydantic/pydantic/blob/v1.10.4/setup.py#L132

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pyiceberg pydantic failed with new typing-extension package
3 participants