Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency on pydantic requires Rust, which is not available on Cygwin. Pre-compiled wheels might also be not available on non-{windows,linux,mac} platforms #195

Closed
abravalheri opened this issue Jul 8, 2023 · 6 comments · Fixed by #199

Comments

@abravalheri
Copy link
Contributor

abravalheri commented Jul 8, 2023

It seems that inflect may have turned platform dependent (accidentally/indirectly). I don't know if that is something the project is OK with, please feel free to close the issue.

Newly released versions of pydantic depend on pydantic-core which is written in Rust (at least as of version 2.1.2), with no pre-built wheels for non-mainstream platforms (like Cygwin, FreeBSD, etc).

As a consequence, a Rust compiler must be present in the system for the installation of inflect to be successful. However, the rust toolchain may not be supported in some platforms (like Cygwin).

In terms of dependency chain, this also affects upstream jaraco.text, pip-run, pytest-perf and setuptools[testing].

The problem was initially reported in pypa/setuptools#3976.

abravalheri added a commit to abravalheri/setuptools that referenced this issue Jul 17, 2023
The dependency chain of `pip-run` includes `pydantic-core`.
The problem is that newer versions `pydantic-core` are written in Rust
and don't have pre-compiled wheels available for all platforms (e.g.
freebsd). It might be the case it is not even currently possible to
compile it to some platforms due to the lack of availability of the
toolchain (seems to be the problem with cygwin).

See `jaraco/inflect#195` and `pydantic/pydantic-core#773`.
@abravalheri
Copy link
Contributor Author

abravalheri commented Jul 20, 2023

After a discussion in pydantic/pydantic-core#773, it does not seem that Pydantic will make plans to support Cygwin (and pre-built wheels on other platforms).

@abravalheri abravalheri changed the title Dependency on pydantic can require Rust, which is not available on Cygwin Dependency on pydantic requires Rust, which is not available on Cygwin. Pre-compiled wheels might also be not available on non-{windows,linux,mac} platforms Jul 21, 2023
@jaraco
Copy link
Owner

jaraco commented Jul 28, 2023

My motivation for using Pydantic comes from wanting to have a way to disentangle the manipulation of inputs from the core logic of the function and make transformation reusable. I attempted to capture this need and presented it at a Python Language summit a couple of years ago. The recommendation at the time was that Pydantic offers this functionality and I should rely on that first instead of contributing it as a language feature.

In this project, I've been aiming to leverage Pydantic to prove the viability and benefits of the approach.

I also bemoan the non-pure dependency. I'm sad to hear that the rust rewrite means there's no pure python fallback.

I want to avoid re-implementing validation and transformation inline in this project (and every other).

I would be open to someone implementing a pure-python API-compatible version of the parts of Pydantic that this project relies on.

I want to be cautious about declaring defeat early and effectively declaring that it's not possible to have this functionality. Maybe this issue should inform an effort to port pydantic-core into the stdlib so that it's no longer a non-pure dependency.

@abravalheri
Copy link
Contributor Author

abravalheri commented Jul 29, 2023

Thank you very much for the reply @jaraco.

The proposal you made at the Python Language summit is really great. It is a pity it did not get incorporated to the stdlib (it seems to be more comprehensive and powerful than what can be achieved with pydantic).

I would be open to someone implementing a pure-python API-compatible version of the parts of Pydantic that this project relies on.

It seems that inflect is (mostly) using pydantic to do runtime type checking of function calls via validate_call (it also seems that the most relevant thing being checked is Word).

Although it is not API-compatible, https://github.com/agronholm/typeguard seems to be a pure-python suitable replacement for the parts of pydantic inflect uses in the project. Would you be interested in this approach? It also uses a decorator (@typechecked), so probably they are not that far from each other. I implemented a proof of concept in https://github.com/jaraco/inflect/compare/main...abravalheri:inflect:issue-195-typeguard?expand=1.

The other caveat is that typechecked does not support pydantic magic annotations, so I tried to use __instancecheck__ instead.

Maybe this issue should inform an effort to port pydantic-core into the stdlib so that it's no longer a non-pure dependency.

It would be nice if the stdlib offers some validation capabilities (maybe pydantic-core is too specific to be part of stdlib). But the proposal you made at the language summit seem to be simpler and more comprehensive/powerful. I would personally favour that over pydantic-core.

@jaraco
Copy link
Owner

jaraco commented Aug 27, 2023

Would you be interested in this approach?

Yes, possibly. The proof of concept looks good.

I'm slightly disappointed that the basic constraint (min length 1) is no longer declarative but requires imperative code to implement, but that's probably acceptable.

My biggest reluctance is that I'll want to implement argument transformation (not just validation) and typeguard will be insufficient.

I haven't yet figured out if even Pydantic offers a facility for the none_is_zero or always_iterable transforms, but it does have built-in converters, so it seems plausible.

I scanned through the inflect code, and I don't see any place that I obviously think a transform/coercion should exist, so I'm going to say sure, let's explore using typeguard for inflect.

Since Pydantic doesn't seem to have plans to support Python broadly, inflect does not want to inherit that limitation.

@abravalheri
Copy link
Contributor Author

Thank you very much @jaraco for the feedback, I will send a new PR using typeguard.

I'm slightly disappointed that the basic constraint (min length 1) is no longer declarative but requires imperative code to implement, but that's probably acceptable.

I had another look on other alternatives like beartype and typical, but I don't think they manage to achieve that either (beartype almost do it, but it still need to use a lambda in the type annotation).

I decided to stick with typeguard in the PR because it is been around for a while (so relatively stable/safe), but I think it is relatively easy to replace it with another solution (please let me know if you would like to do replace typeguard with something else).

@jaraco
Copy link
Owner

jaraco commented Aug 31, 2023

In the pydantic conversation, I was leaning toward downstream consumers should be supporting build from source (e.g. cygwin environments should compile pydantic), but I've since also encountered new regressions due to the rust dependency. It's definitely an extra burden not taken lightly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants