continuous integration: resolve docutils installation step build failures

[jayaddison]

Feature or Bugfix

• Maintenance

Purpose

• Restore continuous integration test success.

Detail

• pip's vendored libraries have begun warning about the deprecation of pkg_resources
• Our build stages intentionally treat warnings as errors by default
• This change temporarily permits this specific warning deprecation warnings to occur and be reported-but-not-considered-failure.
• The first commit on this branch will attempt to replicate the problem, and the second commit will attempt to resolve it.

Relates

• Resolves build failure: DeprecationWarning during docutils dependency install #11330.

[jayaddison]

Lowering the severity of the specific pkg_resources is deprecated as an API message was insufficient to allow the (non-development) docutils package install to succeed with warnings-as-errors; the third commit adds further lenience, allowing any deprecation warnings during (again, non-dev-version) docutils package installation to succeed.

[picnixz]

Instead of suppressing some warnings as errors, why not:

• freeze the pip version, or
• filter the exact DeprecationWarning causing the issue instead of silencing all DeprecationWarning exceptions ? (if possible)

[jayaddison]

Thanks @picnixz - reasonable suggestions. Replying inline:

Instead of suppressing some warnings as errors, why not:

* freeze the pip version, or

This doesn't appear to be a bug/problem with pip, it's a message for our information, so I don't think we need to pause upgrades to the pip version we're building with.

* filter the exact DeprecationWarning causing the issue instead of silencing all DeprecationWarning exceptions ? (if possible)

That's worth more consideration, yep. I attempted a precise, single-message filter in 671bc0e, but it turns out that there was a second deprecation warning (also pkg_resources-related):

Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages

Perhaps it would be better to provide filters for both of those messages.

I did briefly consider a message filter on the single-word string pkg_resources, to catch both - but it felt wrong somehow to use a single message filter to attempt to handle two not-directly-related warnings.

[jayaddison]

Ah, an important detail from the documentation explaining why the message-filtering isn't working as-expected as of 09984f4:

In -W and PYTHONWARNINGS, message is a literal string that the start of the warning message must contain (case-insensitively), ignoring any whitespace at the start or end of message.

• ✔️ pkg_resources is deprecated matches from the start of pkg_resources is deprecated as an API
• ❎ pkg_resources.declare_namespace does not match from the start of Deprecated call to pkg_resources.declare_namespace('sphinxcontrib')

[jfbu]

@jayaddison trying out at jfbu@b95fc0e (it worked in my local testing).

However I escaped the ` out of copy paste from my usage of PYTHONWARNINGS in console.

Which was bad.

[jfbu]

@jayaddison Can you cherry-pick jfbu@6bbe3f6 it lets CI pass

[jayaddison]

Ah, thanks @jfbu! I was experimenting with something similar. Since yours is proven working, let's go with that :)

[jayaddison]

And for completeness to explain why 40e547a didn't work:

The documentation mentions that regular expressions are supported during some warning filtering:

message is a string containing a regular expression that the start of the warning message must match, case-insensitively.

... but that excludes PYTHONWARNINGS environment variables (and I should have noticed that in the text I quoted previously) - there the message filter is evaluated as a literal string only:

In -W and PYTHONWARNINGS, message is a literal string that the start of the warning message must contain (case-insensitively), ignoring any whitespace at the start or end of message.

[jfbu]

LGTM, thanks!

I hesitated about the added comment as we have very few in such files and I am not sure about our policy here about long lines, but well, this does not undergo linting! And it can always be adjusted later.

[jfbu]

Thanks for contribution, I will merge this now because this helps other contributed PRs not end up receiving unjust and unfair red crosses 😄

[jayaddison]

Thanks again @jfbu. About that comment line:

My hope is that we can resolve the ResourceWarning warnings (see #11317), and then to simplify that line and remove the comment at the same time.

[jfbu]

Thanks for the documenting of the rationale behing the comment and other choices. I streamlined the commit message and added hard linebreaks out of habit. Hope ok with you.

(I did not do a great job as I forgot entirely about first one of four items for linewrap and the title...)