Include exceptions.pyi and fields.pyi #1937

savarin · 2020-08-27T06:03:17Z

Addresses #1897, as an alternative to #1931.

The previous PR #1931 was created to incrementally add .pyi files from typeshed, with the following steps:

Create .pyi files incrementally based on dep tree
Port .pyi files in from typeshed as is
Correct mypy errors flagged by nox -s lint
Commit all stubs for all .pyi files created

This PR is created to illustrate another possibility, where the focus would be on porting as much of the .pyi files from typeshed first. This involves only steps 1 - 3, i.e. changes made only where errors are raised.

After all the .pyi files have been ported from typeshed, step 4 would then be completed for all the .pyi files.

savarin · 2020-08-27T06:10:05Z

@sethmlarson I'm a little concerned we might have been trying to do too much with the previous PR. I created this one in light of the objective of moving as all the .pyi files from typeshed first (and making corrections only where there are explicit mypy errors), and once that's done fill out the .pyi files. This should help us move faster (by focusing on the typeshed port first), as well as do so in a systematic way.

Steps 1 and 2 are captured in the commit f1fd1c0 (ports from typeshed as is), and step 3 in the commits b3db9ac for exceptions.pyi and 8429071 for fields.pyi. The mypy errors corrected in step 3 can be found here.

savarin · 2020-08-27T06:19:15Z

src/urllib3/exceptions.pyi

@@ -0,0 +1,55 @@
+from typing import Any, Optional, Union, Tuple, TYPE_CHECKING


From typeshed at exceptions.pyi.

savarin · 2020-08-27T06:20:47Z

src/urllib3/fields.pyi

@@ -0,0 +1,26 @@
+# Stubs for requests.packages.urllib3.fields (Python 3.4)
+
+from typing import Any, Callable, Dict, Optional


From typeshed at fields.pyi.

sethmlarson

Couple of comments

sethmlarson · 2020-08-27T13:57:38Z

src/urllib3/fields.pyi

+from typing import Any, Callable, Dict, Optional
+
+def guess_content_type(filename: str, default: str) -> str: ...
+def format_header_param(name: str, value: str) -> str: ...


I think we're missing types for format_header_param_rfc2231 and format_header_param_html5, shouldn't Mypy catch that issue though?

@sethmlarson Annoyingly with stub files, mypy only catches errors on a subsequent import. For example https://github.com/savarin/urllib3/compare/port-types-03...savarin:port-types-03-example?expand=1 would raise the following error.

src/urllib3/result.py:4: error: Argument 2 to "format_header_param" has incompatible type "int"; expected "str".

Types added for those two.

sethmlarson · 2020-08-27T13:59:20Z

src/urllib3/fields.pyi

+        name: str,
+        data: Any,
+        filename: Optional[str],
+        headers: Optional[Dict[str, str]],


headers: Optional[Mapping[str, str]]

savarin · 2020-08-30T16:34:48Z

@sethmlarson Changes incorporated.

I also have an idea on tracking progress. We can simply count the lines returned by mypy (as per 20d00cc) to get a sense how many errors we're reducing at each change.

# Remove all existing `.pyi` files
nox > all errors count: 1116

# Current master
nox > all errors count: 1105

# This PR
nox > all errors count: 854

codecov · 2020-08-30T16:50:33Z

Codecov Report

Merging #1937 into master will increase coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1937      +/-   ##
==========================================
+ Coverage   99.72%   99.81%   +0.09%     
==========================================
  Files          24       24              
  Lines        2179     2179              
==========================================
+ Hits         2173     2175       +2     
+ Misses          6        4       -2

Flag	Coverage Δ
#unittests	`99.35% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/urllib3/util/ssl_.py	`100.00% <0.00%> (+2.24%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf6ab7a...20d00cc. Read the comment docs.

sethmlarson · 2020-08-31T18:59:29Z

Hey @savarin wanted to thank you for all this work. Want to keep you up to date with what we're planning and how that relates to type hints.

Right now I'm planning on shipping type hint support in v2 which likely won't be released until mid/late 2021. For now you can continue with type stubs and when the master branch becomes the "v2" branch we'll migrate your type stubs into inline type hints. This means that type stubs won't get released in v1.26 because our API will become a lot simpler when we drop Python 2.7 support. So for now:

You can continue as you're doing with .pyi files, we'll handle the rolling into inline type hints via a tool probably
Focus on Python 3.6+ types, don't worry about urllib3.packages.six or code that's written explicitly for Python 2 for example.
If you are discouraged by this I understand, we really value what you're doing and all hope you'll still be motivated to keep contributing. Let me know if you want to discuss more via email (sethmichaellarson@gmail.com)

savarin · 2020-09-02T15:42:13Z

Hey @savarin wanted to thank you for all this work. Want to keep you up to date with what we're planning and how that relates to type hints.

Right now I'm planning on shipping type hint support in v2 which likely won't be released until mid/late 2021. For now you can continue with type stubs and when the master branch becomes the "v2" branch we'll migrate your type stubs into inline type hints. This means that type stubs won't get released in v1.26 because our API will become a lot simpler when we drop Python 2.7 support. So for now:

You can continue as you're doing with .pyi files, we'll handle the rolling into inline type hints via a tool probably

Focus on Python 3.6+ types, don't worry about urllib3.packages.six or code that's written explicitly for Python 2 for example.

If you are discouraged by this I understand, we really value what you're doing and all hope you'll still be motivated to keep contributing. Let me know if you want to discuss more via email (sethmichaellarson@gmail.com)

@sethmlarson Thanks for sharing and the kind words. Re: plans, it's not a problem. Focus on 3.6+ types makes a lot of sense. Let me know if you have any further thoughts/comments on this PR.

sethmlarson

LGTM

savarin added 3 commits August 26, 2020 22:32

move from typeshed - exceptions, fields

f1fd1c0

correct .pyi for exceptions

b3db9ac

correct .pyi for fields

8429071

savarin commented Aug 27, 2020

View reviewed changes

sethmlarson requested changes Aug 27, 2020

View reviewed changes

Seth comments

88b856b

savarin requested a review from sethmlarson August 27, 2020 16:13

include all errors count

20d00cc

sethmlarson approved these changes Sep 4, 2020

View reviewed changes

sethmlarson merged commit 15f0f21 into urllib3:master Sep 4, 2020

savarin mentioned this pull request Sep 9, 2020

Include additional files to be typed checks, up to exceptions.py #1931

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Include exceptions.pyi and fields.pyi #1937

Include exceptions.pyi and fields.pyi #1937

savarin commented Aug 27, 2020

savarin commented Aug 27, 2020 •

edited

savarin Aug 27, 2020 •

edited

savarin Aug 27, 2020

sethmlarson left a comment

sethmlarson Aug 27, 2020

savarin Aug 27, 2020

sethmlarson Aug 27, 2020

savarin commented Aug 30, 2020 •

edited

codecov bot commented Aug 30, 2020 •

edited

sethmlarson commented Aug 31, 2020

savarin commented Sep 2, 2020

sethmlarson left a comment

		@@ -0,0 +1,55 @@
		from typing import Any, Optional, Union, Tuple, TYPE_CHECKING

		@@ -0,0 +1,26 @@
		# Stubs for requests.packages.urllib3.fields (Python 3.4)

		from typing import Any, Callable, Dict, Optional

Include exceptions.pyi and fields.pyi #1937

Include exceptions.pyi and fields.pyi #1937

Conversation

savarin commented Aug 27, 2020

savarin commented Aug 27, 2020 • edited

savarin Aug 27, 2020 • edited

Choose a reason for hiding this comment

savarin Aug 27, 2020

Choose a reason for hiding this comment

sethmlarson left a comment

Choose a reason for hiding this comment

sethmlarson Aug 27, 2020

Choose a reason for hiding this comment

savarin Aug 27, 2020

Choose a reason for hiding this comment

sethmlarson Aug 27, 2020

Choose a reason for hiding this comment

savarin commented Aug 30, 2020 • edited

codecov bot commented Aug 30, 2020 • edited

Codecov Report

sethmlarson commented Aug 31, 2020

savarin commented Sep 2, 2020

sethmlarson left a comment

Choose a reason for hiding this comment

savarin commented Aug 27, 2020 •

edited

savarin Aug 27, 2020 •

edited

savarin commented Aug 30, 2020 •

edited

codecov bot commented Aug 30, 2020 •

edited