Skip to content

Commit

Permalink
A test to demonstrate a misleading error for chunked response
Browse files Browse the repository at this point in the history
If a chunked response is truncated exactly before sending a new chunk
(that is, before the first byte of the chunk size), urllib3 throws
`InvalidChunkLength(got length b'\\n', 0 bytes read)`. This is
confusing as no chunk length was sent (a valid chunk length line would
also be \r\n-terminated and we didn't see those either).

In my opinion it would be better if urllib3 threw an IncompleteRead
error instead.

I've worked up a test that demonstrates the problem but I haven't
figured out the right fix yet. I hope this is at least helpful.
  • Loading branch information
rnewson committed Dec 23, 2022
1 parent 02f5690 commit 55cbf2c
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions test/test_response.py
Original file line number Diff line number Diff line change
Expand Up @@ -1020,6 +1020,22 @@ def test_incomplete_chunk(self) -> None:
orig_ex = ctx.value.args[1]
assert isinstance(orig_ex, httplib_IncompleteRead)

def test_truncated_before_chunk(self) -> None:
stream = [b""]
fp = MockChunkedNoChunks(stream)
r = httplib.HTTPResponse(MockSock) # type: ignore[arg-type]
r.fp = fp # type: ignore[assignment]
r.chunked = True
r.chunk_left = None
resp = HTTPResponse(
r, preload_content=False, headers={"transfer-encoding": "chunked"}
)
with pytest.raises(ProtocolError) as ctx:
next(resp.read_chunked())

orig_ex = ctx.value.args[1]
assert isinstance(orig_ex, IncompleteRead)

def test_invalid_chunk_length(self) -> None:
stream = [b"foooo", b"bbbbaaaaar"]
fp = MockChunkedInvalidChunkLength(stream)
Expand Down Expand Up @@ -1300,6 +1316,11 @@ def _encode_chunk(self, chunk: bytes) -> bytes:
return f"{len(chunk):X};asd=qwe\r\n{chunk.decode()}\r\n".encode()


class MockChunkedNoChunks(MockChunkedEncodingResponse):
def _encode_chunk(self, chunk: bytes) -> bytes:
return b""


class MockSock:
@classmethod
def makefile(cls, *args: typing.Any, **kwargs: typing.Any) -> None:
Expand Down

0 comments on commit 55cbf2c

Please sign in to comment.