Got more than 8190 bytes when reading Header value is too long.

[salmosri]

Hi Team,

When attempting to carry out a move of Spotify -> Google the following error appears which causes the app to hang (but not crash):

Listening on http://0.0.0.0:3132
Please open your browser window to http://localhost:3132
Gathering tracks for playlist Saved Tracks (784)
Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 559, in start
(message, payload) = yield from self._protocol.read()
File "/usr/local/lib/python3.6/site-packages/aiohttp/streams.py", line 509, in read
yield from self._waiter
File "/usr/local/lib/python3.6/site-packages/aiohttp/client_proto.py", line 165, in data_received
messages, upgraded, tail = self._parser.feed_data(data)
File "aiohttp/_http_parser.pyx", line 274, in aiohttp._http_parser.HttpParser.feed_data (aiohttp/_http_parser.c:4364)
File "aiohttp/_http_parser.pyx", line 382, in aiohttp._http_parser.cb_on_header_value (aiohttp/_http_parser.c:6391)
aiohttp.http_exceptions.LineTooLong: 400, message='Got more than 8190 bytes when reading Header value is too long.'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 422, in start
resp = yield from self._request_handler(request)
File "/usr/local/lib/python3.6/site-packages/aiohttp/web.py", line 306, in _handle
resp = yield from handler(request)
File "/usr/local/lib/python3.6/site-packages/pyportify/middlewares.py", line 18, in index_handler
ret = yield from handler(request)
File "/usr/local/lib/python3.6/site-packages/pyportify/app.py", line 117, in transfer_start
yield from transfer_playlists(request, s, g, lists)
File "/usr/local/lib/python3.6/site-packages/pyportify/app.py", line 169, in transfer_playlists
done = yield from asyncio.gather(*tasks)
File "/usr/local/lib/python3.6/site-packages/pyportify/app.py", line 242, in search_gm_track
track = yield from g.find_best_track(search_query)
File "/usr/local/lib/python3.6/site-packages/pyportify/google.py", line 56, in find_best_track
data = yield from self.search_all_access(search_query)
File "/usr/local/lib/python3.6/site-packages/pyportify/google.py", line 48, in search_all_access
'ct': '1,2,3,4,6,7,8,9',
File "/usr/local/lib/python3.6/site-packages/pyportify/google.py", line 113, in _http_get
params=merged_params,
File "/usr/local/lib/python3.6/site-packages/aiohttp/helpers.py", line 97, in iter
ret = yield from self._coro
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 241, in _request
yield from resp.start(conn, read_until_eof)
File "/usr/local/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 564, in start
message=exc.message, headers=exc.headers) from exc
aiohttp.client_exceptions.ClientResponseError: 400, message='Got more than 8190 bytes when reading Header value is too long.'

==============

This issue exists while using the install method of brew or downloading the dmg and running the pyportify-copyall script.

I am using MacOS Sierra 10.12.6

Let me know if there is anything i can do to resolve this or provide more information!

Cheers,
Shadi

[rckclmbr]

It looks like aiohttp limits header length to 8190 in their response parser. I don't see any easy way in their API to change this length, since this is hidden behind a few layers of abstraction. I'll dig into this and see what I can do.

There's a C and python parser, but they both have this param.

https://github.com/aio-libs/aiohttp/blob/e7c9390111932dd6dbb642170d7a0da1876271ec/aiohttp/http_parser.py#L59

[robhazan]

I'm encountering the same error on Ubuntu. Any debugging information I can provide to help get this resolved? Anything I can change to avoid this error?

[robhazan]

@rckclmbr - friendly ping to see if there's anything that can be done about this. Any workarounds?

[rckclmbr]

@robhazan there's not much I can do about this short of moving away from aiohttp, since aiohttp doesn't support what we need it to. We need to wait until aio-libs/aiohttp#2304 is resolved (they set the milestone for 3.1) then we can work around this.

The other approach is to figure out why the header is exceeding 8190 -- it's not all users, and I can't troubleshoot it since it's not happening to me. Maybe there's something we can do to make google not have such a large header, if we could find what it actually is.

The hackish solution is to manually edit aiohttp to allow a longer header. I don't think it's a good idea to make a release with that though, for a variety of reasons...

[dIgor93]

Hello!
I found some solution for same problem. Pass the handler_args parameter when creating app. It will pass to parser.

web.Application(handler_args={'max_field_size': 16380})

[magnus919]

Running into this years later with Home Assistant.