[Bug] Brave error ?

[Rayzen-dev]

Feature Request or Bug or Another
Feature: Bug

Describe the feature request or bug or other
When I try to run python3 theHarvester.py -d exemple.com -b all, I have an error with brave:

An exception has occurred: 400, message='Got more than 8190 bytes (8858) when reading Header value is too long.', url=URL('https://search.brave.com/search?q=%22exemple.com%22&offset=0&source=web&show_local=0&spellcheck=0')

System Information (System that tool is running on):

• OS: Parrot OS
• Version: latest

[NotoriousRebel]

I'm assuming they changed something in the backend to cause this error, I'll have to debug it this week and figure out the root cause.

[NotoriousRebel]

After doing some digging this has been a known bug in aiohttp that has been talked about for close to 6 years. It looks like a solution has been implemented and merged. However, it is not yet in the latest version of aiohttp that you can install via pip, so until that is published this error will continue unless brave changes something in the backend to shrink their reply headers. May have to look into patching it in the meantime or hoping the new release comes soon or make a special exception and use requests or httpx temporarily for brave until this gets figured out. There is also an API for brave search which may yield more results so that is also another option.

[Rayzen-dev]

After doing some digging this has been a known bug in aiohttp that has been talked about for close to 6 years. It looks like a solution has been implemented and merged. However, it is not yet in the latest version of aiohttp that you can install via pip, so until that is published this error will continue unless brave changes something in the backend to shrink their reply headers. May have to look into patching it in the meantime or hoping the new release comes soon or make a special exception and use requests or httpx temporarily for brave until this gets figured out. There is also an API for brave search which may yield more results so that is also another option.

Thanks for your reply, I will try soon with this informations.

[pdblood]

Looks like this is not in aiohttp 3.9.1. Does anyone know what release this fix will appear in?

[NotoriousRebel]

Did not realize this was fixed, I'll update Brave this weekend.

[L1ghtn1ng]

fixed in git