Script for scraping e-sport betting information from website https://loot.bet/ (for educational purposes). (Here You can find some examples of using css selectors and selenium webdriver to obtain information from dynamic generated web pages).
Before running this script, you'll need to install next packets: selenium(with geckodriver if on Windows), bs4 and Firefox web browser.
After importing this script into your own code - use 'crawl_site()' function to start scraping. Obtained information will return as list of lists of dictionaries, where dictionary represent information about single game event, inner list - game events from one site page, outer list - all events. Here is example:
[[{'game': name of the game, 'date': date of the event, 'player1': name of the first participant,'player2': second participant's name, 'odds1': bet rates on 1st player, 'odds2': bet rates on 2nd player},
{another game information},
........
],
[{}, {}, {}, ....]
........
]
You can run this script from the command line using parameter 'show' to print scraped data into terminal window(also parameter 'help' available).
This script will save scraped data by "lootbet_scraper.py" from web site "Loot.bet" to .csv file 'file_name'. Use your own parameters of 'delimiter' and 'quoting' or delete them for standard output.
Scrapy spider for scraping bet information on esport events from 'loot.bet' website(for educational purposes)
This script using Selenium to parse dynamic generated content.
Before running spider - install 'scrapy' packets (also Selenium and Firefox browser). On Windows machine you will need to have appropriate C++ SDK (check Twisted documentation) and geckodriver.
Put this script into 'spiders' folder of your scrapy project and run command(from project directory):
'scrapy crawl lootbet'
Returning dictionary:
{ 'game': name of the game,
'date': date, he the game will be played,
'player1': first participant's name,
'odds1': bet rate on first participant,
'player2': second participant's name,
'odds2': bet rate on second participant
}