Getting the Data
Step 1: In Python, we have Beautiful Soup which is a library to pull out data from HTML files. To install Beautiful Soup, run a simple command;
pip install beautifulsoup4
Similarly, install the Requests module of Python.
pip install requests
We would use the NDTV Sports Cricket Scorecard to fetch the data.
Step 3: Following are the steps for Scraping data from the Web Page. To get the HTML text from the web page;
html_text = requests.get(‘https://sports.ndtv.com/cricket/live-scores’).text
To represent the parsed object as a whole we use the BeautifulSoup object,
soup = BeautifulSoup(html_text, "html.parser")
Note: It is recommended to run and check the code after each step to know about the difference and thoroughly understand the concepts.
Example:
Python
from bs4 import BeautifulSoup import requests html_text = requests.get('https://sports.ndtv.com/cricket/live-scores').text soup = BeautifulSoup(html_text, "html.parser") print(soup) |
We will further find all the required divs and other tags with their respective classes.
Python
from bs4 import BeautifulSoup import requests html_text = requests.get('https://sports.ndtv.com/cricket/live-scores').text soup = BeautifulSoup(html_text, "html.parser") sect = soup.find_all('div', class_='sp-scr_wrp') section = sect[0] description = section.find('span', class_='description').text location = section.find('span', class_='location').text current = section.find('div', class_='scr_dt-red').text link = "https://sports.ndtv.com/" + \ section.find('a', class_='scr_ful-sbr-txt').get('href') |
The next section of the code has our data that is our result. If for any of the reasons that code is not present in the HTML file, it would lead to an error, so including that part in a try and except block.
Complete Code:
Python3
from bs4 import BeautifulSoup import requests html_text = requests.get('https://sports.ndtv.com/cricket/live-scores').text soup = BeautifulSoup(html_text, "html.parser") sect = soup.find_all('div', class_='sp-scr_wrp ind-hig_crd vevent') section = sect[0] description = section.find('span', class_='description').text location = section.find('span', class_='location').text current = section.find('div', class_='scr_dt-red').text link = "https://sports.ndtv.com/" + section.find( 'a', class_='scr_ful-sbr-txt').get('href') try: status = section.find_all('div', class_="scr_dt-red")[1].text block = section.find_all('div', class_='scr_tm-wrp') team1_block = block[0] team1_name = team1_block.find('div', class_='scr_tm-nm').text team1_score = team1_block.find('span', class_='scr_tm-run').text team2_block = block[1] team2_name = team2_block.find('div', class_='scr_tm-nm').text team2_score = team2_block.find('span', class_='scr_tm-run').text print(description) print(location) print(status) print(current) print(team1_name.strip()) print(team1_score.strip()) print(team2_name.strip()) print(team2_score.strip()) print(link) except: print("Data not available") |
Output:
Live score England vs India 3rd Test,Pataudi Trophy, 2021
Headingley, Leeds
England lead by 223 runs
Day 2 | Post Tea Session
England
301/3 (96.0)
India
78
https://sports.ndtv.com//cricket/live-scorecard/england-vs-india-3rd-test-leeds-enin08252021199051
Create Cricket Score API using Web Scraping in Flask
Cricket is one of the famous outdoor sport played worldwide. There are very few APIs providing live scoreboards and none of them are free to use. Using any of the scoreboards available we can create API for ourselves. This method not only works for Cricket Scoreboard but also for any information available online. Following is the flow in which this blog would guide to create an API and deploy it.
- Setting up the App Directory
- Web Scrape data from NDTV Sports.
- Beautiful Soup in Python would be used.
- Create an API.
- Flask would be used.
- Heroku would be used for deployment,
Contact Us