How to Scrape Web Data from Google using Python?

Prerequisites: Python Requests, Implementing Web Scraping in Python with BeautifulSoup Web scraping is a technique to fetch data from websites. While surfing on the web, many websites don’t allow the user to save data for personal use. One way is to manually copy-paste the data, which both tedious and time-consuming. Web Scraping is the automation of the data extraction process from websites. In this article, we will scrape the weather update from google's search result. Modules Required

• BeautifulSoup: This module is used for iterating, searching, and modifying the parse tree over the HTML or XML parser. To download it type the below command in the terminal.

pip install beautifulsoup4
       OR  
pip install bs4

• Requests: Requests library is one of the integral part of Python for making HTTP requests to a specified URL. To download it type the below command in the terminal.

pip install requests

Below is the implementation. 

import requests
from bs4 import BeautifulSoup

# Enter the City Name
city = input("Enter the City Name: ")
search = f"Weather in {city}"

# URL
url = f"https://www.google.com/search?q={search}"

# Sending HTTP request
req = requests.get(url)

# Pulling HTTP data from internet
sor = BeautifulSoup(req.text, "html.parser")

# Finding temperature in Celsius
temp = sor.find("div", class_='BNeawe').text

print(f'Temperature in {city} is {temp}')

Output :

akash_kumar_singh

Follow

Improve

Article Tags :

• Python
• Python Programs
• Python-requests
• Web-scraping
• Python web-scraping-exercises

+1 More

Practice Tags :

• python