BeautifulSoup is a third party Python library that is used to parse data from web pages. It helps in web scraping, which is a process of extracting, using, and manipulating the data from different resources.
Web scraping can also be used to extract data for research purposes, understand/compare market trends, perform SEO monitoring, and so on.
The below line can be run to install BeautifulSoup on Windows −
pip install beautifulsoup4
Following is an example −
Example
from bs4 import BeautifulSoup
import requests
url = "https://en.wikipedia.org/wiki/Algorithm"
req = requests.get(url)
soup = BeautifulSoup(req.text, "html.parser")
print("The href links are :")
for link in soup.find_all('a'):
print(link.get('href'))Output
The href links are : … https://stats.wikimedia.org/#/en.wikipedia.org https://foundation.wikimedia.org/wiki/Cookie_statement https://wikimediafoundation.org/ https://www.mediawiki.org/
Explanation
The required packages are imported, and aliased.
The website is defined.
The url is opened, and data is read from it.
The ‘BeautifulSoup’ function is used to extract text from the webpage.
The ‘find_all’ function is used to extract text from the webpage data.
The href links are printed on the console.