Read Html File In Python Using Pandas
Last Updated :
28 Apr, 2025
In Python, Pandas is a powerful library commonly used for data manipulation and analysis. While it's primarily used for working with structured data such as CSV files, Excel spreadsheets, and databases, it's also capable of reading HTML files and extracting tabular data from them. In this article, we'll explore how to read an HTML file in Python using Pandas, along with practical examples and explanations.
Read HTML Files in Python Using Pandas
Below are the possible approaches to Read HTML Files in Python Using Pandas.
- Using read_html() Function
- Using BeautifulSoup with read_html()
- Using requests with read_html()
- Using lxml parser with read_html()
Read HTML Files Using read_html() Function
This approach directly uses the read_html() function provided by pandas. This function is specifically designed to parse HTML tables and return a list of DataFrames corresponding to the tables found in the HTML content. It's a convenient method when dealing with simple HTML files containing tabular data.
Python3
import pandas as pd
def read_html_with_read_html(file_path):
# Read HTML file into DataFrame using read_html()
df = pd.read_html(file_path)[0]
return df
# File path
html_file_path = 'data/geeks_for_geeks.html'
# Read HTML file using read_html() function
df = read_html_with_read_html(html_file_path)
# Display DataFrame
print("Approach 1 Output:")
print(df)
<!DOCTYPE html>
<html>
<head>
<title>Table Example</title>
</head>
<body>
<table border="1">
<tr>
<th>Name</th>
<th>Topic</th>
<th>Difficulty</th>
</tr>
<tr>
<td>Introduction to Python</td>
<td>Python</td>
<td>Beginner</td>
</tr>
<tr>
<td>Data Structures</td>
<td>Algorithms</td>
<td>Intermediate</td>
</tr>
<tr>
<td>Machine Learning Basics</td>
<td>Machine Learning</td>
<td>Advanced</td>
</tr>
</table>
</body>
</html>
Output:
Approach 1 Output: Name Topic Difficulty
0 Introduction to Python Python Beginner
1 Data Structures Algorithms Intermediate
2 Machine Learning Basics Machine Learning Advanced
Read HTML Files Using BeautifulSoup with read_html()
In this approach, we first use the BeautifulSoup library to parse the HTML file and extract tables from it. BeautifulSoup provides more flexibility in navigating and extracting specific elements from HTML documents. We then pass the extracted tables to the read_html() function to convert them into DataFrames.
Python3
import pandas as pd
from bs4 import BeautifulSoup
def read_html_with_beautiful_soup(file_path):
# Read HTML file
with open(file_path, 'r') as f:
# Parse HTML using BeautifulSoup
soup = BeautifulSoup(f, 'html.parser')
# Find all tables in the HTML
tables = soup.find_all('table')
# Read tables into DataFrame using read_html()
df = pd.read_html(str(tables))[0]
return df
# File path
html_file_path = 'data/geeks_for_geeks.html'
# Read HTML file using BeautifulSoup with read_html()
df = read_html_with_beautiful_soup(html_file_path)
# Display DataFrame
print("Approach 2 Output:")
print(df)
<!DOCTYPE html>
<html>
<head>
<title>Programming Languages</title>
</head>
<body>
<table border="1">
<tr>
<th>Code</th>
<th>Language</th>
<th>Difficulty</th>
</tr>
<tr>
<td>HTML</td>
<td>HTML/CSS</td>
<td>Beginner</td>
</tr>
<tr>
<td>Python</td>
<td>Python</td>
<td>Intermediate</td>
</tr>
<tr>
<td>JavaScript</td>
<td>JavaScript</td>
<td>Advanced</td>
</tr>
</table>
</body>
</html>
Output:
Approach 2 Output: Code Language Difficulty
0 HTML HTML/CSS Beginner
1 Python Python Intermediate
2 JavaScript JavaScript Advanced
Read HTML Files Using requests with read_html()
This approach involves fetching the HTML content from a URL using the requests library and then passing the content directly to the read_html() function for parsing. It's useful when the HTML content is available online and can be accessed via URL. This approach enables automation in data retrieval and is suitable for reading data from remote sources. However, it requires an internet connection to fetch HTML content, dependency on external servers for data retrieval, and potential security risks when fetching data from untrusted sources.
Python3
import pandas as pd
import requests
def read_html_with_requests(file_url):
# Fetch HTML content using requests
response = requests.get(file_url)
# Read HTML content into DataFrame using read_html()
df = pd.read_html(response.content)[0]
return df
# File URL
html_file_url = 'https://media.geeksforgeeks.org/wp-content/uploads/20240213175028/geeks_for_geeks.html'
# Read HTML file using requests with read_html()
df = read_html_with_requests(html_file_url)
# Display DataFrame
print("Approach 3 Output:")
print(df)
<!DOCTYPE html>
<html>
<head>
<title>Topics in Different Categories</title>
</head>
<body>
<table border="1">
<tr>
<th>Category</th>
<th>Topic</th>
<th>Difficulty</th>
</tr>
<tr>
<td>Data Structures</td>
<td>Algorithms</td>
<td>Beginner</td>
</tr>
<tr>
<td>Web Development</td>
<td>HTML/CSS</td>
<td>Intermediate</td>
</tr>
<tr>
<td>Machine Learning</td>
<td>Python</td>
<td>Advanced</td>
</tr>
</table>
</body>
</html>
Output:
Approach 3 Output: Category Topic Difficulty
0 Data Structures Algorithms Beginner
1 Web Development HTML/CSS Intermediate
2 Machine Learning Python Advanced
Read HTML Files Using lxml parser with read_html()
In the approach, we use the lxml parser in the read_html() function to parse the HTML file. XMLlxml parser is known for its speed and ability to handle large HTML files efficiently. This approach is suitable for cases where performance is a concern or when dealing with large HTML files. While it offers fast and efficient parsing and good performance, especially with large datasets, it requires additional installation of the lxml library and has limited control over parsing compared to BeautifulSoup.
Python3
import pandas as pd
# Approach 4: Using lxml parser with read_html()
def read_html_with_lxml(file_path):
# Read HTML file into DataFrame using read_html() with 'lxml' parser
df = pd.read_html(file_path, flavor='lxml')[0]
return df
# File path
html_file_path = 'data/geeks_for_geeks.html'
# Read HTML file using lxml parser with read_html()
df = read_html_with_lxml(html_file_path)
# Display DataFrame
print("Approach 4 Output:")
print(df)
<!DOCTYPE html>
<html>
<head>
<title>Book Information</title>
</head>
<body>
<table border="1">
<tr>
<th>Title</th>
<th>Author</th>
<th>Difficulty</th>
</tr>
<tr>
<td>Python Basics</td>
<td>John Doe</td>
<td>Beginner</td>
</tr>
<tr>
<td>Data Analysis</td>
<td>Jane Smith</td>
<td>Intermediate</td>
</tr>
<tr>
<td>Machine Learning Algorithms</td>
<td>David Johnson</td>
<td>Advanced</td>
</tr>
</table>
</body>
</html>
Output:
Approach 4 Output: Title Author Difficulty
0 Python Basics John Doe Beginner
1 Data Analysis Jane Smith Intermediate
2 Machine Learning Algorithms David Johnson Advanced
Conclusion
In conclusion, Pandas provides multiple methods to read HTML files in Python, offering flexibility based on specific requirements. The read_html() function is a straightforward option for parsing simple HTML tables. Alternatively, utilizing BeautifulSoup allows for more control over complex HTML structures. For web-based content, the integration of requests enables fetching HTML from URLs, while the use of the lxml parser enhances performance with large datasets.
Similar Reads
Read JSON file using Python
The full form of JSON is JavaScript Object Notation. It means that a script (executable) file which is made of text in a programming language, is used to store and transfer the data. Python supports JSON through a built-in package called JSON. To use this feature, we import the JSON package in Pytho
4 min read
Read File As String in Python
Python provides several ways to read the contents of a file as a string, allowing developers to handle text data with ease. In this article, we will explore four different approaches to achieve this task. Each approach has its advantages and uses cases, so let's delve into them one by one. Read File
3 min read
Read a Parquet File Using Pandas
Parquet is a columnar storage format that is optimized for distributed processing of large datasets. It is widely used in Big Data processing systems like Hadoop and Apache Spark. A partitioned parquet file is a parquet file that is partitioned into multiple smaller files based on the values of one
5 min read
Pandas Read CSV in Python
CSV files are the Comma Separated Files. It allows users to load tabular data into a DataFrame, which is a powerful structure for data manipulation and analysis. To access data from the CSV file, we require a function read_csv() from Pandas that retrieves data in the form of the data frame. Here’s a
6 min read
How to import an excel file into Python using Pandas?
It is not always possible to get the dataset in CSV format. So, Pandas provides us the functions to convert datasets in other formats to the Data frame. An excel file has a '.xlsx' format. Before we get started,  we need to install a few libraries. pip install pandas pip install xlrd  For importin
2 min read
Extracting rows using Pandas .iloc[] in Python
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages that makes importing and analyzing data much easier. here we are learning how to Extract rows using Pandas .iloc[] in Python.Pandas .iloc[
7 min read
Reading CSV files in Python
A CSV (Comma Separated Values) file is a form of plain text document that uses a particular format to organize tabular information. CSV file format is a bounded text document that uses a comma to distinguish the values. Every row in the document is a data log. Each log is composed of one or more fie
5 min read
Pandas Extracting rows using .loc[] - Python
Pandas provide a unique method to retrieve rows from a Data frame. DataFrame.loc[] method is a method that takes only index labels and returns row or dataframe if the index label exists in the caller data frame. To download the CSV used in code, click here.Example: Extracting single Row In this exam
3 min read
Convert CSV to Excel using Pandas in Python
Pandas can read, filter, and re-arrange small and large datasets and output them in a range of formats including Excel. In this article, we will be dealing with the conversion of .csv file into excel (.xlsx). Pandas provide the ExcelWriter class for writing data frame objects to excel sheets. Syntax
1 min read
Data Manipulation in Python using Pandas
In Machine Learning, the model requires a dataset to operate, i.e. to train and test. But data doesn’t come fully prepared and ready to use. There are discrepancies like Nan/ Null / NA values in many rows and columns. Sometimes the data set also contains some of the rows and columns which are not ev
6 min read