Scribd
Upload
3 views

how to load dataset in your python program

The document provides methods for loading various types of datasets in Python, including CSV, Excel, JSON, and SQL databases, primarily using the pandas library. It also outlines how to iterate through data line by line for different formats, such as using iterrows() for pandas DataFrames, csv.reader for CSV files, and basic loops for text and JSON files. Additionally, it explains how to fetch and iterate through results from a SQL database using cursor.fetchall().

Uploaded by

idehennosa231
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
3 views

how to load dataset in your python program

The document provides methods for loading various types of datasets in Python, including CSV, Excel, JSON, and SQL databases, primarily using the pandas library. It also outlines how to iterate through data line by line for different formats, such as using iterrows() for pandas DataFrames, csv.reader for CSV files, and basic loops for text and JSON files. Additionally, it explains how to fetch and iterate through results from a SQL database using cursor.fetchall().

Uploaded by

idehennosa231
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

how to load dataset in your python program

ChatGPT said:

Loading a dataset in a Python program depends on the type and format of the dataset (e.g., CSV,
Excel, JSON, SQL, etc.). Here's a general overview using the popular pandas library, which is
great for data manipulation:

🔹 1. CSV File
python
CopyEdit
import pandas as pd

# Load CSV file


df = pd.read_csv('your_file.csv')

# Display the first few rows


print(df.head())

🔹 2. Excel File (.xlsx)


python
CopyEdit
import pandas as pd

# Load Excel file


df = pd.read_excel('your_file.xlsx')

print(df.head())

🔹 3. JSON File
python
CopyEdit
import pandas as pd

# Load JSON file


df = pd.read_json('your_file.json')

print(df.head())

🔹 4. From a URL
python
CopyEdit
import pandas as pd

url = 'https://example.com/data.csv'
df = pd.read_csv(url)

print(df.head())

🔹 5. SQL Database
python
CopyEdit
import pandas as pd
import sqlite3

# Connect to database
conn = sqlite3.connect('your_database.db')

# Load table into DataFrame


df = pd.read_sql_query("SELECT * FROM your_table", conn)

print(df.head())

🔹 6. Using NumPy (for .txt or numerical arrays)


python
CopyEdit
import numpy as np

# Load a text file


data = np.loadtxt('your_file.txt')

print(data[:5])

If you're working with a specific type of dataset or file format, let me know and I can tailor the
code to fit that exactly. Do you already have a dataset you're working with?

You said:

how to iterae through the data line by line


ChatGPT said:

To iterate through data line by line, the approach depends on the type of data structure you're
working with. Below are examples for different types of datasets:

🔹 1. CSV File (Using pandas)

You can iterate through each row in the DataFrame after loading it using pandas:
python
CopyEdit
import pandas as pd

# Load the dataset


df = pd.read_csv('your_file.csv')

# Iterate through each row


for index, row in df.iterrows():
print(row) # This gives you the entire row as a pandas Series
# Access individual column values like this:
# print(row['column_name'])

 iterrows() returns each row as an index and Series object.


 Be cautious using iterrows() on large datasets, as it can be slow.

🔹 2. CSV File (Without pandas, using csv module)


python
CopyEdit
import csv

# Open the CSV file


with open('your_file.csv', newline='', encoding='utf-8') as file:
reader = csv.reader(file)

# Iterate through the file line by line


for row in reader:
print(row) # Each row is a list of values

 This method gives you each row as a list, which you can then access by index.

🔹 3. Text File

If your data is in a simple text file (line-by-line format), you can read it using a basic loop:

python
CopyEdit
# Open the text file
with open('your_file.txt', 'r') as file:
for line in file:
print(line.strip()) # Use .strip() to remove any trailing newlines or
spaces

 This will read each line of the file as a string, and you can then manipulate it as needed.
🔹 4. JSON File (Iterating over JSON objects)

If your JSON data is an array of objects, you can iterate through it as follows:

python
CopyEdit
import json

# Load JSON file


with open('your_file.json', 'r') as file:
data = json.load(file)

# Iterate through the JSON array


for item in data:
print(item) # Each 'item' is a dictionary

 Each item is a dictionary if your JSON is an array of key-value pairs.

🔹 5. Excel File (Using pandas)

If you're working with an Excel file, you can iterate over rows similarly to a CSV file:

python
CopyEdit
import pandas as pd

# Load Excel file


df = pd.read_excel('your_file.xlsx')

# Iterate through each row


for index, row in df.iterrows():
print(row)

This works for Excel files in .xls or .xlsx formats.

🔹 6. SQL Database (Iterating through query results)

If you're pulling data from a SQL database, here's how you can iterate through rows:

python
CopyEdit
import sqlite3

# Connect to the database


conn = sqlite3.connect('your_database.db')
cursor = conn.cursor()
# Execute a query
cursor.execute("SELECT * FROM your_table")

# Iterate through the results


for row in cursor.fetchall():
print(row) # Each 'row' is a tuple

 fetchall() retrieves all rows, and you can iterate through the list of tuples.

Summary:

 Use iterrows() for iterating through rows of a pandas DataFrame.


 Use csv.reader for iterating through a CSV file.
 Use a simple for loop for text files or JSON files.
 Use cursor.fetchall() for iterating through results from a SQL database.

Let me know if you need help with a specific dataset format!

You might also like