Scribd
Upload
17 views4 pages

Flask

The document describes steps to automate the download of finance data from multiple companies, load it into a SQLite database, and create APIs to retrieve and update the data. The steps include: 1. Defining companies and URLs in a config file 2. Scraping data for each company and storing in a DataFrame 3. Merging DataFrames and loading to a SQLite database 4. Creating a trigger to update rows if they already exist 5. Creating Flask APIs to get data by date, company/date, company, and update rows

Uploaded by

KSHITIZ GUPTA
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
17 views4 pages

Flask

The document describes steps to automate the download of finance data from multiple companies, load it into a SQLite database, and create APIs to retrieve and update the data. The steps include: 1. Defining companies and URLs in a config file 2. Scraping data for each company and storing in a DataFrame 3. Merging DataFrames and loading to a SQLite database 4. Creating a trigger to update rows if they already exist 5. Creating Flask APIs to get data by date, company/date, company, and update rows

Uploaded by

KSHITIZ GUPTA
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 4

Data Load -

1. Download the finance data from five companies of your choice:


Sample URL for IBM - https://finance.yahoo.com/quote/IBM/history?p=IBM

2. Download should be automated via code.


a. The list of companies should be configurable using a config file.
b. This is to ensure that there is flexibility to add more companies
with minimal code changes

3. The data should be loaded to a single table in a DB (sqllite, postgres, my


sql).

4. You can add more columns if you need.

5. Data can be inserted or updated if the same rows already exist.


You can write a SQL function to accomplish this.

Solution :-

1. Defining a list of companies and their URLs in a configuration file. You can use
the configparser module to read the configuration file. Here's an example of what
your configuration file might look like:
[companies]
IBM = https://finance.yahoo.com/quote/IBM/history?p=IBM
AAPL = https://finance.yahoo.com/quote/AAPL/history?p=AAPL
MSFT = https://finance.yahoo.com/quote/MSFT/history?p=MSFT
GOOG = https://finance.yahoo.com/quote/GOOG/history?p=GOOG
AMZN = https://finance.yahoo.com/quote/AMZN/history?p=AMZN

2. Here writing a Python code that will read the configuration file and scrapes
the finance data for each company. For each company, you can extract the data using
BeautifulSoup and store it in a pandas DataFrame. Here's an example of how you can
scrape the data for IBM:

import requests
from bs4 import BeautifulSoup
import pandas as pd
import configparser
import sqlite3

# read the configuration file


config = configparser.ConfigParser()
config.read('config.ini')

# scrape the data for IBM


url = config['companies']['IBM']
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')
table = soup.find('table')
df = pd.read_html(str(table))[0]
df['company'] = 'IBM'

3. Merging the data for all companies into a single DataFrame and store it in a
SQLite database.

dfs = []
for company, url in config['companies'].items():
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')
table = soup.find('table')
df = pd.read_html(str(table))[0]
df['company'] = company
dfs.append(df)
df = pd.concat(dfs)

# store the data in a SQLite database


conn = sqlite3.connect('finance_data.db')
df.to_sql('finance_data', conn, if_exists='replace')
conn.close()

4. Writing a SQL function that updates data in the database if the same rows
already exist. You can use a trigger to accomplish this.

conn = sqlite3.connect('finance_data.db')
cur = conn.cursor()
cur.execute('''
CREATE TRIGGER IF NOT EXISTS update_finance_data
AFTER INSERT ON finance_data
BEGIN
UPDATE finance_data
SET open = new.open,
high = new.high,
low = new.low,
close = new.close,
adj_close = new.adj_close,
volume = new.volume
WHERE date = new.date AND company = new.company;
END;
''')
conn.close()

5. With these steps, you can automate the download of finance data for multiple
companies and store it in a SQLite database.

Data APIs-

1. Write below APIs end points using flask or fast framework:


a. Get all companies’ stock data for a particular day (Input to API would
be date)
b. Get all stock data for a particular company for a particular day (Input
to API would be company ID/name and date)
c. Get all stock data for a particular company (Input to API would be
company ID/name)
d. POST/Patch API to update stock data for a company by date.

Solution :-

1. Create a Flask app and import the required modules:

from flask import Flask, jsonify, request


import sqlite3
2. Define the routes for each API endpoint:

app = Flask(__name__)

@app.route('/stocks/<date>')
def get_all_stocks_for_date(date):
conn = sqlite3.connect('finance_data.db')
cur = conn.cursor()
cur.execute('SELECT * FROM finance_data WHERE date = ?', (date,))
rows = cur.fetchall()
conn.close()
return jsonify(rows)

@app.route('/stocks/<company>/<date>')
def get_stock_for_company_and_date(company, date):
conn = sqlite3.connect('finance_data.db')
cur = conn.cursor()
cur.execute('SELECT * FROM finance_data WHERE company = ? AND date = ?',
(company, date))
rows = cur.fetchall()
conn.close()
return jsonify(rows)

@app.route('/stocks/<company>')
def get_all_stocks_for_company(company):
conn = sqlite3.connect('finance_data.db')
cur = conn.cursor()
cur.execute('SELECT * FROM finance_data WHERE company = ?', (company,))
rows = cur.fetchall()
conn.close()
return jsonify(rows)

@app.route('/stocks/<company>/<date>', methods=['POST', 'PATCH'])


def update_stock_for_company_and_date(company, date):
data = request.get_json()
conn = sqlite3.connect('finance_data.db')
cur = conn.cursor()
cur.execute('''
UPDATE finance_data
SET open = ?,
high = ?,
low = ?,
close = ?,
adj_close = ?,
volume = ?
WHERE company = ? AND date = ?
''', (data['open'], data['high'], data['low'], data['close'],
data['adj_close'], data['volume'], company, date))
conn.commit()
conn.close()
return 'Success', 200

if __name__ == '__main__':
app.run()

3. With these steps, you can create APIs that allow you to retrieve and update
finance data for multiple companies using Flask.

You might also like