Web Scraping CryptoCurrency price and storing it in MongoDB using Python
Last Updated :
10 Nov, 2022
Let us see how to fetch history price in USD or BTC, traded volume and market cap for a given date range using Santiment API and storing the data into MongoDB collection.
Python is a mature language and getting much used in the Cryptocurrency domain. MongoDB is a NoSQL database getting paired with Python in many projects which helps to hold details that got retrieved from Python Programs. PyMongo is a Python distribution containing tools for working with MongoDB and it is a very convenient way to work with MongoDB from Python to do Create/Update/Delete/Read operations easily.
Let us see the code in using Santiment's API in getting cryptocurrency prices for a given date range
Few Examples of cryptocurrency are :
bitcoin
ethereum
ripple
bitcoin-cash
litecoin
eos
cardano
stellar
neo
iota
- In the place of id, we can pass bitcoin/ethereum or any cryptocurrency name which the below code can parse and get the data
- For valid cryptocurrency name, data will be retrieved properly
- In the place of from_date and to_date valid dates in valid date formats in yyyy-mm-dd pattern need to be given. For understanding purpose, it is given to take 7 days of data. We can able to get 1 month old data too. API call can able to get data for the given cryptocurrencies in the given date range or else if not available (due to invalid cryptocurrency name/invalid date range specification), we can catch the errors
- In the below program, for easier understanding, taking bitcoin and ethereum in "IndexCoins.idx" file and hence in the place of id, they are passed
Example :
For the following input

For bitcoin above API call, returns data as below

Similarly for ethereum also, we will get value
Retrieved value has to be stored in MongoDB
Database : geeksforgeeks
Collection : Coll_santiment_Price
As we are going to take for 7 days of data, it has to be given in a loop
In MongoDB, _id is the default column which can be got by ObjectId(). Our process has multiple rows where each and every row is identified by means of "cryptocurrencyname" and "time". In our code, we are having it as "id" and "time" respectively.
Let "Coll_santiment_Price" be created with 3 columns namely _id, id and time first. Then upon successful execution of API call, for the id and time, let API call output namely (verify column name from bitcoinprice.png) 'priceBtc', 'priceUsd','volume' and 'marketcap' are updated in a loop.
Code Implementation in Python
Python3
# importing the modules
from datetime import date, timedelta, datetime
from pymongo import MongoClient
import san
from bson.objectid import ObjectId
client = MongoClient('mongodb://localhost:27017/')
db1 = client.geeksforgeeks
data = {}
with open("IndexCoins.idx") as openfileobject:
for line in openfileobject:
# One by one File's cryptocurrency names are read
id = line.strip()
try:
print(id)
# Collecting the data for 7 days and hence range(7) is taken
for idx in range(7):
daystr = str(date.today() - timedelta(days = idx))
# Coll_santiment_Price collection documents need to be created
# It will have columns namely "_id", "time", "id",
# "priceBtc", "priceUsd", "volume", "marketcap"
data['id'] = id
data['time'] = daystr
# _id column for unique key and can be generated by using ObjectId()
data['_id'] = ObjectId()
# Initially create 3 columns in collection as rest of the
# columns are updated after running santiment api call
db1.Coll_santiment_Price.insert(data)
try:
# Santiment API call to get Cryptocurrency prices
daa = san.get("prices/" + id,
from_date = "2020-08-20",
to_date = "2020-08-27",
interval = "1d")
# API call output and for bitcoin it is given in
# https://media.geeksforgeeks.org/wp-content/uploads/20200827191739/bitcoinprice.png
print(daa)
except:
print("URL error")
continue;
# 7 days output
for idx in range(7):
daystr = str(date.today() - timedelta(days=idx))
# API call output for the above chosen date
row = daa.loc[daystr]
# priceBtc, priceUsd, volume and marketcap are
# collected and stored in separate variables
priceBtc = row['priceBtc']
priceUsd = row['priceUsd']
volume = row['volume']
marketcap = row['marketcap']
print(id, daystr, priceBtc, priceUsd, volume, marketcap)
try:
# Update the collection with above details against
# cryptocurrency id and time i.e. bitcoin and 2020-08-27
db1.Coll_santiment_Price.update(
{'time': daystr, 'id': id},
{"$set": {"priceBtc": priceBtc,
"priceUsd": priceUsd,
"volume": volume,
"marketcap": marketcap,
}
},
upsert = True
)
except Exception as e:
print(e)
except:
print("Error")
Sample Output : (For bitcoin)

Sample Output : (For Ethereum)

Santiment API calls for getting prices are very much used across cryptocurrency projects. As we are getting historical data, for data analytics, this is very much useful. Moreover, it is freely accessible and hence any beginner-level projects can use it without any issues. Price variations are not much varying and hence well set for demo and small scale projects.
Similar Reads
Get Top 100 Cryptocurrencies Ranking Using Python
In this article, we are going to see how to get the top 100 coins with Python. This article will provide an overview of interacting with the Live Coin Watch cryptocurrency data API using the Python programming language. By following this tutorial, users will be able to print a ranked list of the top
6 min read
Web Scraping for Stock Prices in Python
Web scraping is a data extraction method that collects data only from websites. It is often used for data mining and gathering valuable insights from large websites. Web scraping is also useful for personal use. Python includes a nice library called BeautifulSoup that enables web scraping. In this a
6 min read
Get Silver and Gold Price in Tkinter Using Python
Prerequisite: BeautifulSoup, requests, tkinter, datetime In this article, we are going to discuss how to get Gold and Silver Price in India Using Python in Tkinter. There is a variation in price with time to time, price of gold and silver is moved by a combination of supply, demand, and investor beh
4 min read
Create And Deploy A Stock Price Web Application using Python and Streamlit
In this article, we are going to see how to create and deploy a stock price web application. To create an amazing Web Application that deals with Data Science, we have a perfect platform to carry out this task. Streamlit and Python package builds and shares the data application in the fastest way po
4 min read
Get Bit Coin price in real time using Python
In this article we will see how we can get the current price of the bit coin. Bitcoin is a cryptocurrency. It is a decentralized digital currency without a central bank or single administrator that can be sent from user to user on the peer-to-peer bitcoin network without the need for intermediaries.
2 min read
Web Scraping Financial News Using Python
In this article, we will cover how to extract financial news seamlessly using Python. This financial news helps many traders in placing the trade in cryptocurrency, bitcoins, the stock markets, and many other global stock markets setting up of trading bot will help us to analyze the data. Thus all t
3 min read
Get current Gold price using Python
In this article we will see how we can get the current gold price in India. There is a variation in price of gold with time to time, price of gold is moved by a combination of supply, demand, and investor behavior. That seems simple enough, yet the way those factors work together is sometimes counte
2 min read
Cryptocurrency Portfolio Tracker Using Django
Cryptocurrency Portfolio Tracker is a Portfolio that provides information about cryptocurrency and we can also create our Portfolio which will implement simple adding functionality. In this article, we will create a Cryptocurrency Portfolio Tracker using Django in Python. Cryptocurrency Portfolio Tr
15+ min read
How to access a collection in MongoDB using Python?
MongoDB is a cross-platform, document-oriented database that works on the concept of collections and documents. MongoDB offers high speed, high availability, and high scalability. Accessing a Collection 1) Getting a list of collection: For getting a list of a MongoDB database's collections list_coll
2 min read
Build a GUI Application to Get Live Stock Price using Python
The stock price is the highest amount someone is willing to pay for the stock. In this article, we are going to write code for getting live share prices for each company and bind it with GUI Application. Module Needed Yahoo_fin: This module is used to scrape historical stock price data, as well as t
3 min read