Web Scraping CryptoCurrency price and storing it in MongoDB using Python

Let us see how to fetch history price in USD or BTC, traded volume and market cap for a given date range using Santiment API and storing the data into MongoDB collection.

Python is a mature language and getting much used in the Cryptocurrency domain. MongoDB is a NoSQL database getting paired with Python in many projects which helps to hold details that got retrieved from Python Programs. PyMongo is a Python distribution containing tools for working with MongoDB and it is a very convenient way  to work with MongoDB from Python to do Create/Update/Delete/Read operations easily.

Let us see the code in using Santiment's API in getting cryptocurrency prices for a given date range

Few Examples of cryptocurrency are :

bitcoin
ethereum
ripple
bitcoin-cash
litecoin
eos
cardano
stellar
neo
iota

1. In the place of id, we can pass bitcoin/ethereum or any cryptocurrency name which the below code can parse and get the data
2. For valid cryptocurrency name, data will be retrieved properly
3. In the place of from_date and to_date valid dates in valid date formats in yyyy-mm-dd pattern need to be given. For understanding purpose, it is given to take 7 days of data. We can able to get 1 month old data too. API call can able to get data for the given cryptocurrencies in the given date range or else if not available (due to invalid cryptocurrency name/invalid date range specification), we can catch the errors
4. In the below program, for easier understanding, taking bitcoin and ethereum in "IndexCoins.idx" file and hence in the place of id, they are passed

Example :

For the following input

👁 Image

For bitcoin above API call, returns data as below

👁 Image

Similarly for ethereum also, we will get value

Retrieved value has to be stored in MongoDB

Database : geeksforgeeks

Collection : Coll_santiment_Price

As we are going to take for 7 days of data, it has to be given in a loop

In MongoDB, _id is the default column which can be got by ObjectId(). Our process has multiple rows where each and every row is identified by means of "cryptocurrencyname" and "time". In our code, we are having it as "id" and "time" respectively.

Let "Coll_santiment_Price" be created with 3 columns namely _id, id and time first. Then upon successful execution of API call, for the id and time, let API call output namely (verify column name from bitcoinprice.png) 'priceBtc', 'priceUsd','volume' and 'marketcap' are updated in a loop.

Code Implementation in Python

Sample Output : (For bitcoin)

👁 Image

Sample Output : (For Ethereum)

👁 Image

Santiment API calls for getting prices are very much used across cryptocurrency projects. As we are getting historical data, for data analytics, this is very much useful. Moreover, it is freely accessible and hence any beginner-level projects can use it without any issues. Price variations are not much varying and hence well set for demo and small scale projects.