Predict Bitcoin Prices with Deep Learning
Use Neural Networks to Forecast Cryptocurrency Prices with Python
I Tried Deep Learning Models to Predict Bitcoin Prices
Click below to see the previous article using machine learning for bitcoin prices:
Not too long ago, we delved into the usage of Machine Learning models to predict the future prices of Bitcoin. There we used two time series models to forecast the direction in which the price of Bitcoin may go in the next few days or weeks. It was pretty straightforward in regards to training and fitting the model to Bitcoin’s historical price data. But, what if there was another way besides Machine Learning to forecast time series data?
The answer to that question is…
Deep Learning
Deep Learning is a subset of Machine Learning but the key difference between the two is that Deep Learning uses Neural Networks which grants the machine the ability to train itself. Machine learning requires algorithms with parameters set by the user or engineer to train the machine. In other words, Machine Learning requires more hands-on tweaking and fixing than Deep Learning. Deep Learning is capable of doing all the tweaking and fixing itself without the need of any hands-on interference by the user/engineer. The main thing that they need to tweak or create is the Neural Network itself, which has its own set of obstacles.
But is Deep Learning better than Machine Learning?
Well that depends entirely on the problem. In some cases, traditional Machine Learning models are much better than complex Neural Networks. One such case we found is the training and fitting of tabular data for classification. In that scenario, the Neural Network’s time to train was significantly longer but performed at a lesser or equal level compared to the results of traditional classification algorithms. But what about time series data?
Well, that is exactly what we are going to be doing today! We are going to create and run our very own, relatively simple, Neural Network on Bitcoin’s historical price data. Then we will be able to see if the Neural Network outperforms the traditional SARIMAX or even Facebook Prophet time series models we implemented before.
As we walkthrough the process of creating a Neural Network, we will be explaining the necessary concepts to better understand the code. Expect to see a Github link at the end showing the entire source-code of the Neural Network.
Let’s Get Started
First, we need to install the necessary libraries in order to create our Neural Network (you may need to install Tensorflow Keras). Next, we load in the CSV file containing Bitcoin’s historical price data, which can be downloaded from Yahoo Finance. Another option for price data would be to use a financial data API such as EOD Historical Data. It is free to sign up and you’ll have access to vast amounts of financial data. Disclosure: I earn a small commission from any purchases made through the link above.
Then, we’ll do a little data preprocessing so that the prices we observe are the "Close" prices and the index is in a datetime format. Finally, in order to improve performance, we will scale the data using scikit-learn’s MinMaxScaler().
Helper Functions
For the next part, we’ll need to create some helper functions, which will enable us to run the Neural Network appropriately and effectively.
visualize_training_results() -plots the accuracy and loss of our Neural Network. This allows us to check on the training progress/results of our Neural Network.
split_sequence() – creates two arrays: X and y. These variables are the input and output variables respectively for our Neural Network. The input is the number of periods to look back and the output is the number of periods ahead. Simply put, the input periods are used to figure out patterns and sequences that lead to the output periods. They both can be any number but it would probably be beneficial if the number of input periods are greater than the output periods.
layer_maker() – creates a specified number of hidden layers (and Dropout layers if necessary) for our Neural Network. This function makes up the bulk of our network. The importance of this function will be explained later on.
Preparing/Splitting the Data
Our next step is to implement our split_sequence() function on our Bitcoin price data.
The number of periods in and out can be any number of our choosing. But for the sake of this example, we will choose to look back on 30 days of price history to predict the next 10 days. But, this doesn’t mean we are only looking back on the most recent 30 days to figure out the next 10 days for a total of 40 days only. It means that for each day of the 1,000 days we selected, the previous 30 days are used to determine any patterns or sequences that lead to the next 10 days. These values are used to train the Neural Network so that we can predict or forecast the next 10 days of Bitcoin prices from today.
A Neural Network
Now that we have our data ready to go, we can move on to the fun part – building the Neural Network. Our Neural Network will be created using a simple Sequential model from the Keras library. From there, we will be using a specific Neural Network called a Recurrent Neural Network by implementing LSTM (Long Short-Term Memory) layers in our model.
Recurrent Neural Network – LSTM
A Recurrent Neural Network (RNN) is a class of Neural Networks and a Long Short-Term Memory (LSTM) Network is a type of RNN. We are using an RNN instead of a normal Neural Network such as Multi-Layer Perceptron (MLP) Network because RNNs are most appropriately used for sequential data such as our Bitcoin historical price data. If you want to know more about RNNs and LSTMs, click here.
Creating our Recurrent Neural Network
With everything ready, we can start constructing the Neural Network for our Bitcoin price data:
To begin, we instantiate the Sequential model and an activation function, which runs throughout the layers in the Neural Network. After much experimentation, the best activation function turned out to be "softsign" or "tanh." This is probably due to the fact that both these functions range from -1 to 1 and handle negative and positive values from inputs better than activation functions that range from 0 to 1.
Input Layer
The first layer in our Network will be the input layer, which can be added to the Network by simply calling model.add(). Inside the .add() method, we have:
- The LSTM layer to create our RNN.
- The number of nodes as the first argument (here it is set as 25).
activationis the activation function we set as a variable before.return_sequenceis set to True so each sequence between the LSTM layers contains the appropriate dimensions.input_shapeis set as the number of inputs and features. This is done so that the network knows what shape to expect.
Hidden Layers
For the next item, we can begin creating our hidden layers within the Neural Network. In order to avoid typing out model.add() multiple times, we will use our layer_maker() function. This creates a simple ‘for‘ loop with a specified range that generates as many layers as we wish. It also has the option to add in Dropout layers, which is important for regularization. If you are not familiar with the term regularization, just know that it is a method of preventing overfitting in our Neural Network. Overfitting, simply put, **** occurs when a model performs poorly when predicting new observations and in our case, future Bitcoin prices.
Next, we finish up the hidden layers with one final hidden layer without return_sequence set to True. This is done so that the dimensions from the final hidden layer can proceed smoothly to the output layer with no errors.
Output Layer and Model Summary
The next two items in our code snippet above contain the final layer and a summary of the Neural Network we just created. The final layer or output layer is not an LSTM layer but instead it’s a normal, densely-connected NN layer, which would be regularly used if we were using a standard, non-recurrent Neural Network. That layer contains the number of periods/days we wish to predict as its number of nodes. The model summary displays basic information regarding the created Neural Network.
Compiling the Model
Before we move on to training our NN, we must compile it with the preferred specifications:
optimizeris the optimization algorithm we want for training our NN (check here for a thorough overview of different optimization algorithms).lossis how we measure our errors within our model. Typically, we want the losses to be as close to zero as possible (see here for more information on different loss functions).metricsis how we measure the performance of our model/NN (click here for more information on different metrics).
Experimenting with Different Parameters
When it comes to creating the NN model, plenty of experimentation will be necessary in order to find the best combination of parameters. The number of nodes and layers are especially important when it comes to how the model will perform in the end. There is no magic number for either and both require their own ideal number that must be discovered through trial and error.
Nearly every NN you create for any new dataset must be composed with their own unique set of parameters. Because of this experimentation process, our results may drastically differ from one NN to the next depending on the dataset used.
Fitting/Training the Neural Network
With everything prepared and good to go, we can finally begin to fit/train our Neural Network! We can accomplish this by writing just one line of code:
res = model.fit(X, y, epochs=800, batch_size=32, validation_split=0.1)
Once we run this code, our Neural Network will begin training on our Bitcoin price data. Depending on the resources we have, the time to train may take several hours. Those with better hardware specifications may arrive at better results than us. In the meantime, while the NN is training, we can explain the arguments within the model.fit() method:
X, yare the variables we assigned that contain our Bitcoin historical price data.epochsare the number of times the NN will train over the entire dataset and contains the number of batches.batch_sizeis the number of samples within the training set from which the model will work through before updating its own parameters.validation_splitis the percentage of the dataset put aside to evaluate the losses and metrics on the data at the end of an epoch.
Visualizing our Model’s Performance
Once we have completed training our model, we can visualize our training results by using our visualize_training_results() function. Running the following line of code will display our model’s performance over the course of the number of epochs.
visualize_training_results(res)
Which will give us the following visualizations…
These visualizations display the Loss and Accuracy for both the Training and Validation set. In order to see if our NN is training well, we will want to see both the Validation and Training __ set’s Loss and Accuracy _converg_e as the number of epochs increases. Otherwise, if our Loss and Accuracy from both sets are _divergin_g from each other, then there may be a sign of overfitting from our NN. This might be solved by introducing some _Dropout layer_s, reducing the _number of epoch_s, altering the _amount of layer_s, etc. This step in our NN training may require additional trials and tuning in order to observe the convergence of the Training and Validation set.
Validating our Neural Network
Once the training is complete and we are satisfied with how the Loss and Accuracy converged. We will need to move on testing our model against actual data to see how well it performs. We can do this by simply visualizing the NN’s predicted Bitcoin prices and graphing them with Bitcoin’s actual values.
When it comes to accurately predicting the price, our model falls short quite a few times. Although, the trend and potential price targets may provide some value. For example:
On day 0, you observed from the model’s predictions that the price will reach $7,700 on day 10. But, in actuality, the price reached slightly higher than that on day 6.
From this example, the information may provide potential investment timing and opportunity to Bitcoin traders.
Forecasting/Predicting with our RNN
When we are finally satisfied with validating our predicted Bitcoin values, then we can move on to the most useful part of our NN – forecasting the future prices of Bitcoin! To predict the next 10 days of Bitcoin prices, all we have to do is input the last 30 days worth of prices in our model.predict() method.
With the following code we can print out the prices for the next 10 days as well as graph those predictions for better interpretability.
Excellent! We now have the Bitcoin prices for the next 10 days. However, are these numbers infallible? Absolutely not! Just like our time series models from before, this NN can also give us the general trend on which the prices may go in the future. If they’re wrong, then that is to be expected because nobody and no machine can correctly predict the future. We can only hope to see where it might go and using a NN is probably better than blindly guessing.
Closing Thoughts
This newly created NN based on Bitcoin historical price data was able to successfully provide us with a forecast of the next 10 days. What you do with that information is entirely up to you but you should be aware that this was still a relatively simple NN. Much more could be done to improve the performance of our NN but for the sake of time and simplicity, we decided to only cover the basics of creating a RNN.
Now, depending on how much time you took to train and tune the NN, your results may vary from ours. Feel free to experiment with the code and test out different approaches in order to improve the NN. Some ideas might be to include technical indicators as another set of input variables or maybe try a different type of RNN. There are just so many possibilities and avenues to explore with Deep Learning and new prospects emerge everyday. So be on the lookout for new discoveries and hopefully you enjoyed and learned a thing or two about building a NN!
Resources:
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
