AI For Trading: What Could Possibly Go Wrong?
Note: the following post is purely to illustrate how to avoid pitfalls in trading. My motivation for this post was to give an example of mistakes that I have made myself all too often and I can fully appreciate how one would come to those conclusions. I’m sure that the authors are excellent in machine learning and AI but in trading specifically, domain-specific knowledge is very important. Furthermore, I learned from their article how to use Bokeh in Jupyter notebooks, which is awesome and I highly value their approach to make their results available to the community. We are all here to learn.
Recently, I came across the following link on the first page of HackerNews:
titled: Stock Price Prediction With LSTMs
I like HackerNews as it has a large and generally very smart audience and the articles there are often very interesting. As I understand it’s not easy to get your article on the front page as a very smart audience votes your articles up. Naturally, I was very intrigued by the title as I have been using LSTMs in for various applications in trading algos for a while. However, pure price prediction from the price series along is hard. The post is laid out in a Latex-like format similar to a scientific paper and has the following chart on the front page:
The second you see this you can’t help getting excited, right? The fact that it’s on the front page of HN adds to that a little bit. However, over the years I have also learned the following mantra, sometimes the hard way:
When things look too good to be true, they usually are.
Luckily, the authors point to their IPython notebook on Github, which can be found here:
and make it easy for us to follow their reasoning. After cloning the notebook I looked at the code and in a nutshell, ANNs are applied to the past time series and a 1-day forward prediction is made providing next days price. Welcome to the intelligent, money-printing machine! What could possibly go wrong with that? Are things really too good to be true?
So, let’s look at this a bit more closely. The author uses various types of machine learning algorithms such as LSTM, BiLSTM and GRU and they show slightly different results. But let’s not be too concerned with the details. For simplicity, I just picked the results of the last one.
The predicted price matches the out-of-sample series very closely, which seems pretty amazing at first. However, let’s look at the returns of both the original and predicted out-of-sample prices.
Starting from the last prediction with the GRU I take the out-of-sample series:
t = pred_test[:,0] o = mean_price_inv[-len(pred_rets)-1:,0]
Just double-checking that we get the same series after transformation. Now, let’s calculate the returns of the series:
pred_rets = np.diff(t)/t[:-1] orig_rets = np.diff(o)/o[:-1]
Looking at the first 20 values we can see the the predicted returns closely follow the original returns. However, the two series have already been adjusted for the lag, so in actual fact, they should be on top of each other. If we plot the correlation between the returns we see the following:
Basically, there is none. However, when the predicted returns are shifted 2 time units to the left as in:
We get almost perfect overlap:
and perfect correlation:
The LSTM is perfect in determining what the returns were two days ago. So, no reason to get too excited over AI in trading just yet.
Again, thanks to the authors for putting their work out there.
In case you are interested, I’m running an advanced algorithmic trading workshop in Sydney on April 14, 2018 and if you’re in New York on April 28, 2018, check out my presentation a Quantcon: Reinforcement Learning for Trading – Practical Examples and Lessons Learned.