Deep learning for Time Series analysis and prediction

[yetanotherquant]

Hello together,

I wonder, whether anyone has experience with deep learning in application to time series?
Not necessarily financial time series, I, for one, have an interest in (short-term) prediction of gas and energy consumption.
There are some papers on subject, for example this one:
http://cs229.stanford.edu/proj2012/BussetiOsbandWong-DeepLearningForTimeSeriesModeling.pdf
However, since they provide no source codes (data are publicly available), it is not a result, it is, according to the manifest of reproducible research an advertisement of a result.


I have an impression that (one-dimensional) time series is just too simple for deep learning since there are few (if at all) hierarchies of abstraction; contrary to, say, face recognition, by which one can at first recognize strokes/pixel patterns then eyes/ears/noses and then Peter/Paul/Sarah.

However, deep learning might probably be applied to pattern recognition.
Just like a chartist looks at first at 5 years chart, then at 6 months chart then at monthly chart and probably at even finer scales...

Any thoughts?

 

[Alexandre Chretien]

Hey yetanotherquant!

This is awesome that you are getting interested with these models!
I am currently conducting research as part of my masters for a financial institution and am implementing neural networks for algorithmic trading. Essentially, I try to forecast the movement direction 1 to 5 days ahead. I can tell you for fact that there are structures in signal that you can't just see with a naked eye. Take for instance the Fourrier transform of a signal. You will realize that it can be decomposed in a variety of much simpler signals.

Now concerning the article, I would be very careful interpreting the results. A lot of the times in these articles the data they try to forecast is included in the training set...

If you are still interested and want to try it out yourself, I recommend Matlab as good starting point. The neural network library is really straightforward even with little knowledge about NN, you will be able to test your own models within several hours.

Good luck,

Feel free to ask if you got any specific questions.

 

[yetanotherquant]

Hi Alexandre,

I do use neural network in practice (for gas consumption prediction), however, these are "shallow" networks with only one hidden layer. They work sufficiently well but they are nothing special: sometimes better than ARIMAX, sometimes worse.

But my question is explicitly about deep architectures, which (as I understand them) extract hierarchical abstractions. But are there such abstractions in stock (bond, fx, commodities) price time series?!

P.S.

1) To be true, I am a pretty skeptical about an ability of automatic short term prediction of returns. But if you get positive results, I would readily have a look at your thesis
(unless you then keep it secret )

2) As to the article I mentioned, yes I also have an impression that Figure 1 is actually an in-sample fit, not a forecast. It is often the case: even a simplest time series model like ARMA(1,1) can be almost perfectly fitted to data but its forecasting ability is far not as good.

3) What one can see with naked eye depends on how well this eye is trained
I, for one, pretty often can say a lot about a time series just looking at its chart.
But of course I also use all analysis tools I have.

4) I prefer the wavelet transform to the Fourier transform.
Have a look at images attached. A smoothed periodogram does not disclose anything interesting. But a CWT reveals peaks between scale 8 and 9, which may be interpreted as weekly periodicity (2000 entries / 2^8 = 7.8). However, this periodicity does not permanently persist in time.

P.P.S.
Matlab is good but R is better. IMO