Historical Bloomberg data has gaps?

[donny]

Hi Quantnet,

I am doing my rounds downloading intraday 1-min index data using Bloomberg API. The request I'm using is "IntradayBarRequest" with eventType as "TRADE". I'm starting my downloaded from about 3 months back, October 2015. This is well within the historical data limit of 140 days from today.

What I've discovered is that when I'm downloading the data for common symbols, the data is complete with almost every minute being populated. However, with the less common symbols, there are gaps in the data when the time goes further back in the past.

Compare data download for DAX:

10/9/2015 9:16:00 AM,10115.72,10118.72,10115.72,10116.12,0
10/9/2015 9:17:00 AM,10116.43,10116.43,10113.74,10113.74,0
10/9/2015 9:18:00 AM,10113.64,10113.78,10110.07,10110.11,0
10/9/2015 9:19:00 AM,10110.04,10110.15,10108.18,10109.32,0
10/9/2015 9:20:00 AM,10109.69,10111.72,10109.39,10111.72,0
10/9/2015 9:21:00 AM,10111.46,10111.46,10106.71,10107,0
10/9/2015 9:22:00 AM,10107.81,10108.19,10105.4,10105.4,0
10/9/2015 9:23:00 AM,10105.45,10111.64,10105.34,10109.99,0
10/9/2015 9:24:00 AM,10110.01,10111.24,10108.67,10110.3,0
10/9/2015 9:25:00 AM,10109.11,10109.2,10105.16,10105.16,0
10/9/2015 9:26:00 AM,10105.61,10108.26,10105.61,10106.33,0
10/9/2015 9:27:00 AM,10106,10106,10096.05,10100.45,0

versus that for BZA:

10/14/2015 4:13:00 PM,48400,48400,48340,48340,10
10/14/2015 4:59:00 PM,48337,48337,48337,48337,0
10/15/2015 2:28:00 PM,48600,48600,48600,48600,35
10/15/2015 2:52:00 PM,48400,48400,48400,48400,150
10/15/2015 4:59:00 PM,48689,48689,48689,48689,0
10/16/2015 10:32:00 AM,48300,48300,48300,48300,45
10/16/2015 10:35:00 AM,48200,48200,48200,48200,10
10/16/2015 12:02:00 PM,48300,48550,48300,48550,20
10/16/2015 2:39:00 PM,48700,48700,48700,48700,25

Clearly data is missing. The perculiar part is that recent data for BZA is complete!

1/8/2016 12:49:00 PM,41130,41135,41110,41120,355
1/8/2016 12:50:00 PM,41130,41130,41065,41065,65
1/8/2016 12:51:00 PM,41065,41065,41055,41055,25
1/8/2016 12:52:00 PM,41065,41085,41065,41075,30
1/8/2016 12:53:00 PM,41085,41085,41080,41080,15
1/8/2016 12:54:00 PM,41075,41090,41050,41055,265
1/8/2016 12:55:00 PM,41060,41060,41025,41045,60
1/8/2016 12:56:00 PM,41040,41060,41035,41050,55
1/8/2016 12:57:00 PM,41055,41055,41045,41055,50
1/8/2016 12:58:00 PM,41050,41060,41050,41055,65
1/8/2016 12:59:00 PM,41055,41055,41025,41025,85
1/8/2016 1:00:00 PM,41025,41045,41020,41045,60
1/8/2016 1:01:00 PM,41050,41050,41020,41040,55
1/8/2016 1:02:00 PM,41040,41080,41040,41070,115
1/8/2016 1:03:00 PM,41070,41070,41035,41040,60
1/8/2016 1:04:00 PM,41035,41060,41035,41060,55
1/8/2016 1:05:00 PM,41050,41050,41030,41030,60

So it seems for uncommon symbols, the historical data going further back in the past, say further than two months, gets fudged and is incomplete.

Is anyone experiencing this problem?

Sincerely Yours,
Donny

 

[mhy]

This is quite common for any data source. Cleaning the data and filling in these missing gaps appropriately is an important part of doing time series analysis, statistics, machine learning, etc.

 

[donny]

That's what I guessed. Would it be any use that I raise a query to Bloomberg as to why there are gaps in their data?

 

[donny]

Hi everyone,

Problem resolved. In case there are those who have had this problem, here is what's happening.

For an index future, the 'A' at the end of the symbol represents the latest contract. As we're in January, this would be the March contract H6. The reason why there is low volume, price gaps in this symbol is that it was traded infrequently months back, i.e. a March 2016 contract was traded infrequently in October 2015.

Instead what I needed to be looking for is the December 2015 contract. So this would be BZZ5. So to summarize:

BZA - takes the latest contract by default.
BZH6 - is the current contract, if you count today as 18 Jan 2016. There will be price gaps a few months back because this contract would be rarely traded.
BZZ5 - is the December 2015 contract.

Therefore, to form a continuous series, you'll need to download separately the BZZ5 and BZH6 data. that's what I'll be doing now.

SPX Index isn't a contract. That's why it has a continuous price series.

Sincerely Yours,
Donny

 

[Alexandre Chretien]

You can also always download data for the front month. Bloomberg offers this option.