Page 1 of 1

bi5 Data format

Posted: Sat Oct 01, 2016 11:05 am
by michaelmichael
Hi, I'm looking for the data format if bi5 tick files after lzma decomp. I would like to process the data myself after having downloaded them. I have found no comprehensive documentation on that.
AFAIK it is:
int milliseconds since epoch
int ask
int bid
float askv
float bidv
Can anyone confirm that?
Is there any plain Java examples on that?
Thanks in advance

Re: bi5 Data format

Posted: Fri Feb 19, 2021 1:29 am
by jdaniele
Hi Michael,

This is the EURUSD tick, 30/12/2019, original file named 00h_ticks.bi5, decompressed (with 7-Zip) file name: 00h_ticks (42,400 bytes)

Text format (CSV)
20191230 00:00:00:218,1.11815,1.11812,1.12000000476837,0.75

The binary record size is 20 bytes long. So the file contains 2,119 records = ticks

Binary (hex)

Code: Select all

  
   mills     -      ask      -     bid       -   ask vol     -    bid vol
00 00 00 DA  -  00 01 B4 C7  -  00 01 B4 C4  -  3F 8F 5C 29  -  3F 40 00 00
The time is expressed as milliseconds since the HOUR of the file.
The hour is included in the filename, so no need to include it into the binary time.
00 00 00 AD = 218 mill = HH:00:00.218

Just one more note about time.
Because time (excluded the HH) can be max 59:59.999 = 3599000 mills = 00 36 EA 98, the first byte of time will be always zero.
I wonder if it is actually used for something else... :roll:
Maybe it is there just for record padding.

The next 2 numbers are expressed as (straight 4 bytes) "integers", i mean like the window calculator shows them to you.
00 01 B4 C7 = 111815 = 1.11815 (divided by 100,000)
00 01 B4 C4 = 111812 = 1.11812 (divided by 100,000)

The next last 2 numbers are expressed as 32 bit floats (IEEE-754): 1 bit (sign) + 8 bits (exponent) + 23 bits (mantissa)
I used this useful converter to check them out: https://www.h-schmidt.net/FloatConverter/IEEE754.html

3F 8F 5C 29 = 0.01111111.00011110101110000101001 = 1.12 (error due to conversion: 0.00000000476837)
3F 40 00 00 = 0.01111110.10000000000000000000000 = 0.75 (no conversion error)
I think the above 2 numbers should represent millions.

Hope this helps.
Bye
jdaniele