Historic Rate Data from Gain Capital

[imported_tickstoryParticipant February 4, 2015 at 1:31 am Post count: 1479]

#20506

Thanks, Fiverr. We’re working on a delimited file importer that should handle this format. Any comments with regard to the reliability of the data?

[FiverrParticipant February 5, 2015 at 5:34 pm Post count: 15]

#20507

Because the FX market is not centralized; hence, it is advisable to test your algo with multiple data sources. If an algo is still profitable with multiple sources, then we can ensure robustness. This will also eliminate curve fitting in our trading system design.

[imported_tickstoryParticipant February 8, 2015 at 10:12 pm Post count: 1479]

#20508

Agreed. So as far as you’re aware the data is complete and accurate for testing strategies at a tick level?

[NaddmrParticipant April 25, 2015 at 10:28 pm Post count: 3]

#20509

Hi folks,

I use their data since 2012 to build up my own tick based data ware house.
Their data is pretty accurate albeit they changed their format a few times so the import script had to deal with the changes accordingly.

Here’s the part of my perl script with the different regexps to break up their formats to get started:

if (/((^d*),([A-Z]*)/([A-Z]*),(d{4}-d{2}-d{2}sd{2}:d{2}:d{2}.000),(d*.d*,)(d*.d*,)D)/) {

my $fract=(split(/,/))[0];

if (length($fract) > 3 ) {

$fract=substr($fract, length($fract)-3,3);

}

$fract=sprintf("%03d", $fract);

$tc_symbol=(split(/,/))[1];

$tc_time=(split(/,/))[2];

$tc_time=(split(/./, $tc_time))[0] . "." . $fract;

$tc_bid=(split(/,/))[3];

$tc_ask=(split(/,/))[4];

$struct_ok=1;

}

# Format 2: 

elsif (/(^(d*),"([A-Z]*)/([A-Z]*)",(d{4}-d{2}-d{2}sd{2}:d{2}:d{2}),("(d*).(d*)s*"),("(d*).(d*)s*"))/) {

$tc_symbol=(split(/,/))[1];

$tc_symbol=~s/"//g;

my $fract=(split(/,/))[0];

if (length($fract) > 3 ) {

$fract=substr($fract, length($fract)-3,3);

}

$fract=sprintf("%03d", $fract);

$tc_time=(split(/,/))[2] . "." . $fract;

$tc_bid=(split(/,/))[3];

$tc_bid=~s/"|s//g;

$tc_ask=(split(/,/))[4];

$tc_ask=~s/"|s//g;

$struct_ok=1;

}

# Format 3: 

elsif (/(^(d*),"([A-Z]*)/([A-Z]*)",(d{4}-d{2}-d{2}sd{2}:d{2}:d{2}),(d*.d*),(d*.d*),"D")/) {

my $fract=(split(/,/))[0];

if (length($fract) > 3 ) {

$fract=substr($fract, length($fract)-3,3);

}

$fract=sprintf("%03d", $fract);

$tc_symbol=(split(/,/))[1];

$tc_symbol=~s/"//g;

$tc_time=(split(/,/))[2] . "." . $fract;

$tc_bid=(split(/,/))[3];

$tc_ask=(split(/,/))[4];

$struct_ok=1;

}

# Format 4: 

elsif (/(^(d*),([A-Z]*)/([A-Z]*),(d{4}-d{2}-d{2}sd{2}:d{2}:d{2}),(d*.d*),(d*.d*),D)/) {

my $fract=(split(/,/))[0];

if (length($fract) > 3 ) {

$fract=substr($fract, length($fract)-3,3);

}

$fract=sprintf("%03d", $fract);

$tc_symbol=(split(/,/))[1];

$tc_time=(split(/,/))[2] . "." . $fract;

$tc_bid=(split(/,/))[3];

$tc_ask=(split(/,/))[4];

$struct_ok=1;

}

# Format 5: 

elsif (/(^(d*),D,([A-Z]*)/([A-Z]*),(d{4}-d{2}-d{2}sd{2}:d{2}:d{2}),(d*.d*),(d*.d*))/) {

my $fract=(split(/,/))[0];

if (length($fract) > 3 ) {

$fract=substr($fract, length($fract)-3,3);

}

$fract=sprintf("%03d", $fract);

$tc_symbol=(split(/,/))[2];

$tc_time=(split(/,/))[3] . "." . $fract;

$tc_bid=(split(/,/))[4];

$tc_ask=(split(/,/))[5];

$struct_ok=1;



}

# Format 6: 

elsif (/(^(d*),D,([A-Z]*)/([A-Z]*),(d{4}-d{2}-d{2}sd{2}:d{2}:d{2}.d{9}),(d*.d*),(d*.d*))/) {

$tc_symbol=(split(/,/))[2];

$tc_time=(split(/,/))[3];

$tc_bid=(split(/,/))[4];

$tc_ask=(split(/,/))[5];

$struct_ok=1;

}

else {

print("$name mismatch: n");

}

The most outstanding feature of their tick data is the time range covered. Some pairs have tick data as early as March 2003.

[imported_tickstoryParticipant May 4, 2015 at 4:05 am Post count: 1479]

#20510

Many thanks for you contribution, Naddmr.

[Author]

Posts