Historic Rate Data from Gain Capital

Have some interesting ideas about how to improve the software? Let us know in this section!

Historic Rate Data from Gain Capital

Postby Fiverr » Wed Jan 28, 2015 2:34 pm

Guys,

Check this out. We can use this as a second tick source.

http://ratedata.gaincapital.com/


lTid,cDealable,CurrencyPair,RateDateTime,RateBid,RateAsk
3520596979,D,EUR/USD,2014-10-05 17:00:07.533000000,1.251190,1.251360
3520596998,D,EUR/USD,2014-10-05 17:00:09.500000000,1.251200,1.251370
3520597062,D,EUR/USD,2014-10-05 17:00:19.767000000,1.251210,1.251380
3520597143,D,EUR/USD,2014-10-05 17:00:29.783000000,1.251230,1.251400
3520597163,D,EUR/USD,2014-10-05 17:00:30.517000000,1.251210,1.251380
3520597231,D,EUR/USD,2014-10-05 17:00:47.767000000,1.251190,1.251360
3520597279,D,EUR/USD,2014-10-05 17:01:07.767000000,1.251200,1.251370
3520597331,D,EUR/USD,2014-10-05 17:01:48.767000000,1.251210,1.251380
3520597352,D,EUR/USD,2014-10-05 17:02:00.267000000,1.251260,1.251430
3520597471,D,EUR/USD,2014-10-05 17:02:44.017000000,1.251270,1.251440
3520597485,D,EUR/USD,2014-10-05 17:02:44.283000000,1.251290,1.251460
3520597539,D,EUR/USD,2014-10-05 17:02:49.267000000,1.251300,1.251470
3520597595,D,EUR/USD,2014-10-05 17:02:53.517000000,1.251310,1.251480
3520597745,D,EUR/USD,2014-10-05 17:03:09.267000000,1.251320,1.251490
3520597753,D,EUR/USD,2014-10-05 17:03:09.767000000,1.251330,1.251500
3520597765,D,EUR/USD,2014-10-05 17:03:10.267000000,1.251340,1.251510
3520597928,D,EUR/USD,2014-10-05 17:03:18.767000000,1.251330,1.251500
Fiverr
 
Posts: 15
Joined: Tue Jan 20, 2015 2:42 pm

Re: Historic Rate Data from Gain Capital

Postby tickstory » Wed Feb 04, 2015 1:31 am

Thanks, Fiverr. We're working on a delimited file importer that should handle this format. Any comments with regard to the reliability of the data?
tickstory
 
Posts: 3688
Joined: Sun Jan 06, 2013 12:27 am

Re: Historic Rate Data from Gain Capital

Postby Fiverr » Thu Feb 05, 2015 5:34 pm

Because the FX market is not centralized; hence, it is advisable to test your algo with multiple data sources. If an algo is still profitable with multiple sources, then we can ensure robustness. This will also eliminate curve fitting in our trading system design.
Fiverr
 
Posts: 15
Joined: Tue Jan 20, 2015 2:42 pm

Re: Historic Rate Data from Gain Capital

Postby tickstory » Sun Feb 08, 2015 10:12 pm

Agreed. So as far as you're aware the data is complete and accurate for testing strategies at a tick level?
tickstory
 
Posts: 3688
Joined: Sun Jan 06, 2013 12:27 am

Re: Historic Rate Data from Gain Capital

Postby Naddmr » Sat Apr 25, 2015 10:28 pm

Hi folks,

I use their data since 2012 to build up my own tick based data ware house.
Their data is pretty accurate albeit they changed their format a few times so the import script had to deal with the changes accordingly.

Here's the part of my perl script with the different regexps to break up their formats to get started:

Code: Select all
                     if (/((^\d*)\,([A-Z]*)\/([A-Z]*)\,(\d{4}-\d{2}-\d{2}\s\d{2}\:\d{2}\:\d{2}\.000)\,(\d*\.\d*\,)(\d*\.\d*\,)D)/) {
                                my $fract=(split(/,/))[0];
                                if (length($fract) > 3 ) {
                                        $fract=substr($fract, length($fract)-3,3);
                                }
                                $fract=sprintf("%03d", $fract);
                                $tc_symbol=(split(/,/))[1];
                                $tc_time=(split(/,/))[2];
                                $tc_time=(split(/\./, $tc_time))[0] . "." . $fract;
                                $tc_bid=(split(/,/))[3];
                                $tc_ask=(split(/,/))[4];
                                $struct_ok=1;
                        }
                        # Format 2: <18616,"USD/JPY",2004-03-09 09:07:00,"111.30    ","111.35    ">
                        elsif (/(^(\d*)\,"([A-Z]*)\/([A-Z]*)"\,(\d{4}-\d{2}-\d{2}\s\d{2}\:\d{2}\:\d{2})\,("(\d*)\.(\d*)\s*")\,("(\d*)\.(\d*)\s*"))/) {
                                $tc_symbol=(split(/,/))[1];
                                $tc_symbol=~s/\"//g;
                                my $fract=(split(/,/))[0];
                                if (length($fract) > 3 ) {
                                        $fract=substr($fract, length($fract)-3,3);
                                }
                                $fract=sprintf("%03d", $fract);
                                $tc_time=(split(/,/))[2] . "." . $fract;
                                $tc_bid=(split(/,/))[3];
                                $tc_bid=~s/\"|\s//g;
                                $tc_ask=(split(/,/))[4];
                                $tc_ask=~s/\"|\s//g;
                                $struct_ok=1;
                        }
                        # Format 3: <192968689,"EUR/JPY",2006-01-15 17:05:27,138.710000,138.770000,"D">
                        elsif (/(^(\d*)\,"([A-Z]*)\/([A-Z]*)"\,(\d{4}-\d{2}-\d{2}\s\d{2}\:\d{2}\:\d{2})\,(\d*\.\d*)\,(\d*\.\d*)\,"D")/) {
                                my $fract=(split(/,/))[0];
                                if (length($fract) > 3 ) {
                                        $fract=substr($fract, length($fract)-3,3);
                                }
                                $fract=sprintf("%03d", $fract);
                                $tc_symbol=(split(/,/))[1];
                                $tc_symbol=~s/\"//g;
                                $tc_time=(split(/,/))[2] . "." . $fract;
                                $tc_bid=(split(/,/))[3];
                                $tc_ask=(split(/,/))[4];
                                $struct_ok=1;
                        }
                        # Format 4: <281982362,EUR/JPY,2006-12-08 16:30:31,153.630000,153.670000,D>
                        elsif (/(^(\d*)\,([A-Z]*)\/([A-Z]*)\,(\d{4}-\d{2}-\d{2}\s\d{2}\:\d{2}\:\d{2})\,(\d*\.\d*)\,(\d*\.\d*)\,D)/) {
                                my $fract=(split(/,/))[0];
                                if (length($fract) > 3 ) {
                                        $fract=substr($fract, length($fract)-3,3);
                                }
                                $fract=sprintf("%03d", $fract);
                                $tc_symbol=(split(/,/))[1];
                                $tc_time=(split(/,/))[2] . "." . $fract;
                                $tc_bid=(split(/,/))[3];
                                $tc_ask=(split(/,/))[4];
                                $struct_ok=1;
                        }
                        # Format 5: <1123609580,D,USD/JPY,2010-04-23 17:00:00,93.960000,93.990000>
                        elsif (/(^(\d*)\,D\,([A-Z]*)\/([A-Z]*)\,(\d{4}-\d{2}-\d{2}\s\d{2}\:\d{2}\:\d{2})\,(\d*\.\d*)\,(\d*\.\d*))/) {
                                my $fract=(split(/,/))[0];
                                if (length($fract) > 3 ) {
                                        $fract=substr($fract, length($fract)-3,3);
                                }
                                $fract=sprintf("%03d", $fract);
                                $tc_symbol=(split(/,/))[2];
                                $tc_time=(split(/,/))[3] . "." . $fract;
                                $tc_bid=(split(/,/))[4];
                                $tc_ask=(split(/,/))[5];
                                $struct_ok=1;

                        }
                        # Format 6: <1489060813,D,EUR/USD,2011-04-24 17:18:14.930000000,1.455900,1.456200>
                        elsif (/(^(\d*)\,D\,([A-Z]*)\/([A-Z]*)\,(\d{4}-\d{2}-\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{9})\,(\d*\.\d*)\,(\d*\.\d*))/) {
                                $tc_symbol=(split(/,/))[2];
                                $tc_time=(split(/,/))[3];
                                $tc_bid=(split(/,/))[4];
                                $tc_ask=(split(/,/))[5];
                                $struct_ok=1;
                        }
                        else {
                                print("$name mismatch: <$line>\n");
                        }


The most outstanding feature of their tick data is the time range covered. Some pairs have tick data as early as March 2003.
Naddmr
 
Posts: 3
Joined: Sun Apr 19, 2015 2:13 pm

Re: Historic Rate Data from Gain Capital

Postby tickstory » Mon May 04, 2015 4:05 am

Many thanks for you contribution, Naddmr.
tickstory
 
Posts: 3688
Joined: Sun Jan 06, 2013 12:27 am


Return to New features & suggestions

Who is online

Users browsing this forum: No registered users and 3 guests

cron