GBPNZD [M15] FXT fails

[jlao]

Hi,
I have a problem with the GBPNZD pair, I will expose my workflow in order to explain better my problem:

1) I downloaded 36 pairs with Tickstory and from Dukascopy database, I downloaded from 2008.01.01 to 2015.06.30, everything was OK, I got all the XXh_ticks.bi5 files for each pair and for these dates. Great!!

2) I exported the data to MT4 and I got all the 36 FXT files (one for each timeframe M15)..... everything was OK also, and Tickstory is incredibly fast!!!

The problem is with the GBPNZD pair, I'm only able to backtest this pair in Every Tick mode from 2008.01.01 until 2010.10.08!!?? and I want until 2015!!.
I tried to do the backtest with my EA and with MT4i – Data Integrity checker and I got the same result!!??. I don’t understand the reason because the other pairs backtesting until 2015!! But GBPNZD (M15) not!!.

I repeated steps 1) and 2) and I got the same!!

I review the integrity of the XXh_ticks.bi5 files for this pair was OK, folders until 2015/05 with .bi5 files inside.

If I do a visual check of the data with MT4 (I work in a offline MT4 instance) I got also candles until 2015!!. So I understand that HST files are correct but FXT files are the problem for this pair and timeframe.

The MT4 journal log to my this message: 2010.10.06 17:11 TestGenerator: invalid file for 'GBPNZD15'

So what’s the problem?, any idea?

Thanks
Jose

[tickstory]

Hi Jlao,

It sounds to me that you're coming across the MT4 4GB FXT size limitation. You can find out more about this in our FAQ under the question "My back-test doesn't complete fully and stops at a certain date. What's happening?":

http://tickstory.com/forum/viewtopic.php?f=4&t=613

Regards.

[jlao]

Hi,

Thanks for the reply.

I’m not sure that the 4Gb limitation is the origin of my problem, because I can perform 8 years backtest (Every Tick mode) with the other 35 pairs, and as you will see on the the enclosed image, these other pairs have bigger size.

By the other hand, if I try to backtest few months (i.e. always after 2011 year) for GBPNZD M15, I get this error message from MT4: MT4i - Data Integrity Checker: no data for testing, so it seems there is a problem inside the FXT file....

Regards,

jlao

[jlao]

sorry, I forgot the image...

[tickstory]

Hi Jlao,

This one is a tough one because we've often seen some odd behaviour when larger than 4GB FXTs are used (although it's impressive that you dont outwardly appear to be having problems with the other pairs).'
If there is something off with the 2011 year, this should be easy to prove by simply exporting only that year (i.e. FXT < '4GB) and theoretically you should get the same error. If you can confirm this and ideally the month it occurs, we can look into this further.

Regards.

[jlao]

Hi,
I have made your directions and something happens in 2010 year.

I followed these steps (always for GBPNZD – M15):
1) I exported the FTX file and for only 1 year (2011) --> the backtest was succesfull!! from January to December!!
2) I exported the FXT file and for only 1 year (this time 2010) --> backtest stopped at 2010.10.08 and I got MT4 Journal error message like this: “TestGenerator: invalid file for ‘GBPNZD15’”

Below you will find MT4i – data Integrity Checker summary data for 2010:

Thanks!!

jlao

[tickstory]

Hi JLao,

Thanks for the information We are currently in the process of trying to reproduce the issue with the information you have given. We'll update you when we have more details.

Thanks.

[tickstory]

Hi JLao,

We've done some further tests and it appears that there is an issue with the underlying data for GBPNZD on the 8th October 2010 at around 06:10 (there could be others). It seems there is a zero price somewhere in the mix which is causing the "invalid data" issue. The fix will most likely involve trying to filter out all anomalous (zero priced) data which will hopefully resolve the issue. We'll let you know when we have an update.

Thanks for assisting us with identifying the problem.

Regards.

[tickstory]

Hi JLao,

This issue has now been resolved by filtering out any tick that has a bid or ask price that is zero. This filtering only occurs during the MT4 export. File exports, charts and data viewer will still show the anomalous data for completeness.

For information about downloading this update, please see http://tickstory.com/forum/viewtopic.ph ... 696&p=3303

Thanks again for taking the time to report the issue.

Kind regards.