A strategy to backtest a strategy

[tickwaiting]

Hi Tickstory


as you know the only winning horse model is every tick backtest, reliable and realistic approach.

however, this model can be extremely slow on short timeframes or scalping timeframes like M1, M5, M15.

it can take days and weeks assuming the server or the device didn't shut down or restart due to updates or maintenance which is inevitable.

so it will be trap to run it over and over once it's disturbed and you will find yourself wasting time for nothing.

so I'm sure you know based on your experience how to do it the right way or let's say the professional way.

in short words give us a guide or a strategy to backtest a strategy of short timeframes based on every tick model.


Best Regards

[tickstory]

Hi Tickwaiting,

I think the best approach here is to use small ranges of (recent) data for your test. As you mention, there is no easy way to quicken the testing cycle unless your specific strategy has some "short" way of testing. For example, there is a "filter duplicate ticks" option when you export from Tickstory. This will remove duplicate ticks in a sequence. If your EA is not sensitive it will help speed up things a little. Similarly, you might decide to try testing your EA on specific dates which you know to give problems/draw-downs and optimize from there.

Check out our Walk-forward analyzer which might also help. It breaks down data into time period chunks and gives it a test with 'in' and out of sample data. This can give you an indication of how the EA performs with "unseen" data. The better it performs, the more likely it will work in other time periods. The manual gives more details on how to use it if you need.

If you can get your strategy consistent on 1-year's worth of recent data (including ensuring it is tolerant to varying spreads, slippage, etc), you can invest the extra time testing it on earlier years.

Hope this helps.

[tickwaiting]

Hi Tickwaiting,

For example, there is a "filter duplicate ticks" option when you export from Tickstory. This will remove duplicate ticks in a sequence. If your EA is not sensitive it will help speed up things a little. Similarly, you might decide to try testing your EA on specific dates which you know to give problems/draw-downs and optimize from there.

i think i will try filter duplicate ticks and random sample of time sounds backtesting hack

Check out our Walk-forward analyzer which might also help. It breaks down data into time period chunks and gives it a test with 'in' and out of sample data. This can give you an indication of how the EA performs with "unseen" data. The better it performs, the more likely it will work in other time periods. The manual gives more details on how to use it if you need.

sure i'm going to check the advanced tester sounds nice but i need to master the baktesting process, to go to next level.

If you can get your strategy consistent on 1-year's worth of recent data (including ensuring it is tolerant to varying spreads, slippage, etc), you can invest the extra time testing it on earlier years.

this is the big issue i'm facing now, is 1 year enough for H1 Timeframe, or how many trades should i have to consider strategy consistent? what about optimization how many years or trades should i use it for? does it fail if optimize many years? because i have found strange thing i could optimize 1 year old data and things looks good but if optimize longer periods of historical data it tends to be bad. what do you know about the optimization limits?

Hope this helps.

[tickstory]

Hi Tickwaiting,

> this is the big issue i'm facing now, is 1 year enough for H1 Timeframe, or how many trades should i have to consider strategy consistent? what about optimization how many years or trades should i use it for?

It's really up to you based what you know about your strategy. For example, if it were a long term strategy that traded every month, then obviously you would need a larger dataset to be confident. With a high frequency strategy, a period such as 3 months sampled from one year and then a similar sized period from another year could give you the confidence you need. If you are finding that your EA is tolerant with various parameters and periods that potentially affect your EA (spread, holiday periods, etc), then it's recommended to put that iteration into forward/demo testing to see if it behaves the same as in your back-test.

>does it fail if optimize many years? because i have found strange thing i could optimize 1 year old data and things looks good but if optimize longer periods of historical data it tends to be bad. what do you know about the optimization limits?

We have seen certain people run across limitations in the MT4 optimizer. We think this might be due to reaching CPU/memory limitations with their particular EA strategy (unfortunately MT4 doesn't seem to raise any errors to give us further information). You are always best off keeping your optimization periods to the "minimum" (as per the guidelines above). If you are optimizing over a number of years, it's likely that your EA is being over-optimized anyway. The best is to take the last year of data and do an optimization on that. After that, you can try your optimized settings on previous years.

Hope this helps.

[tickwaiting]

Hi Tickstory

thanks man this is really helpful, I remember I read the term overoptimized ea. on other forum and I think it's real especially with many parameters over 20 and large historical data.
but you know based on your last reply you made me curious to ask a question but not sure if its right here or you prefer on new thread to be reference for other users. the question is about best device, vps and mt4 or mt5 as for best performance for backtesting. I can go in details about the 3 options each with experience traders notes from previous years.

[tickstory]

Hi Tickwaiting,

Sounds like it might be useful as a new thread. Not sure if we can answer it but perhaps others might be able to pitch in.

Kind regards.

[tickwaiting]

alright cool thanks for your responses here but also looking forward to see your opinion based on your experience on the new thread.