Expected degree of accuracy
-
Emersonian
- Posts: 2
- Joined: Fri Mar 03, 2023 11:27 pm
Expected degree of accuracy
Can someone advise how accurate I should expect the results to be, comparing backtest ATM to forward test in same period and settings please.
Re: Expected degree of accuracy
Hi Emersonian,
The question of "accuracy" is a bit of an open-ended question as it will depend a lot on the nature of your EA. If, for example, your EA is only making one trade every day and the order entry trigger is a simple moving average, then you are likely to get much more similar results in your forward test than a scalper.
What you should be comparing in your back-testing versus forward test comparison is that the performance is largely the same. That is, are you getting similar a similar profit curve to what you back-tested? Are the draw-downs similar? Are you getting roughly the same amount of trades? You'll need to compare these critical metrics to understand if your forward-testing is matching your back-test results.
The settings you ultimately choose should be the most "stable" set across all your forward-tests (i.e. the one that represents the most stable draw-downs, profit curve, etc).
Hope this helps.
The question of "accuracy" is a bit of an open-ended question as it will depend a lot on the nature of your EA. If, for example, your EA is only making one trade every day and the order entry trigger is a simple moving average, then you are likely to get much more similar results in your forward test than a scalper.
What you should be comparing in your back-testing versus forward test comparison is that the performance is largely the same. That is, are you getting similar a similar profit curve to what you back-tested? Are the draw-downs similar? Are you getting roughly the same amount of trades? You'll need to compare these critical metrics to understand if your forward-testing is matching your back-test results.
The settings you ultimately choose should be the most "stable" set across all your forward-tests (i.e. the one that represents the most stable draw-downs, profit curve, etc).
Hope this helps.
-
Emersonian
- Posts: 2
- Joined: Fri Mar 03, 2023 11:27 pm
Re: Expected degree of accuracy
"The settings you ultimately choose should be the most "stable" set across all your forward-tests (i.e. the one that represents the most stable draw-downs, profit curve, etc)...."
Are you suggesting that back testing is not a suitable tool to choose EA settings ???
What purpose does backtesting with ATM serve ?
Are you suggesting that back testing is not a suitable tool to choose EA settings ???
What purpose does backtesting with ATM serve ?
Re: Expected degree of accuracy
Hi Emersonian,
The statement is saying that you should choose the best settings that you have found through the ATM. Your back-tests will get a set of candidate settings and the forward testing you do with the ATM will validate those settings are working on "unseen" data.
There are few a sections on "Walk Forward Analysis" in the ATM manual which go into more detail about how walk-forward analysis can help your testing (see the "A Primer on WFA" and "How does the WFA work?").
Hope this helps.
The statement is saying that you should choose the best settings that you have found through the ATM. Your back-tests will get a set of candidate settings and the forward testing you do with the ATM will validate those settings are working on "unseen" data.
There are few a sections on "Walk Forward Analysis" in the ATM manual which go into more detail about how walk-forward analysis can help your testing (see the "A Primer on WFA" and "How does the WFA work?").
Hope this helps.