Add new export data feature option to detect dishonest EAs

[tickstory]

I agree that if the EA has a legitimate reason to avoid certain dates then a date-shift will not work. That said, how would a price shift alone work in this scenario? It would really be up to the person testing to determine which anti-cheat mechanism to use.

So just to be clear, how would you see the option to change the price working? At the moment there is a check-box to shift time by x days/hours. Is this simply a matter of adding another checkbox to alter the price by y points or are you saying that more complex logic is needed (eg. "If date is x then add y to price")?

Thanks.

[Nauris]

Simply a adding another checkbox to alter the price by y points. To calculate the points you will probably need additional steps, so it would be even better not to use points but a double number (as Srormex777 was suggested - "...add 0.2 to each quote ...").

Cheers.

[tickstory]

OK, thanks. For further clarification, could you explain how you think an EA could cheat with price alone? If you know some pseudocode, that would help just to make sure we're on the same page. It still doesn't quite make sense to me without having a date range involved. If a user decides to enable the price modification feature, the cheating EA would be only exposed if it had code like:

"If the price is between x and y then don't trade"

That logic might hide a failing test period but would also present lots of other problems for the EA if the price ever went to the same range (during the test period or in the future).

If the dishonest EA's code looked like:

"If the date is between <startdate> and <enddate> the price is between x and y then don't trade"

then the existing date shift will already capture this scenario. So why have an option to solely change the price? What do you think the cheating EA code will look like more precisely?

Thanks.

[Nauris]

The ability to shift the price cannot be viewed separately from shift the timestamp. They must be both together. In the current window, where you have days and hours, you should add a one more option - price shift.

"If the date is between <startdate> and <enddate> the price is between x and y then don't trade" - then the existing date shift will already capture this scenario. Yes exactly if <startdate> and <enddate> is <yyy.mm.dd> and end date <yyy.mm.dd> , therefor dishonest EA's date code will look like: startdate <mm.dd> and end date <mm.dd> >.

Do you know the difference between an allowed time filter and an unfair avoid? The filter is a legitimate reason to skip some hard-coded dates and times, such as Non-Farm Employment Change. Just as we can hardcoded the DST time, we can also hardcoded a news event for a specific time, and this will be a filter.

Simple example:
don't trade 2024.03.08 - unfair avoid.
don't trade 03.08 - filter.

In the filter case, the EA will skip trades all the years at 03.08. Unfair avoids will skip trades only 2024.03.08.
In a 10-year period, there will be 10 trades that the filter will avoid, but it will only be 1 case of unfair avoidance. Now imagine how many restrictions there will be if you need to hide trades for the entire 10-year period. For example, if you have to hide 100 lost trades for the entire period, then in the case of the filter, 100 trading days will be missed every year.

Therefore, smart dishonest EA's coder will not use simple avoidance of specific dates (2024.03.08), which can be detected by date shift, but will use filter case with price (03.08 + 1.02300 -1.02400). Of course, if the price went to the same range some transactions will be missed, but definitely not be as many as in the case of the filter (03.08). So therefor is so important to introduce price shift ability.

[tickstory]

Hi Nauris,

Appreciate the additional explanation. In your description, you seem to suggest that both options in Tickstory Advanced Tester would be enabled (i.e. date shift and price shift) but if the date shift is enabled, won't the legitimate filters be incorrect since the date has been shifted? This would mean the back-test results would be incorrect since the EA would potentially end up trading on the data of those days that the EA is trying to legitimately filter out.

In your example, let's say we shifted 7 days forward during our back-test, then the data on March 8th would actually be timestamped as the 15th of March and the EA's logic would allow it to trade at that time, invalidating the test results. If we only activated the price shift feature, then theoretically we would expose any "unfair" filters created by date and price, assuming that the EA is not sensitive to any relative price movements (that is, as long as the price change from 1.02300 to 1.02400 does not affect any of its entry or exit calculations, the back-test should remain the same).

Would you agree with that synopsis?

Cheers.

[Nauris]

The only tick data shift that is realistically possible is to the past by 28 years. The main reason for that is the fact that shifting by this value will result in a 100% match between the dates and the days of the week.

[tickstory]

Understood, but if you add 28 years, the <mm.dd> date would be the same and the filter would work in the same manner including the "unfair avoid" which specified "Don't trade on <mm.dd> between x and y price". This means that only the price adjustment would be meaningful, not the date shift?

Thanks again for your time talking this through - we're just trying to make sure we're adding these features in a sensible way, otherwise it can cause more questions for users than it solves. For example, if a user decides to adjust the price alone, they will need to comprehensively know that the EA is not sensitive to price changes in terms of its trading behaviour, otherwise they will wrongly conclude that it is cheating if they see differences in the results.

[Nauris]

Example, to avoid the signal when the position opens on 2024.03.08 1.02350 and there is a loss, a dishonest EA coder uses the hard coded settings (03.08 + 1.02300 -1.02400).

If only data shift 28 year is applied, then the loss will be at 1996.03.08 1.02350, then loss avoidance will work (03.08 + 1.02300 -1.02400). 1996.03.08==03.08 . The lost position will be hidden.

If will be applied data shift 28 year + price shift 0.7, then the loss will be at 1996.03.08 1.72350, then the dishonest EA will not work (03.08 + 1.02300 -1.02400), because it will not does not match to the price range 1996.03.08==03.08 && 1.72350 != (1.02300 -1.02400). The lost position will be visible.

[tickstory]

Thanks, Nauris. I think we're on the same page, except as I figure, the date-shift doesn't play any part in this particular example because you should get the same result with and without shifting the date. In any case, I can see the potential for a price shift function to be of use in this case and it's now on our TODO list.

Appreciate your feedback on this one. We'll update this thread when we have the feature ready.

Thanks.