Help with Walk Forward (etc)

General discussion about the Tickstory Lite software package.
Darkdoji
Posts: 9
Joined: Thu Jan 10, 2019 4:18 pm

Help with Walk Forward (etc)

Post by Darkdoji »

I have just acquired Tickstory Standard and as an initial step reviewed the guidance on use. This review has
left me a bit confused about how to proceed. While I now have access to all the advantages the Tickstory tool provides,
my immediate aim is not exactly to undertake regular back testing.

My aim is simply to download as much data as is available for specific symbols I want to run
within Walk Forward Pro (WFP), a stand alone application that runs an MT4 tester instance to generate
optimizations and walk forward analysis for the selected symbol.

This means a) once activated, the WFP program takes over control of the the MT4 strategy tester to undertake its operations
b) as such, and for instance, launching MT4 from Tickstory would be in conflict with how Walk Forward Pro
works, etc. In other words, I need to use Tickstory in a mode that simply downloads tick data, exports it to MT4 for use
as I would broker provided data in the MT4 database (while maintaining Tickstory quality and range of data). Therefore the point of this post is to request urgent assistance or help in guiding me to how I might achieve this without running Tickstory in full mode, as clearly one can anticipate serious conflicts with WFP workings. I shall be most grateful for a step by step listing of what to do to reach the stated aim (take it that not only am I a newbie to Tickstory
usage but that I am also slow on the uptake). Finally, please note that it should not be taken as an over investment (i.e. Tickstory
Standard given my immediate aim). Rather, that I fully intend to learn and employ Tickstory capabilities once my immediate
project is concluded. But my pressing need is to succeed with my immediate project using Tickstory quality.

Thanks in advance

tickstory
Posts: 4883
Joined: Sun Jan 06, 2013 12:27 am

Re: Urgent Help with Tickstory Standard

Post by tickstory »

Hi Darkdoji,

We haven't had much experience with the software you mention, however to distill your question to what we can help with, you should be able to export data from Tickstory to MT4. To confirm this has been successful, make sure that you get 99% modelling quality from the MT4 Strategy Tester by following the steps described in our video here:

https://www.youtube.com/watch?v=084k9fJCeCY

Once you've confirmed this, we don't know of any reason why you cant then use your external program such that it uses the same data in its own tests (if that's what its designed to do).

Thanks.

Darkdoji
Posts: 9
Joined: Thu Jan 10, 2019 4:18 pm

Re: Urgent Help with Tickstory Standard

Post by Darkdoji »

Ok thanks - watched the vedio and exporting the first of 3 intended symbols now (looks like it will take the rest of this year at this rate though). Three quick questions if you do not mind (to help me manage the process best practice).
a) I left the default time zone shift as is, hope that is fine (I live in West Africa, trade with alpari (EET server time)). My understanding of your online (help) manual is that adjusting that otherwise depends in part on EA requirements for matching GMT offset with broker server time. I have no such need for my EA.
b) The speed has me thinking there is some kind of mistake somewhere. I am exporting from the earliest available date (2003 - now) for the first of 3 symbols and it has taken about an hour to hit 7% would that be something usual? One reason I opted for the TS Standard was speed (but seems slower than was my experience with free lite and certainly no comparision with downloading from JForex where I actually downloaded the same 3 symbols X all time frames in less than an hour)*. Hope there is no mistake with the version in your delivery (everything reads tickstory lite in that delivery though I paid for standard).
c) Finally, getting lots of error messages in log view of the form, "13-01-2019 20:09 Error occurred processing file C:\Users
\Darkdoji\AppData\Roaming\Tickstory\Tickstory Lite\Data\EURUSD\2003\07\26\05h_ticks.bi5. Attempting redownload."
Do I take this as normal and mere notifications (as the process is continuing)? Your online (help) manual indicates that errors
will print to that view but makes no mention of what the user should read into such or what the user should do in response. My
only concern is that data quality is not impinged by this or any other lapse.

Thank you

*PS: Only reason I did not simply use the JForex data is that for some crazy reason I could not get them to import via history centre though I still have them safely stored on disk.

tickstory
Posts: 4883
Joined: Sun Jan 06, 2013 12:27 am

Re: Help with Walk Forward (etc)

Post by tickstory »

Hi Darkdoji,

In response to your questions;

a) I left the default time zone shift as is, hope that is fine (I live in West Africa, trade with alpari (EET server time)). My understanding of your online (help) manual is that adjusting that otherwise depends in part on EA requirements for matching GMT offset with broker server time. I have no such need for my EA.

That's fine if you think your EA is not dependent on time. If you're unsure, you can select "European time" at the top of the time-selection combo box which would be advisable if you are trying to replicate your broker environment as best as possible.

b) The speed has me thinking there is some kind of mistake somewhere. I am exporting from the earliest available date (2003 - now) for the first of 3 symbols and it has taken about an hour to hit 7% would that be something usual? One reason I opted for the TS Standard was speed (but seems slower than was my experience with free lite and certainly no comparision with downloading from JForex where I actually downloaded the same 3 symbols X all time frames in less than an hour)*. Hope there is no mistake with the version in your delivery (everything reads tickstory lite in that delivery though I paid for standard).

Tickstory Standard is definitely faster than the Lite version since it uses additional processing threads. If you are comparing against the JForex platform, however, this isn't a one-for-one comparison since Dukascopy have a different delivery method for JForex users. You can use the same repository/cache location if you wish so you don't need to re-download everything twice. Once the data is exported, the processing within Tickstory will get much faster. For those users who are maintaining an active subscription, we are also working on speeding up their data access.


c) Finally, getting lots of error messages in log view of the form, "13-01-2019 20:09 Error occurred processing file C:\Users
\Darkdoji\AppData\Roaming\Tickstory\Tickstory Lite\Data\EURUSD\2003\07\26\05h_ticks.bi5. Attempting redownload."
Do I take this as normal and mere notifications (as the process is continuing)? Your online (help) manual indicates that errors
will print to that view but makes no mention of what the user should read into such or what the user should do in response. My
only concern is that data quality is not impinged by this or any other lapse.

This error can sometimes occur when the data is corrupt (a temporarily broken internet connection, for example). In these cases Tickstory will re-attempt the download. As long as you don't see this error repeatedly you can safely ignore it.

Regards.

Darkdoji
Posts: 9
Joined: Thu Jan 10, 2019 4:18 pm

Re: Help with Walk Forward (etc)

Post by Darkdoji »

Hi again - this is to confirm that in fact I managed to complete the export this morning (started in the afternoon of yesterday).
On completion I did launch MT4 and run an EA to confirm model quality and that came in at 99.90%. This I assume was the test you required to affirmatively answer my initial (original) query.
However, let me ask some related questions to help my mastery of the tickstory tool.
First I noted 2 significant gaps in the downloaded data (by visual inspection of the test chart). Significant because the first in 2008 was for about a week and the second in 2016 was for about six months. Is this usual or does this imply problems with the export process? On seeing such data gaps what is best practice (in terms of what to do to correct such)? I ask because of what happened next (see next).
Thinking that I could correct the gaps by reexporting for just those isolated periods and starting with the first (2008 Feb 2 - Feb 12) I selected the desired range and hit export. But that action appears to have wiped out the entire effort over the last 24 hours. The system did export the specified period but wiped out the entire rest. This made me mad because I had not slept just monitoring the snail speed download. So next time what should I know about that circumstance and how best to respond? And why would it have missed data the first time if it easily exported them the second time? How can I ensure this does not happen in the future?
Finally, for this first trial, I chose to ignore weekend data but that was because I did not fully understand the effect of "flats" mentioned by your online (help) manual which it states will replace unavailable data when this is case for weekends. I get about one or two hours data on Sundays given broker server timing so I am wondering a) would omitting weekends not have significant implications for quality especially in the lower time frames? b) what is meant by "flats" and do they distort data in any way?
Thank you so much for your patience and support.

Darkdoji
Posts: 9
Joined: Thu Jan 10, 2019 4:18 pm

Re: Help with Walk Forward (etc)

Post by Darkdoji »

Hi,
Now I have a most distressing situation as a result of the last (reported) attempt at exporting data via Tickstory into the database*. MT4 will simply not work even after an attempt at uninstalling previous instance and reinstalling a new instance. This was the same problem the very first time, i.e. when I botched an attempt to download data with free tickstory. Now I have a licensed version and imagined such would not happen again. Is there something you can tell about this? I mean (and given my earlier post on the failure to fill a gap in exported data by isolating the specific (gap) period). Is there a way to avoid this disastrous effect on MT4?
Also, given I have data from Dukascopy (downloaded via JForex) and stored on disk, is there a safe way I can use tickstory to export from the stored file location into history centre should I succeed in reinstalling a new instance of MT4? This will save me the time and hassle of having to download/export data all over again especially since I already have the required data on disk for the 3 symbols I wish to download/export via tickstory?

Kindly, treat as urgent and critical given that the reported situation is inhibiting me from working and quickly progressing my project which was the initial aim of taking all the trouble I have to get things going via a licensed version of TS Standard.

Thanks in advance

PS*: I had decided that while I wait to get a response from you on my last post before this, I would simply replace the exported data with alpari history centre data. Once I did that the MT4 would simply not open - i.e. on launching it appears on screen and then fails to activate (similar to what we experience when a new build updates the platform except that this is not metaquotes trying to send in an update it is a problem related to my use of Tickstory).

Darkdoji
Posts: 9
Joined: Thu Jan 10, 2019 4:18 pm

Re: Help with Walk Forward (etc)

Post by Darkdoji »

Further to my earlier notes and as I await help, please note a) I now think that the data gaps reported by me may have been caused by local internet connection failures - would that explain such visible gaps in data exports? b) I am desperate for a response on all the questions I seek of you since the reported events in earlier notes. This is because my project is at an absolute standstill which is extremely worrisome as it is costing me fair amount of money per day for each day I am unable to use data I had aimed gain via tickstory (as my subscription to WFP is running regardless), c) most critical is to help me understand why on any failure with export/download my mt4 becomes so fowled up I am forced to reinstall (happened with tickstory free version and now). This implies loss of data, setups and settings in various areas and programs that invariably force a restart from the scratch in terms of setup and runs in a lot of areas. Kindly, help provide me (if you can) with an understanding of how to safeguard mt4 stability with tickstory usage (I simply cannot understand why failures in basic operations (such as I have reported) should cause such hassles or what I need to do to assure that I do not lose mt 4 functionality each time things don't go well with downloads/exports).

Thank you in advance (-_-)

tickstory
Posts: 4883
Joined: Sun Jan 06, 2013 12:27 am

Re: Help with Walk Forward (etc)

Post by tickstory »

Hi Darkdoji,

Appreciate if you could could outline the problem in point-form as sometimes it's not easy to extrapolate the issues you are describing. From what I understand:

Problem: You have installed Tickstory Lite instead of Tickstory Standard.
Solution: Just uninstall Tickstory Lite and re-install Tickstory Standard. All your data and settings should be unaffected. This includes all the Dukascopy tick data you downloaded and the MT4 data you have exported.

Problem: You have gaps in your data. It seems that you might have internet connection issues.
Solution: Correct the issue with your internet connection and try exporting again. If you are still having issues, use the "Download" option first in Tickstory to download all the data from the web. Make sure you get no errors (otherwise re-try the download again). Once you have no errors, you can proceed to export. You should not no longer have any major gaps in the data.

If you are getting any specific errors, could you please post them or a screenshot. This should help identify the issue more quickly than a description.

Thanks!
P.S. Appreciate your patience as our response times can vary depending on how busy things are. Rest assured however that we do prioritize our responses to subscribers.

Darkdoji
Posts: 9
Joined: Thu Jan 10, 2019 4:18 pm

Re: Help with Walk Forward (etc)

Post by Darkdoji »

Ok thanks very much for your assistance so far which is very appreciated. Please find below point by point the specific areas and questions I have. I hope this helps to make clear my needs and allows you to address each area fully in order to help me use/understand the tickstory tool more effectively.


a) How might I download/export data for a specific date range without overwriting existing data? For example, say I have existing data for 2003.x.x - 2018.x.x in the database and wish to download/export data for 2008.x.x - 2010.x.x ONLY (but without affecting existing data outside that specific range). Is this possible? How do I do that?

b) I have JForex data stored on file downloaded using JForex tools. Can I use tickstory to export that data into mt4? How can I do that?

c) In your online (help) manual, and with regard to weekend data it is stated that where no data is available "flat bars" will register instead. What are flat bars? Will they affect data quality and or bar display in any way? My broker starts its sessions Sunday every new week and I believe it is useful to capture that in exports/downloads but not sure how having "flat bars" might affect quality/display.

d) How can I validate your software delivery to be correct. That is to say, how can confirm that the link you provided delivers tickstory standard and NOT tickstory lite? I worry that the speed of exports is slow and that both the setup icon and the launch icon are blue and a shade of purple respectively. Also the store location reads C:\Users\Darkdoji\AppData\Roaming\Tickstory\Tickstory Lite\Data. All of that leads me to feel (correctly or incorrectly) that I may have had the wrong product delivered and hence the slow speed. I need to assure that this is not the case.

e) Further, each time I have experienced a process failure using tickstory, e.g. exporting data for a limited period that appears to have overwritten the overall range of existing data (for a single symbol), i) mt 4 tester fails to run (for that symbol) and ii) on shutting down mt4, I am unable to restart the platform again. This problem is so severe that on the two occasions I have had this problem, I have had to reinstall a fresh instance of mt4 losing all the data and work on the earlier platform. What do I have to do in case of a process failure not to lose the platform?

Much obliged (-_-)

tickstory
Posts: 4883
Joined: Sun Jan 06, 2013 12:27 am

Re: Help with Walk Forward (etc)

Post by tickstory »

Hi Darkdoji,

To answer your questions:

a) How might I download/export data for a specific date range without overwriting existing data? For example, say I have existing data for 2003.x.x - 2018.x.x in the database and wish to download/export data for 2008.x.x - 2010.x.x ONLY (but without affecting existing data outside that specific range). Is this possible? How do I do that?

At the moment this is not possible (although we intend to implement it). As such, for the moment please select the entire date range you wish to export.

b) I have JForex data stored on file downloaded using JForex tools. Can I use tickstory to export that data into mt4? How can I do that?

The JForex cache directory is specified in the JForex settings - the base folder will look something like C:\(AppDataFolder)\JForex\.cache. If you browse to that directory, it should contain a structure such as "(AppDataFolder)\JForex\.cache\(symbol)\(year)\(month). If you set the base directory as your "Dukascopy Data Folder" in Tickstory, Tickstory should load all the information. You will know it worked if Tickstory correctly displays the "begin" and "end" dates beside each available symbol in the main screen.

c) In your online (help) manual, and with regard to weekend data it is stated that where no data is available "flat bars" will register instead. What are flat bars? Will they affect data quality and or bar display in any way? My broker starts its sessions Sunday every new week and I believe it is useful to capture that in exports/downloads but not sure how having "flat bars" might affect quality/display.

A "flat bar" is simply a bar of data with the same OHLC, basically indicating the price hasn't moved during that period. It is personal preference whether you wish to have weekend bars or not. Some people omit weekends as it skews their analysis.

d) How can I validate your software delivery to be correct. That is to say, how can confirm that the link you provided delivers tickstory standard and NOT tickstory lite? I worry that the speed of exports is slow and that both the setup icon and the launch icon are blue and a shade of purple respectively. Also the store location reads C:\Users\Darkdoji\AppData\Roaming\Tickstory\Tickstory Lite\Data. All of that leads me to feel (correctly or incorrectly) that I may have had the wrong product delivered and hence the slow speed. I need to assure that this is not the case.

The Standard version is denoted in the Help->About menu with an "" beside the version number (eg. v1.9.2.7 ). The Standard version also has no banner at the top of the application.

e) Further, each time I have experienced a process failure using tickstory, e.g. exporting data for a limited period that appears to have overwritten the overall range of existing data (for a single symbol), i) mt 4 tester fails to run (for that symbol) and ii) on shutting down mt4, I am unable to restart the platform again. This problem is so severe that on the two occasions I have had this problem, I have had to reinstall a fresh instance of mt4 losing all the data and work on the earlier platform. What do I have to do in case of a process failure not to lose the platform?

As per question a), when you export you need to specify the entire date range otherwise it will seem like you're "missing" data. When you say that the "tester fails", are you getting an error? The MT4 platform should not need to be reinstalled in normal circumstances and if it does, it is highly unlikely that Tickstory causing is this issue.

Hope this helps.

Post Reply