Data Validation Routine

Have some interesting ideas about how to improve the software? Let us know in this section!
Post Reply
sryoung12001
Posts: 2
Joined: Fri Jan 02, 2015 12:56 am

Data Validation Routine

Post by sryoung12001 »

TickStory Data Validation Process
TickStory Data Validation Process
I would like to suggest TickStory include a separate Data Validation Routine.

Reason: When downloading data, I had a number of data errors show up during the process (possibly a poor internet connection, or Dukscopy was busy, or some other reason), so that, at the end I had no idea whether my data set was complete. A manual examination of the folders showed my suspicion to be correct - I had folders with missing days or missing months, even missing years, and day folders that did not have 24 of the data files. So I 'cobbled' together a process which examined the data files created by Tickstory and gave me a summary of any missing data (i.e. any one day did not have 24 hour files). A Screen shot of my process is attached.

I would like to suggest that Tickstory include such a data validation program, so their uses can be confident that their tick data files are complete prior to transferring to mt4 or some other target format

tickstory
Posts: 4899
Joined: Sun Jan 06, 2013 12:27 am

Re: Data Validation Routine

Post by tickstory »

Hi Sryoung,

Thanks for the suggestion. We should be tackling this two-fold in the later versions. Firstly, we will most likely stop the download process on encountering any error. Secondly, we will be providing a gap analysis function that should highlight data gaps similar to the way your program is doing.

For the moment, I would highly recommend the use of the "Download" feature to re-check and re-download missing information where necessary. You can make this feature can go through your entire date range and it will only download information that is missing or corrupted. Of course, you should ensure no errors are generated during the operation as it indicates some problem during the download process. Note also that whenever you attempt an export, the same verification & repair process is applied.

Regards.

Post Reply