Forum Replies Created
-
AuthorPosts
-
@ratsh wrote:
Longterm AmiBroker user speaking here:
FYI AmiBroker is not dependent on any predefined fix format to get data imported to it. There is no such thing as “fix format”. AB works with any format defined. It also provides millisecond data which is not even covered in those two predefined formats. In short those two formats supposed to work with Amiquote are just two of thousands possibilities as there are in addition any other standard data file extensions supported also. Also you don’t need Amiquote to get data files imported since you can use any custom data plugin or OLE import script or the two manual file import options in File menu of AB. This is just meant to clarify to remove misunderstanding of how AB works. It’s not like MT or NT who only work with fix datafile formats. It is highly flexible. So saying this or that 3rd party software now supports Amibroker for data import sounds quite funny to insiders.
Side note: aqi, aqd, aqh,aqfe other Amiquote file extensions are just standard comma separated ascii data files also and are readable with any texteditor. So there is no mystery about them and no hidden protection.
Hope this clears up a few tings.
##################################################################################################################
Here is one (out of many) examples showing how to import bid/ask price and volume with millisecond timestamp to AmiBroker at once.
##################################################################################################################In Tickstory choose:
Timeframe: Tick
Timezone: whatever you need
Output format: CustomCheck include header
Header info:
Timestamp,Bid price,Bid volume,Ask Price,Ask volumeData format (timestamp is again just one of many possiblities):
{Timestamp:yyyyMMdd HH:mm:ss.fff},{BidPrice},{BidVolume},{AskPrice},{AskVolume}File name: for example use {Symbol}.csv
and start downloading
##################################################################################################################
With notepad++ create a file with following content (this is a “to be done just one time” procedure for each format you need):
##################################################################################################################$FORMAT Date_YMD,Time,Close,Volume,Aux1,Aux2
$SKIPLINES 1
$SEPARATOR ", "
$CONT 1
$GROUP 255
$AUTOADD 1
$DEBUG 1
$TICKMODE 1
$TIMESHIFT 0
and save it with file extension .format (for example TickstoryBidAsk.format) and copy that file to …AmiBrokerFormats subfolder
Note:
Timshift line is set to zero but if you want to shift to other time zone during import then for example change to $TIMESHIFT 2 which would shift the time stamp by +two hours while $TIMESHIFT -4 would shift by 4 hours backwards (for example if you want import different timezone as the one you have set in tickstory download. Of course you need to take care of summer/winter time, so you would have to split data file if summer/winter time range is part of same file and need to adjust $TIMESHIFT line for summer and winter time import. Anyway $TIMESHIFT is not needed if you are happy with downloaded timezone. Just showing that it’s possible to change time zone after download also.)Go to same subfolder AmiBrokerFormats, open import.types file placed in there and add line (for example):
TickstoryTickBidAsk (*.csv)|*.csv|TickstoryBidAsk.format
(you can see that the format file info at the end of the line needs to have same file name as you have saved your format file to before)save that import.types file
Now you are finished with preparations.
Open AmiBroker, create a Tickstory database, go to File –> Import Ascii, choose file format “TickstoryTickBidAsk” in drop down menu and browse to your tickstory download folder to open your previously downloaded Bid/Ask data file(s) separated or all at once (by marking them).Of course you can use programmatic import too but that’s another short story.
One thing to do while creating your Database and being related to upper example is setting data source to local database since it is off-line data and setting base time interval to tick there. In addition go to File –> Database settings –> Intraday settings of our database and set Aux1 to Last and Aux2 to Sum (since latter one is ask volume). Also set start minutes to “00” from “30” to start hourly TF at 00.
see picture

I followed your instructions. Importation goes well, no errors. Takes some time, but its obviously because of the size of these tick files which average atleast around 10gigs each. I imported just 1 for a test. Worked. So to save it, I shut down Amibroker, which took a long, laggy time, about 10 minutes or so to shut down on my pc with 4 cores @ 4ghz and 16 gigs of ram. Then I restarted the program, start up took equally long with similar lag. Then I tried to pull up the chart and this seems to be an impossible task. Memeory usage skyrockets to max, program becomes unresponsive, pc lags. I let this go on for about to hour to see if it would load eventually and then had to end process.
Do you not experience any issues with your tick files? I download the fill length, 2003-now for EURUSD. That was my test file. I thought Amibroker was even better than MT4 @ handling data, backtesting, and optimizing. It seems I’m unable to even pull up a chart.
Thoughts?
in reply to: Large data jumps #19807I’m back from the dead.
So was this implemented into the “duplicate ticks” tab?
If not-
1 are these anomalies in the dukascopy BI5 files or are they an export/tickstory issue?
We can solve this by seeing if multiple users, just 2 instances of the same anomaly will be enough, to prove its the BI5 files or tickstory export.If we reach the conclusion its the BI5 file (but I for some reason don’t think it is) then the best solution would be to delete those ticks (since its only very few which actually cause this) if the standard deviation is more than 10 or something huge like that, since these spikes tend to all be enormous. This will stop 99.9% of all real world events from being filtered out. The only exception that comes to mind is the CHF currency peg removal earlier this year but it wouldn’t be very difficult to add an exception rule for extremely rare events such as that.
I believe that it might not be the BI5 files is because SQ Tick Data downloader doesn’t have these, but its possible it has a built in filter. (?)
My reason of preference toward tickstory though is the FXT export and MT4 launcher, which SQ Tickdata downloader doesn’t support.in reply to: Large data jumps #19805A side query: You said latest versions of tickstory has filters implemented to help eliminated some of these anomalies. Does this have options? Can you tell me a bit about this, how it works?
Do you know of any spreadsheet programs that can handle files of this size?Excel stops importing @ 1,048,576 rows and will not import any more after that even on additional sheets. I know there are some solutions such as file splitting I think there is a better solution.
I think the best solution, if I cannot edit the .CSV as easily as a .HST, would be to just edit my .HST files in MT4 and make my own .FXT files and pretty much backtest the old fashioned way since Duka’s default data is too unreliable for good tests. I must say tickstory is brilliant and I do love it! The duka data is a real shame however.
in reply to: Large data jumps #19803Am I able to remove these from .FXT or the original Duka BI5 files?
I can edit .HST as you know but then I would have to overwrite tickstory’s FXT and create new ones via MT4.
Has there been any solutions to getting these out?Should I edit .HST, produce new FXT with MT4 backtest, then run from tickstory in the future after?
Currently its impossible to test on this data.
[attachment=0:2uuff1zs]Untitled.png[/attachment:2uuff1zs]I am running MT4 build 610 windows 7 UAC OFF and latest version of Tickstory.
Tickstory is on E:Tickstory
MT4 on G:MetaTrader
Windows C:
I am able to select the location of my MT$ in Tickstory settings and set up the server as well.
I have exported HST and FXT files and to their proper folders-
G:MetaTrader 4historyFXCM-AUDDemo01
G:MetaTrader 4testerhistoryFXT are set up as read only.
I press F8 in Tickstory to boot MT4.
It boots.
I get this message from Tickstory as many others have mentioned:
[attachment=2:vmnnbmm7]Untitled.png[/attachment:vmnnbmm7]I close the message. My MT4 has normal title title:
[attachment=1:vmnnbmm7]Untitled1.png[/attachment:vmnnbmm7]Run a test on READ ONLY fxt.
Fails to load anything. Errors:
[attachment=0:vmnnbmm7]Untitled3.png[/attachment:vmnnbmm7]When I turn OFF read only it works. But the FXT is overwritten.
-
AuthorPosts