Historical Data Formats in Timing Solution (updated November, 2009)

Overview

I often get a question: "Can we use historical price data from other programs?" For example, you use Trader
Station system or something else. And you want to use the price information from this program.

Right now Timing Solution software supports:

1. a lot of different Text formats; 

2. CSI format;
3. Metastock format; 
4. e-Signal data feeding
5. downloading from Yahoo website
6. MetaTrader 4 data feeding
7. AmeriTrade service

Actually, almost all data feed systems allow saving price history data into text formats (Excel). Usually Timing
Solution is able to read these files (now, in April, 2009, it supports more than 1000 different text formats). If we
face with something very specific, you can always send me the example of this data (e-mail it to
[email protected]), and this format will be added to a special program called "Rare Format". This
program converts specific text formats into the format readable by Timing Solution software. Therefore,
Timing Solution covers practically all available data formats. To download the "Rare Format" program, click
here.

When you open the price chart, you will get this window:

 
Choose one of the files, and the program automatically finds the appropriate format for this text file. You can
use any price data files with *.csv, *.txt, *.prn or *.asc extensions.

The program supports MORE THAN 1000 DIFFERENT TEXT FORMATS. Here are some samples of text
files:

01.01.1900 68.13 68.13 68.13 68.13 7.00 0.00

02.01.1900 66.61 66.61 66.61 66.61 8.00 0.00
03.01.1900 67.15 67.15 67.15 67.15 7.00 0.00

02.01.1962,7.69,7.69,7.60,7.60,8100
03.01.1962,7.60,7.67,7.60,7.67,6100
04.01.1962,7.67,7.67,7.59,7.59,5500

730101, 132.850, 133.000, 132.850, 132.850,0

730102, 132.600, 132.750, 132.600, 132.750,0
730103, 132.800, 133.000, 132.800, 133.000,0

Date,Open,High,Low,Close,Volume
17-Apr-03,8255.79,8388.28,8172.52,8337.65,14306000
16-Apr-03,8405.72,8459.15,8217.27,8257.61,15876000
15-Apr-03,8347.66,8443.11,8251.38,8402.36,14602000 

03/08/1991,264.56,267.46,262.80,262.86,0
03/11/1991,262.86,262.86,254.75,255.05,0
03/12/1991,255.18,255.18,250.31,250.67,0

15/02/1881 12.57366693
15/03/1881 12.83411791
15/04/1881 12.74243939

02/25/03,06:45,820.5,820.5,820.5,820.5,1
02/25/03,07:45,823,824.75,823,823,23
02/25/03,08:45,823.5,823.5,822.25,822.25,21

18.06.2001 08:30 AM 67.9500 67.9600 67.5900 67.7100 1329900.00

18.06.2001 08:35 AM 67.7100 67.9200 67.5000 67.8000 754200.00
18.06.2001 08:40 AM 67.8000 67.8500 67.5100 67.5700 589000.00

And many, many others...  

Actually, this module works as a mini expert system that automatically analyses the content of the file and finds
the appropriate format for it. We worked on this module for 7 years, starting with few formats and adding new
ones. For the last 6 months, I have not heard that somebody has had any problem with reading some text
format.

Here are more examples:

Downloading from www.yahoo.com website.

If you are downloading the price history from free Yahoo financial service, follow these steps:

a) click this button:

 

b) run Yahoo service:

c) put the content of this table into the clipboard:

 

d) download the price history from the clipboard:

Yahoo downloaded troubleshooting :

1)  If your operating system is Vista, you may get a  stand alone Excel application (with "Navigation to the
webpage was canceled" prompt): 
This is Windows security system.

2) Sometimes it displays ##### symbols instead of data information in Excel window:

You need to extend A column to have enough room to display the data (drag the mouse to extend this column):
Also I recommend to expand F column with Volume information:

Interactive Brokers

To download through Interactive Brokers software follow these steps:

a) Run Interactive Brokers software (Trader Workstation -TWS)

b) Follow this:

 
c) Choose All API Settings:

and click there "Create" button:

type there 127.0.0.1 and click OK:

You should do this procedure once only.

 
d) In Timing Solution click here:

 click this button and choose "Interactive Brokers" item:

Fill out this form and click these buttons:

 

Downloading from Trader Station

 Run Omega Downloader program:

 Load any securities (if you have not loaded them before), like this:
 

 File => Export => Security

  It gives you this dialog window:

  

 Choose any security and press “Export”

You will get this window:

   Click “OK”.

 Input the file name

and click “OK”.

 You can use this text file as a data file for Timing Solution program.

PS. Sorry, the samples are for Trader Station v. 4. I know that there are newer versions. And I know that
there have been no problem with them as well.

 Real time/data feeding in Timing Solution

Data feeding
Updating

Real Time

Before discussing anything, we need to distinguish between the data feeding and real time. The data feeding is
the way to link with any data feed system, to be able to obtain historical prices either from some text file, or
through Yahoo/e-Signal/any other system. The real time is the possibility to upgrade the price history data
online during the working process. In other words, you do your regular tasks while at the same time the
program automatically upgrades the price history. 

The policy of Timing Solution team is: we do not charge for data feeding, though we charge for the real time
version (which is one time payment of $150 US).

Data Feeding

Let's start with the data feeding option. Now (February, 2007) Timing Solution software is able to provide data
feeding (and real time as well) from different text files (more than 400 formats are supported) and e-Signal.
Also, there is a pilot version for data feeding through Quote Tracker software.

 Let's start with e-Signal system. Click this button: 

Here you can choose the data feeding system (one of the two). Highlight e-Signal:

Then click "Data Manager" button:

Wait a few seconds till you get this message informing you that the program is ready for data feeding:

If you will get this message:

it means that:

1) either e-Signal is not installed on your computer;

2) or somehow your Operating System does not allow to run e-Signal software. In this case, run e-Signal before
running Timing Solution.

Now you are able to open the price history file. Click on this button: 

Another button becomes available:

Click it. Type the symbol, choose the bar,  and the program will download the price history:
 

Updating

If you already have downloaded some price history and would like to update it, click this button:

Or you may prefer using TS Work Sheets. The idea is simple: you are doing your work and save it into some
file. Next day you run your computer, open your yesterday's work - and you need to get the price chart with the
latest upgraded price history. This is the general idea of worksheets. You can create them using these buttons: 

To force the program to upgrade the price history, first define the data feed source. Do it while saving your
worksheet: 
As an example, let us say that you update you price history through QCollector program, using its text file.
Choose "from File" option and click the button "Data Feed File". There select the file to upgrade your data.

Real Time

Real time is another feature. Pushing the "Active" button, you command the program to do regular upgrades of
the price history. Timing Solution will automatically upgrade all its windows like Neural Net, Astronomy
modules, etc. Do not forget to sign for real-time data provider service (like e-Signal); the subscription to this
service is separate from TS and is governed by the rules of that provider. 

There are some parameters that you should know about. The "Interval" parameter defines the periodicity of
updating. Set it at your convenience:

You can activate the real time regime here:

Important: If you have decided to work in real time, set the prediction horizon big enough:
 

There is the option "Set LBC on last bar". If you set it ON, the program will move the LBC to the last price bar
when it comes. This new position of the LBC is used in all modules of Timing Solution.

For example, you run Neural Net in real time regime. The program will upgrade the forecast in respect to the
new available price history:

Intraday data: Daily, Daily-Intraday and Weekly-Intraday and Time

Zones
 
  

Daily, Daily-Intraday, Weekly-Intraday 

In Timing Solution, it is possible to work with different kinds of financial data - like daily data, daily intraday
and weekly intraday. Please be careful with this issue because the setting related to a specific kind of data can
strongly affect the projection line. For example, when working with daily-intraday data, you should define the
non trading hours, and the program will not display the projection line on non trading hours. To do that, the
program needs to know what type of data you are dealing with.These parameters are defined here:

Sometimes, when you download the price history file, the "Trade Days/Hours" label is flashing; it means that
the data you are working with and your chosen settings do not match.

Click  button. Here you can set the kind of data you are working with.

Timing Solution operates with three types of price data. It is very important to distinguish these data because
the manner of calculating the projection line by Neural Net module strongly depends on it.

1) Daily  data - when we have one price bar per one day. The sample of such data looks like this:

Working with this type of data, we need to exclude weekends and holidays:
The projection line generated for this data sample is shown here:

In this case, the program generates a projection line on a daily basis and skips weekends/and holidays
automatically. 

2) Daily Intraday data - when we have intraday data stream during the day. For example, if it is measured by 5
min ticks, we will get for just one day many 5 minutes price bars. In other words, this type of data allows to
observe the intraday dynamics of the price change, and this dynamics is limited by trading hours (i.e., from 9:30
am till 4:00 pm).

This is the typical example of such kind of data:

For any day (except weekends and holidays), we have 6:30 hours bunches of price data. In this case, you should
set up the time when trading starts and ends during the day

Look how the Neural Net will generate the projection line under these options:

The program will skip non trading days and non trading hours (these periods are marked by red stripes under
the time scale). If you will try to set daily options for this kind of data, you will not get the intraday projection
curve. The program must know what kind of data it operates with.

3) Weekly Intraday data - this kind of data correspond to intraday data within trading hours. The trading
begins (as an example) at 9:30 am Monday and stops at 4:00 pm Friday. During this five day period, we have
continued intraday data stream (like for Forex). This is the typical example of weekly intraday data:
  

Here we have the data where the trading begins at 3:00 am every Monday and ends Saturday at 7:00 am (local
time) every week.

For this particular example, set these options:

This is the projection line generated by Neural Net under these options:
We have series of continuous 5 days data. If you try to use Daily-Intraday for this kind of data, you will not get
the projection line for non trading hours (like on Wednesday from 4:00 pm to 9:00 pm).

Time Zones

When you download the price history in Timing Solution, you will get this window:

As you see, here you define the time when trading begins (for intraday data this option is grayed), Time Zone
and coordinates of the trading place.

Pay attention to that time zone. See the sample setting below, it is 5 hours West of Greenwich: 

The problem is that during the year we usually switch to summer or winter time, like for New York we have
4W summer time and 5W winter time.

To set up these options, click on this button:

and choose one of the files of Time Zones history (we provide  data for New York, Chicago, Moscow, London,
Hong Kong and Taiwan time zones histories).

Now the time zones are taken from this table:

This option: is useless now, because we have a detailed price history for this exchange.
However, the program uses it if we do not download the file with time zone history. 

You can also define these options here:

If you use the same time zones for all data you are working with, you can check the "Auto" option:
Thus the program will not display this window when you download the new price history while taking this
information into account.

Downloading price chart and screen manipulations

Before making any forecast, you need to decide what you will forecast and find the data for your financial
instrument. It is your responsibility to find the data that you will use for your forecast. Together with the
program, you will get the price history files for some financial instruments. They are good enough to use while
you are learning how to use the program. However, for a better forecast, you need a source of the most recent
data. You may get this info as some file, or from some data feed, or you may use Yahoo data.

The program supports more than 500 different text formats, Metastock and CSI formats. Also you can
download the price from Yahoo financial service and e-Signal system.

To download the price history from some previously saved file, click this button:

And follow these steps to download the price history from your text file:

You need to choose the directory where this file is located, highlight the file and click "Load" button. Also
define the forecast horizon, i.e., the length of your forecast.

To download CSI or Metastock data, use these tabs:

 

If you are downloading the price history from free Yahoo financial service, follow these steps:

a) click this button:

b) run Yahoo service:

c) put the content of this table into the clipboard:

 

d) download the price history from the clipboard:

If you are using intraday data, you need to set trading hours; this information is necessary to generate the
projection line. In order to do that, click on this button:
Choose the kind of intraday data (daily or weekly intraday):

And set trading hours here:

When the price history is downloaded, you can do any screen manipulations with your price chart.

For example if you need to view not all price chart but its small part with the details, do this:
To move this interval to see the price history prior/after selected interval, do this:

Also you can use these buttons to manipulate the viewed interval:

Move the mouse to any of these buttons. You will see there what this button is for.
To draw any charting tool, follow these steps:

Here I draw the Andrews pitchfork:

You can modify this pitchfork this way: