Openquant 2014 Faq: Frequently Asked Questions
Openquant 2014 Faq: Frequently Asked Questions
03.11.2015
SmartQuant Ltd
Dr. Anton Fokin
How to build bars from bids, asks or middle prices .................................................................................. 3
How to download historical data in scenario............................................................................................ 3
How to save downloaded historical data to OpenQuant database .......................................................... 3
How to backtest a strategy with downloaded historical data .................................................................. 4
How to access data series in OpenQuant database .................................................................................. 5
How to pass additional parameters to instruments ................................................................................. 6
How to define a number of digits displayed in instrument prices ............................................................ 6
How to develop and register a custom object streamer .......................................................................... 7
How to access n-bars ago bar in a bar series ............................................................................................ 9
How to work with multiple time frames ................................................................................................... 9
How to filter market data and other events ...........................................................................................11
How to develop complex custom indicators ...........................................................................................12
How to set parameters for a particular instance of Instrument Strategy...............................................14
How to change the FileServer port .........................................................................................................15
How to control event bus latency in the real time mode .......................................................................16
How to change path to local data files (instrument and market database) ...........................................16
How to do things periodically (on every session open, close and so on) ...............................................17
How to change chart attributes ..............................................................................................................18
How to plot bids, asks, trades and bars on the same plot ......................................................................20
How to control reminder event order ....................................................................................................22
How to use ExecutionSimulator.Queued property .................................................................................22
How to build bars from bids, asks or middle prices
There are certain cases where you have to build bars on the fly using bids and asks or their combination.
Particularly, FX brokers such as IB don’t provide trade information in their price feed.
You can use BarInput parameter of the BarFactory.Add() method to indicate what data type should be
used for bar construction for this particular BarFactoryItem (or Instrument).
The following code line tells to the BarFactory to build one minute bars for MSFT instrument from
bid/ask middle prices
Use DataManager.GetHIstoricalData .
The Scenario class (as well as the Strategy) class references all managers of the framework
(InstrumentManager, DataManager, etc.) for your convenience, so that you can use them directly to
simplify your code.
The following code snippet downloads historical one minute (60 second) bars from Interactive Brokers
(IB) for Microsoft (MSFT) instrument for last seven days. Indeed you should have MSFT instrument
properties properly set to work with IB (instrument exchange should be set to SMART or other valid IB
exchange, currency to USD and so on).
Use DataManager.Save(IDataSeries) method to save a series of data objects into OpenQuant database.
Since TickSeries and BarSeries (as well as many other data series classes in SmartQuant framework)
inherit IDataSeries interface, you can call this method to save series of ticks and bars downloaded in the
previous FAQ.
The following code snippet downloads a bar series of historical one minute bars from IB and stores them
in the local OpenQuant database.
DataManager.Save(series);
The DataManager.Save(IDataSeries, SaveMode) method has an option to either add all data from
IDataSeries (SaveMode.Add) or append data (SaveMode.Append). If you select SaveMode.Append
option, only objects with datetime greater than datetime of the last object in the OpenQuant database
will be added to the database.
DataManager.Save(series, SaveMode.Append);
If you don’t want to add all objects from the data series, you can add objects one by one using
DataManager.Save(IDataObject, SaveOption) method
You don’t necessarily need to save downloaded historical data to OpenQuant database to backtest a
strategy with this data. You can simply add downloaded series of historical bars or trades to the list of
external data series of the data simulator and run a backtest.
strategy.AddInstrument("AAPL");
DataSimulator.Series.Add(series);
DataSimulator.SubscribeAll = false;
ExecutionSimulator.FillOnBar = true;
StartStrategy();
}
Perhaps if you run the code above, you will see a mixture of data stored in OpenQuant database and
data from downloaded series. In order to avoid this, you can disable subscriptions to data series stored
in OpenQuant database changing DataSimulator.SubscribeTrades, DataSimulator.SubscribeQuotes, etc.
or DataSimulator.SubscribeAll properties
strategy.AddInstrument("AAPL");
BarSeries series = DataManager.GetHistoricalBars("IB", "AAPL",
DateTime.Now.Date.AddDays(-7), DateTime.Now.Date, BarType.Time, 60);
DataSimulator.Series.Add(series);
DataSimulator.SubscribeAll = false;
StartStrategy();
}
Finally it’s a good idea to check that you have FillOnBar enabled in the execution simulator, since now
you don’t have any other data than one minute bars in your simulation. Otherwise there will be no
order fills in your strategy.
strategy.AddInstrument("AAPL");
DataSimulator.Series.Add(series);
DataSimulator.SubscribeAll = false;
ExecutionSimulator.FillOnBar = true;
StartStrategy();
}
Use DataManager.GetDataSeries(name). The following code snippet returns a reference to series of one
minute bars of AAPL instrument.
Data series name can be constructed for bar series using the following rule
or
You can also use DataManager.GetDataSeries(Instrument, dataType, barType, barSize) method. The
following code returns the same one minute bar series as the code above.
DataSeries series = DataManager.GetDataSeries("AAPL", DataObjectType.Bar,
BarType.Time, 60);
Instrument.Fields property is persistent. Objects added by users to the object table are stored in the
instrument data base when InstrumentManager.Save(Instrument) method is called. Note that objects of
user defined classes should have corresponding streamers and such streamers should be registered in
the StreamerManager of SmartQuant Framework. Standard classes, such as double, int, boolean and
string already have registered streamers.
You can learn how to develop custom object streamers in the corresponding FAQ in this document.
Set parameters
instrument.Fields[0] = 10.25;
instrument.Fields[1] = 5;
instrument.Fields[2] = "Shortable";
InstrumentManager.Save(instrument);
Get parameters
You can also use GetDouble(), GetInt(), GetString() methods of ObjectTable class
You may want to show prices in different formats depending on instrument type. For example you may
want to display six digits after the dot for forex instruments.
You can use Instrument.PriceFormat property to set a format string used in GUI or in your own
applications to display instrument prices. Default value of this property is “F2”, which means two digits
after the dot. The following code line changes it to four digits after the dot for MSFT instrument.
Actually the price format parameter is a standard C# ToString() numeric format string, so that you can
follow C# rules to display instrument prices as you need:
http://msdn.microsoft.com/en-us/library/0c899ak8(v=vs.110).aspx
In order to demonstrate how to store custom objects in SmartQuant Framework, we create custom
FIXField class that contains an integer FIX tag and a string value, and a simple FIXMessage class, which is
just a dictionary containing FIXField objects.
We also create corresponding streamers for these classes, FIXFieldStreamer and FIXMessageStreamer
and register these streamers in the framework.xml configuration file.
FIXField class
using System;
namespace CustomStreamer
{
public class FIXField
{
int tag;
string value;
FIXMessage class
using System;
using System.Collections.Generic;
namespace CustomStreamer
{
public class FIXMessage : Dictionary<int, FIXField>
{
public void Add(FIXField field)
{
Add(field.Tag, field);
}
FIXFieldStreamer class
using System;
using System.IO;
using SmartQuant;
namespace CustomStreamer
{
class FIXFieldStreamer : ObjectStreamer
{
public FIXFieldStreamer()
{
typeId = 201;
type = typeof(FIXField);
}
writer.Write(field.Tag);
writer.Write(field.Value);
}
}
}
FIXMessageStreamer class
using System;
using System.Collections.Generic;
using System.IO;
using SmartQuant;
namespace CustomStreamer
{
class FIXMessageStreamer : ObjectStreamer
{
public FIXMessageStreamer()
{
typeId = 202;
type = typeof(FIXMessage);
}
for(int i=0;i<count;i++)
{
FIXField field = (FIXField)streamerManager.Deserialize(reader);
message.Add(field);
}
return message;
}
writer.Write(message.Count);
Now we register our custom streamers in the framework by adding the following lines in the Streamers
section of the framework.xml configuration file:
<Streamer>
<TypeName>CustomStreamer.FIXFieldStreamer, CustomStreamer</TypeName>
</Streamer>
<Streamer>
<TypeName>CustomStreamer.FIXMessageStreamer, CustomStreamer</TypeName>
</Streamer>
You can simply use BarSeries.Ago(n) method, which is equivalent to BarSeries[BarSeries.Count – 1 – n].
In this FAQ we will modify SMA Crossover example so that instead of crossing two SMA with different
lengths of 1 min bar series, it will cross SMA of 1 min and 5 min bar series.
First of all we change Scenario.cs file so that the BarFactory builds 1 min and 5 min bars on the fly from
trade data.
Define two BarSeries, bars1min and bars5min. We will use these series to store bars of different time
frames.
BarSeries bars1min;
BarSeries bars5min;
Create BarSeries objects in OnStrategyStart method and use them as input series of corresponding SMA
indicators. Note that we use one Length parameter, which we set to 14 in our demo
Then all we need to do is to add right bars into right series in the OnBar method
if (bar.Size == 60)
bars1min.Add(bar);
if (bar.Size == 300)
bars5min.Add(bar);
Now we are ready to run the strategy. The screenshot below shows strategy chart where you can see
bars of two different time frames as well as SMA indicators and strategy transactions.
How to filter market data and other events
EventManager.Filter can be used to filter (or modify) event stream in the SmartQuant framework. An
event filter is an object of a class derived from the EventFilter class.
Here is an example of a simple event filter that filters out of session market data events.
using System;
using SmartQuant;
namespace OpenQuant
{
public class SessionEventFilter : EventFilter
{
private TimeSpan time1;
private TimeSpan time2;
return e;
}
}
}
All you need to do to develop a simple custom indicator is to derive your indicator class from Indicator
class of SmartQuant Framework and override its Calculate(int index) method.
OpenQuant 2014 installation includes a demo solution with several indicators that you can use as a
template to develop your own custom indicators.
Below we publish source code of EWM (Exponential Weighted Mean) indicator which you can study as a
reference and which we will use in this chapter to develop a complex indicator.
for
(see http://en.wikipedia.org/wiki/Moving_average)
using System;
using SmartQuant;
namespace SmartQuant.Indicators
{
/// <summary>
/// Exponential Weighted Mean (EWM)
/// </summary>
public class EWM : Indicator
{
double length;
double initValue;
A complex indicator is an indicator that depends on another indicator. For example a EWV (Exponential
Weighted Volatility) indicator uses EWM (Exponential Weighted Mean) to calculate volatility as
(see http://en.wikipedia.org/wiki/Variance )
In order to understand issues related to the development of such complex indicator, we should
understand how indicators get calculated in the SmartQuant Framework.
An indicator has its input series, which is passed in the indicator constructor. When you create an
indicator its instance is added to the list of indicators of its input series. I.e. when you call new
SMA(barSeries, 14); SMA indicator instance is added to barSeries.Indicators list. When a new bar is
added to the barSeries, the Indicator.Calculate method is called for all indicators in the
barSeries.Indicators list. This means that indicators are updated automatically behind the scene when
new entries are added to their input series. Since Indicator class is derived from TimeSeries class,
Indicator also has a list of Indicators. This means that you can easily construct indicators that use other
indicators as input series (construct indicators on indicators).
Note that indicators are updated one by one in the order in which they were added to the Indicators list
of their input series. This may create problems in situations where you need to update indicators in
certain order.
using System;
using SmartQuant;
namespace SmartQuant.Indicators
{
public class EWV : Indicator
{
double length;
double initValue;
EWM ewm;
EWM ewm2;
TimeSeries input2;
public double Length
{
get { return length; }
}
ewm.AutoUpdate = false;
if (index > 0)
{
Console.WriteLine(ewm2.Last + " " + Math.Pow(ewm.Last, 2));
1. Call Strategy.Init() - this creates strategy tree and instances for instruments added in the
Strategy.AddInstrument.
InstrumentStrategy.GetStrategy("SMACrossover (MSFT)");
4. Set parameters.
Code:
strategy.Init();
((MyStrategy)strategy.GetStrategy("SMACrossover (MSFT)")).Qty = 100;
Code:
Note that in this case your parameter (field or property) should be marked with [Parameter] attribute in
your strategy code.
Also note that parameter values for parameters marked with [Parameter] attribute get copied among
instances when you call InstrumentStrategy.Init or run the strategy (which calls init behind the scene if
you don't call it in your code).
((MyStrategy)strategy).Qty = 100;
strategy.Init();
or
((MyStrategy)strategy).Qty = 100;
StartStrategy();
It might happen so that FileServer default port (1000) is already in use on your computer. You can
change the port number, which the FileServer uses to accept client connections, by modifying FileServer
configuration file. When you run the FileServer for the first time, it deploys the configuration.xml file
into AppData/Roaming/SmartQuant LTD/FileServer /config directory. You can modify this file to change
the port number
Next time you run the FileServer, it will accept clients on this new port (245).
Don’t forget to change InstrumentFilePort and DataFilePort accordingly in the OpenQuant 2014
configuration file.
If you use an old version of OpenQuant 2014 and don’t have these lines in your configuration.xml file,
you should add them manually to connect to the FileServer on 245 port.
The Event Dispatcher of the SmartQuant Framework constantly tries to dequeue an event (market data,
execution report and so on) from the Event Bus and pass it to your trading strategy or application. If the
Event Bus contains no events, the bus turns into the idle mode. Such idle mode can be organized in
several ways. The event bus thread can go to sleep for a while, it can spin until it has an event to
dequeue, or it can wait until it gets a notification about a new event added to the bus. This behavior can
be controlled with the EventBus.IdleMode option, which can be Spin, Sleep or Wait.
EventBusIdleMode.Spin adds almost no latency but it eats 100% of one core of your CPU since it
constantly checks for a new event in [while empty do nothing] loop.
EventBusIdleMode.Sleep calls Thread.Sleep(1) and adds 1ms latency if you have a stream of rare events,
but on the other hand it almost completely releases the CPU.
EventBusIdleMode.Wait is the default idle mode, which is optimal and adequate for almost every task,
including UHF trading. You can change it to Sleep if you don’t care about millisecond latency but don’t
want to load your CPU. Alternatively, you can change it to Spin if you have a multicore CPU and require
the lowest possible latency.
How to change path to local data files (instrument and market database)
You can configure SmartQuant framework and OpenQuant application by editing configuration.xml file
that can be found in your AppData\Roaming\SmartQuant Ltd\OpenQuant 2014\config folder.
Note that OpenQuant installs this file when you run OpenQuant application for the first time (as well as
demo solutions and GUI settings), so that you should run OpenQuant at least once to be able to find this
file.
Data access settings are particularly interesting for us and we are going to discuss these settings in this
and next chapters.
OpenQuant setup installs instrument definition and market data files into
AppData\Roaming\SmartQuant Ltd\OpenQuant 2014\data folder. These files are instruments.quant and
data.quant. If you want to change location(s) of these files, you can move them to another place on your
computer and then edit <InstrumentFileName> and <DataFileName> entries in the configuration file to
reflect this relocation.
Note that you don’t need to type .quant extension in the instrument.quant and data.quant file names in
the configuration file.
Note that OpenQuant application should be closed when you edit the configuration file, otherwise you
changes will be overridden with the current OpenQuant configuration settings when you close
OpenQuant.
How to do things periodically (on every session open, close and so on)
....
AddReminder(Clock.DateTime.Date.AddDays(1).Add(SessionOpen));
}
You can change colors, styles and other attributes of objects that you draw on a chart using
corresponding group parameters. Assume you create a group for a bar series to draw it on chart pad 0
using instrument symbol as a selector key (to switch between charts for different instruments). You can
find this code in the SMACrossover demo strategy:
Bar series will be drawn with default attributes. You will see a candle chart with white and black candles.
barsGroup.Add("CandleBlackColor", Color.Green);
barsGroup.Add("CandleWhiteColor", Color.Yellow);
barsGroup.Add("ChartStyle", "Bar");
You can use the following attributes for different drawing objects:
BarSeries
FillSeries
BuyColor
SellColor
TextEnabled
TickSeries
TimeSeries
How to plot bids, asks, trades and bars on the same plot
GroupManager.Add(barsGroup);
GroupManager.Add(bidsGroup);
GroupManager.Add(asksGroup);
GroupManager.Add(tradeGroup);
Note that if you add bar group first, ticks will be plotted on top of bars and vice versa, so the order is
important.
Log asks, bids, trades and bars in the corresponding event handlers in your strategy
In the simulation mode reminders are triggered by time stamps of data events. If you have a data event
with the same datetime as you set in the reminder, there are two possibilities - reminder event can be
emitted before or after this data event. Assume you set a reminder to be fired at 10:30 and you have a
tick (let’s say a bid) coming exactly at 10:30. The framework can process this bid first and you get OnBid
call in your strategy and then your reminder will be executed. Alternatively the framework can execute
your reminder first, call OnReminder in your strategy and then call OnBid.
You can control this order using ReminderOrder property of the EventBus.
What potentially may happen if you do something “too fast”, like sending an order and then
replacing/cancelling it in the same OnBar or OnReminder, is that you send an order to the Execution
Simulator, the simulator accepts the order, fills it, and places execution report into the Event Bus queue.
This execution report is in the queue, but not in the Order Manager yet, so that the order still has New
status although it has already been filled in the Execution Simulator. Then you can try to replace/cancel
the order in your strategy but the Execution Simulator rejects your request sending out cancel/replace
reject report.
This is pretty normal and reflects real life. Your order can already be filled on the broker server but you
don’t know about it yet and ask broker to cancel it. Indeed your cancel request will be rejected.
You can control this scenario using ExecutionSimulator.Queued = false. Then your order will be executed
directly without sending reports via EventBus queue.