LS Retail Data Director User Guide
LS Retail Data Director User Guide
May 2012
© Copyright 2010-2012, LS Retail. All rights reserved. All trademarks belong to their respective holders.
Contents
1 Introduction....................................................................................................................... 1
2 How the Data Director Works .......................................................................................... 1
3 Hardware and Software Requirements .......................................................................... 2
3.1 Hardware ................................................................................................................. 2
3.2 Software .................................................................................................................. 2
4 Installation ......................................................................................................................... 3
4.1 Prerequisites .......................................................................................................... 3
4.2 Security Considerations ........................................................................................ 3
4.3 Installing the Data Director ................................................................................... 4
4.4 TransAutomClient Control .................................................................................... 5
4.5 Change of Computer Host name .......................................................................... 5
5 Configuration .................................................................................................................... 6
5.1 DD Hosts ................................................................................................................. 6
5.2 Configuration Data for DD Host............................................................................ 7
6 Distribution Location Setup and Ports ........................................................................ 17
6.1 Port and Network Mode Setup ............................................................................ 17
6.2 Routing via Other DD ........................................................................................... 18
6.3 Routing via backup DD Host ............................................................................... 18
7 Offline & Manual Transfer.............................................................................................. 19
7.1 Dynamic IP Setup ................................................................................................. 19
7.2 Manual Transfer ................................................................................................... 19
8 Job Monitoring................................................................................................................ 20
9 Event Viewer ................................................................................................................... 22
10 Maintenance .................................................................................................................... 23
A. Data Director Connection Strings................................................................................. 23
Microsoft SQL Server – Standard Connection ............................................................ 24
Microsoft SQL Server – Trusted Connection .............................................................. 24
Microsoft Dynamics NAV Native Server – Standard Connection .............................. 24
Microsoft Dynamics NAV Native Server – Trusted Connection ................................ 24
Microsoft Dynamics NAV SQL Server – Standard Connection ................................. 24
Microsoft Dynamics NAV SQL Server – Trusted Connection .................................... 24
Microsoft Dynamics NAV SQL via SQL Driver – Standard Connection ................... 24
Microsoft Dynamics NAV SQL via SQL Driver – Trusted Connection ...................... 24
Microsoft Dynamics AX 2009 – Standard Connection ............................................... 24
Microsoft Dynamics AX 2009 – Trusted Connection .................................................. 24
OleDB – Standard Connection ...................................................................................... 24
OleDB – Trusted Connection ........................................................................................ 24
B. Install Options................................................................................................................. 25
C. Definition of Terms ......................................................................................................... 25
D. Field Type Mapping ........................................................................................................ 26
1 Introduction
The Data Director is a product of LS Retail ehf, the developers of the LS Retail Back Office
system and the LS POS point-of-sale application. The Data Director is closely tied to these
applications and plays an integral part in enabling the system to function as a whole in a
distributed environment.
The Data Director is an application specialized in moving data between databases in a fast
and efficient way. It works intimately with the Microsoft Dynamics NAV platform but it is not
limited to that platform. It can just as easily move data between other databases such as MS
SQL Server & MySQL. Generally, if the database supports the Microsoft OleDB or ODBC
interface the Data Director should be able to use it.
The Data Director is a flexible tool that can be adapted to a variety of data transfer scenarios.
The most commonly used configurations are for:
Moving data between the head-office, stores and Point-of-Sale Terminals (POS
Terminals) in a Dynamics NAV based retail organization.
Moving data from Dynamics NAV based databases into a data warehouse running
on Microsoft SQL Server.
The Data Director (DD) is run as a service and listens for incoming requests or packages. The
client component (like the TransAutomClient) is used from LS Retail in order to interact with
the Data Director Service and tell it what to do. There are two types of interactions: Request
to read data from the database, and request to write data to the database.
If the DD receives a read instruction, it will start by connecting to the source database. It then
proceeds to read data from the database and stores it in a package. The package can contain
data from more than one database table.
Once the requested data has been read, the DD has two possibilities. The first is to write the
data in the package directly into another database or to forward the package to one or more
DDs. Once the package has been received by the receiving DD, it can proceed to write the
contents of the package into one or more destination databases, all according to instructions
contained in the package. This provides an easy and convenient way to transfer data
between different databases, since the DD converts the data along the way, making sure the
destination database understands it.
The feature of being able to forward one package to more than one DD is very useful in the
retail environment since you often have to distribute price changes or updates to some
product items to all or some of the stores. This is done automatically by the DD once it has
been configured, enabling the user to focus on running the stores.
The DD and client components do not need to be installed on the same computer. The client
component can connect to a DD running on other host computers. This makes it easy to
create a network of Data Directors that can be controlled from one central location.
Chapter 1 - Introduction 1
User Guide - LS Retail Data Director 3.02
3.1 Hardware
At least 1 GB of available RAM. The base processes use around 100-200 MB of
RAM for normal operation. The amount of RAM required depends on size of
database queries that DD needs to perform.
100 MB disk space for the base application. This is the absolute minimum space
required. The additional disk space required when moving data depends on the
amount of data and how frequently it is moved. You should have at least 10 GB free
on your hard drive for the temporary data generated by the DD.
A Dual Core processor or better. The DD is a CPU intensive application, especially
when processing data, so a faster processor usually means improved performance.
3.2 Software
Refer to the release notes of the DD for supported operating systems and
databases.
The DD needs 2-3 client sessions in the database it is running from. The number of
client sessions depends on the functionality used and Data Thread setting in the
Configuration tool. For normal operation DD uses 1 session for query data, one
session for insert/update and then 1 or more sessions for incoming TS requests.
The databases which the DD should connect to need to be accessible using TCP/IP.
4 Installation
4.1 Prerequisites
There are a few things, as listed below, that you need to keep in mind before setting up the
Data Director on Dynamics NAV. For Dynamics AX installations, please contact LS Retail
support for further information.
You should have a solid understanding of the TCP/IP networking protocol. You
should know how IP addresses can be assigned and be able to use tools such as
ping and tracert. You must be able to assign names to an IP number, preferably
using a DNS server or the local hosts file.
You should know how to work with Microsoft Windows Services and view events
from the Event Log.
You should have a working knowledge of Microsoft Dynamics NAV. The setup of the
DD might require an installation of a Dynamics NAV database server.
Make sure that you have a Dynamics NAV License file (.flf) that allows you to access
the DD application objects within Dynamics NAV and the CFront API.
Make sure that you have the necessary permissions to install programs and be able
to start and stop services on the computer running the DD service.
Most database systems allow database administrators to set user access permissions
relatively easily. It is strongly recommended that you spend some time specifying access
permissions for the user account that the DD will use to access your database. For example,
the Dynamics NAV security system provides a powerful feature that limits a user’s access to
database tables only, making the user account useless to regular users since they do not
have access to the database’s graphic user interface. Similar features can be found in most
other database systems. If this feature is available, it should preferably be used for all user
accounts the DD uses.
Chapter 4 - Installation 3
User Guide - LS Retail Data Director 3.02
DD Service: This is the actual DD Service that handles all the data transfer. If you are
installing only client controls for Navision to be able to connect to DD Service on another host,
then deselect this option and only the Client files will be installed.
Client Tools: This will install Configuration Tool and Monitoring Tools for Data Director. If
you don’t want the tools to be available on this host, then deselect this option. DD can be
configured and monitor remotely.
DD2 Client for NAV: If you are installing this on Navision host and Code Unit uses either
CFront controls or DD Controls from previous version of DD (DD2), then check this option.
This installs all the DD2 Client files that are needed to NAV to communicate with DD. There
is no need to install LSRetailDDClient-2.28.x.exe anymore. If you already have DD2 installed
on the host, then there is no need to check this option.
SC Scheduler Plugin: This is the Scheduler Plugin for LS Retail Store Controller. Check this
if you plan to run Scheduler and DD in the Store Controller.
SC Monitor Plugin: This is the Monitor Plugin for LS Retail Store Controller. Check this if you
plan to use Enterprise Monitoring in the Store Controller.
DD3 can also run with DD2 Service installed, if you want to have both DD2 & DD3 on same
computer, you need make sure that either one of them is running or change the Incoming Port
on DD2 to another than the Default port 16750.
No configuration is needed to start up DD3 with default settings. After install, the DD will start
up with default configuration. If DD should connect to database with trusted connection, you
need to add Login to the DD Service to log on as a User that has access to read and write the
data to the database.
Open Configuration Tool and upload the DD license, a demo DD license is installed when DD
is installed. If DD is connected to NAV, upload the NAV Licenses file also under App Config
tab.
Chapter 4 - Installation 4
User Guide - LS Retail Data Director 3.02
Changing the control should be possible without any other change of code; all functions are
the same and should stay the same after the control swap.
If you decide to use TransAutomClient that comes with DD3, the name of the new control in
the Automation list is TransAutomClient and the Class is TransAutomClient. You should also
change out this call in the Code unit to get better error messages from the DD.
This change will display the real error message from the DD System, and not some number
that usually does not tell us much.
Chapter 4 - Installation 5
User Guide - LS Retail Data Director 3.02
5 Configuration
No special configuration is needed to start up the DD with default settings. When the DD
Service is started for first time, the DD takes care of setting up default values. To access the
configuration data, open up the Configuration Tool. The Configuration Tool takes care of all
setting for the DD Service, Debugging and Status Monitoring. This tool can also connect to
the remote DD Host and manage the configuration and debug the remote side.
When you open up the Configuration Tool, you get the configuration for the DD at the Local
Host. To get live configuration from remote system, press Get Config button. The host name
will have green color if configuration has been loaded, otherwise it’s red. Black just means
that the host is selected. If you hover with the mouse over the field you can find information
on what each field does.
5.1 DD Hosts
The fields above the Configure area are to add more DD3 hosts that you want to be able to
remotely administrate. Enter the Host name of the DD host you want to add, and click on the
Update button. If the DD Service is not running on the default port and network mode, enter
the port number and then select the network mode. If you need to pass through another DD
Host, to be able to access this Remote host, select the Passing through host in the Host list,
and click on the Set as Parent button to add it as Parent for this DD Host. The Connection
string is only used for testing connections to databases from the DD Service. To test a
connection to the DD and DB, click on Test Connection button.
To change any information for a DD Host, double-click on the host in the host list and the
information for that host will be loaded into the fields. To update the data, click on the Update
button.
Chapter 5 - Configuration 6
User Guide - LS Retail Data Director 3.02
To remove a host from the list, select the host in the list and press
the Remove button.
To Import long list of hosts into the list, press Import button and
select file to import and press OK. The File to import needs to
have the Hosts one per line, with no other information.
To Filter Hosts or look for specific hosts, you can type in the filter field above the host list
some filtering text, this filters by text contains, so if you type POS, you will get all Hosts with
POS in the host name.
You can also connect to remote DD3 Services and fetch the configuration setup from the
remote host, by selecting a host from the list, and clicking on the Get Config button. You can
change the settings, and then resend the configuration back to the host by clicking on Send
Config button. When the configuration is sent back it will be reloaded live on the remote host,
so there is no need for a restart on the remote host.
You can select more than one host in the list to get configuration data from or an update.
Press the Shift or Ctrl button while you select more hosts. All values that are same for all
selected host are open for editing. You can also send DD and NAV license files to many hosts
at once.
Chapter 5 - Configuration 7
User Guide - LS Retail Data Director 3.02
Archive Folder When the DD is done processing a job, it will move the Job data file
to this location. Files are stored here for long as the Keep Archive
Data value is set to. This data can also be picked up my Monitoring
service for reporting in the Monitoring system.
Aliases By default, the DD works on current Computer Host name. You can
assign Alias names to the DD Service so if a Job that is assigned to
DDTEMP comes in and DDTEMP is included as Alias, the DD will
process the job. If you are upgrading current DD2 setup, then add
all previous DD2 Service names in here.
DD Class Id Class Id used in Monitoring system. This is to identify group of DD
network together under one Class id. This can be used to group
them together in the Monitoring system.
TCPS Port Secured Connection port for incoming connections. The DD service
needs to run on an account that is available on both locations, like
same domain.
TCP Port Basic Connection port for incoming connections. This is standard
open Connection between hosts, and is useful if data is going from a
Domain network to a workgroup network where no trusted
connection is available. This is the default mode to communicate
between hosts.
Errors in Memory The DD stores in memory the latest error messages that it has
reported. Indicate here how many messages it should keep. Even if
there are errors in memory it does not mean that the DD currently
has some problem, it may be that the problem has already been
solved. The purpose of this is just to look at the latest errors that
have popped up in the DD. These errors can also be seen in the
Event viewer under LS Retail Events folder.
Job Memory Timeout The DD stores all jobs it has processed in memory for x min. This is
to set for how long the DD should hold on to this information. When
using the Job Monitor to monitor the DD, it loads this list into the
grid, when it connects, so you can see list of processed jobs from
the last x min. This value only applies to Jobs that are Done
processing not any unprocessed jobs or jobs that are in an error
state.
Error Wait This tells the DD how long it should wait till it tries to process error
job again.
Max Error Retry If this value is set to higher value than 0, then DD will try to process
Job that has error until the Max count is reached, and then put the
job On Hold. If Value is set to 0, DD will try to process the job until it
can or someone Cancels the job. To release job off hold, go to Job
Monitor and release the job.
Keep Archive Data Days to keep the Job data files in Archive folder. If Monitor
Collection is setup, then the Archive data files will be deleted as
soon as the data has been collected, so then this value is needed.
Status Check Time This is the time between when DD Service checks for statuses on all
unprocessed jobs at Destination hosts. If the network connection
goes down, and Source DD misses the status message from the
destination that the job has been processed, the Source DD will
send status request to the Destination and check for update. This is
same status check as can be done manually from Job Monitor.
Host Conn. Timeout This is a Timeout for Connection to a Destination host. After DD has
forwarded a Job to Destination, it keeps the connection alive and if
there is no activity to that host, it will disconnect after this time. This
can be useful if the connection is not stable.
Send DD License Select Hosts that you want to send updated or new DD License file
to and then press this button, choose the license file and press ok.
This will generate a Job that will be sent via DD Network with
updated DD License file.
Show Lic Show DD License information at the DD Host
Chapter 5 - Configuration 8
User Guide - LS Retail Data Director 3.02
Update Program To update DD at all locations, select Hosts that you want to send
update to from the host list, press the Update Program button.
Choose the new setup file and choose the install options that you
want the install to include when the installation job is executed at the
destination host. A Job will be generated that sends the setup file to
all destinations. At the destination location, the DD System will be
shut down, Setup executed and System started up again. As the
setup does not report back whether it was ok or not, you need to
check the location to see if it has the new Version number after the
update. You can see the Version number under Debugging tab.
Allow Program Update Allow Remote program update. If not checked, Update Program
does not run on this host.
Chapter 5 - Configuration 9
User Guide - LS Retail Data Director 3.02
Old DD Support Activate Support for communication with DD version 2 and DD2
Client controls.
Data Port Incoming Port for DD2 Job Connections. If you are upgrading
current DD2 setup, you need to add in here all ports that were used
in the previous setup, including all TS ports.
NOTE: DD needs to be restarted for this to take effect.
Mon Port Incoming Port for DD2 FOB Import Jobs and File Transfer Jobs (old
DD2 monitor port). This port does not support other Monitoring
commands or queries.
Timeout DD2 Socket Timeout for send and connect.
Allow File Transfer Allow File Transfer via Monitor port to support DD2 File replication.
Offline / Pull Support Activate Job Pulling and/or Manual importing of Jobs for offline
locations. If DD is only supposed to send Manual/Offline job, then
this setting is not needed, it’s only needed on host that is supposed
to import Manual/Offline job.
Pull Host DD Host to connect to for Job Offline pulling.
Pull Time Time between Pull requests to DD Host or Check Manual import
files.
Pull Path Path where Manual Job files are located.
Scheduler Support Activate Scheduler Service that runs the jobs created in Store
Controller application.
Job Id Database Id of a job that you want to execute in Scheduler. This is
useful to test a Scheduler job, if something is wrong, error message
will be displayed.
Reload Trig / Run Job If Job id field is left blank this will generate Reload message in
Scheduler Service to reload all Scheduler triggers. Use this if a
trigger has been added or changed manually, not via the SC
Interface. If Job Id is entered, this will execute Trigger to run that job.
Test Conn Use to test the connection string for Scheduler Service.
Conn String Database Connection string where Scheduler Tables are stored.
Chapter 5 - Configuration 10
User Guide - LS Retail Data Director 3.02
Plugins Select which plugin to run for collection and on what interval:
ArchiveDataCollector: Collects all Done Jobs and data from
Archive folder.
WorkDataCollector: Collects data for all Jobs running in DD
memory.
Route Path Distribution Service Host and Path to send Job to.
DB String Database Connection string for Database where Data should be
inserted into.
Add Path Add Database destination (Route Path & DB String) to Destination
list. Job can be sent to more than one location.
Monitoring Service Activate Monitor Service Process to process Monitoring data.
Plugins Select which plugin to run for Monitor Service and on what interval:
DataProcessor: Check Host statues and check for Email
Triggers.
Test Conn Use to test the connection string for Monitor Service.
Conn String Database Connection string where Monitor Tables are stored.
SMTP Srv SMTP Email Server to send Trigger Emails
Email From From Email address used in Trigger Emails
Base Port The base Port used for all local DD Processes. Each process has an
id and the Port number for that process will be Base Port + Process
Id. This generates the incoming port number for the process that will
be used for debugging and other communication. It’s not necessary
to open these ports in firewall unless you want to be able to remotely
debug the system. It’s not needed for normal operation.
Data File Size The maximum Size of a Job Data File to be transferred between DD
hosts.
Message Size The maximum Size of Messages that the DD sends between
Processes and Hosts. This includes TS Requests, Configure
messages and Result Messages. If you get error message when
sending large TS Request, you need to increase this value to handle
the whole data transfer required for the TS Request.
Chapter 5 - Configuration 11
User Guide - LS Retail Data Director 3.02
Compress Mode Compression Mode to use on Data Files. Default mode is ZipStream
which is same as DD2 uses. This does not affect DD2 compatibility.
Mode File Size Process Speed
ZipStream Medium Medium
ZipFile Large Fast
SevenZip Small Slow
None Large Fast
It can take some load off the Store server to set Compression mode
to None on POS that is sending Transactions in DD mode to a Store
server on same network. These jobs are small and no need to
compress the data. Compression is only needed on slow
connections via WAN. If jobs are large and you are running 32bit
OS (x86) then you may run into Memory issues with compression.
ZipFile mode allows larger files on 32bit OS so you can use that
mode for larger jobs.
Shutdown Wait The time the DD Service waits between each Process when it is
shutting the system down.
Router Wait The time the DD Service waits to start all other processes after it
starts the Router. This may be changed if there are a lot of jobs
waiting and the Router takes a long time to load the Jobs back into
memory before it is able to open incoming connections.
Kill Wait The time the DD Service waits for processes to shut down before a
kill command will be sent out to kill remaining processes.
Process Thread limit Sets the limit of threads used when forwarding jobs to next host.
This limit is used so the Server where the DD is running on does not
get overloaded.
Data Thread limit Sets the limit of threads used when updating destination database
servers. This limit is used so the Server where the DD is running on
does not get overloaded.
Thread Priority Sets the Priority on Database Thread for Job Processing. This does
not affect the Interactive (TS) processing.
Lock Code Set Password so DD Configuration cannot be changed remotely.
Open Timeout The time provided for a connection to open before the transport
raises an exception.
Send Timeout The time provided for a write operation to complete before the
transport raises an exception.
Receive Timeout Time that a connection can remain inactive, during which no
application messages are received, before it is dropped.
Max Connections Value that controls the maximum number of connections to be
pooled for subsequent reuse on the client and the maximum number
of connections allowed being pending dispatch on the server.
Timeout values
There are few timeout values to adjust. To let POS not hang if Store server is down you can
set the TS Timeout value in the Distribution Location card. This affects the total allowed
transfer time that POS can take when sending request to DD/TS. If the value is not set there
then the Open and Send Timeouts in System Config will apply. If DD/TS is not finished
processing the request before TS Timeout expire, the request will be canceled. So be sure to
have this value long enough for the request to finish otherwise it will end good request.
These values need to be set in the Configuration on the POS Host. You can run the DD
Configuration tool to set these values there, even if the Host is not running any DD Service.
When setting these values on a host running DD Service, then make sure the send value is
set to long enough time for the largest job to be able to be forwarded to next host. Also do
not set TS Timeout values in Distribution card for DD Jobs send by scheduler, as it may let
the scheduler fail sending jobs to DD Service.
Chapter 5 - Configuration 12
User Guide - LS Retail Data Director 3.02
DD uses 2 threads to process Database jobs. One is for Query jobs and another for
Insert/Update. This means that DD can be processing both jobs at same time. DD does this
for each host that its connecting to, so if the jobs are set to use databases on 2 different
hosts, DD will have 2 Database threads for each host, one doing query an done doing
insert/update.
The Data Thread Limit value, limits DD on how many Databases DD will process at same
time. If the value is set to 2, DD can process 2 Databases at same time, and can process
both query and insert jobs for each host, so total 4 threads can be running. If this value is set
to high value, DD will take up most of the CPU Power on the host.
You can also set Thread Priority for the Database threads. If you set the priority on low, then
even if DD has taken up 100% CPU load, other programs on the computer should not
become slow in operation. This priority only affects the Job Processing it does not affect
interactive requests like TS Queries.
When playing with these values, there is no one correct way of doing this, it all depends on
what kind of data is being replicated and how powerful the computer is that DD is running on,
do you should monitor the CPU Load, Disk Load and Network Load when adjusting this, and
set the values to something that the computer, database and network can handle.
5.2.5 Debugging
Here you can look at the Debug stream from each DD Process and also look at the Job
Queue in the Router. If you run into a problem, this is the place to check what is going on.
Local DD version and build numbers are also displayed here.
When contacting LS Retail because of problems, you may be required to send the debug files
and/or some other info that you can get from the Debugging tab.
Queue Show Processing Queues within the Router. To update the status
click Queue button again. (see 5.2.5.1)
Global Queue Show List of all Jobs that the DD has been processing and also all
jobs waiting to be processed. To update the list, click the button
again.
Dump Mem If there are problems is with the Router: it does not send jobs to
processing or something locks up, press this button to dump the
Queue Memory to file for debugging.
Chapter 5 - Configuration 13
User Guide - LS Retail Data Director 3.02
Errors Shows the latest errors detected by the system. The number of
errors shown here is based on the Errors in memory settings in
Base Config.
App Status Shows the Running status of DD Processes.
Job Monitor Starts up Job Monitor for selected host.
Debug File Size The maximum Size of each Debug file. When a Debug file reaches
this size a new file will be created.
Debug File Count The maximum number of debug files to create. When the limit is
reached, the DD will override the first Debug file.
Client Ctrl Level File Debug level for Client Control Debugging. This is used when
Client app starts up and if the value is greater than 0, the DD will log
data to file.
Keep Data Files Keep Job Data files after processing. Useful if you need to look at
data files if data is replicated, but the data is not correct.
Startup Levels File Debug level for each DD Process. This is used when the DD
starts up and if value is greater than 0, the DD will log data to file.
In order to find out which debug level value you should put in the Startup Level & Client Level,
you can use the Level Calculator to calculate the value for you. Check the levels you want to
log and then you can double-click on the Startup Level field and it will fill out correct value for
you. See more detail about debug levels and DD Processes in 5.2.5.3
If you look at the XML Data files from the work folder you may see that the text strings look as
if they only contain garbage text. You can copy this text and paste it into the Base64 String
field and press Convert to see what it actually means.
5.2.5.1 Queue
The Router has few Queues where it stores all jobs waiting to be processed. Here you can
see the status of these queues and if any jobs are waiting there to be processed.
If no Job is waiting, you see list of available Queues and no data within the queue.
Chapter 5 - Configuration 14
User Guide - LS Retail Data Director 3.02
If a Job is waiting in the queue, you see Host sub queues within the processing queue, and
then Job sub queues within the Host queue. Click on either the Host queue or Job queue to
see how many packages or jobs are waiting and overall status of the queue.
Offline If Host is offline host, the Job will wait until that host will connect and
collect the jobs waiting to be forwarded to that host.
Error If there is any error and error message associated with the queue.
Waiting If the job is doing Database access or being forwarded to next host, the
router waits for status update on the job process. If the queue get stuck in
processing mode with waiting as true, the queue can be reset by pressing
the Reset button.
On Hold Queue has been put on hold and will not be processed until taken off hold.
Data Cnt Number of Jobs waiting in queue.
ReRoute button is to forward a whole forwarding queue to another host. This can be useful if
the host running DD is down, but Database server is still running, the whole queue can be
rerouted to anther host that has also access to the database server. This will not modify the
Database Connection string. Type in new DD Host in the field and then press ReRoute
button.
Check mark Word Wrap to wrap the information text, if a long error message is present.
Refresh button will refresh one queue node and Refresh All will refresh all Queues.
From here you can shut down each process and start it up again. They may be useful if one
process gets stuck or has problem, or if a new update is available for that process you can
shut down one process without interfering with other processes and do the update.
Service Main Data Director Service application. This is what you have in Windows
services and it takes care of keeping other processes alive.
Router Heart of Data Director. Takes care of routing jobs to correct DD processes
and other DD hosts.
Chapter 5 - Configuration 15
User Guide - LS Retail Data Director 3.02
This will activate File Debugging on the host where DD is running. If the
Log to File on
check mark is left on after you disconnect from the DD Process, debugging
Host
will continue logging into the file.
Max Lines Maximum number of debug lines that Live debugging will show.
Clear Clear Debug stream window.
Save Save data in Debug stream window to file on local host. You can also select
debug lines from the Debug stream window and press CTRL-C to copy them
to clipboard so you can past them into other applications like email or
notepad.
Connect Connect to DD Process and start Debugging.
Chapter 5 - Configuration 16
User Guide - LS Retail Data Director 3.02
LS Retail NAV 6.3 has new form that includes setup for these modes and ports. You can
select what DD Mode to use and then type in port number in the field next to Distribution
Server name field.
If you are using older LS Retail NAV versions, you can still set this values, you need to type
them into the Distribution Server name field.
Below is list of all Network Modes the DD Uses and the Default Port for each of them:
Mode NetMode Port Description
TCP TCP 16860 Standard TCP
TCPS TCPS 16850 Secured TCP
DD2 DDOLD 16750 Old Socket Connection for DD2
OFFLINE OFF For Pulling host
MANUAL MAN For Manual Job Transfer
If the DD host is behind Firewall, make sure you open up access for both DD3 (16860) & DD2
(16750) ports, or give add DDRouter.exe and DDIncoming.exe to the Firewall allowed
programs.
As mentioned in the Configuration Tool section, a location can have a parent location and in
this case a parent location for POS01 is STSRV. When the DD sends a job or any other
request to POS01, the Job will first go to STSRV and then be forwarded to POS01.
There is no limit on how many child hosts can be added, just separate the hosts with
semicolon ; The order how job will be routed is the same order as the hosts in the list, and
the last host it the host that will process the job.
In Distribution Location Card, add the backup host to the Distribution Server name with % as
separator. You can have more than one backup hosts. If at Head Office you want to have 2
different computers running DD3 for incoming jobs from Stores, then at the Stores, the Head
Office card should include both DD3 hosts at Head Office like this HODD1%HODD2.
If a Primary host goes down, the job will be routed to Backup Host, and from that point all job
will go to the Backup Host as long as it stays alive. To Re-Route the jobs back to Primary
Host, stop the Backup Host DD, and jobs will go back to the Primary Host.
If any error occurs on either of the hosts, you can see from which one of them the error
comes from by looking at the Error Host value in Job Monitor.
If you are routing jobs via one DD (PASSDD) to another (STDD) and want to have backup
host for both locations, the routing path setup would be like this:
At the Distribution Location at HO for the store, you need to set the OFF property to the DD
Host, so the DD Host looks like this: “STR01:16860:OFF”. HO setup at the store can just be
normal setup with the TCP/TCPS Property.
At the store activate DD the Offline / Pull Support in the Configuration Tool. Enter the HO host
name as Pull Host and indicate how often the Store should check for data.
When HO has a Job ready for a Host that has OFF Property, it holds on to the data For that
location until a Pull request comes in from the store.
At the Distribution Location at HO for the store, you need to set the MAN property to the DD
Host, so the DD Host looks like this “STR01:16860:MAN”. The same goes for the HO setup at
the store, it needs the MAN Property also.
Activate at both locations the Offline / Pull Support in the Configuration Tool. Enter in the Pull
Path field where the DD should look for incoming data files and how often the Store should
check for data.
When a job is sent to MAN location, the data files will be copied to Outgoing folder where
Data Files for the DD are stored. These files should be moved out of the folder when they are
transferred over to the destination host. At the destination host, put the files into the incoming
folder specified in Pull Path. The DD will then import the files and remove them when the job
has been processed. There are 2 files that need to be transferred, xxxxHEAD.XML & xxxxx-
RDAT.ZIP. Both these files need to be put into the Pull Path on the destination host.
Note: The Job Monitor at Source Host will not get any report back for this job so you will see
Status Unknown for the Manual locations. The Job Monitor at Destination host shows if Job
was processed or not.
8 Job Monitoring
This tool listens to all Jobs that are being processed and passing through the DD Service.
The upper grid shows you Source Processing, and the lower grid shows Destination
Processing.
When the program is minimized, it will go down to the Tray area, and the
LS Icon at the tray area will change color depending on current status. If
Error message occurs a window with error message will pop up. If you
right click on the Tray Icon, you get menu where you can disable any
popup messages (Enable Tooltips) set if Job Monitor shows also in
Taskbar when minimized (Show In Taskbar) Reset Error status, open up
the Configuration Tool and Convert XML Data file so the values are
readable.
Default Job Monitor view jobs by Pack Id, but you can also switch the view to view Hosts.
Select Host View in the Tray Menu. You will get view where there is one line by host and if
there is error you see latest error and error message for that host.
Enter the Host name and Port of the DD Host to connect to. When you press the Connect
button, all jobs that are stored in memory for the DD service are loaded to the grid. The
number of jobs you get when you connect is based on the value Jobs Memory Timeout that
you can find in the Configuration Tool. After that, the grid will update by itself when new jobs
come in or a current job status is updated if the Live Check box is selected. If you want to
stop the live feed, uncheck the Live Check box and the grid will not be updated with the latest
Jobs and Statuses. This can be useful if the system is very active and you need to work with
the data in the grid, and you do not want it to update all the time.
At any time you can Disconnect from the DD and Clear the List, by clicking on the Clear
button in disconnected mode, and when connecting again, you will get all jobs again that
remain in DD Memory.
Load Done loads all done job from DD on Connection, this only applies when connecting to
DD. This should not be checked if there is large amount of Done jobs in memory as it slows
down DD while the list is being loaded into the monitor.
If you want only to see Unprocessed Jobs or Jobs with error, place a check mark in the Hide
Done check box. If you want to see all done jobs again, uncheck the Hide Done check box.
If you want to clear out Done Jobs, place a check mark in the Clear Done check box. Note
that if you place a check mark in the Clear Done, these jobs will not show up again in the grid.
To see processed jobs again, you need to disconnect from the DD Host, Clear the list, and
reconnect.
If job has an error, the Icon will change color. Green is when you have connection to the DD
Host and no errors have been reported. Yellow is Warning and Red is Error. A job with error
will also color the job line Red, so it is easier to find all jobs with error. If a job has an error, all
jobs with the same time, will just wait and not turn red, so only the first job will have the red
warning color. If job is On Hold, it will show up in Yellow-Green Color.
To only show Error Jobs, check Error only check box, when unselected all jobs will show
again.
You can press the Reset button to reset the Tray Icon back to Green, and when Job is
updated and it still has an error, the Tray icon will go back to a warning color.
Snooze Error messages for a Job is only available in upper grid and it will tell the monitor to
not update the warning color or status of that job. This is useful if a POS is offline and you
already know about it, and it will be so for a few hours. Then you will not get a warning for
every time the DD tries to process the job to that POS.
In both grids you can right-click on the job line to cancel the processing of a job if the status
allows you to do so. NOTE: You will be asked if you want to cancel just the job you selected
or all jobs of same time for that Host. To cancel only one job, select NO. If you have a Job
going to a POS that is not going to go online again, you can cancel all Jobs by selecting YES.
In the upper grid you also have the option to Cancel Job on all Hosts. This will cancel the
job on all remaining hosts, left to process the job, so there is no need to go on each host in
the lower grid to cancel each one of them.
In the lower grid you have option to put job On Hold. If job is on error because host is offline,
you can select to put the job on hold and DD will not try to send the job to the host until the
job is released again from on hold. When job is on hold the color of the line will turn into
Yellow-Green color. If Max Retry count is set, and job reaches Max Count, job will be
automatically put on hold, and you care release the job here to let DD retry job processing.
Ignore Monitoring will stop all further status update from that Host and job status on the
monitoring host will get status Done, no matter if the job is still at error state on the destination
host.
If job has an error or has not yet been processed, you have the option of looking at the Data
files for that job by right-click on the grid and select View DataFile. The upper grid shows you
the Query file, and the lower grid shows the Result file. The data file will be opened in the
default XML Viewer. If the Keep Data Files option is check marked in Configuration, then you
can also look at the Data Files after the Job has been processed. Make sure to clean up the
Data files when you do not need them any more to free up space.
Modify DB String allows you to change the Database Connection string for either the Source
or Destination Location. This will update the Connection string for either one job or all job of
same type for same host.
You will notice in the Destination grid that some Hosts have the status Unknown. This status
is shown for example on a POS/Store that is part of a list of POS/Stores that the job was sent
to and the DD at that location will never get status update of what is going on at other
POS/Stores, only HO or the Host that sent the job will get that update. All other locations will
have the status Unknown. Also, you will see this status if a job is sent to DD2 or Manual
transfer, and it is because DD2 does not send status report back so DD3 will never know
what is going on there.
9 Event Viewer
DD and TransAutomClient control report all errors into Event Viewer under the folder LS
Retail Events.
This can be used to collect error into Monitoring system like MS System Center or HP Open
View. In the Error text are more details about at what host the Error comes from, where the
job is origin from and Pack Guid. The DD Event IDs are:
1001 – Warning
1002 – General Error
1003 – Network Connection Error
1004 – Database Processing Error
9999 – Information message
10 Maintenance
Once installed, the DD servers should require very little maintenance. Nevertheless there are
a few things to keep in mind in order to keep them running smoothly.
Keep the Dynamics licenses and DD license up-to-date. If the customer wants to
add new granules to his license, the license needs to be updated throughout the DD
network.
Monitor the free disk space. The DD stores the packages on the local disk for
temporary storage. If this space runs out, the DD cannot continue to process new
requests.
Check the status of the Data Folder (see path in Configuration Tool). This is where
the DD keeps all Job data files, temp files and other files it uses for its operation.
This folder can grow fast if something is not working as it should.
Clean up the Archive Files. All processed Job files are archived under the Archive
path, set in the Configuration Tool. Each data file is not big but this folder can grow
in size with the number of files in there, so if history of jobs is needed, his should be
backed up. If not, these files can be deleted. If you are not interested in keeping any
history of processed jobs other than the list that the DD keeps in memory, you can
remove the Archive path from the Configuration Tool, and then the DD will delete the
Job files when done processing them.
Chapter 10 - Maintenance 23
User Guide - LS Retail Data Director 3.02
For information on how to put together connection string for your database, see
http://www.connectionstrings.com/ this site has samples of most common database servers
and data providers that are out there.
Chapter 10 - Maintenance 24
User Guide - LS Retail Data Director 3.02
B. Install Options
Data Director Install can be executed with parameters to make to use for scripted installation.
Detail of what options are available can be found here
http://unattended.sourceforge.net/InnoSetup_Switches_ExitCodes.html
To install DD in silent mode and include DD Service and Client files you add these
commands: /VERYSILENT /SUPPRESSMSGBOXES /TASK=”srvfiles,clitools”
C. Definition of Terms
Following is a list of terms and titles often used in connection with the Data Director along with
a brief explanation:
DD Service – This is the server component of the Data Director that keeps all other
DD Processes alive and monitors the health of the system. If any process goes
down it restarts the process, if possible. The DD Service runs as a service on the
host computer.
TransAutomClient.dll – Used by LS Retail for sending queries to the Data Director.
Package – A package contains the data being transferred between the databases. A
package has a destination Data Director and a destination database which can be
one or many, depending on how it was created. This essentially represents a unit of
work for the Data Director.
Host Computer – The computer running the service part of the Data Director.
Source Database/Data – This is the database that we want to read data from.
Destination Database/Data – This is the database that we want to write data into.
Scheduler Database – This is the Dynamics NAV or Dynamic AX database that
contains all the details regarding tasks that the Data Director should perform (when
to send, which tables, which fields etc.). The Data Director log and the Scheduler
can also be configured to reside in the same database.
Scheduler – The Scheduler is used for scheduling Data Director Activities at
predefined intervals, such as sending sales data to a Head Office database every
hour. The Scheduler can be run either in a Dynamics NAV client, via Dynamics
Navision Application Server (NAS) or from Dynamics AX. The Dynamics AX
scheduler will not be described in this document.
DD – An acronym for the Data Director.
Chapter 10 - Maintenance 25
User Guide - LS Retail Data Director 3.02
Chapter 10 - Maintenance 26