The Ultimate Guide To Systems Integration46 Min Read
The Ultimate Guide To Systems Integration46 Min Read
buildingautomationmonthly.com/systemsintegration/
Systems Integration is a tough topic. I’ve been planning to write this series for a long time.
This series is going to take you from zero to systems integration hero.
In this series you will learn the process I created when I first started systems integration way
back in 2008. Since 2008 I’ve refined this process to align more closely with enterprise
architecture models but at the end of the day it’s still my same process, just a little bit
cleaner
Well, over the next 8 weeks I will cover the following topics.
Each week I will create an individual post for each topic, except topic #1 which is included in
this post.
As I create these topics I will continue to add to this series and will re-release this series
when it is fully complete.
I hope you all are ready to take a deep dive into the topic of Systems Integration!
Lesson # 4: How to Use a Gap Analysis to Detail out the New Systems
In lesson 3 I discussed how to identify existing systems. Once this process is done you may
realize you need additional systems. I will talk you through how to perform a gap analysis
and how to identify the new systems you may need.
2/33
Lesson 4 described how to identify all of the systems you need to enact your use case. In
this lesson I will detail out how to handle the physical and logical connections that are
needed to make your use case work.
The Lessons
Therefore, we are going to start this series by defining each of these terms and then
detailing out how they are different.
Do you want to bring your lighting systems into your BAS so you can see their status? That is
interfacing.
At the end of the day, folks will spend thousands of dollars to get “systems integration”
done that is really simple systems interfacing that can be accomplished for quite a low-cost.
3/33
How low a cost you ask? Let me give you an example.
Say I have a lighting system with a BACnet/IP interface and a BAS that supports BACnet/IP as
well. How much should I budget to integrate these two?
You should be saying it depends on the use case. If you said that, good job it does depend
on the use case. And the use case is what determines if you need an integration or an
interface.
For the sake of simplicity let’s say we simply want to see the lights on the graphic for each
room. That is a simple interface, we are simply mapping data from a separate system.
So then how much would that cost? Would you be surprised to know that I can map 1000
light fixtures into a BAS in under an hour? Before you think that I am some integration
genius, let me tell you that hundreds of other folks have done the same thing.
How do I know that? I’ve worked on their systems and asked them how long it took!
A proper protocol will make a system interface super quick and easy.
Now here’s the deal, when is tying two separate BAS’s together considered systems
integration?
If we can connect two BAS’s via BACnet/IP isn’t that simply systems interfacing? Good
question!
Well in this particular scenario we have a customer who really likes system A. Unfortunately
due to procurement rules, budget, their manager (you pick!) they have been told they need
to go with system B.
How can this customer get system A to work seamlessly with system B?
This is the challenge folks face everyday, with different naming conventions, data structures
and all the other nuances systems integration can be a nightmare!
4/33
This is the scenario that systems integration handles. When trying to get a BAS to act as a
single front-end for multiple other BAS’s most folks just map in points and call it a day
(systems interfacing).
This leads to finger-pointing, late night job site visits, and high service bills. At the end of the
day you want it to just work!
Well, that was weird, making a trumpet noise announcing the start of the article doesn’t
work quite as well in text..
We in the BAS world tend to have a bad habit. We like to do things are way and often do not
look outside our industry for new methods and approaches. The use case is a perfect
example of this.
How many times have you seen use cases formatted according to the CSI MasterFormat.
Sure all the parts and pieces are there, but at the end of the day, most of these glimmering
pieces of structural perfection are… junk.
They are about as useful as me at an art show, sorry art lovers, I don’t get art, never will.
So, why don’t we suspend disbelief for a moment and assume that we in the BAS world can
do use cases a different way, and by different way I mean the way the rest of the world
“does” them.
Our first step to creating this use case is to identify a few key things.
So at this point you may be thinking an actor is a person, you’re partially correct. An actor,
according to most use case documentation, defines a role played by a user or system that
interacts with the subject.
A subject is the system or systems that should perform a task based on the use case
scenario.
Ok, so time for our first question. Who are our actors and who are our subjects? I want you
to take a second and think this through, a lot of people mess up here and its important to
identify the correct actors and subjects up front.
So, pause for a second look a way from the screen and think through who your actors and
subjects are. Remember we are thinking through the use case of two BAS tying into a single
user interface.
Ok you done?
At this point we have our actors and systems identified now we need to determine how to
put these pieces together.
We know that we want a user to be able to use the new GUI to be able to access both BAS.
Do you know what we call that? We call that our main success scenario (MSS). Our MSS is
our success criteria.
If you write a use case and you have more than one MSS you are most likely combining use
cases. It is critically important that you separate your use cases to focus on one MSS.
6/33
You see, systems integration is complicated enough, the idea is that your use cases will be
built for a single purpose. That way when you create the integrations you can segment your
use cases. This will allow you to troubleshoot, code, and support your use cases individually.
So then, how many use cases do we have with the single GUI scenario. Off the top of my
head I can think of several.
Now you may be thinking, oh come on Phil, it’s a simple integrated BAS GUI. I’d caution you
to pause anytime you start to think this way. You need to think through each scenario.
For example, if the new GUI needs a login, which login does it use?
If so where?
And those are just the set of questions for the first scenario.
7/33
Description: The BAS Technician will be prompted for a login at the single GUI. The user can
use his/her login credentials from BAS 1 or BAS 2. Upon the GUI authenticating with BAS 1
or BAS 2 the user will be granted access to the application.
Description: Once the BAS technician is logged into the single GUI (see use case 1 user logs
into the single gui), the user will be able to access BAS point data and modify that data. The
single GUI will confirm and log any data modifications.
Ok, this use case has something glaringly wrong with it. Can you tell me what that is? I’ll give
you a hint, it occurs in the last sentence.
Give up? Well, here is the problem. Use case #2 is actually two use cases. The modification
use case is good, but at the end of the use case I added confirm and log any data
modifications.
Now at first glance this may not seem like a big deal. It actually may seem like I am splitting
hairs and being anal retentive. However, do you know how the single GUI will log data
modifications?
The BAS technician for BAS #1 says “It wasn’t my job to log any data modifications, your use
case clearly says the new GUI will log data modifications.”
In the BAS techs defense the use case does say that, but is that what you want?
Wait! Stop!
8/33
We aren’t ready yet! Do you know why? Because we haven’t gotten our stakeholders buy-in.
Stakeholder, there’s a hundred-dollar word for you! A stakeholder is a person who can be
impacted by the use case. To often I’ve seen, cause I would never do this right :-D, people go
and create use cases that are absolutely beautiful.
And these use cases float their beautiful selves right to the garbage can! This happens
because the use case creator did not take the time to meet with stakeholders in order to get
“buy-in”.
In order to avoid a ton of headache after the fact I encourage you to meet with your
stakeholders in order to ensure that your use cases have their “buy-in”.
This is awesome! You are one step away from being part of the BAM Nation. BE SURE TO
CHECK YOUR EMAIL AND SPAM FOLDER. You will get a confirmation e-mail to confirm that
you are a part of the BAM Nation!
Powered by ConvertKit
Yet, there is a problem. No formal process exists for systems integration in the BAS world!
But, why should we reinvent this process when a perfectly good process exists in the IT
world.
In, the IT space this process is called mapping the AS-IS architecture. AS-IS architecture
means the current architecture as it currently is in the business. There are four main
domains, or focus areas, that an IT mapping session will focus on.
9/33
Those are the Business, Data, Application, and Technology domains. These domains are
often abbreviated to the acronym BDAT. I believe, based on my experience, that any
systems integration effort should focus on all 4 domains.
BDAT in BAS
Believe it or not, there are business, data, application and technology systems in the BAS
space. What might this look like in our simple 2 BAS into a GUI scenario. Let’s move through
each of the systems.
Business
Believe it or not BAS can have business value. For example, does a certain business group
rely on reports from the BAS?
There’s nothing quite as fun as going through an integration effort only to find out that you
missed a key report that the facilities or energy management groups require.
Data
It’s all about that data!
Seriously, the whole reason we are pursuing a single GUI is to make data easier to access
and manipulate! So then, what do we need to know in this section?
We need to understand:
Application
At the end of the day the two BAS are applications. However, each one of the BAS have their
own applications as well. Some BAS allow you to configure their databases and
programming directly from the BAS and some require external software.
10/33
In this category we want to identify:
Technology
Technology makes the world go round! Well, it at least lets the BAS run!
By answering these questions we will have an accurate picture of our AS-IS environment.
With this information we are able to understand what our constraints are within our current
environment. Constraints are hard limitations that we must adhere to because of our
technologies.
For example, since both of our BAS use SQL databases we are constrained to using a SQL
database or using some sort of database translator. Knowing these constraints will become
invaluable as we move through the next several steps of the systems integration process.
Lesson #4: How to Use a Gap Analysis to Detail out the New
Systems
11/33
In a gap analysis you take the functional specifications you created in your use case and you
compare your existing system(s) to the functional specifications. In laments terms you look
at the capabilities required for the use case and the capabilities you currently have.
But, you go and buy this game and you don’t check if your computer can run it. You come
home and all you have is a $60 dollar brick because the game won’t run on your computer.
Now, for most $60 dollars isn’t the end of the world. Imagine though, if you had went and
purchased a $300,000 analytics package.
But…
You didn’t go and check if your BAS could provide the required information for the analytics
software to run. Now, here you are with this $300,000 software package and you don’t have
the points in your system that the analytics software needs! So, now you have two choices,
cancel the contract and absorb any termination fees OR go and spend the time, money, and
effort to add all the necessary points to your BAS.
Quite simply you take the requirements of your use case and you put them next to your
current capabilities.
Let’s see this in action, by continuing our scenario, laid out at the beginning of this series,
where we are integrating two BAS into a single graphical user interface (GUI).
12/33
Our ultimate goal is to have a single GUI that the two existing BAS can tie into to provide full
access and control of real-time systems and historical data. This GUI shall use existing logins and
not require new logins.
Real-Time Systems
We need a way to capture and control the existing data points from the two BAS. The new
GUI supports BACnet/IP
Historical Data
We need a single database or a way to access the existing two databases and pull them into
the new GUI. The new GUI supports SQL
Existing Logins
We need to allow the users to continue to use their current logins and passwords. If the
user changes their login or the user is removed then the GUI needs to reflect this. The new
GUI stores its user name and passwords in a SQL database but does not support external
user names and passwords.
The big gap here is, how do we get and manage the new user logins into the new GUI?
Simple right?
In any database you have what is called the state, the state is the current condition, or
snapshot if you will of the database. In order to achieve our scope of using the user
accounts and passwords that exist in the BAS we need to make sure that the user database
for the GUI reflects, is consistent with the accounts that exist in the new BAS.
Minor problem, what if two users accounts with the same name exist?
This is quite easy to solve, we can simply import both user names and using some SQL
magic we can make them unique. When someone tries to login the GUI will check both
passwords to see if that is correct. Naturally this assumes that Bob on system A is the same
Bob on system B.
Because of this, it’s often a good practice to pull all user names look for identical names and
request the user change their name. This is done in the beginning of the integration
process.
Now we need to document our resolution for these two issues. Let’s say that in order to
maintain the database state it will take 8 hours a week of a Database Administrator at
$60/hr.
This is where alternate solutions come in play. The facility manager will ask the integrator or
rather the integrator will present alternate options to the facility manager.
The facility manager can now go and chose between accepting the cost of managing
database state OR going and looking at a second option, like making folks create new user
names and passwords on the single GUI…
But, even if you do the perfect use case and identify every single gap in your design, you still
haven’t created anything until the wires are connected and the code is written..
So, what is our next step? How can you go from concept to working systems integration?
In order to do this you need to detail out your physical and logical integration points.
But what are physical and logical integration points? And what exactly does detailing them out
mean?
You can write the best code in the world. You can take two completely disparate data
models and make them one. But none of that “stuff” matters if the two systems can’t talk to
one another.
Physical integration points are the physical connections between the systems you identified
earlier in your gap analysis and use case. These physical connections can be direct cabling,
networking, wireless connections, the list goes on and on.
The important point is that physical connections, physically connect the systems…
Logical integration points are the connections between the systems that utilize the physical
connections.
15/33
For example, you may physically connect a lighting system to a building automation system.
But, without a logical connection to tell the two systems how to communicate and to resolve
data and all sorts of other “programming” problems, your integration may be a dud.
Well, here’s the deal, every systems integration job is unique, can we all agree on that?
So, if every job is unique it makes sense that we would need to plan out each project and
“detail” out the plans for each step.
Well, in this section I am going to teach you my 3 step process for “detailing” out physical
and logical integrations. You might want to go grab a pen and pencil for this, don’t worry I’ll
wait for you…
Using our use case from the previous posts, we can map out our connections.
If you don’t remember our original use case, or you haven’t read my other posts (naughty,
naughty…) then here it is.
We want to integrate two building automation systems into a single graphical user interface
(GUI) on a new system.
Do you connect the two building automation systems directly to the new GUI?
You need to look at your use case and follow how the systems are supposed to interact.
Then, you need to draw your physical connections on a piece of paper. Specifically you
should cover:
Using the use case from step 1 maybe we are routing some information from the first
building automation system into the second building automation system.
Maybe the first building automation system controls the air handlers which then send data
to the second building automation system that controls the central plant. The important
part of this step is that we show that connection.
Could you imagine the chaos if we broke that connection and just simply directly connected
the building automation systems to the single GUI?
It’s important that we figure out how our data is going to be connected to the other
system(s). In the case of our single GUI, one of our design constraints is to have a new single
database.
We’ve got to map out how and what data flows. In order to that we need to identify what
data we are looking to communicate and diagram out where that data goes to.
In lesson number five I discussed the concept of physical and logical integration
connections, man is that ever a mouthful.
I described that in any systems integration project you will have either physical, logical or
both connections.
I also discussed how you need to go and map out the connections between your
integrations. However, I did not show you the exact process for doing this.
An integration map, is a method, for mapping out the connections, dataflow, and process
flow. Usually this is done by use case.
In lesson two I discussed, how you can create a use case for your integrations. I also
showed you how to list out the systems for this use case.
Since each use case is specific systems you’re going to want to create an integration map for
each use case. In any systems integration project you want each integration to be as
standalone as possible.
This will help you and troubleshooting as well as supporting and upgrading the use case
because it will eliminate any interdependencies between use case
An interdependency is where one use case depends on logical and/or physical systems
from another use case.
Now it’s not possible to eliminate all interdependencies because some use cases require a
previous use case before they can take place.
For example, if you were to go and say that you wanted to have use case on creating a
schedule. That scheduling use case may be dependent on a login logout use case.
18/33
Ok, so back to the question “what is an integration map?”.
I mean, saying it’s a process for mapping out your connections, data flow, and use case flow,
isn’t exactly the clearest way of defining what an integration map is used.
When you go to execute an integration project, you’re going to have to perform some
specific tasks. These tasks commonly are:
Now if you only had two systems to integrate these tasks would most likely be quite simple.
However, with the data center example I provided, I had to perform the following tasks:
I had over a dozen different use cases, and several dozen data centers, could you imagine
the chaos of trying to map out all of those systems in my head?
I’m hope that you see, why I would want to create an integration map.
So quite simply, an integration map is a way of structuring your physical connections, logical
connections, and data flow in a way that allows you to execute and document your projects
step-by-step.
19/33
When and when not to use an integration map?
At this point you may be wondering when you should be using an integration map?
I’m not going to tell you that you should use an integration map on every project because
it’s simply not true.
If you are connecting two systems, then as long as you identify the physical and logical
connections and any data normalization that has to take place you should be fine.
So the general rule of thumb, is whenever you have to connect more than two systems or
your project involves multiple use cases, with interdependencies, you should look at
creating an integration map.
So you been following along, and you now agree that you should create an integration map,
but how? If you go and scour the Internet, you can find resources on systems integration
but it’s focused on IT systems.
Over my I’ve come up a step-by-step process for mapping out your integrations by use case.
There are 6 steps creating an effective integration map. The steps are:
1. Step 1: Identify the use case that mapping out the integrations for (I described this in
lesson 2)
2. Step 2: Identify your physical connections, logical connections, systems, and data flow
(by the way I went through how to do this in lessons 3, 4 and 5 )
3. Step 3: Create an integration drawing to list out your systems
4. Step 4: Draw the physical and logic connections between the systems and number
those connections
5. Step 5: Draw the direction, priority, and type of data that flows across the physical
and logical connections, number your data flows.
6. Step 6: Take your numbered connections and data flows and list them in a
spreadsheet
Step 1: Identify the use case that you are mapping out the integrations for (I
described this in lesson 2)
20/33
As I mentioned in lesson 2, during the use case creation process you identify the systems
that will be used. It is important now that you identify the use case that you will be mapping
out integrations for.
Otherwise you may not identify the proper systems and connections that are required.
Step 2: Identify your physical connections, logical connections, and data flow
between your systems (by the way I went through how to do this in lessons
3, 4 and 5)
Speaking of systems and connections, it is important now that you identify the physical
connections, logical connections and data flow between your systems.
Once you have this information you’ll be able to perform the next step
That being said, the important thing is to pick the format and software that you’re most
comfortable with using.
Once you’ve picked the software that you’re going to create your system drawing in, you will
need to place each individual system on the integration map.
Step 4: Draw the physical and logic connections between the systems and
number those connections
21/33
Next, using the information you collected in step two, you will want to add the physical and
logical connections to your system integration map.
I personally recommend that, physical connections be solid lines and logical connections be
dashed lines ( – – -).
In most cases I recommend using a color code for the connections to show what protocol,
physical medium (Ethernet, RS-485, RS-232, etc.) is being used.
You would typically list this in the upper right-hand corner as a legend.
Step 5: Draw the direction, priority, and type of data that flows across the
physical and logical connections, number your data flows
Now that you have the physical and logical connections added to the integration map, I
recommend that you go and add the direction in type of your data flows.
For data flows I recommend using dots ( ● ● ●), instead of dashes or solid lines.
You will want to list out the direction that your data flows in, the priority level of the data
(four commands and quality of service), the size of the data being sent, and the type of data
that is being sent.
I will be talking more about data flow, data models, and system requirements in lesson 7.
Step 6: Take your numbered connections and data flows and list them in a
spreadsheet
Now that you have your connections in data flows numbered, you will want to list them out
in a spreadsheet. In this spreadsheet you will want to show, what physical connections,
logical connections, and data flows connect between what systems.
with all of this information defined you’re now ready to move on to laying out your data
model and system requirements. Some of you may be asking why I waited to have the data
model after the integration.
I have found that if you don’t map out should your data flows and logical connections prior
to creating the data model you can often miss or add data that you do not need.
This is awesome! You are one step away from being part of the BAM Nation. BE SURE TO
CHECK YOUR EMAIL AND SPAM FOLDER. You will get a confirmation e-mail to confirm that
you are a part of the BAM Nation!
Powered by ConvertKit
23/33
A data model organizes data and details out how that data is formatted. When you are
exchanging data between systems it is important to understand the format that the data is
in. This is represented visually below:
In the programming world, a string, is a string of text and a float, is typically a number, with
decimal precision.
Now, if I had, another system, that I was going to be integrating the VAV box data into and
that system had a different data type then there would be a conflict. This is illustrated below:
As you can see, the VAV box data is not of the same data type as the third-party BAS data.
This will cause a data conflict.
What is happening is the third-party BAS, has a primary key of device ID, which is of type int.
An INT, stands for an integer. Unlike a float, an integer, does not have decimal precision. This
means that the zone temperature and damper positions would lose their “precision” or
accuracy.
But the bigger issue, is that the primary keys conflict both in type and in name. This will
cause a mismatch. In order to resolve this the data needs to be normalized. You can see
how data is normalized in the image below:
24/33
What I just did in the above image, is I created a higher level data set that bridges the two
data sets utilizing what is called foreign keys. This allows the third-party BAS and VAV box
data sets to access the data from the normalized data set.
When you purchase a device like a field server gateway, this is what is happening. Inside
that gateway there is software that is performing this data normalization and then allowing
the two data sets to connect.
When you look at an individual system you will want to go and create the data sets for each
system. Now you may not know the data types that each system has. This is where you will
have to turn to the system vendor to provide you that information.
Now I take a quick second the make sure I’m clear. If you’re performing a protocol
integration using similar protocols, you most likely will not have to go this deep. The
examples I am providing are geared more towards dissimilar protocols, for example
integrating a LON system into a BACnet/IP system.
You can ask the vendor to provide you the data model for their specific system. Then you
need to understand the points that you want to share between the systems. Once you have
defined the points you will need to map up the data types.
25/33
Once you have done this you will need to determine if a normalization schema is needed.
For some of you the word schema may be unfamiliar. When you hear the word schema,
think format.
CPU
Memory
Storage
Bandwidth
Network Throughput
In an earlier post we discussed how to identify your systems. Now that you know what your
systems are its time to formally document the system requirements. This step is not very
complex but it is time-consuming.
If you do not have an IT person on staff than is important that you retain the services of
someone with an information technology background.
Step 1: Define the network, storage, and compute requirements for each
system
Using the list of systems that you created in (earlier post), you will need to go and work with
your IT staff and vendors to determine the network (also known as bandwidth), storage, and
compute (also known as processing) requirements of each individual system.
It seems like I’ve said that about every post in this series. Well folks, I only say that, because
it is true!
Look, I’m walking you through the mine field but it’s up to you if you want to stay by my side
or run ahead and play with your projects life.
A sequence of operations is like a use case on steroids. Whereas a use case walks you
through the how the sequence of operations is the how, the what, the when and even more.
For example, I’m going to describe a use case where the main success scenario is the user
being able to use credentials from another BAS login into a system, hint, hint this may look
familiar to a use case I described in an earlier post.
27/33
communicate to the back-end database and will perform a select query where the
username and hashed password match. If a matching data set can be located then the user
will be granted access to the graphical user interface.
Do you see the difference? The sequence of operations is the end result of information
from the physical and logical integration map, the data model and the use case.
This happened because I came up with a step-by-step process that empowered me to take
what was in my head and put it on paper.
I am going to walk you through the steps using the example of tying in a lighting system.
A use case may say, the lighting system will be integrated to the building automation system.
The building automation system will turn the lights on when the building is occupied.
From the integration map I will be able to identify that the building automation system and
the lighting system are connected via BACnet/IP. I will also know that the building
automation system will be commanding the lighting system have priority level 10.
28/33
I will also be able to identify, that the building automation system syncs its timeclock with
the building’s local time-server.
Step 4: Chunk out your actions and build out a flow map
I will then take each one of these actions and build a flow map. This flow map will visually
represent how and in what order the different actions will take place.
The buildings time-server will send a time synchronization message to the building
automation system every 15 minutes (adj). The time synchronization message will be
sent using the NTP protocol to the building automation server located in the data
closet.
The building automation system will have a set schedule that will command the
building to occupied. This schedule will be from 7AM CST to 5pm CST (adj).
The building automation system will command the lighting enable object to on (adj)
within 30 seconds (adj) of the building automation system occupancy signal switching
to the occupied state. This command will be sent via the BACnet/IP protocol from the
BAS server to the lighting server at priority level 10.
Upon receiving the on command from the BAS, the lighting system will command all of
the buildings interior lights to on.
The buildings time-server will send a time synchronization message to the building automation
system every 15 minutes (adj). The time synchronization message will be sent using the NTP
protocol to the building automation server located in the data closet.The building automation
system will have a set schedule that will command the building to occupied. This schedule will be
from 7AM CST to 5pm CST (adj). The building automation system will command the lighting
enable object to on (adj) within 30 seconds (adj) of the building automation system occupancy
signal switching to the occupied state. This command will be sent via the BACnet/IP protocol from
the BAS server to the lighting server at priority level 10. Upon receiving the on command from the
BAS, the lighting system will command all of the buildings interior lights to on.
30/33
Well, everyone has a different way of executing their systems integration projects. If you
have a way that you like and you have followed the steps up to this point then I encourage
you to use your method.
However, if you want a proven process that has helped me to execute some crazy complex
projects, then check out my 5 step process below:
Once you have the systems installed you will then want to make sure that any software
associated with the systems set up including, drivers, databases, and dependencies.
Using our lighting example, you would want to install the lighting and building automation
servers, controllers, and end devices.
Continuing our lighting example, you would want to set up any IP addresses, device
instances, VLANs, databases, and system names.
For our lighting solution, you would want to build out, the interlock logic along with the
prioritization scheme for the integration points.
You will want to set the timers and schedules for the lighting integration.
You would also want to change the master time-server and verify that the building
automation system adjusts accordingly.
Finally, you would want to make sure that the lighting system cannot override the building
automation system.
Conclusion
So there you have it folks. That is my step-by-step process for approaching systems
integration projects.
If you started this series with limited information … if after reading this information you
realized that there is much more to building automation systems and you’d like a simple
resource to learn more then I have a solution for you.
Click on the image below to find out more about our solution
Powered by ConvertKit
33/33