UNIT II

BUSINESS INTELLIGENCE

Data Warehouses and Data Mart - Knowledge Management –Types of Decisions - Decision

Making Process - Decision Support Systems – Business Intelligence –OLAP – Analytic

functions

DATA WAREHOUSES AND DATA MART

Data warehouse refers to the process of compiling and organizing data into one common
database, whereas data mining refers to the process of extracting useful data from the databases.
The data mining process depends on the data compiled in the data warehousing phase to
recognize meaningful patterns.
companies capture information every time a customer contacts any of their departments,
whether via purchase, a service call, an online query, or a mail-in rebate card. Banks and credit
card companies, telephone companies, catalog marketers, and many other companies have a
great deal of information about their customers, including transaction history and enhanced data
on age, family size, income, and other demographics.
These data are collected by the company’s contact center and organized into a data
warehouse where marketers can capture, query, and analyze them to draw inferences about an
individual customer’s needs and responses.
Through data mining, marketing statisticians can extract from the mass of data useful
information about individuals, trends, and segments.
Data mining uses sophisticated statistical and mathematical techniques such as cluster
analysis, automatic interaction detection, predictive modeling, and neural networking.
In general, companies can use their databases in five ways:
1. To identify prospects—Many companies generate sales leads by advertising their
product or service and including a response feature, such as a link to a home page and building a
database from customer responses. The company sorts through the database to identify the best
prospects, then contacts them by mail, e-mail, or phone to try to convert them into customers.
2. To decide which customers should receive a particular offer—Companies
interested in selling set up criteria describing the ideal target customer for a particular offer. Then
they search their customer databases for those who most closely resemble the ideal. By noting
response rates, a company can improve its targeting precision. Following a sale, it can set up an
automatic sequence of activities: One week later e-mail a thank-you note; five weeks later e-mail
a new offer; 10 weeks later (if the customer has not responded) e-mail an offer of a special
discount.
3. To deepen customer loyalty—Companies can build interest and enthusiasm by
remembering customer preferences and sending appropriate gifts, discount coupons, and
interesting reading material.
4. To reactivate customer purchases—Automatic mailing programs (automatic
marketing) can send out birthday or anniversary cards, holiday shopping reminders, or off-season
promotions. The database can help the company make attractive or timely offers.
5. To avoid serious customer mistakes—A major bank confessed to a number of
mistakes it had made by not using its customer database well.
1.The bank charged a customer a penalty for late payment on his mortgage, failing to
note he headed a company that was a major depositor in this bank. The customer quit the bank.
2. Two different staff members of the bank phoned the same mortgage customer offering
a home equity loan at different prices. Neither knew the other had made the call.

Data warehouses and databases both are relative data systems, but both are made to
serve different purposes. A data warehouse is built to store a huge amount of historical data and
empowers fast requests over all the data, typically using Online Analytical Processing (OLAP).

A database is made to store current transactions and allow quick access to specific
transactions for ongoing business processes, commonly known as Online Transaction
Processing (OLTP).

Important Features of Data Warehouse

The Important features of Data Warehouse are given below:

1. Subject Oriented

A data warehouse is subject-oriented. It provides useful data about a subject instead of

the company's ongoing operations, and these subjects can be customers, suppliers, marketing,
product, promotion, etc. A data warehouse usually focuses on modeling and analysis of data that
helps the business organization to make data-driven decisions.

2. Time-Variant:

The different data present in the data warehouse provides information for a specific period.

3. Integrated
A data warehouse is built by joining data from heterogeneous sources, such as social databases,
level documents, etc.

4. Non- Volatile

It means, once data entered into the warehouse cannot be change.

Advantages of Data Warehouse:

o More accurate data access

o Improved productivity and performance
o Cost-efficient
o Consistent and quality data
o A data warehouse architecture is a method of defining the overall architecture of data
communication processing and presentation that exist for end-clients computing within
the enterprise. Each data warehouse is different, but all are characterized by standard vital
components.
o Production applications such as payroll accounts payable product purchasing and
inventory control are designed for online transaction processing (OLTP). Such
applications gather detailed data from day to day operations.
o Data Warehouse applications are designed to support the user ad-hoc data requirements,
an activity recently dubbed online analytical processing (OLAP). These include
applications such as forecasting, profiling, summary reporting, and trend analysis.

Operational System

An operational system is a method used in data warehousing to refer to a system that is used to
process the day-to-day transactions of an organization.
Flat Files

A Flat file system is a system of files in which transactional data is stored, and every file in the
system must have a different name.

Meta Data

A set of data that defines and gives information about other data.

Meta Data used in Data Warehouse for a variety of purpose, including:

Meta Data summarizes necessary information about data, which can make finding and work with
particular instances of data more accessible. For example, author, data build, and data changed,
and file size are examples of very basic document metadata.

Metadata is used to direct a query to the most appropriate data source.

Lightly and highly summarized data

The area of the data warehouse saves all the predefined lightly and highly summarized
(aggregated) data generated by the warehouse manager.

The goals of the summarized information are to speed up query performance. The summarized
record is updated continuously as new information is loaded into the warehouse.

End-User access Tools

The principal purpose of a data warehouse is to provide information to the business managers for
strategic decision-making. These customers interact with the warehouse using end-client access
tools.

The examples of some of the end-user access tools can be:

o Reporting and Query Tools

o Application Development Tools
o Executive Information Systems Tools
o Online Analytical Processing Tools
o Data Mining Tools

Data Mart

A Data Mart is a subset of a directorial information store, generally oriented to a specific

purpose or primary data subject which may be distributed to provide business needs.
 Data Marts are analytical record stores designed to focus on particular business functions
for a specific community within an organization.

Data marts are derived from subsets of data in a data warehouse, though in the bottom-
up data warehouse design methodology, the data warehouse is created from the union of
organizational data marts.

● Data Warehouse is focused on all departments in an organization, whereas Data Mart

focuses on a specific group.

The fundamental use of a data mart is Business Intelligence (BI) applications. BI is used
to gather, store, access, and analyze record. It can be used by smaller businesses to utilize the
data they have accumulated since it is less expensive than implementing a data warehouse.

Reasons for creating a data mart

o Creates collective data by a group of users
o Easy access to frequently needed data
o Ease of creation
o Improves end-user response time
o Lower cost than implementing a complete data warehouses
o Potential clients are more clearly defined than in a comprehensive data warehouse
o It contains only essential business data and is less cluttered.

Types of Data Marts

o Dependent Data Marts

o Independent Data Marts
Dependent Data Marts

A dependent data marts is a logical subset of a physical subset of a higher data

warehouse. According to this technique, the data marts are treated as the subsets of a data
warehouse. In this technique, firstly a data warehouse is created from which further various data
marts can be created. These data mart are dependent on the data warehouse and extract the
essential record from it. In this technique, as the data warehouse creates the data mart; therefore,
there is no need for data mart integration. It is also known as a top-down approach.

Independent Data Marts

The second approach is Independent data marts (IDM) Here, firstly independent data
marts are created, and then a data warehouse is designed using these independent multiple data
marts. In this approach, as all the data marts are designed independently; therefore, the
integration of data marts is required. It is also termed as a bottom-up approach as the data marts
are integrated develop a data warehouse.

Other than these two categories, one more type exists that is called "Hybrid Data Marts."
 Hybrid Data Marts

It allows us to combine input from sources other than a data warehouse. This could be
helpful for many situations; especially when Adhoc integrations are needed, such as after a new
group or product is added to the organizations

Parameter Data Warehouse Data Mart

A Data Warehouse is a large
A data mart is an only subtype of a Data
repository of data collected from
Definition Warehouse. It is designed to meet the
different organizations or departments
need of a certain user group.
within a corporation.
It helps to take tactical decisions for the
Usage It helps to take a strategic decision.
business.
The main objective of Data
Warehouse is to provide an integrated A data mart mostly used in a business
Objective
environment and coherent picture of division at the department level.
the business at a point in time.
The designing process of Data The designing process of Data Mart is
Designing
Warehouse is quite difficult. easy.
Data warehousing includes large area Data marts are easy to use, design and
Data Handling of the corporation which is why it implement as it can only handle small
takes a long time to process it. amounts of data.
Data warehousing is broadly focused
all the departments. It is possible that Data Mart is subject-oriented, and it is
Focus
it can even represent the entire used at a department level.
company.
The data stored inside the Data
Data Marts are built for particular user
Data type Warehouse are always detailed when
groups. Therefore, data short and limited.
compared with data mart.
The main objective of Data
Warehouse is to provide an integrated Mostly hold only one subject area- for
Subject-area
environment and coherent picture of example, Sales figure.
the business at a point in time.
Dimensional modeling and star schema
Designed to store enterprise-wide
Data storing design employed for optimizing the
decision data, not just marketing data.
performance of access layer.
 Parameter Data Warehouse Data Mart

Mostly includes consolidation data

Time variance and non-volatile
Data type structures to meet subject area’s query
design are strictly enforced.
and reporting needs.

Read-Only from the end-users Transaction data regardless of grain fed

Data value
standpoint. directly from the Data Warehouse.

Data mart contains data, of a specific

Data warehousing is more helpful as
department of a company. There are
Scope it can bring information from any
maybe separate data marts for sales,
department.
finance, marketing, etc. Has limited usage

In Data Warehouse Data comes from In Data Mart data comes from very few
Source
many sources. sources.

The size of the Data Warehouse may The Size of Data Mart is less than 100
Size
range from 100 GB to 1 TB+. GB.

The implementation process of Data

Implementatio The implementation process of Data Mart
Warehouse can be extended from
n time is restricted to few months.
months to years.

KNOWLEDGE MANAGEMENT

Knowledge Management is the process of identifying, organizing storing and

disseminating information with in an organization. When knowledge is not easily accessible
within an organization, it can be incredibly costly to a business as valuable time is spent seeking
out relevant information versus completing outcome-focused tasks.

A knowledge management system (KMS) harnesses the collective knowledge of the

organization, leading to better operational efficiencies. These systems are supported by the use of
a knowledge base. They are usually critical to successful knowledge management, providing a
centralized place to store information and access it readily.

Companies with a knowledge management strategy achieve business outcomes more

quickly as increased organizational learning and collaboration among team members facilitates
faster decision-making across the business. It also streamlines more organizational processes,
such as training and on-boarding, leading to reports of higher employee satisfaction and
retention.
Components of knowledge management

Types of knowledge
The definition of knowledge management also includes three types of knowledge—tacit,
implicit, and explicit knowledge. These types of knowledge are largely distinguished by the
codification of the information.
Tacit knowledge: This type of knowledge is typically acquired through experience, and
it is intuitively understood. As a result, it is challenging to articulate and codify, making it

● difficult to transfer this information to other individuals. Examples of tacit knowledge

can include language, facial recognition, or leadership skills.
● Implicit knowledge: While some literature equivocates implicit knowledge to tacit
knowledge, some academics break out this type separately, expressing that the definition
of tactic knowledge is more nuanced. While tacit knowledge is difficult to codify,
implicit knowledge does not necessarily have this problem. Instead, implicit information
has yet to be documented. It tends to exist within processes, and it can be referred to
as “know-how” knowledge.
● Explicit knowledge: Explicit knowledge is captured within various document types such
as manuals, reports, and guides, allowing organizations to easily share knowledge across
teams. This type of knowledge is perhaps the most well-known and examples of it
include knowledge assets such as databases, white papers, and case studies. This form of
knowledge is important to retain intellectual capital within an organization as well as
facilitate successful knowledge transfer to new employees.
Knowledge management process and tools
Knowledge management process:

The knowledge management process as involving knowledge acquisition, creation,

refinement, storage, transfer, sharing and utilization. Effective knowledge management system
typically goes through three main steps:

1. Knowledge Creation: During this step, organizations identify and document any
existing or new knowledge that they want to circulate across the company.
2. Knowledge Storage: During this stage, an information technology system is typically
used to host organizational knowledge for distribution. Information may need to be
formatted in a particular way to meet the requirements of that repository.
 3. Knowledge Sharing: In this final stage, processes to share knowledge are communicated
broadly across the organization. The rate in which information spreads will vary
depending on organizational culture. Companies that encourage and reward this behavior
will certainly have a competitive advantage over other ones in their industry.
Knowledge management tools:

There are a number tools that organizations utilize to reap the benefits of knowledge
management. Examples of knowledge management systems can include:

● Document management systems act as a centralized storage system for digital

documents, such as PDFs, images, and word processing files. These systems enhance
employee workflows by enabling easy retrieval of documents, such as lessons learned.

● Content management systems (CMS) are applications which manage web content
where end users can edit and publish content. These are commonly confused with
document management systems, but CMSs can support other media types, such as audio
and video.

● Intranets are private networks that exist solely within an organization, which enable the
sharing of enablement, tools, and processes within internal stakeholders. While they can
be time-consuming and costly to maintain, they provide a number of groupware services,
such as internal directories and search, which facilitate collaboration.

● Wikis can be a popular knowledge management tool given its ease of use. They make it
easy to upload and edit information, but this ease can lead to concerns about
misinformation as workers may update them with incorrect or outdated information.

● Data warehouses aggregate data from different sources into a single, central, consistent
data store to support data analysis, data mining, artificial intelligence (AI), and machine
learning. Data is extracted from these repositories so that companies can derive insights,
empowering employees to make data-driven decisions.
Benefits of knowledge management
Companies experience a number of benefits when they embrace knowledge management
strategies. Some key advantages include:

● Identification of skill gaps: When teams create relevant documentation around implicit
or tacit knowledge or consolidate explicit knowledge, it can highlight gaps in core
competencies across teams. This provides valuable information to management to form
new organizational structures or hire additional resources.
● Make better informed decisions: Knowledge management systems arm individuals and
departments with knowledge. By improving accessibility to current and historical
enterprise knowledge, your teams can upskill and make more information-driven
decisions that support business goals.
● Maintains enterprise knowledge: If your most knowledgeable employees left
tomorrow, what would your business do? Practicing internal knowledge management
enables businesses to create an organizational memory. Knowledge held by your long-
term employees and other experts, then make it accessible to your wider team.
● Operational efficiencies: Knowledge management systems create a go-to place that
enable knowledge workers to find relevant information more quickly. This, in turn,
reduces the amount of time on research, leading to faster decision-making and cost-
savings through operational efficiencies. Increase productivity not only saves time, but
also reduces costs.
● Increased collaboration and communication: Knowledge management systems and
organizational cultures work together to build trust among team members. These
information systems provide more transparency among workers, creating more
understanding and alignment around common goals. Engaged leadership and open
communication create an environment for teams to embrace innovation and feedback.
● Data Security: Knowledge management systems enable organizations to customize
permission control, viewership control and the level of document-security to ensure that
information is shared only in the correct channels or with selected individuals. Give your
employees the autonomy access knowledge safely and with confidence.

TYPES OF DECISIONS
 Decision-making is the action or process of thinking through possible options and
selecting one. It is important to recognize that managers are continually making decisions, and
that the quality of their decision-making has an impact—sometimes quite significant—on the
effectiveness of the organization and its stakeholders. Stakeholders are all the individuals or
groups that are affected by an organization (such as customers, employees, shareholders, etc.)

Members of the top management team regularly make decisions that affect the future of the
organization and all its stakeholders, such as deciding whether to pursue a new technology or
product line. A good decision can enable the organization to thrive and survive long-term, while
a poor decision can lead a business into bankruptcy. Managers at lower levels of the organization
generally have a smaller impact on the organization’s survival, but can still have a tremendous
impact on their department and its workers. Poor decision-making by lower-level managers is
unlikely to drive the entire firm out of existence, but it can lead to many adverse outcomes.
Therefore, increasing effectiveness in decision making is critical, and using a model can help.

DECISION TYPES

⮚ structured decision
⮚ unstructured decision
⮚ semi-structured decision
⮚ Strategic decisions
⮚ operational decisions

Structured decision

An organization has a wide variety of decisions to make, ranging from highly structured
decisions to unstructured decisions. A structured decision is one that is made quite often, and
one in which the decision is based directly on the inputs. With structured decisions, once you
know the necessary information you also know the decision that needs to be made. For example,
inventory reorder levels can be structured decisions. Once your inventory of widgets gets below
a specific threshold, there is a need to order more. Processes like this, based on structured
decisions, are good candidates for automation. They can also be referred to as programmed
decisions.

For programmed decisions, managers often develop heuristics, or mental shortcuts, to help reach
a decision. For example, the retail store manager may not know how busy the store will be the
week of a big sale, but might routinely increase staff by 30% every time there is a big sale
(because this has been fairly effective in the past). Heuristics are efficient—they save time for
the decision maker by generating an adequate solution quickly. Heuristics don’t necessarily yield
the optimal solution, but a good solution. Heuristics are often used for programmed decisions ,
because experience in making the decision over and over helps the decision maker know what
to expect and how to react. Programmed decision-making can also be taught fairly easily to
another person.

Unstructured decision

In contrast, an unstructured decision or non programmed decision involves a lot of

unknowns. They are generally based on criteria that are not well-defined, and information is
more likely to be ambiguous or incomplete. The decision maker may need to exercise some
thoughtful judgment and creative thinking to reach a good solution. An information system can
support these types of decisions by providing the decision makers with information gathering
tools and collaborative capabilities. An example of an unstructured decision might be dealing
with a labor issue or setting policy for the implementation of a new technology.

Semi-structured decision

A semi-structured decision is one in which most of the factors needed for making the
decision are known but human experience and other outside factors may still impact the decision.
A good example of a semi-structured decision is the hiring process. Part of the decision is
structured (years of experience, education, etc.) and part of the decision is based on human
experience (for example: social skills, problem solving skills etc.) Take a look at how companies
are trying to program this decision and use algorithms. Semi-structured and unstructured
decisions are more challenging and systems may not be able to assist in the process fully.
However, advances

in artificial intelligence are helping to change

this.
Decision Type and System Type Pyramid
Decisions can also be classified into three categories based on the level at which they occur.

Strategic decisions set the course of an organization. Tactical/Managerial decisions are

decisions about how things will get done.

Finally, operational decisions refer to decisions that employees make each day to make
the organization run. For example, think about the restaurant that routinely offers a free dessert
when a customer complaint is received. The owner of the restaurant made a strategic decision to
have great customer service. The manager of the restaurant implemented the free dessert policy
as a way to handle customer complaints, which is a tactical decision. Finally, the servers at the
restaurant are making individual decisions each day by evaluating whether each customer
complaint received is legitimate and warrants a free dessert. Different information systems are
used at each level of the company structure to support the different decision types.

Level of Who Typically Makes

Examples of Decision
Decision Decisions

Should we merge with another company? Top Management Teams,

Strategic
Should we pursue a new product line? CEOs, and Boards of
Decisions
Should we downsize our organization? Directors

What should we do to help facilitate

employees from the two companies
Tactical working together?
Managers
Decisions How should we market the new product
line?
Who should be let go when we downsize?

How often should I communicate with my

new coworkers?
Operational Employees throughout the
What should I say to customers about our
Decisions organization
new product?
How will I balance my new work demands?
 DECISION MAKING PROCESS

Decision-making is the action or process of thinking through possible options and

selecting one. It is important to recognize that managers are continually making decisions, and
that the quality of their decision-making has an impact—sometimes quite significant—on the
effectiveness of the organization and its stakeholders. Stakeholders are all the individuals or
groups that are affected by an organization (such as customers, employees, shareholders, etc.)

Members of the top management team regularly make decisions that affect the future of
the organization and all its stakeholders, such as deciding whether to pursue a new technology or
product line. A good decision can enable the organization to thrive and survive long-term, while
a poor decision can lead a business into bankruptcy. Managers at lower levels of the organization
generally have a smaller impact on the organization’s survival, but can still have a tremendous
impact on their department and its workers. Poor decision-making by lower-level managers is
unlikely to drive the entire firm out of existence, but it can lead to many adverse outcomes.
Therefore, increasing effectiveness in decision making is critical, and using a model can help.

The Decision-Making Process

Decision makers should use a systematic process for making decisions. The decision-making
process can be broken down into a series of six steps, as follows:

1. Recognize that a decision needs to be made.

2. Generate multiple alternatives.
3. Analyze the alternatives.
4. Select an alternative.
5. Implement the selected alternative.
6. Evaluate its effectiveness.

While these steps may seem straightforward, individuals often skip steps or spend too little time
on some steps. In fact, sometimes people will refuse to acknowledge a problem (Step 1) because
they aren’t sure how to address it.
Steps in Decision-Making Process
 DECISION SUPPORT SYSTEMS

A decision support system (DSS) helps managers make decisions using interactive
computer models that describe real-world processes. These systems are designed to take inputs
regarding a known (or partially-known) decision making process and provide the information
necessary to make a decision. The DSS uses data from the internal database but looks for
specific data that relate to the problems at hand.

DSSs generally assist a management level person in the decision-making process. It is a

tool for answering “what if” questions about what would happen if the manager made certain
changes. With models, the manager enters into the computer the values that describe a particular
situation, and the program computes the results. A DSS can be looked at as a tool for competitive
advantage because it can give an organization a mechanism to make wise decisions about
products and innovations.

DSS’s can come in many different formats. A nicely designed spreadsheet that allows for
input of specific variables and then calculates required outputs could be considered a DSS. A
manager could create a spreadsheet to show the amount of overtime required if the number of
workers increases or decreases. Another DSS might be one that assists in determining which
products a company should develop. Input into the system could include market research on the
product, competitor information, and product development costs. The system would then analyze
these inputs based on the specific rules and concepts programmed into it. The system would
report its results with recommendations and/or key indicators to be used in making a decision.
Companies can use a predictive analytics program to improve their inventory management
system and use big data to target customer segments for new products and line extensions.

Common DSS Analysis Techniques

The following are common analysis techniques in a decision support system.

Outlines the impact of a change in a variable or assumption on the

What if analysis
model.
 Sensitivity Analysis Outlines the impact when one variable is changed repeatedly.

Goal Seek Analysis Goal seeking analysis-finds the inputs necessary to achieve a goal

Optimization Optimization analysis -finds the optimum value for a target variable
Analysis by repeatedly changing other variables

Executive Information Systems

Although similar to a DSS, an executive information system (EIS) is customized for

executives. These systems provide specific information for strategic decisions. For example, a
CEO’s EIS may include special spreadsheets that present financial data comparing the company
to its principal competitors and graphs showing current economic and industry trends.

Most EIS’s have a business intelligence (BI) dashboard. A BI dashboard is a tool that
shows all of an organization’s key metrics and performance indicators on one screen. This
makes it easy for management to quickly see how the company is performing and address any
outliers or negative trends while also being able to identify positive trends. It also provides a
consolidated overview with the ability to drill down or filter when needed. During the COVID
pandemic many dashboards were created to track the virus as well as vaccine rates. One
example is the GIS Hub provided by Esri Canada that shares all information about the pandemic.
Expert Systems

An expert system gives managers advice similar to what they would get from a human
consultant. Artificial intelligence enables computers to reason and learn to solve problems in
much the same way humans do, using what-if reasoning. Although they are expensive and
difficult to create, expert systems are finding their way into more companies as more
applications are found. Lower-end expert systems can even run on mobile devices. Top-of-the-
line systems help airlines appropriately deploy aircraft and crews, critical to the carriers’ efficient
operations. The cost of hiring enough people to do these ongoing analytical tasks would be
prohibitively expensive. Expert systems have also been used to help explore for oil, schedule
employee work shifts, and diagnose illnesses. Some expert systems take the place of human
experts, whereas others assist them.
 BUSINESS INTELLIGENCE

With the rise of Big Data and a myriad of new tools and techniques at their disposal,
businesses are learning how to use information to their advantage. The term business
intelligence describes the process that organizations take to collect and analyze data in the hopes
of obtaining a competitive advantage.
Business intelligence is about getting the right information, to the right decision makers, at the
right time.
Business intelligence is really an organizational process as much as it is a set of technologies.
Besides using their own data, stored in data warehouses (more on this later), firms often purchase
information from data brokers to get a big-picture understanding of their industries and the
economy. The results of analysis can drive organizational strategies and provide competitive
advantage.

BI is used for

● Fact based decision making

● Single version of truth
● 360 degrees perspective on your business
● Visibilty into enterprise performance
Data Brokers

Data brokers collect and sell data based on online activity. Digital marketing
company WebFX estimates that there are more than 4,000 data brokers worldwide. Digital
marketers use this information to help direct their advertising and product development efforts.
The data broker market in Canada is flourishing due to less safeguards on personal
information. [1]

All the terminology around business intelligence can be confusing, as often the terms are
used interchangeably. What is the difference between business intelligence, data science and data
analytics? Much like information technology and information systems, the terms are
differentiated on the components involved. In the data world, the components consist of
engineering, analysis, and communication. Combining the terms in different ways will yield
various ‘areas of focus’.

Data Engineering Refers to the design, creation, and structure of data and datasets.

Refers to the techniques used to refine and manipulate data into

Data Analysis
information.

Data
Is defined as the flow of data from a source(s) to the user.
Communication
Venn Diagram of Data Engineering, Analysis and Communication.

Data Science

Combining all three areas (engineering, communication and analysis) results in the field
of data science. Data Science is the analysis of large data sets to find new knowledge. The field
of data science is constantly changing, and data scientists are on the cutting edge of work in areas
such as artificial intelligence and neural networks. While a data scientist does many different
things, their focus is generally on analyzing large data sets using various programming methods
and software tools to create new knowledge for their organization. For example, data scientists
look at building new models and algorithms to extract data in different ways. Data scientists are
skilled in machine learning.

Data Analytics

Combining analysis and communication results in data analytics. Data analytics looks at the
past data to try and understand what happened and can also make predictions for the future. Data
analytics is focused on finding answers to questions by analyzing trends and creating visual
representations, like dashboards whereas data science is a broad concept that focuses on figuring
out the questions that should be asked of the data. Whereas business intelligence focuses on what
happened in the past up until now. It helps to identify trends in the past, and data analysis can
look to the future. To learn more about the history of business intelligence and its evolution see
this see this infographic.
Data Analytics Types

Analytics is a very broad term, but no matter the type of data, or size of data you are
using, generally all analytics can be categorized in four ways: describe, diagnose, predict, and
prescribe. These four categories range from summarizing historical information to providing
insight for the future by investigating the data and attempting to answer the following questions:

Descriptive
What happened?
Analytics

Diagnostic
Why did it happen?
Analytics

Predictive Analytics What might happen?

Prescriptive
What to do next?
Analytics

Calculating some summary statistics such as mean, median, mode and standard deviation
is an example of descriptive analytics. Similarly matching, or creating a scatter plot diagram are
examples of diagnostic techniques that can help identify outliers. Predictive techniques involve
using regression analysis and attempts to find similarities between historical data sets to be able
to forecast into the future. Lastly, prescriptive analytics involve using decision support systems,
machine learning and artificial intelligence to analyze the information and recommend a course
of action. Multiple predictions are generated and an optimized solution is recommended. For
example, if you use a GPS on a long road trip, you may have experienced ‘dynamic route
updates’, or the suggestion of new alternate routes that may be quicker due to slowing traffic or
an accident.
 OLAP – ANALYTIC FUNCTIONS

The idea behind analysis tools that allow queries and reporting is to present users with a
subset of requested data, selected, sorted, ordered, calculated, and compared, as needed.
Managers use these tools to see and explore what’s happening inside their
organizations. Canned reports provide regular summaries of information in a predetermined
format. They’re often developed by information systems staff and formats can be difficult to
alter. By contrast, ad hoc reporting tools allow users to dive in and create their own reports,
selecting fields, ranges, and other parameters to build their own reports on the fly.

Online Analytical Processing (OLAP)

A subcategory of reporting tools is referred to as Online Analytical Processing (OLAP)

(pronounced “oh-lap”). Data used in OLAP reporting is usually sourced from standard relational
databases, but it’s calculated and summarized in advance, across multiple dimensions, with the
data stored in a special database called a data cube. This extra setup step makes OLAP fast
(sometimes one thousand times faster than performing comparable queries against conventional
relational databases). Given this kind of speed boost, it’s not surprising that data cubes for OLAP
access are often part of a firm’s data mart and data warehouse efforts.

A manager using an OLAP tool can quickly explore and compare data across multiple
factors such as time, geography, product lines, and so on. In fact, OLAP users often talk about
how they can “slice and dice” their data, “drilling down” inside the data to uncover new insights.
And while conventional reports are usually presented as a summarized list of information, OLAP
results look more like a spreadsheet, with the various dimensions of analysis in rows and
columns, with summary values at the intersection.

Data Visualization

Data visualization is the graphical representation of information and data. These graphical
representations (such as charts, graphs, and maps) can quickly summarize data in a way that is
more intuitive and can lead to new insights and understandings. Just as a picture of a landscape
can convey much more than a paragraph of text attempting to describe it, graphical
representation of data can quickly make meaning of large amounts of data. Many times,
visualizing data is the first step towards a deeper analysis and understanding of the data collected
by an organization. Examples of data visualization software include Tableau and Google Data
Studio.

Dashboards are a type of data visualization that provide a heads-up display of critical indicators,
letting managers get a graphical glance at key performance metrics. Some tools may allow data
to be exported into spreadsheets. Even the lowly spreadsheet can be a powerful tool for
modelling “what if” scenarios and creating additional reports. However, if data can be easily
exported, then it can potentially leave the firm dangerously exposed, raising privacy, security,
legal, and competitive concerns.

Data Mining

Data mining is the process of analyzing data to find previously unknown and interesting trends,
patterns, and associations in order to make decisions. Click to enlarge. Data Mining
Process adapted from Matthew Pauley CC-BY-NC-SA 4.0

While reporting tools can help users explore data, modern data sets can be so large that it might
be impossible for humans to spot underlying trends. That’s where data mining can help. Data
mining is the process of analyzing data to find previously unknown and interesting trends,
patterns, and associations in order to make decisions. Generally, data mining is accomplished
through automated means against extremely large data sets, such as a data warehouse. Some of
the key areas where businesses are leveraging data mining include the following:

Figuring out which customers are likely to be the most

Customer segmentation
valuable to a firm.

Marketing and promotion Identifying which customers will respond to which offers at
 targeting which price at what time.

Determining which products customers buy together, and

Market basket analysis how an organization can use this information to cross-sell
more products or services.

Personalizing an individual customer’s experience based on

Collaborative filtering the trends and preferences identified across similar
customers.

Determining which customers are likely to leave, and what

Customer churn
tactics can help the firm avoid unwanted defections.

Fraud detection Uncovering patterns consistent with criminal activity.

Financial modeling Building trading systems to capitalize on historical trends.

Identifying characteristics consistent with employee success

Hiring and promotion
in the firm’s various roles.

Model Considerations

For data mining to work, two critical conditions need to be present:

1. Clean and consistent data. Deceive your systems with bad data and your models are worthless.
It is important that the data that is input into the system is accurate and consistent
2. Data should reflect current and future trends. As well historical consistency should be
considered. Computer-driven investment models can be very effective when the market behaves
as it has in the past. But models are blind when faced with extreme and unusual events that never
occurred in the data used to build the model.

Data mining presents a host of other perils, as well. It’s possible to over-engineer a model,
building it with so many variables that the solution arrived at might only work on the subset of
data you’ve used to create it. You might also be looking at a random but meaningless statistical
fluke. One way to test to see if you’re looking at a random occurrence in the numbers is to divide
your data, building your model with one portion of the data, and using another portion to verify
your results. This is the approach Netflix has used to test results achieved by teams in the Netflix
Prize, the firm’s million-dollar contest for improving the predictive accuracy of its movie
recommendation engine. Finally, sometimes a pattern is uncovered but determining the best
choice for a response is less clear. First findings don’t always reveal an optimal course of action.

Skills Required

These modelling considerations highlight the importance of recruiting a data mining and
business analytics team that possesses three critical skills:

Information For understanding how to pull together data, and for selecting
technology analysis tools.

For building models and interpreting the strength and validity of

Statistics
results.

For helping set system goals, requirements, and offering deeper

Business knowledge insight into what the data really says about the firm’s operating
environment.