Q1. What are the different types of Digital Data? Brief upon each of them with an example each.

DEFINITION -
Unstructured data is a generic label for describing any corporate information that is not
in a database. Unstructured data can be textual or non-textual. Textual unstructured data is
generated in media like email messages, PowerPoint presentations, Word documents,
collaboration software and instant messages. Non-textual unstructured data is generated in
media like JPEGimages, MP3 audio files and Flash video files.
If left unmanaged, the sheer volume of unstructured data that’s generated each year within an
enterprise can be costly in terms of storage. Unmanaged data can also pose a liability if
information cannot be located in the event of acompliance audit or lawsuit. The information
contained in unstructured data is not always easy to locate. It requires that data in both electronic
and hard copy documents and other media be scanned so a search application can parse out
concepts based on words used in specific contexts. This is called semanticsearch. It is also
referred to as enterprise search.
In customer-facing businesses, the information contained in unstructured data can be analyzed to
improve customer relationship management andrelationship marketing. As social media
applications like Twitter and Facebookgo mainstream, the growth of unstructured data is
expected to far outpace the growth of structured data.

Unstructured data defined

Understanding what unstructured data is and how it is used enables organizations to move beyond
traditional data analysis. Structured data is identified as the data stored in databases (i.e., in DBMSs),
whereas unstructured data is the information stored in word processing documents, e-mail, audio,
PowerPoint presentations, etc. Within the category of unstructured data there exists the concept of semi-
structured data. Semi-structured data includes zipped files, TCP/IP packets and RSS feeds. For the
purposes of this series of articles, both types of data will be classified as unstructured as they are handled
similarly when being transformed into a structured data set used for analysis.
Data used for BI and performance management are generally structured. However, text fields such as
product descriptions and customer addresses are unstructured but stored in a structured way, making the
data easier to handle. The difficulty arises in collecting and analyzing text, audio, and visual data sets that
do not reside in databases and how to transform these forms of data into a structured format that can be
used by the organization. This is where search and text analytics come in.
The world of BI assumes that people know their data. Although this may be true about IT, generally
business units know what information they require, but don’t know where that data exists or how to turn
that data into actionable results. Search allows end users to use text strings to identify what they are
seeking, eliminating the need for intimate knowledge of organizational data structures. For end users
looking to identify sales trends over time or sales by region, a simple search of pre-canned reports allows
quick and easy access to data. Use of text analytics is not as simple. Text analytics involves the
identification of patterns within the data and the development of analyses to identify performance gaps,
trends, benchmarking against competitors, product issues, etc. Where search involves looking for
information that is already available, text analytics uses unstructured data as a springboard for analysis
that can be tied to organizational performance. Either way, both forms of unstructured data identification
are essential to the discussion of unstructured data and contribute to the way that data is currently
leveraged within organizations.

Importance of unstructured data

Because the category of unstructured data is broad and encompasses many types of information, it may
be challenging for organizations to identify the actual importance of utilizing unstructured data to help
drive organizational performance. The use of unstructured data goes beyond the ability to search for

1
reports, documents, PowerPoint presentations and e-mail. Identifying customer complaints,
benchmarking marketing campaigns and identifying insurance claims fraud are just a few of the areas that
can tie the analysis of unstructured data to organizational profit. If general estimates of organizational
unstructured data are correct (between 53 and 85 percent) then organizations that only capture structured
data for analysis are missing potential opportunities for performance optimization as they are utilizing less
than half of their information resources.
Business examples of how the analysis of unstructured data benefits organizations abound. Call centers
provide a good example of how unstructured data can be leveraged to improve internal processes and
performance. Customer complaints may be monitored, but the type of complaint and levels of customer
satisfaction may not be. Freeform text fields within CRM applications can provide decision makers the
information they require to identify trends in customer dissatisfaction and recurring issues. These can
then be used to enhance the overall customer experience, thereby increasing satisfaction and reducing
customer churn rates. In general, freeform text within any organizational software application can be
analyzed to identify trend-based information to help identify areas for improvement.

Conclusion
The importance of unstructured data to the world of BI and BPM can’t be underestimated. As
organizations compete to maintain competitive advantage the way they leverage their data becomes key.
The importance of identifying customer complaints, quality issues within manufacturing, and how to
benchmark against competitors’ marketing campaigns are just a few of the applications of unstructured
data analysis and mark the start of the transition towards the optimization of analysis tools to help drive
organizational performance.

STRUCTURED
Structured information means information that has been traditionally classed as a report. Here,
data, characteristics, key figures, assignments and other attributes are presented in table or
diagram form. The use of trees, grids, and other graphics is also usual. These structures enable
diverse analyses. Reports do not always have to be created individually - a large proportion of
report creation can be carried out automatically. Searching, sorting, filtering, highlighting and
exceptions can be used as desired on individual attributes.

UNSTRUCTURED
In contrast, documents that contain, for example, body texts, pictures, films, are unstructured
information. They are often stored in different ways and created individually and manually rather
than automatically. The search – apart from that of attributes in the document master record or
document folders – is usually a free-text search using a text index that has been created with a
special indexing program.

BLURRING THE BOUNDARIES

Now, attributes in structured data can also contain unstructured information. Shorter and longer
text attributes, and even longer body texts, are not unusual in practice. Moreover, unstructured
documents do not exist in a vacuum, but are usually also linked to structured data (not only by a
document master record and document storage).
Moreover, a text document can contain tables and diagrams, and tables in reports often contain
cells with body text. Users often want to insert text into their reports that give background

2
information about the displayed data. Here the transition is blurred, and it is unclear what has the
higher status in a concrete document: the body text or the tables and diagrams.
Information is often not useful alone, but needs to be explained by its context. This is, of course,
also true for the interaction of structured and unstructured information. Therefore, here too the
operation of both types of information should remain as similar to each other as possible.
With information exchange, structured reports, as well as other documents, also have to be
exchanged and distributed, and it users have to be able to add comments afterwards, not only on
paper. Here it is also important that the information can be assigned in a targeted way within the
document. In addition to this, comments that have already been made in the data source should
also be accessible, without ruining the view of current data.
This means that reports and (in the case of report creation) their fundamental components, are
searched for by attributes and parts as well as by their description. A data source can, for
example, be searched for according to its fields or attributes, such as author, as well as by the
descriptive text.
Lastly, it is must be said that searching for documents, when not using a text index but using
document master record attributes, can be seen as master data reporting.

Q2. Discuss the problems with Unstructured Data. How is it dealt with.

Ans: Dealing with Unstructured Data

Take for example the statements "Jim rode in his Mustang" and "Jim rode on his mustang."
There is little difference in the wording but a vast difference in meaning. A human would correctly
recognize that one is talking about a car and the other a horse. He would also know that the first
sentence must have taken place in the last forty years, since Ford started selling Mustangs in 1964,
and odds are that it occurred on a paved road. The other sentence is more likely to have happened on
a dirt path in the western United States in the latter half of the nineteenth century. There is also a
high degree of probability that Jim refers to a male, adult human. We also recognize that the sentence
might contain a typo, the "I" and the "O" are right next to each other on the keyboard, or someone
might have bad grammar. By reading other sentences in the same document, we convert these
probabilities into certainties.

We constantly engage in those types of analyses and decisions when we speak or read. Very
simple and fast for a human. But there is one problem.

"Humans are better than computers when it comes to less structured data," says Gartner, Inc.
(Stamford, CT) research vice president Alexander Linden. "The problem with humans is that they can't
scale well for large masses of data."

Text Mining Tools

To overcome this scaling problem, companies such as ClearForest Corporation (New York,
N.Y.), Inxight Software, Inc. (Sunnyvale, Cal.),Megaputer Intelligence Inc. (Bloomington, Ind.)
and SPSS Inc. (Chicago, Ill.) have created products to analyze vast quantities of text information and
convert it into actionable intelligence.

3
 The first step typically involves applying "natural language processing" algorithms which
determine the meaning of the sentences taking into account context, grammar, synonyms and
colloquialisms. It can then categorize the documents and group similar ones. Some tools allow
extraction of certain type of data such as all company names or cities. Others present the information
in a graphic form making it easier to spot relationships.

Although this technology is still fairly new and is not as accurate yet as traditional data
mining, its use is expanding. Dow Chemical, for example is using it to conduct patent searches and
manufacturers are using it to mine call center reports for common complaints. The Global Aviation
Information Network (GAIN), an international consortium of airlines, government agencies and
manufacturers, is developing tools to gather data from mechanics, pilot and flight attendant reports to
spot common mechanical problems so they can be corrected before a disaster - a far better method
than sorting through airplane wreckage to try to determine the cause.

"We are trying to get smarter by looking at events that happen relatively frequently, but are
innocuous by themselves because of the robustness of the systems," says Christopher Hart, Systems
Administrator for System Safety for the Federal Aviation Administration. "But if they are part of the
links in an accident chain, we are trying to stop those links before they cause an accident."

Q3. What are the practical applications of Unstructured Data.

Q4. Define Business intelligence. What are its applications.

Ans: Definition -Business intelligence (BI) is a broad category of applications and technologies
for gathering, storing, analyzing, and providing access to data to help enterprise users make
better business decisions.

BI applications include the activities of decision support systems, query and reporting,
online analytical processing (OLAP), statistical analysis, forecasting, and data mining.

Business intelligence applications can be:

• Mission-critical and integral to an enterprise's operations or occasional to meet a special

requirement
• Enterprise-wide or local to one division, department, or project
• Centrally initiated or driven by user demand

Q5. Describe the different components of BI.

Ans: COMPONENTS OF BI.
OLAP (On-line analytical processing): It refers to the way in which business users can slice and dice
their way through data using sophisticated tools that allow for the navigation of dimensions such as time or
hierarchies. Online Analytical Processing or OLAP provides multidimensional, summarized views of business data
and is used for reporting, analysis, modeling and planning for optimizing the business. OLAP
techniques and tools can be used to work with
data warehouses or data marts designed for
sophisticated enterprise intelligence systems.
These systems process queries required to
discover trends and analyze critical factors.
Reporting software generates aggregated views

4
of data to keep the management informed about
the state of their business. Other BI tools are
used to store and analyze data, such as data
mining and data warehouses; decision support
systems and forecasting; document warehouses
and document management; knowledge
management; mapping, information visualization,
Journal of Theoretical and Applied Information Technology
© 2005 - 2009 JATIT. All rights reserved.
62 and dash boarding; management information
www.jatit.org
systems, geographic information systems; Trend
Analysis; Software as a Service (SaaS).

Advanced Analytics: it is referred to as data

mining, forecasting or predictive analytics, this
takes advantage of statistical analysis techniques
to predict or provide certainty measures on facts.

Corporate Performance Management (Portals,

Scorecards, Dashboards): this general category
usually provides a container for several pieces to
plug into so that the aggregate tells a story. For
example, a balanced scorecard that displays
portlets for financial metrics combined with say
organizational learning and growth metrics.

Real time BI: It allows for the real time

distribution of metrics through email, messaging
systems and/or interactive displays.

Data Warehouse and data marts: The data

warehouse is the significant component of
business intelligence. It is subject oriented,
integrated. The data warehouse supports the
physical propagation of data by handling the
numerous enterprise records for integration,
cleansing, aggregation and query tasks. It can
also contain the operational data which can be
defined as an updateable set of integrated data
used for enterprise wide tactical decision-making
of a particular subject area. It contains live data,
not snapshots, and retains minimal history. Data
sources can be operational databases, historical
data, external data for example, from market
research companies or from the Internet), or
information from the already existing data
warehouse environment. The data sources can be
relational databases or any other data structure
that supports the line of business applications.
They also can reside on many different platforms
and can contain structured information, such as
tables or spreadsheets, or unstructured
information, such as plaintext files or pictures
and other multimedia information. A data mart
as described by (Inmon, 1999) is a collection of
subject areas organized for decision support

5
based on the needs of a given department.
Finance has their data mart, marketing has theirs,
and sales have theirs and so on. And the data
mart for marketing only faintly resembles
anyone else's data mart. Perhaps most
importantly, (Inmon, 1999) the individual
departments own the hardware, software, data
and programs that constitute the data mart. Each
department has its own interpretation of what a
data mart should look like and each department's
data mart is peculiar to and specific to its own
needs. Similar to data warehouses, data marts
contain operational data that helps business
experts to strategize based on analyses of past
trends and experiences. The key difference is
that the creation of a data mart is predicated on a
specific, predefined need for a certain grouping
and configuration of select data. There can be
multiple data marts inside an enterprise. A data
mart can support a particular business function,
business process or business unit.
A data mart as described by (Inmon, 1999) is a
collection of subject areas organized for decision
support based on the needs of a given department.
Finance has their data mart, marketing has theirs,
and sales have theirs and so on. And the data
mart for marketing only faintly resembles
anyone else's data mart.
BI tools are widely accepted as a new
middleware between transactional applications
and decision support applications, thereby
decoupling systems tailored to an efficient
handling of business transactions from systems
tailored to an efficient support of business
decisions. The capabilities of BI include decision
support, online analytical processing, statistical
analysis, forecasting, and data mining. The
following are the major components that
constitute BI.

Data Sources
Data sources can be operational databases,
historical data, external data for example, from
market research companies or from the Internet),
or information from the already existing data
warehouse environment. The data sources can be
relational databases or any other data structure
that supports the line of business applications.
They also can reside on many different platforms
and can contain structured information, such as
tables or spreadsheets, or unstructured
information, such as plaintext files or pictures
and other multimedia information.

Q6. Describe the Issues involved in Business Intelligence.

6
Ans : ISSUES IN BI

Experts View: Experts view BI in different

ways. Data warehousing experts view BI as
supplementary systems and is very new to them.
These experts treat BI as technology platform for
decision support application. The author is of
opinion that to data mining experts BI is set of
Journal of Theoretical and Applied Information Technology
© 2005 - 2009 JATIT. All rights reserved.
www.jatit.org
63
advanced decision support systems with data
mining techniques and applications of algorithms.
To statisticians BI is viewed as a forecasting and
multidimensional analysis based tool.

Approaches in Data Warehousing: The main

key to successful BI system is consolidating data
from the many different enterprise operational
systems into an enterprise data warehouse. Very
few organizations have a full-fledged enterprise
data warehouse. This is due to the vast scope of
effort towards consolidating the entire enterprise
data. (Berson et.al, 2002) emphasizes that in
view of emerging highly dynamic business
environment, only the most competitive
enterprises will achieve sustained market success.
The organizations will distinguish themselves by
the capability to leverage information about their
market place, customers, and operations to
capitalize on the business opportunities.

Analysis of right information: Several surveys

including Gartner, Forrester and International
Data Centre report that most of the firms
throughout the globe are interested in investing
in BI. It is to be noted that despite major
investments in enterprise resource planning
(ERP) and customer relationship management
(CRM) over the last decade businesses are
struggling to achieve competitive advantage. It is
due to the information captured by these systems.
Any corporate would look forward for one goal
called ‘right access to information quickly’.
Hence, the firms need to support the analysis and
application of information in order to make
operational decisions. Say for marking seasonal
merchandise or providing certain
recommendations to customers, firms need right
access to information quickly. Implementing
smarter business processes is where business
intelligence influences and influences the bottom
line and returns value to any firm.

Q7. Explain the Business Intelligence Framework.

7
Ans: A framework for implementing Business Intelligence (BI) solutions at an organization
level is provided. The framework includes
1) .an architecture module,
2) a process module,
3) an operations
4) and support module,
5) a governance module
6) and a delivery module.
The architecture module includes an execution architecture, an operations architecture
and a development architecture. The process module manages processes implemented by
Business Units (BUs) to provide BI solutions. The operations and support module manages the
infrastructural requirements of the framework. The governance module manages the
organizational requirements related to the implementation of the framework. The delivery
module manages the delivery of the processed business data to the BUs. The process module, the
operations and support module, the governance module and the delivery module are
implemented, based on the architecture module. The framework is used as a Business
Intelligence Competency Center (BICC) for providing real-time BI solutions to BUs.

Q8. Describe the Business Intelligence Infrastructure.

Ans: Business Intelligence Infrastructure
Business organizations can gain a competitive advantage with a well-designed business intelligence (BI)
infrastructure. Think of the BI infrastructure as a set of layers that begin with the operational systems information and
meta data and end in delivery of business intelligence to various business user communities. These layers are
illustrated in Figure 1.

Figure 1: Business Intelligence Infrastructure

Business Benefits

The payback achieved by building the business intelligence infrastructure is a function of how efficiently it
operates, how well the infrastructure is supported and enhanced by the business organization as well as its capacity

8
for producing business insight from raw operational data. The business intelligence infrastructure delivers key
information to business users. For maximum impact, standards and procedures must be in place to provide key
business information proactively. This business intelligence infrastructure enables the organization to unlock the
information from the legacy systems, to integrate data across the enterprise and empower business users to become
information self- sufficient.

ADVERTISEMENT

Providing managers and knowledge workers with new tools allowing them to see data in new ways
empowers them to make faster and better decisions. Rather than responding to continuous stream of report requests,
the business intelligence platform provides business users self-service decision support via the Web or at the
desktop. The quantifiable benefits of providing such a business intelligence platform are decisions which increase
revenue by identifying and creating up-sell and cross-sell opportunities, improve "valued customer" profitability,
decrease costs or expenses by leveraging infrastructure and automating processes, decrease investment in assets
such as inventory, or improve productivity with better decision making and faster response-to-market changes or
other business events.

The following sections examine more closely each of the layers of the business intelligence infrastructure.

Data Integration. Based on the overall requirements of business intelligence, the data integration layer is required to
extract, cleanse and transform data into load files for the information warehouse. This layer begins with transaction-
level operational data and meta data about these operational systems. Typically this data integration is done using a
relational staging database and utilizing flat file extracts from source systems. The product of a good data staging
layer is high-quality data, a reusable infrastructure and meta data supporting both business and technical users.
Improved data quality can entail matching against third-party name/address/location databases, merging information
from disparate sources into the same information structure and eliminating duplicate, null and outlier values. Building
an efficient data integration process is a key component to delivering powerful business intelligence solutions. Often
this infrastructure needs to bring data across on a daily basis. In order to accomplish this, the processes need to be
efficient and automated. Operator alerts should automatically be generated when exception conditions occur. This
layer will generate significant meta data that must be captured and leveraged in the other layers to ensure proper
delivery, support and guidance to both system administrators and business intelligence users.

Distinguishing characteristics of business intelligence versus operational systems information are primarily along the
lines of purpose, data content, usage and response time requirements. The purpose of operational systems is to
support the day-to-day business processes instead of supporting long-term strategic decision-making which is the
purpose of business intelligence. Data content for operational systems is current and has real-time values as
opposed to historical data that is accurate as of a point in time for business intelligence. Operational data is highly
structured and repetitive, whereas business intelligence is highly unstructured, heuristic or analytical. Operation
support requires information within seconds, yet business intelligence response time requirements range from
seconds to minutes.

Information Warehouse. The information warehouse layer consists of relational and/or OLAP cube services that
allow business users to gain insight into their areas of responsibility in the organization. Important in the warehouse
design the definition of databases that provide information on confirmed dimensions or business variables that are
true across the whole enterprise. The information warehouse is usually developed incrementally over time and is
architected to include key business variables and business metrics in a structure that meets all business analysis
questions required by the business groups. A common practice is to architect the information warehouse into a
sequence of data marts that can be developed within 90 days. The essential responsibility of the first data mart is to
build out the conformed dimensions for the enterprise and the business facts for a specific subject area of the
business. The infrastructure established during that initial data mart effort can be leveraged in subsequent data mart
development efforts.

In order to architect this information warehouse layer correctly, the business requirements and key business
questions need to be defined. When this information is available, there will be additional insight into the business
derived from the underlying data that cannot be fully anticipated before the data is actually available. Key areas to

9
consider in defining requirements relate to the major functional areas of the organization. There are a few key
categories of business intelligence that should be considered: Customer, operational and clickstream intelligence.

Customer Intelligence relates to customer, service, sales and marketing information viewed along time periods,
location/geography, product and customer variables. Business decisions that can be supported with customer
intelligence range from pricing, forecasting, promotion strategy, competitive analysis to up-sell strategy and customer
service resource allocation.

Operational Intelligence relates to finance, operations, manufacturing, distribution, logistics and human resource
information viewed along time periods, location/geography, product, project, supplier, carrier and employee. Business
decisions that can be supported with operational intelligence include budgeting, investment, pricing, hiring, training,
promotion, cost control, scheduling, service levels, defect prevention and capacity planning.

Clickstream Intelligence relates to Web sessions, sales and service information viewed along time periods,
products, customer, Web pages and request type. Business decisions that can be supported with clickstream
intelligence include website optimization, ad space pricing, promotion communication and up-sell strategy.

Automating the warehouse administration is essential for shortening the cycle time for bringing business intelligence
updates into the information warehouse. Features of an automated warehouse administration include operator alerts
for exceptions, automated exception handling based on predefined business rules, DBA alerts for key service
outages and a production scheduled of tasks run according to business requirements.

BI Applications. The most visible layer of the business intelligence infrastructure is the applications layer which
delivers the information to business users. Business intelligence requirements include scheduled report generation
and distribution, query and analysis capabilities to pursue special investigations and graphical analysis permitting
trend identification. This layer should enable business users to interact with the information to gain new insight into
the underlying business variables to support business decisions. Another important application is the balanced
scorecard that displays key performance indicator current values and the targets for financial, customer, internal
systems and human capital categories. The balance scorecard is a summary of key business analytics rolled up to
the appropriate level for the user with capabilities to drill down into more detail. This detail relates to operational,
customer and clickstream intelligence described in the previous section.

In order to achieve maximum velocity of business intelligence, continuous monitoring processes should be in place to
trigger alerts to business decision-makers, accelerating action toward resolving problems or compensating for
unforeseen business events. This proactive nature of business intelligence can provide tremendous business
benefits.

Portals. Presenting business intelligence on the Web through a portal is gaining considerable momentum. Web-
based portals are becoming commonplace as a single personalized point of access for key business information. All
major BI vendors have developed components which snap into the popular portal infrastructure. Portals are usually
organized by communities of users organized for suppliers, customers, employers and partners. Portals can reduce
the overall infrastructure costs of an organization as well as deliver great self-service and information access
capabilities.

Organizational Requirements. There are several organizational considerations that contribute to an efficient
business intelligence infrastructure. First is to have a core business intelligence implementation and support team
dedicated to the principles of optimizing the business intelligence infrastructure. This core team minimally consists of
the following roles:

• BI architect responsible for the overall design and implementation of the business intelligence infrastructure
with special focus on the information architecture in the information warehouse layer
• ETL developer responsible for the design and development of the data staging layer
• BI analyst responsible for identifying the key business intelligence questions of the business decision-
makers and the key requirements of the BI applications layer

10
 • Database administrator responsible for the physical implementation and support of the information
warehouse layer
• Business content managers responsible for delivering the required information to various user communities
of the portal layer

Additionally, there needs to be great organizational support for business intelligence across the subject areas
comprising the business intelligence infrastructure. This is facilitated by a business intelligence steering committee
comprised of the technical managers and business sponsors. This committee ensures that the risks of the business
intelligence projects are mitigated with adequate scope control and good communication to the business sponsors.

Finally, there should be objective measures in place to track the effectiveness of the business intelligence
architecture. These measures should include areas such as business intelligence usage, ratings of business and
technical meta data available, service level metrics, business impacts metrics and efficiency improvement metrics.

How Well is Your Business Intelligence Infrastructure Implemented and Supported?

To determine the completeness and adequacy of your BI infrastructure, answer the following questions. Any "no"
answers indicate opportunity areas for improvement.

1. Do you have an effective data integration process in place to create required business intelligence on a
daily basis?
2. Are continuous monitoring processes in place to allow alerts to be communicated immediately to those who
need to take action?
3. Is your information delivery process automated?
4. Is your warehouse administration infrastructure completely automated?
5. Are alerting techniques used to communicate exceptions quickly so decisions are accelerated?
6. Are the key business questions being answered about your business areas of responsibility?
7. Is information available on standardized dimension such as customer, product and geography?
8. Do you have adequate competitive information to answer key business questions?
9. Have you delivered scorecards on key performance indicators to top decision-makers?
10. Do you leverage your enterprise portal infrastructure to deliver business intelligence?

Mark Robinson manages Greenbrier & Russel's Business Intelligence practice. With more than 20 years of
experience in business and technology fields, Robinson has performed traditional leadership roles in IT
management, product management, practice management and solutions delivery as well as leading companies in
strategic transformation efforts through investments in business intelligence. As a consultant, he has been involved
in business transformation efforts that have begun with critical success factors studies, assessments and discovery
workshops that focus on the value of business intelligence as it aligns with the business strategy. In addition to
teaching, Robinson has been a speaker/educator on various topics in business intelligence with industry associations
including The Data Warehouse Institute and the Finance Executives Institute. You can reach him at
[email protected].

For more information on related topics, visit the following channels:

• Enterprise Application Integration (EAI)

• Business Intelligence (BI)

Q9. What are the different processes in Business Intelligence.

Q9. What are the important roles and responsibilities of a business intelligence
(BI) team?

11
 Ans: The BI team typically is responsible for data definitions, source system
identification and connectivity, working with end users to define business
requirements and needs, performing data validation, writing BI policies and
procedures for steering committee approval, overseeing the implementation of
approved projects and assuring that the promised benefits of BI projects are
achieved. The business intelligence team is also tasked with reporting on the
progress of the BI program and defined projects to senior executives.

Meanwhile, a company’s data management team will work with BI team

members or consultants assigned to the BI program to assure that guiding
principles are met pertaining to organizational policies related to data
procurement, analysis and reporting. In addition, it will assign specific projects to
implementation teams and prioritize data access requests when the availability of
resources is insufficient to meet user demand. The size of a BI team is variable
and will depend on how many departments and/or functional areas are
represented.

Q10. Discuss the Business Intelligence Technology.

Ans: Three Trends in Business Intelligence Technology
This year is shaping up to be one of great changes in the business intelligence (BI) technology arena. Will these be
for good or ill? That remains to be seen. In any case, here are three trends that are already making a huge impact on
the technological landscape and some thoughts about whether these are good or bad for the industry.

Trend # 1: Consolidation of BI Companies

You would have to live under a rock to not have read or heard something about a business intelligence company
gobbling up or forming exclusive relationships with another BI company. These acquisitions / relationships seem
to fall into one of three categories:

• Data quality vendors are being scooped up. There are fewer standalone data quality vendors these days.
Many have now become part of full suite vendors (those offering everything from ETL processing to the
ultimate delivery of BI analytics) or part of ETL packages. These purchases add to data integration
capabilities by improving the data that is being integrated.

• EII (enterprise information integration) vendors are also being purchased or forming exclusive partnerships
with other BI vendors. These strategic allegiances broaden the data integration capabilities of either the
standalone ETL or full-service vendors by adding the ability to deliver real-time integration capabilities.

• Small niche players are being acquired before they even make a name for themselves. IBM and Oracle, in
particular, seem to have a propensity to purchase these innovative and bright point-solution vendors. These
additions give these big companies quick solutions such as master data management (MDM) or customer
data integration (CDI) applications.

12
So – is this a good thing? On the plus side, consolidating your business intelligence solution into the hands of a single
vendor makes life simpler in terms of contracting, a single vendor interface and a reduction of finger-pointing when
things don’t go according to plan. There is no doubt an appeal to the one-stop-shopping mentality.

The goal of the full-service vendor is to provide a fully integrated, end-to-end solution that satisfies the majority of
your needs. These vendors state that their offerings significantly reduce the cost and implementation time. These
products simplify maintenance, make enhancements easier and shorten the learning curve for implementers.

On the negative side, just because a company bought a technology does not mean that the technology is instantly
integrated into the rest of the vendor’s offerings. You may still struggle with integration issues for several years after
the technology was acquired. Second, there is a question of focus. The best-of-breed vendors have a single mission
in life – to make the best solution for a particular problem. Their solution is usually a surgical strike – complete, fast
and elegant. The full-service vendors usually can’t afford to expend that kind of energy on a point solution, especially
when they are applying a lot of their energy on integrating their technologies. Their compromise may be a suite that
meets the majority of your needs, but not all. When shopping for best-of-breed versus full-service technologies, look
at the partnerships between companies. You should get a pretty good idea of who may be next on the auction block.

Finally, no single business intelligence vendor has a commanding lead in terms of sales or market share. That means
that there is still plenty of room for both types of vendors. It is still a place where small, innovative technologies can
effectively compete against their much larger and more established brethren.

Trend # 2: Moving from Strategic BI to Operational or Right-Time BI

The big push in BI analytics is to shorten the latency between when a business event happens and an action is taken.
According to Dick Hackathorn, this latency has three components – data preparation latency (the time it takes to get
the data ready for analysis), analysis latency (the time it takes to get the results of an analytic operation) and decision
latency (the time it takes for the person receiving the results to understand what action must be taken). For
operational business intelligence to be effective, these three latencies must be reduced to nearly zero time.

To reduce data latency, we see more and more virtual BI components being created, including virtual operational
data stores (ODSs) and data marts using enterprise EII technologies. If your operational data is in fairly good shape
(minimal integration and data cleanup required), a virtual ODS or even an oper mart may be a solution to reducing
data latency. However, it is mandatory to monitor the effects of this environment on the operational systems.

For analysis latency, we are seeing technologies that offer business activity monitoring (BAM) or operational
dashboards as inline operational analytic engines that constantly serve the results to the business user and send
alerts or alarms immediately when thresholds are exceeded. Key performance indicators (KPIs), important metrics
delivered hourly or even more frequently, and consolidated current operational results can be displayed through the
dashboards or portals, giving the operations personnel insight into key events that are occurring.

While speeding the collection, analysis and display of operational data is certainly useful to the business, it must be
remembered that not all business intelligence data must be included. Many IT implementers do not perform thorough
due diligence to determine precisely what data must be included in an operational BI application. They make the

13
catastrophic error of including as much as they can, forcing all data to be “real time,” thereby creating an unwieldy
and unmanageable BI world.

The message is to carefully evaluate the push for real-time analytics. Understand the business need completely, and
you may find that it is really only a very small percentage of data that must be rushed into the hands to the business
consumers. Most analytic data can be hours, days or months old and still be relevant to the decision-making process.

Trend # 3: More Sophisticated Analytics and Data Visualization

At first there were simple reports and queries; then came multidimensional analysis. While cubes and star schemas
are still very popular, there is a movement now toward more complex, deeper analytics using data mining,
statistical methods and technologies that support pure ad hoc, unplanned forms of analytics. Perhaps it is just the
natural evolution of business intelligence toward a more mature environment. Certainly, the level of business
expertise is much more sophisticated and advanced than it was 10 years ago, and the ease with which even non-
skilled users can perform these complicated analyses increases their adoption and utilization.

These technologies have raised the awareness of predictive analytics or guided decision-making capabilities, making
it possible to embed these in operational flows. Companies are now able to perform operational or right-time BI,
giving the front-line workers the ability to access and use the results of these analytics, combined with operational
data, for their daily activities.

As exciting as these are, it must be remembered that these capabilities don’t just come into being serendipitously.
They must fit into the enterprise’s overall BI architecture and technological infrastructure or chaos will surely reign.
Operational BI also requires a thorough understanding of the business processes or workflow that it will enhance.
Without this understanding, BI implementers cannot know how or where to embed these valuable insights for
maximum benefit.

There are other technological trends in our business intelligence marketplace, but these seem to be the ones having
the highest impact on business intelligence today. It would be interesting to revisit these trends in six months to see if
other trends have emerged to captivate our attention and change the direction of our still young and growing industry.

14