Scribd Logo
Upload

Welcome to Scribd!

  • 36 views

    An Overview of Data Warehousing and OLAP Technology What Is Decision Support?

    Uploaded by

    Rajani Nain
    This document provides an overview of data warehousing and OLAP (online analytical processing) technology. It defines a data warehouse as a collection of data marts containing integrated, subject-oriented data from multiple sources used to support management decision making. The document discusses the differences between OLTP and OLAP systems and why it is beneficial to separate them into different databases. Common OLAP operations like roll-up, drill-down, slice, dice, and pivot are also summarized.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd

    An Overview of Data Warehousing and OLAP Technology What Is Decision Support?

    Uploaded by

    Rajani Nain
    36 views4 pages
    This document provides an overview of data warehousing and OLAP (online analytical processing) technology. It defines a data warehouse as a collection of data marts containing integrated, subject-oriented data from multiple sources used to support management decision making. The document discusses the differences between OLTP and OLAP systems and why it is beneficial to separate them into different databases. Common OLAP operations like roll-up, drill-down, slice, dice, and pivot are also summarized.

    Original Title

    OLAP

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document provides an overview of data warehousing and OLAP (online analytical processing) technology. It defines a data warehouse as a collection of data marts containing integrated, subject-oriented data from multiple sources used to support management decision making. The document discusses the differences between OLTP and OLAP systems and why it is beneficial to separate them into different databases. Common OLAP operations like roll-up, drill-down, slice, dice, and pivot are also summarized.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    36 views4 pages

    An Overview of Data Warehousing and OLAP Technology What Is Decision Support?

    Uploaded by

    Rajani Nain
    This document provides an overview of data warehousing and OLAP (online analytical processing) technology. It defines a data warehouse as a collection of data marts containing integrated, subject-oriented data from multiple sources used to support management decision making. The document discusses the differences between OLTP and OLAP systems and why it is beneficial to separate them into different databases. Common OLAP operations like roll-up, drill-down, slice, dice, and pivot are also summarized.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 4

    An overview of Data Warehousing and OLAP Technology

    What is decision support?


    Decision support systems are a class of computerized information systems that support decision making activities.

    Presenter: Pooyan Fazli Discussion by Nguyet Department of Computer Science University of British Columbia

    Decision support systems usually require consolidating data form many heterogeneous sources: these might include external sources. -Such as stock market feeds.

    What is a Data Warehouse?


    Defined in many different ways: In simplest terms Data Warehouse can be defined as collection of Data marts A data warehouse is a subject-oriented, integrated, timevariant, and nonvolatile collection of data in support of managements decision-making process.W. H. Inmon A data warehousing is a collection of decision support technologies, aimed at enabling the knowledge worker to make better decisions

    Data Warehouse vs. Operational DBMS


    OLTP (on-line transaction processing) Major task of traditional relational DBMS Day-to-day operations: purchasing, inventory, banking, manufacturing, payroll, registration, accounting, etc. OLAP (on-line analytical processing) Major task of data warehouse system Data analysis and decision making

    Difference between OLAP and OLTP


    OLTP OLAP Knowledge worker Decision support Subject-oriented Historical, summarized, multidimensional, Ad-hoc Lots of scans Complex query Millions Hundreds 100 GB-TB Query throughput

    Why Separate Data Warehouse?


    High performance for both systems DBMS tuned for OLTP: access methods, indexing, concurrency control, recovery Warehousetuned for OLAP: complex OLAP queries, multidimensional view, consolidation Different functions and different data missing data: Decision support requires historical data which operational DBs do not typically maintain data consolidation: DS requires consolidation (aggregation, summarization) of data from heterogeneous sources data quality: different sources typically use inconsistent data representations, codes and formats which have to be reconciled

    Users Function DB Design Data Usage Access Unit of work # rec accessed # users DB size Metric

    Clerk, IT professional Day to day operations Application-oriented Current, up-to-date detailed. repetitive Read/write Short, simple transaction tens thousands 100 MB-GB Transaction throughput

    A Data Warehousing Architecture


    Tier1: Data Warehouse Server Tier2: OLAP Server OLAP Server Tier3: Clients

    Data Warehouse Back-End Tools and Utilities


    Data extraction
    get data from multiple, heterogeneous, and external sources

    External Sources

    Data cleaning
    detect errors in the data and rectify them when possible

    Data transformation
    Operational Databases
    Extract Transform Load Refresh

    Data Warehouse

    Serve

    Analysis Query/Reports Data mining

    convert data from legacy or host format to warehouse format

    Load
    sort, summarize, consolidate, compute views, check integrity, and build indicies and partitions

    Refresh
    propagate the updates from the data sources to the warehouse
    Data Marts

    Data Sources

    Data Storage

    OLAP Engine Front-End Tools

    Typical OLAP Operations


    Roll up (drill-up): summarize data by climbing up hierarchy or by
    dimension reduction

    From Tables to Data Cubes


    A data warehouse is based on a multidimensional data model which views data in the form of a data cube In a multidimensional data model, there is a set of numeric measures that are the objects of analysis. Each of the numeric measures depends on a set of dimensions, which provide the context for the measure.

    Drill down (roll down): reverse of roll-up


    from higher level summary to lower level summary or detailed data, or introducing new dimensions

    Slice and dice: taking a projection of the data on a subset of dimensions


    for selected values of the other dimension

    Pivot (rotate): reorient the cube, visualization, 3D to series of 2D


    planes

    From Tables to Data Cubes


    ct

    A Sample Data Cube


    TV PC VCR sum

    The dimensions together are assumed to uniquely determine the measure. Each dimension is described by a set of attributes. The attributes of a dimension may be related via a hierarchy of relationships.

    Pr od u

    1Qtr

    2Qtr

    Date
    3Qtr 4Qtr sum

    Total annual sales of TV in U.S.A. U.S.A Canada Mexico sum

    Country

    Database Design Methodology


    Time
    Database Designs
    T_key T_day T_day_week T_month T_quarter T_year

    Star Schema
    Sales Fact Table
    Time_key Item_key item
    I_key I_name I_brand I_type I_supplier_type

    Star Schema

    Snowflake Schema

    A fact table in the middle connected to a set of dimension tables

    A refinement of star schema where hierarchy is normalized into a set of smaller dimension tables, forming a shape similar to snowflake

    Branch
    B_key B_name B_type

    Branch_key Location_key Measures Units_sold Dollars_sold Avg_sales

    location
    location_key street city province country

    Star Schema

    Snowflake Schema
    Time
    T_key T_day T_day_week T_month T_quarter T_year

    Materialized Views In a Warehouse


    Challenges in exploiting materialized views

    Sales Fact Table


    time_key item_key Item
    I_key I_name I_brand I_type I_supplier_type Supplier S_key S_type

    identify the views to materialize exploit the materialized views to answer queries, efficiently update the materialized views during load and refresh.

    Branch
    B_key B_name B_type

    branch_key location_key Measures units_sold dollars_sold avg_sales

    Location
    location_key street city City C_key C_city C_province C_country

    Snowflake Schema

    Materialized Views In a Warehouse


    The currently adopted industrial solutions to these problems consider materializing views that have a relatively simple structure. Such views consist of joins of the fact table with a subset of dimension tables with the aggregation of one or more measures grouped by a set of attributes from the dimension tables. The selection of views to materialize must take into account workload characteristics, the costs for incremental update, and upper bounds on storage requirements

    Metadata Requirements
    Administrative metadata
    Source database and their contents Source database and their contents Back-end and front-end tools Definitions of the warehouse schema Pre-defined queries and reports Data mart locations and contents Data refresh and purging policies User profiles and user access control policies

    Metadata Requirements
    Business metadata
    Business terms and definitions Ownership of data Charging policies

    Operational metadata
    Data lineage: history of migrated data and sequence of transformations applied Currency of data: active, archived, purged Monitoring information: warehouse usage statistics, error reports, audit trails

    You might also like