Scribd
Upload
233 views

Database Migration Case Study

A telecom company was experiencing performance issues with their SQL Server 2005 data warehouse as data volumes increased. They migrated to a Netezza appliance to gain parallel processing capabilities. This reduced their daily ETL cycle from 6 hours to 40 minutes and improved BI report performance from 2 minutes to 30 seconds. However, scripts needed optimization for Netezza and debugging was challenging due to limited error messages.

Uploaded by

Swapnil Deshmukh
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
233 views

Database Migration Case Study

A telecom company was experiencing performance issues with their SQL Server 2005 data warehouse as data volumes increased. They migrated to a Netezza appliance to gain parallel processing capabilities. This reduced their daily ETL cycle from 6 hours to 40 minutes and improved BI report performance from 2 minutes to 30 seconds. However, scripts needed optimization for Netezza and debugging was challenging due to limited error messages.

Uploaded by

Swapnil Deshmukh
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Database Migration

SQL Server to Netezza


Case Study

Created by InfoCepts

Database Migration Case Study

Introduction Our client, who is major data analysis provider for telecom sector, used SQL Server 2005 as DW database for their products. However they experienced performance degradation in database as the volume of data and the processing involved started increasing. Business Challenges Faced ETL process slow down Every month there is an ETL run that runs daily for 10 days called Production Cycle, which processes and loads data into the Data Warehouse. This Production Cycle took almost 6 hrs to complete every day. The time taken was mostly due to o Heavy processing during ETL on the database server. o Sequential processing done by the SQL Server. Degrade in Performance of BI Reports used for data quality testing by QC team, daily This was mainly due to execution time spent by database for complex queries retrieving voluminous data and large number of parallel active users to database. Large number of database objects and complex data storage structures to handle and process voluminous data with maximum efficiency.

Solution Architecture and Approach After studying the challenges and processing needs the problem found was rotating around entire architecture of database and solution was selected based on following criteria Parallel Processing at database level. Efficient storage of voluminous data. Efficient processing of data especially for Data warehouse architecture. Support for Large Parallel connections.

Netezza appliance supported all the above criteria, cost effectively, hence client introduced it to replace SQL Server. After this entire production cycle ETL was modified to perform on Netezza DW architecture.

InfoCepts Case Study

Page 2 of 3

Database Migration Case Study

Achievements The Production cycle time was brought down from 6 hrs to 40 min. This improved the performance of BI Reports for data quality testing which were run after the completion of the daily load in Production cycle. This improved the turnaround time of the teams which used these reports for analysis and QC, phenomenally. The performance of the BI Reports increased (Report which ran for 2 mins on SQL Server now runs within 30 secs in Netezza) In the existing SQL Server Production cycle code, there were quite a few steps that were required for breaking down huge data into smaller subsets and then processing this data. Since Netezza is good at handling huge data and processing queries in parallel, all these additional objects and processing were removed from the Production cycle code.

Constraints Since Netezza appliance stores and handles data differently than traditional RDBMS like SQL server and Oracle, scripts and SQLs need to be optimized differently for which there is Need for technical expertise for converting existing T-SQL and SQL Server scripts to Netezza NZSQL. SQL Server scripts may not work as is on Netezza. These may need modifications so as to achieve optimum results. Debugging of Netezza code is difficult since Netezza error messages are not very detailed. Compilation of Netezza code is not very strong. Syntactical errors in the code are also uncovered during procedure run time.

InfoCepts Case Study

Page 3 of 3

You might also like