CH 18
CH 18
18-1
Introduction
Anyone who has ever been responsible for the management and administration of a database knows that the ability to import and export data is essential. Microsoft SQL Server 7.0 introduced a whole new way of performing data migrations with DTS (Data Transformation Services).
What Is DTS?
DTS can be used to import, export, and transform data between data sources. Prior to SQL Server 7.0, the only data migration tool that shipped with SQL Server was BCP (Bulk Copy Program). This tool was efficient for moving large amounts of data in and out of SQL Server. However, its DOS interface was limiting. Microsoft re-engineered BCP and integrated it into DTS with the release of SQL Server 7.0 to provide users with a much more robust data transformation tool that included a Windows interface. In SQL Server 2000, DTS has been upgraded and even more functionality has been added.
18-2
18-3
This is where you select your data source. Select the type of data source from the Source drop-down list box. Some available choices are SQL Server, Microsoft Access, Excel, Sybase System 11, and text file. The rest of the options on this page will vary depending on your type of source. You may be asked to enter the user id and password for accessing your source, the file location of the source, or the specific database to be used within your source. This is where you select your data destination. This screen functions the same as the Choose a Data Source screen. This is where you specify whether your source data is coming from a specific table or is the result of a query that can pull data from several tables. If both your data source and destination are SQL Server, version 7.0 or later, you have an additional option on this page called Copy Objects Between SQL Server databases. Selecting this option allows you to copy database elements such as tables, views, users roles, and indexes between your source and destination. This screen only appears if you selected that your data will be coming from a query on the previous screen. This is where you can type the SQL query to be used for pulling your data out of the source object. You can load an existing query into the screen by clicking the Browse button and selecting the existing .SQL file. This is where you select the specific table or tables from your source to be imported into your destination. You can also select views as your source. If you are using a query as your data source, the query you entered on the previous page will show up as your only source option on this page. Clicking the Transform button for a specific source table will show you how DTS plans to perform the column mappings between the source and destination. You can modify the column mappings and the transformation script here.
18-4
Table 1. The screens in the DTS import wizard and their functions.
NOTE
The screens that appear in the wizard can vary slightly depending on your type of data source and destination. For example, if you are using a text file as either your source or destination, you will be presented with an additional screen in the wizard that allows you to specify the format of your text file.
Figure 1. The Save, schedule, and replicate package screen in the DTS Wizard.
18-5
Try It Out!
Here are the steps to import the authors table from the pubs database into the Northwind database using the DTS Import/Export Wizard: 1. Right-click on the Data Transformation Services node in the Enterprise Manager tree view and select All Tasks|Import Data. This launches the DTS Import Wizard opening dialog box. 2. The welcome screen appears. Click the Next button to continue. 3. This screen is where you enter information about your data source. Select the Microsoft OLE DB Provider for SQL Server since youre importing data from a SQL Server database. 4. Select the server where the pubs database resides from the Server drop-down list box. Click the Use Windows Authentication radio button if you are using integrated security. Otherwise, click the Use SQL Server Authentication radio button and enter a valid username and password for accessing the server. Select the pubs database from the Database drop-down list box and click the Next button. 18-6 Microsoft SQL Server 2000 Professional Skills Development
Copyright by Application Developers Training Company and AppDev Products Company, LLC All rights reserved. Reproduction is strictly prohibited.
6. To copy the table verbatim from the data source, check the Copy Table(s) and View(s) from the Source Database radio button. Click Next to continue.
18-7
8. By default, DTS assumes that you want to create a new table in the Northwind database (also named authors) in which to import the data. Unless you want to rename the table, do not alter the Destination column. Click the Next button to continue. 9. The next screen is where you specify the scheduling and saving options for the wizard. To execute the transformation right away, click the Run Immediately option. To save as a package, select the Save as DTS Package and SQL Server options. Click Next to continue. 10. You will be prompted to enter a name for the DTS packagetype DTSWizardImport and select the default options. Click Next to continue.
18-8
Figure 4. The DTS Import/Export Wizard executing the import and saving the package.
Once the package has been saved, it can be executed and modified from within SQL Servers DTS Designer. The next section describes working with DTS Packages from within the DTS Designer.
18-9
DTS Packages
Once you save the data in a DTS Import or Export Wizard, it becomes a DTS package. A DTS package is an organized collection of connection objects, tasks, transformations, and workflow constraints. Packages contain one or more steps that are executed when the package is executed. Packages can be scheduled, edited, password-protected, and retrieved by version.
Key Terms
Package A program that executes one or more data transformations in addition to other tasks associated with the migration of data. Task Connection object A work item incorporated into a DTS package. A component of a DTS package that contains information about a data source. Transformation task A component of a DTS package that handles the migration of data from one data source to another.
18-10
DTS Packages
NOTE The DTS Designer is simply a graphical interface for writing to the DTS COM components. It is possible to create DTS packages from scratch using any programming language that supports COM such as Visual Basic or C++. DTS packages created programmatically in this way bypass the use of the DTS Designer. Additional programming skills are required for creating DTS packages programmatically, but programmatically created packages can provide additional control and flexibility that is not available when you create a package using the DTS Designer.
A DTS package typically consists of one or more combinations of the following three types of components: Connection objects: A connection object is a built-in component that allows you to establish a connection to a data source or destination. A 18-11
Each of these three types of components is discussed in detail in the following sections.
Connection Objects
Connection objects are used to gather information about your data source or destination. You must have at least one connection object in your package in order to perform a data migration using a transformation task. The Data section of the toolbar in the DTS Designer lists all of the available types of connection objects. Once you have added a connection object to your package by clicking on it in the toolbar, you have the opportunity to set its properties. You can also set or modify the properties for a connection object after adding it to your package.
Try It Out!
Follow these steps to view the Connection properties for the package the wizard created: 1. Select the DTSWizardImport package in the Local Packages window. Right-click and choose Design Package from the menu.
18-12
DTS Packages
2. Right-click on Connection1 and select Properties from the menu. This loads the Connection Properties dialog box, as shown in Figure 6.
3. Click OK and display the properties for Connection2. All of the information in the Connection1 and Connection2 Properties dialog boxes was initially collected by the wizard.
18-13
18-14
DTS Packages
The different connection objects found on the toolbar are described in Table 2.
Type of Connection Microsoft OLE DB provider for SQL Server Microsoft Access Description Uses OLE DB to establish a connection to a SQL Server database. Properties include the server name, login name and password, and the database name. Establishes a connection to an Access MDB file. Properties include the path and file name of the MDB and the user name and password. Establishes a connection to an Excel XLS file. Properties include the path and file name of the XLS and the user name and password. Establishes a connection to a dBase 5 file. Properties include the path and file name of the dBase 5 file and the user name and password. You can also connect to a dBase III or IV file by making the appropriate selection from the data source drop-down list box in the Connection Properties dialog box. Establishes a connection to an HTML file to be used as a data source. Properties include the path and file name of the file and the user name and password. Establishes a connection to a Paradox 5.x file. Properties include the path and file name of the Paradox 5.x file and the user name and password. You can also connect to a Paradox 3.x or 4.x file by making the appropriate selection from the data source drop-down list box in the Connection Properties dialog box. Establishes a connection to a text file. Properties include the path and file name of the text file. Advanced properties include specifying whether the file is fixed length or delimited. You can further specify the fixed length for each column or the characters that represent row and column delimiters. Establishes a connection to an Oracle database using an ODBC driver. Properties include the server name and user name and password. Establishes a connection to a data source using a Microsoft Data Link. You can either load an existing UDL or define a new data link using an OLE DB provider. Establishes a link to a data source using an ODBC driver. Properties include the name of the DSN and the user name and password.
Microsoft Excel
DBase 5
Microsoft ODBC driver for Oracle Microsoft Data Link Other connection
Table 2. The types of connection objects that are available on the DTS Designer toolbar.
In addition to the types of connection objects that are available on the toolbar, there are also a few other types of connection objects. These include Microsoft SQL Server 2000 Professional Skills Development
Copyright by Application Developers Training Company and AppDev Products Company, LLC All rights reserved. Reproduction is strictly prohibited.
18-15
Try It Out!
Follow these steps to view the transformation task for the package the wizard created: 1. Select the arrow connecting Connection1 and Connection2 in the Designer window. Right-click and choose Properties from the menu. This loads the Properties dialog box.
18-16
DTS Packages
2. The dialog box has five different tabs to show the options available for the transformation task, which are described in Table 3. Click the Transformations tab. This displays the transformations for the task, as shown in Figure 8.
3. Right-click on the join line between the Source and the Destination and choose Edit from the menu. This allows you to further modify the properties of the task. Click OK or Cancel when finished.
18-17
Destination
Transformations
Lookups
Options
Table 3. The five tabs on the Transform Data Task Properties dialog box.
Try It Out!
Here are the steps to create a simple package that imports the titles table from the pubs database into the Northwind database. This is essentially the same task you accomplished before with the Import Wizard, but this time youll do it without the aid of the wizard and youll import a different table.
18-18
DTS Packages
1. Expand the Data Transformation Services node in the Enterprise Manager. Right-click on Local Packages and select New Package from the menu. The DTS Designer loads with an empty screen. 2. Click the Microsoft OLE DB Provider for SQL Server icon in the Data toolbar to add a Connection object to the package to represent the data source. This opens the Connection Properties dialog box. 3. Type Pubs as the name of the connection. Enter the information about the data source by selecting the appropriate authentication information for your server. Select pubs from the Database drop-down box. Click OK. This adds a new connection to the Designer window. 4. Repeat step 2 to add a connection object to represent the data destination. This time name the connection Northwind. Make sure Microsoft OLE DB Provider for SQL Server is selected in the Server drop-down list box. And that Northwind is selected from the Database drop-down box. Click OK. This adds a second connection to the Designer window. 5. Click on the Pubs Connection object. Hold the SHIFT key down, and click on the Northwind Connection object. Both connection objects should be highlighted. 6. Click on the Transform Data Task button in the Task section of the toolbar shown in Figure 9. This adds a transformation going from the Pubs connection to the Northwind connection. The transformation task is represented by a gray arrow.
18-19
Figure 10. Naming the connection and specifying the source table.
18-20
DTS Packages
8. Click the Destination tab. Since the Titles table doesnt exist in the Northwind database, you need to have DTS create one. Click the Create button, which loads the dialog box shown in Figure 11. This is the script that will be used to create the table in the destination database. Click OK.
9. Click the Transformations tab. By default, DTS assumes that you are doing a straight column copy from the source to the destination. If you want to change any of the column mappings, double-click the join line between them. Otherwise, click OK to complete the task. 10. Click the Save toolbar button or choose Package|Save from the menu. Type Titles to Northwind in the Package Name text box and fill in the other properties. Click OK. This saves the package. 11. Run the package by clicking the Execute toolbar button or selecting Package|Execute.
18-21
Bulk Insert Execute Package Message Queue Transfer Error Messages Transfer Databases Transfer Master Stored Procedures Transfer Jobs Transfer Logins Dynamic Properties
Table 4. The built-in tasks that are available in the DTS Designer.
18-22
Precedent Constraints
Precedent constraints allow you to specify the tasks to be executed sequentially. There are three kinds of precedent constraints. They are listed in Table 5.
Constraint Type On success On failure On completion Description Specifies that the first task must complete successfully before the second task can begin. Specifies that the first task must fail to complete successfully before the second task can begin. Specifies that the first task must complete before the second task can begin. The second task will execute regardless of whether the first task succeeded or failed.
To add a precedent constraint between two tasks, select the task you want to execute first. Holding down the Shift key, select the task you want to execute second. Then select On completion, On success, or On failure from the Workflow menu depending on which of the three types of precedent constraints you wish to create.
DTS Transactions
Without transactions, there is no way to undo transformations that have already completed successfully if an error occurs later in the package execution. When transactions are implemented, they allow you to group package steps together so that they can be committed or rolled back as a unit. There are two rules to remember when implementing transactions. First, in order for transactions to work with DTS you must use a data provider that supports transactions. Second, only one transaction can be active at a time within DTS. Microsoft SQL Server 2000 Professional Skills Development
Copyright by Application Developers Training Company and AppDev Products Company, LLC All rights reserved. Reproduction is strictly prohibited.
18-23
18-24
Error Handling
As your packages begin to get more robust, the ability to track errors or problems that may occur becomes increasingly important. DTS provides you with the ability to create package logs and exception files to help you handle any errors that may arise.
Package Logs
Package logs provide an overview of all the steps in a package. Each step in the package is recorded in the log. Since any information as to whether or not the step executed successfully and any errors that might have been encountered are also recorded, the package log can be helpful in determining where and why a package error may have taken place. Follow these steps to generate a package log in the DTS Designer: 1. Make sure no items are selected in the designer window, and choose Package|Properties from the menu. Click the Logging tab of the Package Properties dialog box. 2. If you would like your package log information to be stored in an external file, enter the name and path of the external file in the Error File text box. 3. If you would like your package log information to be stored within SQL Server, check the Log Package Execution to SQL Server check box. Click OK when finished.
18-25
Exception Files
Exception files can be generated as a data transformation is taking place. The purpose of the execution file is to provide specific information on any row of data that encounters a problem when attempting to migrate it into the data destination. Follow these steps to add an exception file to a transformation task: 1. Right-click on the transformation task and select Properties. 2. Click the Options tab and enter the name of the exception file in the Name text box. 3. Select the file type and file formatting options from the File Type and File Format sections and click OK when finished.
Global Variables
Before discussing the three types of ActiveX scripts, its important to discuss the role of global variables in DTS Packages. Global variables allow you to store data and object references that can be passed between different types of scripts. There are two kinds of global variables: those declared at design time and those declared programmatically at run time. Global variables that are created at design time have the ability to retain their values in between package executions. This can be crucial if you need to carry data from one package execution to the next. Follow these steps to create a global variable at design time: 1. Right-click anywhere in the DTS Designer and select Package Properties. 2. Click the Global Variables tab. Any existing global variables will be displayed along with their data type and current value. 3. Click the New button and type in the name of the global variable, select its database, and enter a starting value (optional).
18-26
Transformation Scripts
The first script location or kind of script is the transformation script. A transformation script is executed as data is being moved from one location to another.
18-27
Row transform
Follow these steps to add a new script to your transformation task: 1. Right-click on the transformation task and select Properties. 2. Click the Transformations tab. 3. Click the New button. The Create New Transformations dialog box appears. 4. Select ActiveX script and the type of transformation and click OK. 5. Use the Transformation Options dialog box to set the options for the script. Use the Source and Destination columns to select the column or columns from the data source and destination that you will be working with in your script. On the Phases tab, select which phase in 18-28 Microsoft SQL Server 2000 Professional Skills Development
Copyright by Application Developers Training Company and AppDev Products Company, LLC All rights reserved. Reproduction is strictly prohibited.
The second unique characteristic of transformation scripts is that they are contained within functions. These functions must return specific values called return codes. Return codes tell DTS how to proceed after running this script. The return codes are referenced by constants.
18-29
DTSTransformStat_SkipRow
DTSTransformStat_SkipFetch
DTSTransformStat_AbortPump
Table 8. A sample of the return code constants available for transformation scripts.
NOTE
If you are not familiar with how to write code using either VBScript or JScript, you will have to gain a basic understanding of the programming languages involved before you will be able to write effective scripts in DTS.
The following code example written in VBScript is an example of a Row Transform transformation script. In this example, the data source stores a customers name in two fields, first_name and last_name. However, in the destination, there is only a single customer_name field that stores both the first and last name. The script combines the first_name and last_name fields to create the data to be inserted in the customer_name field in the destination.
Function Main() Dim NewName NewName = DTSSource(first_name) & " " & _ DTSSource(last_name) DTSDestination(customer_name) = NewName Main = DTSTransformStat_OK End Function
18-30
Workflow Scripts
The second kind of script is the workflow script. A workflow script is associated with a transformation task and runs prior to the transformation task. This gives you the ability to do things such as verify external conditions and dynamically decide whether or not to cancel the transformation task prior to its execution. Unlike transformation scripts that have four custom collections, workflow scripts only have access to one custom collection, the DTSGlobalVariables collection. That means that the global variables for your package can be accessed and manipulated by workflow scripts. Similar to transformation scripts, workflow scripts are contained within functions and must pass back a specific return code. The return code values available for workflow scripts are different from the return code values that are available for transformation scripts. Table 9 lists the only three return codes that are valid for workflow scripts.
Return Code DTSStepScriptResult_ExecuteTask DTSStepScriptResult_DontExecuteTask Description This return code instructs DTS to proceed with the associated transformation task. This return code instructs DTS to skip over the associated transformation task without executing it. This script instructs DTS to loop back to the beginning of the workflow script. The resulting loop will continue until one of the other two return codes is passed. This looping buys you time to do things such as complete external processing.
DTSStepScriptResult_RetryLater
Table 9. The return code constants that are available for workflow scripts.
18-31
Set fso = CreateObject("Scripting.FileSystemObject") IF NOT(fso.FileExists("C:\Biblio.mdb")) THEN DTSStepScriptResult_DontExecuteTask ELSE Main = DTSStepScriptResult_ExecuteTask END IF End Function
Follow these steps to add an ActiveX scripting task to your project: 1. Click the ActiveX scripting task icon in the Task section of the DTS Designer toolbar. 2. Enter your script in the ActiveX Script Task Properties dialog box. 3. After you have added the task to your project, you can view or modify the script by double-clicking on the ActiveX script task.
Try It Out!
Here are the steps to enhance the Titles to Northwind package by adding an ActiveX scripting task to write to an external log file. You will also add a precedent contraint so that the ActiveX task only executes after the transformation is completed successfully: 1. Open the saved Titles to Northwind package in the DTS Designer. 2. Click the ActiveX Script Task icon in the Task section of the toolbar to add an ActiveX script task. This loads the ActiveX Script Task dialog box. 3. Enter Log info to file in the Description text box for this task.
18-33
18-34
6. Click on the Northwind connection object, hold down the SHIFT key, and click on the ActiveX script task. Both the connection object and the task should be highlighted.
18-35
18-36
Summary
DTS can be used to set up simple or complex data transformations between any two OLE DB, ODBC, or text file objects. The DTS import and export wizards are a good way to perform single step transformations. Using the wizards is also a good way to start building your package if you are not comfortable with the DTS Designer. DTS packages can be saved and scheduled to run at specific time intervals. DTS packages are usually made up of a combination of one or more of the following three elements: connection objects, transformation tasks, and other built in DTS tasks. Adding precedent constraints to your package allows you to enter conditional logic into your package and enforce the sequential execution of tasks. Implementing transactions in your DTS packages allows you to group transformation tasks together so that they can either be committed or rolled back as a group. You can generate package logs and exception files to help track any errors that may occur in your package. Writing custom ActiveX scripts allows you to validate, transform, and scrub your data along with adding custom code for additional processing. There are three locations or types of ActiveX scripts: transformation, workflow, and ActiveX scripting tasks. Global variables allow you to pass data between your ActiveX scripts.
18-37
18-38
Questions
1. True/False: SQL Server must be either your data source or destination when using DTS to migrate data. 2. What is the function of a connection object? 3. What is the function of a transformation task? 4. True/False: Using a precedent constraint allows you to specify that a certain task should only be executed if another task fails to execute properly. 5. What are the three types or locations of custom ActiveX scripts in a DTS package? 6. Name one of the important things that can be accomplished with transformation script. 7. True/False: All Global variables lose their values once a package has finished executing.
18-39
Answers
1. True/False: SQL Server must be either your data source or
4. True/False: Using a precedent constraint allows you to specify that a certain task should only be executed if another task fails to execute properly.
True. You can use a precedent constraint to specify that one task should only be executed after another task completes either successfully or unsuccessfully, completes successfully, or fails to complete successfully. 5. What are the three types or locations of custom ActiveX scripts in
a DTS package?
Transformation, workflow and Active X scripting task
6. Name one of the important things that can be accomplished with transformation script.
Some of the things that transformation scripts can be used for are: validate data before inserting it into your destination, change the format of your source data, combine several fields from your source into one field for your destination and executing lookup queries.
7. True/False: All Global variables lose their values once a package has finished executing.
False. Global variables declared at design time can retain their values between package executions.
18-40
18-41
Lab 18 Overview
In this lab youll learn how to create a DTS package using the DTS Designer that will export data out of the authors table in the pubs database to a text file. Once the export has completed successfully, you will use an ActiveX scripting task to write a header line to the newly created text file. To complete this lab, youll need to work through two exercises: Setting Up a Transformation Task Adding an ActiveX Scripting Task
Each exercise includes an Objective section that describes the purpose of the exercise. You are encouraged to try to complete the exercise from the information given in the Objective section. If you require more information to complete the exercise, the Objective section is followed by detailed step-bystep instructions.
18-42
Things to Consider
How do you launch the DTS Designer? How do you create a new package? How do you add connections and tasks?
Step-by-Step Instructions
1. Expand the Data Transformation Services node in the Enterprise Manager. Right-click on Local Packages and select New Package from the menu to load the DTS Designer. 2. Click the Microsoft OLE DB Provider for SQL Server toolbar button in the Data section of the toolbar. This adds a connection object for the data source and loads the Connection Properties dialog box. Choose the following options: New Connection: Data Source: Server: Authentication: Database: Pubs Microsoft OLE DB Provider for SQL Server (local) or your server name Either SQL Server or NT pubs
Click OK when finished. 3. Click the Text File (Destination) button in the Data section of the toolbar. This adds a connection object for the destination and loads the Connection Properties dialog box. Choose the following options: New Connection: Author List Data Source: Text File (Destination) File Name: c:\AuthorList.txt Microsoft SQL Server 2000 Professional Skills Development
Copyright by Application Developers Training Company and AppDev Products Company, LLC All rights reserved. Reproduction is strictly prohibited.
18-43
Figure 15. Add a transformation task between Pubs and Author List.
7. Right-click on the arrow and select Properties from the menu. On the Source tab enter Migrate Authors to Text File as the description for this task. 8. Select [pubs].[dbo].[authors] from the Table/View drop-down list box and click the Destination tab. 9. Accept the default definitions and click Execute.
18-44
12. Click the Execute button or choose Package|Execute from the menu to execute the package. The AuthorList.txt file will show up on your c:\ drive.
18-45
Things to Consider
How do you add an ActiveX script task? How do you retrieve the current date? How do you add a precedent constraint?
Step-by-Step Instructions
1. Open the package in the DTS Designer if it is not already open. Click the ActiveX Script Task button in the Task section of the toolbar to add an ActiveX script task. This loads the ActiveX Script Task Properties dialog box. 2. Enter Add footer file in the Description textbox.
18-46
18-47
18-48