Updating IBM Cognos 8 Models Using Code and XML
Updating IBM Cognos 8 Models Using Code and XML
Copyright
Licensed Materials - Property of IBM IBM Cognos Products: DOCS (C) Copyright IBM Corp. 2010 US Govenment Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Contents
1 INTRODUCTION ................................................................................... 4 2 ASSUMPTIONS ..................................................................................... 4 3 OVERVIEW .......................................................................................... 5 4 USING FRAMEWORK MANAGER TO REPLAY XML ACTION LOGS .................... 6 5 USING BMTSCRIPTPLAYER TO REPLAY XML ACTION LOGS .......................... 7 6 CHANGING MODELS USING JAVA AND XML ACTION LOGS ........................... 8 6.1 RUNNING BMTSCRIPTPLAYER FROM AN EXTERNAL APPLICATION.............................................................8 6.2 USING IBM COGNOS 8 APIS TO APPLY CHANGES TO A MODEL USING XML............................................9 7 JAVA APPLICATIONS USING THE JAVA FRAMEWORK MANAGER API ........ . . . . 11 8 PUTTING IT ALL TOGETHER ................................................................. 12 8.1 OPTION 1 XML 8.2 OPTION 2 XML
IN CODE
..............................................................................................12
9 CRITERIA FOR SELECTING A TECHNIQUE ............................................... 14 APPENDIX A BMTSCRIPTPLAYER EXAMPLE ............................................ 16 APPENDIX B MDPROVIDERHELPER EXAMPLE ......................................... 18 APPENDIX C RETRIEVE A MODEL ALREADY PUBLISHED IN IBM COGNOS 8 22 APPENDIX D HOW TO PUBLISH A MODEL. ............................................ 22 APPENDIX E THE FRAMEWORK MANAGER API UPDATEMETADATA METHOD . 22 APPENDIX G DATASOURCE CONNECTION RELATED EXAMPLES .................. 23 A. UPDATE A CONNECTION STRING..............................................................................................23 B. CREATE OR UPDATE A DATASOURCE AND CONNECTION. ...................................................................24 C. SETTING A CONNECTION STRING AT RUNTIME................................................................................24 APPENDIX H DOCUMENTATION ON ERROR CODES. ................................. 24
1 Introduction
This document describes various automation techniques that can be used to create and modify Framework Manager models. These automation scenarios would be common for situations that have dynamic models that change often or, as with OEM organizations, are different for each customer but are based on a common set of modeling operations. Another common situation occurs when a model would require many man hours of repetitive operations to create or modify and automation would free resources to address more specific tasks. Additionally, there may be scenarios where a model needs to be modified in a Unix or Linux environment where the full Framework Manager modeling tool is not available. The focus on this document is to demonstrate capability with examples. This document should be used in conjunction with in the Framework Manager Developer Guide. This document was based on testing with IBM Cognos 8.2, 8.3, and 8.3 SP1. As indicated in the Framework Manager Developer Guide, Framework Manager is a data modeling product. It lets users import metadata from one or more data sources and transform it into a business-oriented model for creating reports. The Framework Manager Developer Guide is for developers interested in using the collection of cross-platform Web services, libraries, and programming interfaces provided with the SDK API, to access the full functionality of Framework Manager. You can use the Framework Manager SDK to model metadata and publish packages without the use of the Framework Manager application. For more details on the Framework Manager API and the structure of the log files please refer to the Framework Manager Developer Guide provided with the Cognos documentation.
2 Assumptions
This document assumes that persons attempting this are familiar with Framework Manager, XML, writing Java code and the necessary infrastructure requirements such as starting and stopping the IBM Cognos 8 server. Administrator or developer privileges will be required as well as a working knowledge of how to test Framework Manager Models by creating reports in IBM Cognos 8 in order to test the results. This document was created and tested with IBM Cognos 8.2, 8.3, and 8.3 SP1. Persons using this document should also have the The Framework Manager Developer Guide as a reference. The code utilized in this document is not considered production ready. It is intended to a demonstration of various techniques available for updating and or creating Framework Manager models with IBM Cognos 8. Modification or support for these samples is the responsibility of the user. However, feedback on these samples is appreciated.
NOTE : An SDK licence for Cognos 8 is required if modifications are made to the Java source code.
3 Overview
IBM Cognos 8 provides customers the ability to create and or modify Framework Manager models in three different ways. 1. The Framework Manager modeling tool 2. BMTScriptPlayer 3. The Framework Manager API All three techniques utilize XML to implement changes to a model (which is itself an XML document). The Framework Manager modeling tool is a Windows-only thick client application that is used to create a Framework Manager model. This modeling tool generates and modifies XML files whenever any add, change or deletes are done to a Framework Manager model. As an example, the action log files for the sample model of gosales_goretailers are located at1:
<IBM Cognos 8 location>\webcontent\samples\Models\gosales_goretailers\logs
Any time a change is made to a model you should see a file created that contains the package name along with a date time stamp such as
gosales_goretailers-20070828122354-log.xml
For IBM Cognos 8.3 the log files are located in the following directory:
<Cognos 8 location>\webcontent\samples\Models\gosales_goretailers
Starting with 8.3 there is only one log maintained and the name of the file is log.xml Note: Some functions such as creating a datasource or a datasource connection can only performed using the Framework Manager API rather then with XML scripts. Classes such as: DataSource, DataSourceConnection and DataSourceSignon are used to implement this capability. See the example in Appendix G for more details.
The samples for 8.3 are provided as a separate install option and must be made available by the administrator.
Once you select a file to be run you should see the following screen. This screen has a run button to the right side of the screen. After running the script you should see a number of messages indicating changes that were made or errors that were encountered when replaying the actions2. You can then accept or reject the changes.
Errors may occur for several reasons. For example an action may fail if the referenced objects are no longer accessible or security restrictions prevent access to specific objects.
where <projectname> is the name of the project and <actionlogname> is the name of the action log. For example,
BmtScriptPlayer -m goSales.cpf -a import.xml
The Framework Manager Developer Guide contains additional details on how this utility can be used to implement change to models using xml action logs. The full command syntax for BmtScriptPlayer.exe can be obtained by supplying an invalid argument such as -? when executing BmtScriptPlayer from a command line.
The BmtScriptPlayer utility was written to process one file at a time. If a group of files need to be applied to a model it is important that they be processed in order because actions in the log files may depend on the results of actions in earlier logs. When log files are created by Framework Manager they are time stamped and this will allow for easy determination of the appropriate script order. Action logs created by external applications may not contain such information in the file names and care would need to be taken in such situations to process the scripts appropriately. See Appendix A for an example of using BmtScriptPlayer to process multiple files.
The Framework Manager SDK supports two methods of modeling metadata in addition to the Framework Manager application: BmtScriptPlayer and the Metadata Service. Each of these methods allows you to perform metadata modeling and create a Framework Manager package that you can then publish for report authors. At the core of each of these methods is the action. An action is a request that is sent to the Framework Manager Engine. Actions can be grouped together to perform certain modeling activities. Actions are recorded as elements of an XML document. These actions are the same elements that are written to the action logs created by Framework Manager. For more information, see "Action Logs" in the Framework Manager Developer Guide. When using a programming language to implement these actions there are two main methods that can be used: calling BmtScriptPlayer from the external application or using the Framework Manager APIs directly. 6.1 Running BmtScriptPlayer from an external application. The first technique for an external application to implement changes to a model using XML is to invoke the BmtScriptPlayer from an external application such as a Java program. This can be done using the Java class Runtime.getRuntime().exec, Here is an example:
String locationofBMT = pfile.getProperty("BMTScriptcmdString");
See Appendix A for a sample program. 6.2 Using IBM Cognos 8 APIs to apply changes to a model using XML The second technique for an external application to implement changes to a model using XML is to invoke the IBM Cognos 8 APIs from an external application. The Framework Manager API will allow the external application to act as a client to an IBM Cognos 8 server, communicating through the BI Bus API. The client interacts with the server via Simple Object Access Protocol (SOAP) requests using the HTTP transport protocol. The Metadata Service running on the IBM Cognos 8 server receives and processes the SOAP requests from the client application and sends them to the Framework Manager Engine. Responses from the Framework Manager Engine are fed back to the Metadata Service which encapsulates the response as a SOAP message and returned to the client application through the BI Bus. The Metadata Service can query existing models for information or apply actions to modify a model. The responses for action results and queries will be returned as an XML document. There are two types of requests that can be made to the Metadata Service to manipulate an unpublished model: Send generic requests to create, open, save, and close the model. Send action requests to modify the metadata or publish a package.
Both types of request use the Framework Manager API Request element. There are a number of different classes to support the creation of a model using Java code. The most common Java class is the MetadataServiceStub used along with the updateMetadata method. When using these classes and methods, actions are still defined in the same XML format but must be wrapped within an mdprovider XML element. Please refer to Chapter 1 of the Framework Manager Developer Guide for details on the mdprovider element and examples of its use. Here is an example of a method that implements a generic request to open a project:
private String openProject(String projectLocation) throws Exception { String sXML = "<mdprovider type=\"generic\" action=\"openModel\" model=\""+ projectLocation + "\"/>"; XmlEncodedXML encodedXML = new XmlEncodedXML(sXML);
10
return getMetadataService().updateMetadata(encodedXML.toString()); }
11
Here is an example of a Java class that makes a modification to the namespace. Notice a pair of String variables were created to make this class more generic then the original XML which hard-coded the actual name of the original namespace and the new name of the namespace in the XML:
public class ModifyNamespaceName extends Action { private String oldName;
12
The idea is to take some very specific XML that was generated by Framework Manager and then make it more generic so you can then pass program parameters to the methods and implement your model in this manner. Another best practice is to build a framework that can allow for code reuse and maintainability. A framework that encourages the use of patterns, good object oriented design practices and implementation of utility classes is desirable so that time to develop and maintain application can be optimized. See Section 8 for an example. Another idea for a potential framework is to use the concept of templates. This approach takes the raw XML action log generated by Framework Manager and manipulates the XML content using java classes such as JDOM to rewrite components of the action log. For example, you could create an action log that imports a table, renames it and then moves it to another namespace within the model. With this template you can manipulate the XML to import a different table, rename the resulting query subject with another name and then move it to yet another final namespace. By using a template approach you can eliminate the need to duplicate actions for each permutation and simply make the appropriate substitutions in the action log before processing the actions in the file. See Section 9 for an example.
13
There are a number of Java classes that are used to implement many different functions. Every customer has their own unique requirements so this Java framework may be too much for some and not enough for others. The requirements in your environment will dictate the best approach for your applications and this example is meant as a sample rather than a hard rule. However, this example does use some common elements such as object oriented practices including inheritance and some of the new Java 1.5 collections that are useful in creating reusable and flexible code. Someone using this framework should be familiar with these concepts. Here is a brief review of the main method in the SampleApplication class:
String workingNamespace = new String("Model"); String modelPath = new String("C://NewProject1//bcdtest.cpf"); CognosConnect cognosConnect = new CognosConnect("localhost"); cognosConnect.connect(); Project project = new Project(cognosConnect, modelPath); project.create(); project.open(); project.setLocale(); project.setName("TEST"); project.setCoreNamespace(workingNamespace, "Analysis"); workingNamespace = "Analysis"; Vector<String> tables = new Vector<String>(); tables.add("titleauthor"); tables.add("titles"); tables.add("authors"); project.importTables(workingNamespace, "Analysis", "pubs", "dbo", DBType.OL, tables); project.createRelationship(workingNamespace, workingNamespace, "titleauthor", "title_id", RelationshipCardinality.ONE2ONE, workingNamespace, "titles", "title_id", RelationshipCardinality.ONE2MANY); project.createNamespace(workingNamespace, project.createNamespace(workingNamespace, project.createNamespace(workingNamespace, project.createNamespace(workingNamespace, "Import View"); "Dimensional View"); "Business View"); "Development View");
14
The above code will create a model (if it does not exist3), import from a data source that points to the MS SQL Server pubs database4 and import the titles, titleauthor, and authors tables. It then creates a relationship between the titles and titleauthor tables based on the respective title_id columns. The code will do this in the Analysis namespace. It will also create additional namespaces: Import View, Dimensional View, Business View, and Development View. 8.2 Option 2 XML in external files as templates. FileName: fmcodeintemplates.zip This example is the beginning of a Java framework that can be used to implement the building of a metadata model using external XML files as templates rather then embedding XML in code. This example utilizes the JDOM classes (www.jdom.org) to parse the XML5. Listed below are some examples of how this can be used: createNameSpaceTest: - This method will create a new XML file that will create a namespace. This example utilizes an XMl template file called dbimport_template.xml. dbImportTest : - This method will create a new XML file that will demonstrate how to perform a database import. This example utilizes an XML template file called importnewtable-log.xml. deletefolderTest : - This method will create a new XML file that will demonstrate how delete a folder. This example utilizes an XML template file called delete_template.xml. Notice that in this example there is no XML embedded in code. The idea with this version is to isolate the XML in files to be parsed with code and then update the relevant entry to perform a new similar action on another object. The resulting XML files can be executed as scripts in Framework Manager, through BMTScriptPlayer or through the Framework Manager API.
If the class is modified to point to an existing model then be aware that several of the methods are defined to use specific namespace names and these may need to be modified to locate the appropriate elements within the existing model. 4 This sample database is available from Microsoft. To use the sample code a data source connection named Analysis must be configured within the Cognos 8 environment to access the pubs sample database. 5 SAX, DOM, or any other XML processing package could be used to perform the XML document manipulation. JDOM just happens to be the one chosen for this example.
15
Typically, BmtScriptPlayer is used to reproduce a set of changes made in one environment to the same base model in another environment. The XML action logs produced by Framework Manager in a development environment can be used as-is to quickly update a model in a training or production environment. The Framework Manager API is normally used for situations when there are many new objects or modifications that need to be implemented in a model. In such a scenario, automation would be useful in freeing modelers from having to perform repetitive actions and also reduce the time required to implement a package. Another scenario where the Framework Manager APIs are useful is for OEMs or consulting partners. Here a single base model may exist and automation would be used to create customized models for the individual OEM customer solutions. So, if the project scope calls for simple playback of recorded actions with minor variations then BmtScriptPlayer is ideal for executing a series of actions from a file. On the other hand, if the project calls for more detailed automation where many objects are manipulated in complex ways then the Framework Manager API is likely the better choice as it will allow for smaller code blocks to be reused in these multiple and varied actions.
16
Updating IBM Cognos 8 Models Using Code and XML The sample model contains a logs directory with 3 log files. These are the action log files that contain the necessary XML to add the two tables to the model. Using run.bat for the first time will add these two tables to this view. If you want to delete the two tables then delete the exiting files out of the log directory and then copy the file deletelog.xml from the main model directory to the log directory and execute the run.bat again. (Note the backup directory contains the original log files).
17
18
The reportNetEndPoint identifies dispatcher target for the IBM Cognos 8 server. It should be the same value as the External Dispatcher URI setting that was specified in Cognos Configuration. The username, password, and nameSpaceID properties are the logon information for the IBM Cognos 8 server. The value for the namespaceID must be the value of the NamespaceID setting that was specified in Cognos Configuration. The value is not necessarily the same as the display name of the namespace. If no authentication provider has been configured for the IBM Cognos 8 environment then set authenticate=no in the properties file. If the environment is secured and a logon is required then set authenticate=yes in the properties file. The projectLocation is the location of the Framework Manager model and the scriptLocation is the location of the action log that is to be applied to the model.
19
Finally, updateonly is a property that allows you to identify whether the action log is to be applied to an existing model or that the model should be created before applying the actions in the log file. By setting this property to yes the script will apply the actions to an existing model. Using no will cause the script to create the model before applying the action log. Sample scenario 1 Model from Machine 1 needs to be created on Machine 2 and Machine 2 does not have Framework manger installed on it. The steps to do so using the MDHelperAPIDemo would be: 1. Prepare the target environment a. Create a directory for the new model on Machine 2. b. Copy the action log from Machine 1 to Machine 2. c. Modify the MDHelperAPIDemo.properties file to define the connection information, location for the new project file (from step a) and the location of the action log (from step b) In this case, since a new model is to be created, set updateonly=no in the properties file. 2. Call MDHelperAPIDemo by executing run.bat Expected results The model from Machine 1 is now on Machine 2. Note: It is still necessary to publish the model for report authors to use the updated information. See Appendix D on how to publish the model using automation. Sample scenario 2 Model changes from Machine 1 needs to be applied to an existing model on Machine 2 and Machine 2 does not have Framework manger installed on it. The steps to perform this operation with MDHelperAPIDemo would be: 1. Prepare the target environment a. Copy the action log for the changes from Machine 1 to Machine 2. b. Modify the MDHelperAPIDemo.properties file to define the connection information, location of the existing project file on Machine 2,and the location of the action log (from step a). In this case, since the model already exists, set updateonly=yes in the properties file. 2. Call MDHelperAPIDemo by executing run.bat Expected results
Updating IBM Cognos 8 Models Using Code and XML All the changes from the development Machine 1 are now applied to the model on Machine 2.
20
Note: It is still necessary to publish the model for report authors to use. See Appendix D on how to publish model using automation. Notes on example The tables and data source connection used in the samples were created from the IBM Cognos 8.1 sample SQL Server database. Subsequent releases of IBM Cognos 8 have updated the sample databases and the tables and schemas may be different than those available with IBM Cognos 8.1. To make this example work in your environment the tables from the 8.1 sample database have been included with this sample application in the script GOSL.sql. To use this content with this sample application the ddl must be executed to create the tables in a SQL Server database named gosl. The associated IBM Cognos 8 data source must be named gosales. This example includes sample models located at the following locations: \yourziplocation\MDHelperAPIDemo\sourcemodel \yourziplocation\MDHelperAPIDemo\targetmodel
An empty directory structure is also present to satisfy the create option available with this application at the following location: \yourziplocation\MDHelperAPIDemo\targetnewmodel
When applying changes to the sample target model the target model must be upgraded to the current Framework Manager release that you are using. To ensure that the model is at the correct release, open the model, respond to any upgrade messages and then save the upgraded model. Before using the sample, use Framework Manager to open the provided models. Notice that the target model does not have any tables imported. There are two configuration files provided with this sample. MDProviderHelper.properties.create.txt MDProviderHelper.properties.update.txt
Use these files for the different scenarios. The Java code looks for the MDProviderHelper.properties file. To test the update scenario then copy MDProviderHelper.properties.update.txt to MDProviderHelper.properties. Similarly, if testing the create scenario then copy the MDProviderHelper.properties.create.txt to MDProviderHelper.properties. Modify the properties files to identify the source and target locations of the extracted models and to identify the appropriate logon information.
21
After executing run.bat you should see the new tables imported into the model (within targetmodel or targetnewmodel depending on the properties file that you use).
22
java.rmi.RemoteException
As indicated above, the method takes in a XmlEncodedXML class. Here are some examples of how it can be used:
23
Sample XML within the updateMetadata method for opening an existing project:
private String openProject(String projectLocation) throws Exception { return getMetadataService() .updateMetadata(new XmlEncodedXML("<mdprovider type=\"generic\" action=\"openModel\" model=\""+ projectLocation + "\"/>")).toString(); }
Sample XML within the updateMetadata method for saving an existing project:
private String saveProject(String projectLocation) throws Exception { return getMetadataService() .updateMetadata(new XmlEncodedXML("<mdprovider type=\"generic\" action=\"saveModel\" model=\""+ projectLocation + "\"/>")).toString(); }
For an example of this method and for more details please refer to MDProviderHelper in Appendix B.
Updating IBM Cognos 8 Models Using Code and XML Expected results The connection string is updated to reflect the parameters defined when calling the Java application.
24
Note that for migration of packages from one environment to another it would be more appropriate to create the deployment export package without including data source definitions. After importing the package into a new environment the content will use the same data source name reporting but will use the connection information that has been defined in the new environment. Different connection definitions can be present in the different environments as long as the data source names are the same. B. Create or Update a Datasource and Connection. This example can be used to either create a new datasource or update the properties of an existing data source and connection.
Sample scenario A custom application needs to create a new datasource and datasource connection. This would be most useful when creating many new data sources or to automate application development for serviced environments. Expected results This example will create a datasource and a datasource connection in IBM Cognos 8. C. Setting a connection string at runtime This example will demonstrate how to set a connection used by a report at runtime.
Sample scenario A custom application needs to run a report with a specific connection string. Expected results: This will example will run the sample report Add Color with a specific connection string.