Department of Veterans Affairs Open Source Electronic Health Record (EHR) Services Web Application Automated Testing Framework

(WAATF) Software Design Document (SDD)

Version 1.0 September 2013

Contract: VA118-12-C-0056

Revision History

Date 09/23/2013 09/24/2013 09/24/2013 09/24/2013 09/25/2013 09/25/2013

Version 0.1 0.2 0.3 0.4 0.5 1.0

Description Initial Draft Peer Review Updates per Peer Review Formal Review Updates per Formal Review Final Review

Author Andal FeQuiere Meredith Watkins Andal FeQuiere Paul Bradley Andal FeQuiere Paul Bradley

Open Source EHR Services WAATF SDD

September 2013

Table of Contents
1. Introduction...................................................................................... 3
1.1. 1.2. 1.3. Purpose ......................................................................................................... 3 Scope and Audience..................................................................................... 3 Acronyms and Definitions ........................................................................... 4

2. 3. 4.

Tools ................................................................................................. 5 Tool Application............................................................................... 6 Framework Architecture.................................................................. 6

4.1. 4.2. 4.3. 4.4. High level Architecture ................................................................................. 6 AutomatedTestingSuite Object.................................................................... 7 Selenium Tests.............................................................................................. 7 Logging System .......................................................................................... 10 Clone Automated Functional Testing Source Code ................................ 13 Test Structure and Setup ........................................................................... 13 Testing Execution ....................................................................................... 15

5.

Installation and Setup.................................................................... 13

5.1. 5.2. 5.3.

Open Source EHR Services WAATF SDD

September 2013

1.

Introduction

The Department of Veterans Affairs (VA) has contributed the latest U.S. Department of State Freedom of Information Act (FOIA) release of the Veterans Health Information Systems and Technology Architecture (VistA) codebase to Open Source Electronic Health Record Agent (OSEHRA), the custodial agent that serves as the central governing body of a new open source community. The VA has begun to make VistA more modular and easier for developers to work with, and has also begun the deployment of the open source version of VistA inside the VA. In order to accelerate this work, the VA is in need of a comprehensive testing framework to verify refactored code, enhancements, and bug fixes. The Web Application Automated Testing Framework (WAATF) was developed, in response to a need, to provide automated testing support for web applications. The WAATF uses cutting edge build tools and architecture to provide ease in use and development of test scripts. The WAATF also provides scripts to perform functional testing of any web application on remote driven web browsers.

1.1. Purpose
The purpose of this document is to define the tools, concepts, assumptions, and structure used within the automated testing framework for web application automated testing. The approach described below is a hybrid framework using both data-driven and modularized code techniques. This framework is not restricted to only the code being produced by this specific project; eventually these concepts will be integrated into the automated testing framework currently under development by OSEHRA, incorporating the testing for all code received by OSEHRA. This plan aims at outlining an automated testing infrastructure that supports the creation, management, and execution of a rich set of functional tests constructed for a variety of purposes including functional verification, regression testing, and certification of the abovementioned code.

1.2. Scope and Audience

The following text describes the automated testing framework used for functional verification of the MyHealtheVet (MHV) application. The WAATF can be used to test any web application. MHV is provided as a reference. The framework described here can be used to test functionally of any feature within the MHV application. The test framework is the set of tools, concepts, assumptions, software, and structure that enables automated software testing. This framework has the overarching objective of simplifying test creation, configuration, and modification.

Open Source EHR Services WAATF SDD

September 2013

1.3. Acronyms and Definitions

Acronym AUT EE EHR FOIA IDE JDK ME MHV MVC OSEHRA SDD SDK SE VA VistA WAATF Term Automated Unit Test Enterprise Edition Electronic Health Record Freedom of Information Act Integrated Development Environment Java Development Kit Micro Edition My HealtheVet Model View Controller Open Source Electronic Health Record Agent Software Design Document Software Development Kit Standard Edition Department of Veterans Affairs Veterans Health Information Systems and Technology Architecture Web Application Automated Testing Framework

Open Source EHR Services WAATF SDD

September 2013

2.
Name CMake

Tools
Version 2.8.7 Description CMake is a family of tools designed to build, test, and package software. CMake is used to control the software compilation process using simple platform and compiler independent configuration files. CMake generates native makefiles and workspaces that can be used in the Compiler environment of your choice. The JDK is an implementation of either one of the Java Standard Edition (SE), Java Enterprise Edition (EE), or Java Micro Edition (ME) platforms released by Oracle Corporation in the form of a binary product aimed at Java developers on Solaris, Linux, Mac OS X or Windows. Since the introduction of Java platform, it has been by far the most widely used Software Development Kit (SDK). Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. It's comprised of two parts that include: Git: A version control tool Bash: A shell that runs commands once you type the name of a command and press Enter on your keyboard http://www.eclipse.org/downloads/pack ages/eclipse-ide-java-eedevelopers/indigosr2 Information http://www.cmake.org/ http://www.cmake.org/cmake/resource s/software.html

Java Development Kit (JDK)

http://www.oracle.com/technetwork/jav a/index.html

http://www.oracle.com/technetwork/jav a/javase/downloads/index.html

GitBash

1.8.0

http://git-scm.com/downloads

Eclipse (optional Integrated Development Environment [IDE])

Indigo

Eclipse is a community for individuals and organizations who wish to collaborate on commercially-friendly open source software. Its projects are focused on building an open development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software 5

Open Source EHR Services WAATF SDD

September 2013

across the lifecycle. The Eclipse Foundation is a not-for-profit, member supported corporation that hosts the Eclipse projects and helps cultivate both an open source community and an ecosystem of complementary products and services. EGit Git Eclipse Plug-in Git Plug-in http://www.eclipse.org/egit/

3.

Tool Application
CMake Used to build the automated test script Java Create and test scripting language GitBash Bash shell used for test execution and Git operations Eclipse Optional development IDE that includes Git plug-in (EGit) Selenium Used to record and replay test scripts in a web browser

4.

Framework Architecture 4.1. High level Architecture

The WAATF project Architecture is composed of four main packages: Controller AllTestSuites.java used for launching all tests Utils Contains files to assist with logging and other utility functions Templates Templates which represent a subset of data in the properties file; modeled for a specific functionality Suites Consists of a library of individual test case scripts to be executed

Figure 1 below displays a high level diagram of how the Controller, Templates, and Suite objects are related. The architecture for this framework greatly relates to that of Model View Controller (MVC), which is a software architecture pattern which separates the representation of information from the user's interaction. The Controller object is AllTestSuite class, which sends commands to its associated view and controls the flow of the application. The Controller object launches each individual Suite object. The View consists of each Suite objects requests for information from the model that it needs for generating an output representation to the user. Each View Suite object correlates to a specific test case and functionality to be executed. Although the tests themselves are unique, there are similar functions such as login, logout, and future functions that are shared amongst all test cases. To allow for code reusability, each major action throughout the test framework has been modularized such that it can be referenced in any test. The Model consists of Template objects and properties files that notify its associated views and controllers when there has been a change in its state. The Template object is used to
Open Source EHR Services WAATF SDD

September 2013

model data from the corresponding properties file and represent a major function for the particular test case. Currently there exists only two templates: Login and Profile. Each Suite Object also has a properties file which is used to provide data to the templates needed for that particular test. The naming convention of the properties file is the name of the Suite Object followed with a .properties extension. The properties file in conjunction with the templates provides a layer of abstraction such that the tester can make changes to the data passed to the application without having to modify any code. Thus, allowing the test framework to operate as data driven while also implementing MVC architecture.

Figure 1: Architecture Diagram

4.2. AutomatedTestingSuite Object

The AutomatedTestingSuite class is an abstract class which can be extended by any test suite class. This class was created as a means to provide an additional layer of abstraction for building tests and also to be used as an object which holds reusable functions. The AutomatedTestingSuite class manages logging, Selenium web drivers, test properties file management, and test verification through verification statements. Utilizing the AutomatedTestingSuite class allows extended test classes to simply test case creation as well as reduced the complexity and number of lines of code generated.

4.3. Selenium Tests

The example automated tests, that are included in the framework, are for the MHV application and uses Selenium to emulate user interaction. Selenium is an application tool, primarily used for automating web applications for testing purposes. To display Selenium functionality, two tests were developed, LoginLogout and Profile tests. Displayed below is an image of the file system structure with the tests included in the org.automatedtesting.suites package:

Open Source EHR Services WAATF SDD

September 2013

Figure 2: Test Framework packages

The LoginLogout test, performs logging in to the MHV application, verifies that the user is logged in and then logs the user out. Using Selenium the system can emulate a user typing their username and password credentials and logging in the application followed by logging out. Displayed below is an example of the test code:

Open Source EHR Services WAATF SDD

September 2013

Figure 3: LoginLogout Test Script

The Profile test, performs logging in to the MHV application, verifies that the user is logged in, goes to My Profile, updates the following fields: street, city, state, and zip code, saves changes and then logs the user out. Using Selenium the system can emulate a user logging in and updating their Profile information as well as other actions. Displayed below is an example of the test code:
Open Source EHR Services WAATF SDD 9 September 2013

Figure 4: Profile Test Script

4.4. Logging System

The results of each test will be maintained in a result file which will display Pass/Fail indications for each test. If a test failure occurs, a brief error message may be included in the result file. In addition to the result file, one or more log files will be created for each test to record the
Open Source EHR Services WAATF SDD 10 September 2013

interaction between the test harness and the automated unit test (AUT). Detailed error messages will be located in the log files. The Logging system included the usage of SLF4J and standard Java logging utilities. Currently Selenium utilizes standard Java logging utilities, but with technology advancements come logging advancements that currently include SLF4J. SLF4J is a Java library which serves as an abstraction for various logging frameworks (e.g. java.util.logging, logback, log4j) allowing the end user to plug in the desired logging framework at deployment time without any code redesign. The following Java jar files were added to the project to allow the application to view logs that Selenium generated: SLF4J-APIi-1.7.5.jar JUL-to-SLF4J-1.7.5.jar Logback-core-1.0.7.jar Logback-classic-1.0.7.jar

Logback is an implementation of SLF4J, which was used to assist in generation of individual log files for each test suite. The configuration was implemented in a LogbackFileUtils.java file and as well as the AllTestSuites object to bridge the gap in Java standard logging and SLF4J. The following code displayed below made these enhancements possible:

Figure 5: JUL to SLF4J and Logback

Open Source EHR Services WAATF SDD

11

September 2013

Figure 6: LogbackFileUtils Class Part 1

Open Source EHR Services WAATF SDD

12

September 2013

Figure 7: LogbackFileUtils Class Part 2

5.

Installation and Setup 5.1. Clone Automated Functional Testing Source Code
1. Setup Git https://help.github.com/articles/set-up-git 2. Get ATF Source Code a. Source Code Option Git clone [email protected]:afequ871/Web-Automation-Testing-Framework.git

5.2. Test Structure and Setup

The test harness utilizes a test structure that organizes the build, configuration, and execution methods. This structure also defines the file organization. As shown in Figure 8, the CMAKE and CTEST scripts are used to control the build, configuration, and execution of tests. The first step in the testing process is to build the platform specific test scripts. This is accomplished with the CMAKE-GUI (see Figure 9), a configuration and generation tool. Currently this configuration and generation step is manual but will be automated in the future. After producing the platform specific test scripts, CMAKE is used to
Open Source EHR Services WAATF SDD

13

September 2013

create and configure the AUT and then populate it with test data. After the AUT is populated with test data, each test is invoked using CTEST. Test results are then captured in log files.

Figure 8: Build and Test Summary

Figure 9: CMAKE-GUI Screenshot

Open Source EHR Services WAATF SDD

14

September 2013

5.3. Testing Execution

To run the automation tests perform the following steps: a) Run CMake-GUI to configure and generate CMake files making a /binC or /build folder b) From the /binC (/build) folder do the following: a. CMake -P BuildProject.cmake b. CTest -R WEB_ (Test) Please note the following: The CMake -P BuildProject.CMake must be run before every CTest -R WEB due to the fact that after every code change the application needs to generate the appropriate binary class files. Test result logs are located in the /binC/Testing/execution/logs New test suites can be added by placing the "AllTestSuites main" .java file in the /src/org/automatedtesting/controller directory and related test classes should be placed in /src/org/automatedtesting/suites directory.

Open Source EHR Services WAATF SDD

15

September 2013