Improving Enterprise System Implementation (ESI) Project Outcomes: An ESI Project Assessment and Framework for Incorporating Best

Practices

PM 685 Thesis Project Spring 2009

Master of Science In Project Management

Weishaar First Draft; submitted 12/5/08

ABSTRACT
Enterprise systems (ESs) are software packages that offer seamless integration of all the information flowing through a company. Examples of recent AAG ESs are TOTEM, Single-Source, AIMS, SOLAR, and TRAX. Implementing an ES (ES Implementation or ESI), is a complex and dynamic process, often characterized by occurrences of unplanned problems and events. As with ESI efforts in many organizations, the outcomes of a number of AAG ESI projects have fallen short of expectations. Outcomes of recent AAG ESI projects are assessed using qualitative and quantitative data collection methodologies. Factors hindering ESI project success are identified based on literature review, analysis of interview and survey findings. Best practices for improving ESI outcomes are identified based on literature review. Recommendations, including a framework for integrating best practices into current ESI project processes, are presented, based on correlations between interviews, survey findings and ESI best practices.

TABLE OF CONTENTS LIST OF EXHIBITS..6 1.0 Introduction....8 1.1 Background ..9 1.2 Objective & Approach.......13 1.2.1 Research Questions....14 1.2.2 Approach..14 1.2.3 Method.....15 1.3 Report Structure16 2.0 Literature Review.19

Weishaar First Draft; submitted 12/5/08

Enterprise Systems (ES)20 ES Implementation (ESI).27 Organizational Context30 Project Management (PM) Context38 3.0 Problem Analysis..41 3.1 Research Design41 3.1.1 Study Population.42 3.1.2 Sampling Methods..42 3.2 Instrument Selection and Design.43 4.0 Data Collection..45 4.1 Step One: Key Leader Interviews45 4.2 Step Two: AAG ESI Project Survey.47 4.3 Step Three: Project Sponsor Interviews50 5.0 Data Analysis51 5.1 Key Leaders Interview Data.51 5.1.1 Company Affiliation..51 5.1.2 Company Tenure.52 5.1.3 AAG ESI Project Selection/Perceived Outcomes53 5.1.4 Factors Hindering AAG ESI Project Success54 5.1.5 Value of Improving AAG ESI Project Success.55 5.2 AAG ESI Project Survey Data..56 5.2.1 Section 1: Self Identification.56 5.2.2 Section 2: Pre-planning/Project Outcomes..57 5.2.3 Section 3: Practice/Effectiveness of CSFs.72 5.2.4 Section 4: Value/Significance of CSFs..80 5.2.5 Section 5: Other Factors Hindering AAG ESI Project Success..83 5.3 Project Sponsor Interviews.85 5.3.1 Study Findings Feedback 5.3.2 Preliminary Recommendations Review 6.0 Conclusions/Recommendations 6.1 CSF Gaps/Recommended Best Practices 6.1.1 Clear Goals/Definition of Success 6.1.2 Top Management Support 6.1.3 User Involvement 6.1.4 Communication 6.2 Strategy for Implementation 6.2.1 Post project follow-up activities 1.1.1.1 CSF Framework 1.1.1.2 AAG ESI Project Orientation PowerPoint presentation 1.1.1.3 Risk Management Toolbox 6.3 Project Contribution 6.4 Lessons Learned 6.5 Final Words REFERENCES

2.1 2.2 2.3 2.4

Weishaar First Draft; submitted 12/5/08

1.0

INTRODUCTION

All work in an organization is performed through some kind of process. Support processes such as accounting, human resource management, maintenance, management information systems, and supply chain management, are driven by internal customer needs. They provide the infrastructure for value-creation processes,

Weishaar First Draft; submitted 12/5/08

which are driven by the external customer, and considered key to running the business and achieving and maintaining a sustainable competitive advantage. Acquiring customer and market knowledge, strategic planning, conducting research and development, developing new products or services, fulfilling customer orders, measuring and analyzing performance, and training employees, are just a few examples of value-creation processes (Evans & Lindsay, 2008).

To be successful in todays hot, flat, crowded, increasingly transparent, ubercompetitive, and rapidly changing world, an organizations strategy must include continuous improvement of business processes. Through an enterprise-wide commitment to continuous improvement, companies increase productivity and effectiveness in the use of resources; reduce waste, defects, errors, and costs; increase speed to market of products and services, and much more. Through the practice of continuous improvement, businesses enhance value to the customer, improve competitive advantage, and increase their bottom line.

In many companies, continuous improvement of support processes that facilitate value creation takes place in the form of projectstemporary work structures that start up, produce products or services, and then shut down (Marten and Tate, 1998). Projects that deliver enterprise-wide solutions to support and enhance organizational infrastructures are referred to as enterprise system implementation (ESI) projects.

This thesis focuses on the role of project management processes, tools, and techniques in executing a strategic plan that includes continuous improvement of

Weishaar First Draft; submitted 12/5/08

business processes in support of value creation through ESI projects. A case study of one organizations ESI project management practices and outcomes is presented.

1.1

Background

AAG, Inc. operates as a commercial airline with combined annual revenues of $3.69 billion. As all commercial airlines face the daunting challenge of recent dramatic and historic downturns in the economy, AAGs outlook remains relatively strong compared to its peers. Despite a quarterly net loss of $86.5 million in Sep 09, AAGs financial viability was ranked first among U.S. legacy airlines, as seen in Exhibit 1, an excerpt from an annual report published in September by Aviation Weekly. (See Appendix A.1 for study methodology). While almost all competitors suffered double digit declines in market capitalization, AAG stock has experienced less volatility, as seen in Exhibit 2, which reflects North America share price movements for November 12, 2008.

As overall airline performance continues a five-year slide (2008 Wichita State/Saint Louis University study based on information compiled by the Department of Transportation) and overall customer satisfaction fell to its lowest level in three years (J.D. Power and Associates 2008 North America Airline Satisfaction Study), Alaska Airlines ranked first in five of seven key measures, and received the coveted top rating in overall customer satisfaction, in a tie with Continental Airlines.

Weishaar First Draft; submitted 12/5/08

Exhibit 1 Aviation Week Top Performing Companies: 2008 Airline Rankings

Exhibit 2: North America daily share price movements (% change) Nov 10, 08

Weishaar First Draft; submitted 12/5/08

This accomplishment is made even more impressive by the fact that toward the end of 2007, AAGs on-time performance had dropped to among the worst, (number eighteen out of the twenty major airlines surveyed by the U.S. Department of Transportations Air Travel Consumer Report), with an on-time performance of 67.1 percent.

This remarkable turnaround can only be attributed to a 2008 strategic plan that entailed a laser-like focus on the improvement of operational reliability. Focusing so much of the 2008 strategic plan on operational reliability signals the airlines resolve to commit resources that will ensure sustained improvement, said Peggy Willingham, [then] director of strategic planning. In the past, promising efforts such a TANGO (Turn Aircraft N Go) faltered from insufficient accountability and long-term resources to support them (McElroy, P. Getting to the Gate On-Time, Alaskas World, February 1, 2008).

Recognizing the need for a more disciplined approach to process improvement, after a number of 2007 initiatives aimed at achieving the carriers on-time goal of 80 percent resulted in a disappointing average of 72.4 percent (about one point behind its performance for all of 2006), a single leader, was appointed to jumpstart the 2008 strategic plan. With a focus on data and process, the leader and his team identified ten key initiatives to improve the operation including: a single-source manual for aircraft technicians, streamlined A check, timetable of tasks during turns, better employee recruiting and retention efforts, and a revamped flight schedule.

Weishaar First Draft; submitted 12/5/08

In addition to ten projects devoted to the key initiative of operational reliability, eleven Running the Business Initiatives (RBIs) were targeted to support the airlines other strategic goals for safety, compliance, diversity and inclusion, easy flying, and profitability. The 21 projects, down from 38 in 2007, reflected a philosophy by senior leadership to tackle fewer, higher impact efforts that have sufficient resources to support them. The six-month process of selecting the initiatives included a critical assessment of project benefits, costs, and possible constraints. An Investment Council (IC), comprised of nine senior managers, was installed to help ensure coordination among the selected initiatives and available resources.

It is clear that Project Management maturity has progressed significantly within AAG, since the results of Peggy Willinghams UAA MSPM thesis on project management maturity (2006) were presented. The AAG PMO is a formalized organizational entity, headed up by Peggy Willingham, now managing director of strategy management. Critical process improvement and value-creation projects are managed by experienced, senior project managers from the PMO, with resources allocated and coordinated by the IC.

Key initiative projects undergo a rigorous, non-negotiable process consisting of the use of robust organizational process assets, such as standardized, non-negotiable project management policies and standards, templates, communication tools and requirements, and knowledge base for storing and retrieving information. But while project outcomes have improved (in no small part as a result of the knowledge, training, and resourcefulness of UAA MSPM graduates such as Peggy, Diana Shaw, and

Weishaar First Draft; submitted 12/5/08

others), room for improvement remains, particularly in terms of the outcomes of enterprise system implementation projects.

Enterprise systems (ESs) are software packages that offer seamless integration of all the information flowing through a company. Examples of ESs are Enterprise Resource Planning (ERP), Customer Requirement Management (CRM), and Product Data Management (PDM). Examples of recent AAG ESs are TRAX, TOTEM, Single-Source, AIMS and SOLAR. Implementing an ES (ES Implementation or ESI), is a complex and dynamic process, often characterized by occurrences of unplanned problems and events. As with ESI efforts in many organizations, there is an overarching sense that outcomes of a number of AAG ESI projects have fallen short of expectations. Is this sense of ESI project outcomes valid? If so, what are the key factors that hinder AAG ESI project success, and what are some best practices that might be incorporated into existing project management processes to improve ESI project outcomes?

1.2

Objective & Approach

This thesis will evaluate outcomes of recent AAG ESI projects using a multi-method approach, combining both qualitative and quantitative data collection methodologies. Objectives include: (1) identify key factors hindering ESI project success (based on literature review and case studies); (2) Identify key factors hindering AAG ESI project success (based on analysis of survey findings and interviews); (3) Identify best practices for improving ESI outcomes (based on literature review and case studies); and (4) develop recommendations, including a framework for integrating best practices into current ESI project processes, to help continue progress toward project

Weishaar First Draft; submitted 12/5/08

10

management maturity and to support the companys strategic plan for 2010 and beyond.

1.2.1 Research Questions 1. How do organization leaders and other key stakeholders characterize typical outcomes of AAG ESI projects? 2. What are the typical factors that hinder ESI success? 3. What are the primary typical factors that hinder AAG ESI success? 4. What are best practices for improving EFI project outcomes? 5. How can the organization incorporate best practices to improve ESI project outcomes?

1.2.2 Approach General inductive approach entailing: (1) summarization of textual data (from literature survey, review of selected AAG ESI project notebooks, as well as interview and survey findings); (2) establishment of defensible links between summary findings and research objectives; and (3) development of a causal framework categorizing findings based on common themes. Categories are incorporated into a causal framework, from which recommendations for improvement are extrapolated.

Assumption that a gap indeed exists between AAG ESI project expectations and outcomes are checked (based on analysis of interview and survey findings). Factors hindering ESI project success are identified (based on literature review). Factors hindering AAG ESI project success are identified (based on analysis of interview and survey findings). Best practices for improving ESI outcomes are identified (based on

Weishaar First Draft; submitted 12/5/08

11

literature review). Recommendations, including a framework for integrating best practices into current ESI project processes, are provided, based on correlations between interview/survey findings and ESI best practices.

1.2.3 Method To determine valid answers to research questions listed in Section 1.2.1, in addition to careful analysis of literature and case study review findings, a sampling plan utilizing both quantitative and qualitative data gathering techniques is developed and executed. Descriptive as well as inferential statistics are used to generate analysis summaries. To answer the first research question: Key leaders are interviewed to: a) validate thesis premise and, b) identify appropriate AAG ESI projects for study.

Project notebooks of selected AAG ESI projects are reviewed; sampling framework established based on project RAMs (roles and responsibility matrices).

Questions designed to obtain AAG ESI perceptions of project success are included in AAG ESI project survey.

After survey is conducted, interview and survey responses are analyzed and aggregated. A gap analysis and other evaluative processes are performed.

To answer the second research question: A literature review is conducted to identify factors that typically hinder ESI success.

To answer the third research question: A survey is developed to identify primary typical factors that hinder AAG ESI success (based on findings of literature review).

Weishaar First Draft; submitted 12/5/08

12

The survey is administered to key ESI project team members (as determined through project notebook reviews and collaboration with project advisors.)

Responses are aggregated and analyzed.

To answer the fourth research question:

A literature review is conducted to identify best practices for improving ESI

project outcomes.

To answer the fifth question:

Correlations are determined between interview/survey results and best

practices for improving ESI project success (as determined through literature review).
A framework is developed and recommendation to incorporate specific best

practices (based on relevance as determined by interview and survey results) into existing AAG ESI project processes is presented.

1.3

Report Structure

Review of recent literature pertaining to ESs (definition & historical context) and related topics, such as ESI challenges and critical success factors are presented in Section 2, along with relevant information regarding Organization Context & Project Management Context. Description of original research, including Problem Analysis (entailing study design and rationale for selection of data collection methodologies); Data Collection (including qualitative and quantitative methods), and Analysis of study findings (including a gap analysis of actual versus desired critical success factors) is presented in Sections 3, 4, and 5. Section 6 consists of recommended best practices, (including a framework for integrating best practices into current & future ESI efforts at AAG), along with project contribution, lessons learned and final words.

Weishaar First Draft; submitted 12/5/08

13

As seen in Exhibit 3 , the work breakdown structure (WBS) for this project was organized into eight deliverables at the first level, including key sections of the thesis itself, (i.e., introduction, literature review, original research, best practices, gap analysis and conclusion), as well as thesis presentation and project management deliverables.

Exhibit 3 Original Thesis Project WBS As writing progressed, the structure of the WBS changed slightly. While the number of first level deliverables remained the same, Original Research was decomposed into 3 first level deliverables, Problem Analysis, Data Collection, and Data Analysis. Problem Analysis was further decomposed into work packages, Research Design and Instrument Selection/Design, and work packages Qualitative Data and Quantitative Data was decomposed into Leader Interviews, AAG ESI Survey, and Thesis Project Sponsor Interviews. First level deliverables, Gap Analysis and Best Practices

Weishaar First Draft; submitted 12/5/08

14

became a work package and moved under Recommendations which was combined with Conclusions. Changes, (highlighted in yellow), are shown in Exhibit 4.

Exhibit 4 - Revised Thesis Project WBS

Weishaar First Draft; submitted 12/5/08

15

2.0

LITERATURE REVIEW

Starting with a survey of literature about enterprise systems, (i.e., definition, history, and context in todays business environment), and the challenges of implementing them (Section 2.1), the search expanded to key issues referenced in ES and ESI literature, including but not limited to: business process management, methodology in action, enterprise resource planning, enterprise project management, strategic decision-making, post-implementation reviews, and organization change management. Then, based on findings from interviews with eleven key company leaders, along with readings of the masters theses from the 2005 UAA MSPM AAG cohort, focus of the search was narrowed down, based on relevancy to AAG, to two key issues organization context (Section 2.2); and project management context (Section 2.3).

A literature search was conducted, primarily via the internet, through web-based search engines such as Google and MS Live Search. DOI numbers of relevant journal articles were obtained through searches via assorted portals, some entailing subscription or membership, including: ACM Digital Library, IT Business Edge Online, Institute of Electrical and Electronics Engineers (IEEE) Xplore Library, TechRepublic Online, Knowledge Storm Research [email protected], and others. Access to full text journal articles was obtained through ABI/INFORMS Global, Business and Management Practices, LexisNexis Academic, Academic Search Premier, and Business Source Premier through the UAA Consortium Library Consortium. When available, contents of most often cited books were accessed through Google Book Search, and, when necessary, purchased through Barnes and Noble Online.

Weishaar First Draft; submitted 12/5/08

16

2.1

Enterprise Systems (ESs)

As a tool that enables businesses to streamline work-related activities and allocate resources to maximize profit, it could be argued that the roots of ES can be traced along the same lines of business process management (Lusk, Paley, and Spanyi, 2005) or business process reengineering (Soliman and Youssef, 1998), that is, back to the beginning of whats referred to as the Industrial Revolution, when Honore Le Blanc first introduced a system for manufacturing muskets from a standard pattern using interchangeable parts. Since that time through the second Industrial Revolution of the early 1900s, (with the introduction of Fords automobile assembly lines and Taylors principles of scientific management, among other developments), organizations came to realize the benefits of improving the efficiency of their processes to decrease costs (Lindsay and Evans, 2008). Exhibit 5 presents a timeline (2005), tracing the evolution of enterprise systems as one of a number of significant business drivers contributing to the development of the ES forms we know today.

The significance of the connection between the categories of business drivers depicted in Lusk, Paley, and Spanyis timeline can not be overstated. The evolution of ES as determined by a combination of management perspectives, methods and tools, and information communication technology (Lorincz, 2007) is a dominant theme in the literature on ES. In what is now considered a seminal work on ESs, Davenport (2000, p. 5) asserts, Ill argue throughout this book that an enterprise project is as much about changing the way a business operates as it is about technology. As seen in Exhibit 6, in reviewing important contributions to the study of ES, Lorenzo (2004) organized

Weishaar First Draft; submitted 12/5/08

17

selected works into four topic categories: business implications, technical issues, managerial issues, and implementation issues. Three Waves of Process Evolution

Weishaar First Draft; submitted 12/5/08

18

Exhibit 5 Three Waves of Process Evolution, (Lusk, Paley, and Spanyi, 2005)

Weishaar First Draft; submitted 12/5/08

19

Weishaar First Draft; submitted 12/5/08

20

Exhibit 6 Sample of ES Research by Main Topics and Areas, (Lorenzo, 2004) As depicted in the Lusk, Paley, and Spanyi timeline, it wasnt until the 1970s, largely as a result of the ground-breaking work of Feigenbaum, Juran, Deming, Crosby, Ishikawa, and other pioneers of the quality revolution, that focus shifted from streamlining processes involving repeatable, sequential tasks within functional silos to improving processes occurring between and across functional areas. It was this shift in focus, from task productivity to continuous flow (systems approach), combined with the advent of computer technology, that marked the beginning of the Information Age, starting with what many ES experts refer to as the first wave, which

Weishaar First Draft; submitted 12/5/08

21

emphasized process improvement (Lusk, Paley, and Spanyi, 2005) and deployed mainframe and minicomputers to automate and organize back-office functions (Papows, 1998 and Stephens, 2004).

The second wave of the information age (late 1980s to early 1990s) is marked by the advent of the personal computer, which enabled workers to automate a number of front-office, white-collar tasks, and lead the way to company-wide computing handled by client/server systems consisting of local area networks (LANs), organization-wide electronic mail, and document management systems. As use of software applications, such as word-processing, spreadsheets, desktop publishing, and graphics presentations, became increasingly widespread, personal productivity improved (Papows, 1998). As these gains in productivity lead naturally to decreased costs and increased speed to market, managers began to recognize the potential of these new technologies for gaining competitive advantage. Organization leaders became increasingly interested in leveraging the new technology to reduce complexity and redundancy, and improve productivity across the entire enterprise by integrating systems, data, and information (Stephens, 2004). Material requirement planning I (MRP I) was an early example of an ES, integrating data related to inventory, manufacturing/production, and demand management. Driven by the need for more precise data about other business processes such as financial information on inventories, labor, and overhead, the next phase, manufacturing resource planning II (MRP II), integrated business planning, sales, distribution, supply logistics, and other functions within silos. While these were considered early enterprise resource planning (ERP) systems, and often accompanied by business process re-engineering (BPR) efforts, focus was still on processes taking place within functional areas (Lorincz,

Weishaar First Draft; submitted 12/5/08

22

2007). During this time, organizations shifted focus to total quality management, employing cross-functional teams, and replacing the why of accomplishing tasks with the how (Lusk, Paley, and Spanyi, 2005).

According to most experts, we are now experiencing the third wave in the evolution of the Information Age, particularly in terms of Business Process Management (BPM), (Lusk, Paley, and Spanyi, 2005; Smith & Fingar, 2003), which began in the mid 1990s with the rise and global spread of the Internet and World Wide Web. With the increasingly robust networking capabilities of integrated enterprises (begun during the second wave), and the transparency and accessibility provided by the Web and the Net, the business process reengineering movement has largely been replaced by ES initiatives.

To meet the demands of todays highly-competitive and rapidly-changing world, ESs are expected to address more than the processes taking place within the walls of the enterprisefrom better management decision-making, improved customer service and retention, ease of expansion/growth, increased flexibility, faster, more accurate transactions, cost reduction, and increased revenue (Davenport et al., 2002; Yang & Seddon, 2004). Exhibit 7 provides a framework of 21 business benefits across five benefit dimensions as presented by Shang and Seddon (2000).

Weishaar First Draft; submitted 12/5/08

23

Exhibit 7 ES Business Benefits Framework, adapted from Shang & Seddon, (2000)

To realize the promise of todays ESs, Organizational structure and culture, the behaviors of workers throughout the company, and even business strategy itselfhave to be restructuredWe have gotten what we wished for, says Davenport, now we only need to make the business and organizational changes necessary to take advantage of our fulfilled dreams (2000, p 6). Since publication of the Lorenzo literature survey in 2004, an increasing number of additional studies have been added to the research base too many to enumerate in this paper. Even so, in considering many of these more recent works, Lorenzos main

Weishaar First Draft; submitted 12/5/08

24

topics and areas framework remains relevant and useful. Some of the more recent studies not included in Lorenzos review are categorized below based on Lorenzos framework:

business implications Lengnick-Hall, Abdinnour-Helm and Lengnick-Hall (2004) examined the importance social and intellectual capital in leveraging ES capabilities to achieve competitive advantage; Mathrani and Viehland (2005) assert the need to move from a focus on efficiency to effectiveness through knowledge-based processes managerial issues Liu and Dai (2002) conducted a study of the relationship and interaction between ESI and business processes and presented a model of managing ESI for competitiveness; Sedera (2002), presented the results of a research process aimed at deriving an ES benefits measurement instrument; Rikhardsson, Rohde, and Rom (2004) studied the relationship between ES and management control; McAdam and Galloway (2005) explored the organizational issues involved in implementing an enterprise resource planning (ERP) system implementation issues Sawyer (2001) use data from a five year study to understand the role of various users in ESI efforts; Sedera (2002) examined the common and specific requirements of stakeholders as antecedent to implementation of an ES; Diaz and Lorenzo (2004) presented an empiricallysupported framework for process modeling of ES; Ward, Hemingway and Daniel (2005) propose a framework for addressing the organizational issues of ESI; Kansal (2006) presented an integrative framework for ESI, with a focus on decision-making prior to selection of the ES system; Gulla (2007) discusses the use of modeling to configure application work and devise more efficient work processes as part of ESI project initiation; Kenett and Raphaeli (2008) consider the roles of risk management and chance management in ESI projects; Parthasarathy and Anbazhagan, (2008) applied the analytical hierarchy process (AHP) to a framework for evaluating various customization possibilities prior to ERP implementation.

While the list above is nowhere near exhaustive, it does serve to illuminate the emerging focus of ES researchimplementation. Additionally, while often considered as a mainly technical endeavor, ESI projects should be considered as organizational change and evolution processes (Davenport, 2000; Orlikowski; 1992, and LeonardBarton, 1988). Examined in more detail in the following section, the challenge of successfully implementing an enterprise system is the subject of more study than all other ES topics and areas (Esteves, et al. 2000; Bhati, 2004; Chetcuti, 2008).

Weishaar First Draft; submitted 12/5/08

25

2.2

Enterprise System Implementation (ESI)

In a 2003 article by Judson Puterbaugh, partner with The Progress Group, a logistics and supply chain consulting firm, the author describes an all-too-familiar scenario: A project team spends six months of late nights and cold pizza working to go live with a new system. Then the go live date occurs, the new system is up and running, and reality sets in. In this real world, implementing a system by a given date frequently requires compromises and tradeoffs, and these usually translate into a reduction in the system's functionality. Those involved assume that the omitted features will eventually be implemented. However, it often happens that after the climactic moment of reaching live status, the project is deemed complete, and both resources and the organization's focus move on to deal with the next formal hurdle.

Because the scenario described by Puterbaugh is so familiar to so many, its no wonder the challenge of implementing an ES is by far the most prevalent subject of ES research. According to Esteves and Pastor, The number of publications that are related to the implementation phase is greater than the number related to all other phases. Exhibit 8 illustrates the preponderance of articles on ES implementation relative to other ES research topics.

Weishaar First Draft; submitted 12/5/08

26

Exhibit 8 Publications related to ES by category (Esteves and Pastor, 2001)

In their annotated bibliography of ERP (as Davenport states, just another name for ES) research, Esteves and Pastor organize published studies of ESI into four categories: implementation approaches, implementation success, other implementation issues, and implementation case studies (see Exhibit 9).

Exhibit 9 Study categories related to ES implementation (Esteves & Pastor, 2001)

Weishaar First Draft; submitted 12/5/08

27

In terms of implementation approach, considered a seminal work on the subject, Parr & Shanks (2000) present a taxonomy based essentially on the scope of the ESI project, i.e., comprehensive, middle-of-the-road, and vanilla. Shang & Seddon (2002) categorized approaches based on four strategies along a spectrum, from an organizations preparedness to change processes, at one end, and focus on customizing the ES software to adapt to existing processes at the other end. Their conclusion echoes a common theme across the literaturecompanies realize the most benefit from ESI when they conduct a robust system exploration of all opportunities for better process performance, and then implement based on a combined approach that includes tailoring some parts of the software to fit key business requirements and changing some organizational processes to exploit attractive software features.

In terms of implementation success, a number of studies have focused on critical success factors (CSFs) of ESI project success. As Moon reports in his 2007 literature review, One of the popular topics in the ERP implementation is to identify or develop Critical Success Factors. The idea is that some important factors determining the success or failure of an ERP implementation can be learned from prior implementation experiences. Some articles focus on generating the list of the critical success factors and others conduct data analysis regarding those factors. Moon also references a set of articles that address the importance of change management to ESI project success why it is important, how to do it effectively, lessons learned, and various change management strategies.

In a very recent paper prepared for the first national WICT conference at the University of Malta, (September 2008), Hein Chetcuti presented a conceptual model

Weishaar First Draft; submitted 12/5/08

28

consisting of key critical success factors based on the latest comprehensive review of ESI literature and findings of a robust, empirical study (quantitative as well as qualitative methodology) to gain input from 31 stakeholders from 11 organizations. As seen in Exhibit 10, Project Management (along with Strategic Intent, Top Management Support, and organizational issues such as Change Management) were the top four concerns.

Exhibit 10 Overall Hypothesis Results (Significant ESI CSFs, Chetcuti, 2008)

Across all categories depicted in Lorenzos organization of ES research (2004), the majority of researchers, including Davenport (1996 & 1998), Gibson et. al. (1999), Somers (2000), Parr & Shanks (2000), Frantz, Southerland & Johnson (2002), Shang & Seddon (2002), Mihailescu, Carlsson, & Mihailescu (2007), and, most recently, Kenett & Raphaeli (2008), Morton & Hu (2008), Chetcuti (2008), and Pollack & Williams (2009) seem to agree there is no such thing as a one size fits all solution. In addition to business rules and processes, disparate visions and beliefs, techniques and practices, the personalities and agendas of the various players involved in the development, implementation, use, and governance of the new system must all be considered.

Weishaar First Draft; submitted 12/5/08

29

Context then, (or the effective accommodation thereof), becomes a critical factor in ESI project success, specifically in terms of organization and project management. (An argument could be made that both topics are actually subsets of the implementation category, however, enough research has been conducted, and the relevance of the two topics to challenges associated with AAG ESI projects is such that they merit additional scrutiny.)

2.3

Organization context

According to Strong & Volkoff (2004), In part, [ES] implementation is a technical taskBut the larger part is organizationalreconciling inconsistent requirements and managing changethus, implementation must align different parts of the organization and support the change process. Davenport (2000) identifies four organization components as critical to ESI project success, including: (1) the role of organization structure in supporting essential collaboration across silos; (2) realignment of reward systems to encourage new behaviors that leverage the new ES as a resource; (3) orientation to change (ability to release emotional investment in old ways of doing things); and (4) a data-oriented culture (norms that encourage decision-making based on data). As seen in Exhibit 11 below, Mihailescu, Carlsson & Mihailescu (2007) expanded on the work of Shang and Seddon (2002) identifying five organization characteristics as key influencers of ESI project success.

Exhibit 11 Organization influences on ESI success, (Mihailescu, et al., 2007)

Weishaar First Draft; submitted 12/5/08

30

Morton and Hu (2008) propose that the fit between organizational dimensions and characteristics of ERP [ES] has significant implications for the success or failure of ERP implementation in organizations (shown in Exhibit 12), presenting a set of propositions regarding the likelihood of ESI project success in different types of organizations (see Exhibit 13).

Exhibit 12 Framework for organizational fit and ESI success (Hu & Morton, 2008)

Exhibit 13 Fit between ESI project and organizational types (Hu & Morton, 2008)

Weishaar First Draft; submitted 12/5/08

31

While Hu and Morton advise caution in observing their findings as their paper dealt with ERP in its pure form (a single solution deployed across the entire enterprise) and Mintzbergs organizational structural types in their ideal forms, the proposed relationships are theoretically based and supported by findings of other studies, i.e., Hammer and Champy (1993), Davenport (2000), Hong & Kim, (2001), Strong & Volkoff (2004), and Lengnick-Hall, et al. (2004).

Organizational structure is one aspect of organizational fit, which is part of the organizational context that must be considered in determining a strategic ESI approach. Another critical organizational characteristic (sometimes associated with culture and sometimes associated with standardized processes) is organization learning and knowledge management. Robey, Ross & Boudreau (2000), Lengnick-Hall, et al. (2004), Sedera, (2007) and Mathrani and Viehland, (2007), among others, have contributed to the literature on this aspect of ESI success.

Robey, et al. (2000) conducted a rigorous analysis of the motivations, implementation processes, and process outcomes of numerous ESI initiatives and found that aside from brief mentions of organizational culture, most of the language used [by survey respondents] reflected a concern with learning and knowledge. As seen in Exhibit 14, specific actions most often correlated with overcoming knowledge barriers included: core team cohesion, emphasis (including budget) for training, effective consulting relationship, incentives to retain ERP-experienced employees, incremental approach to implementation, and change management training. Exhibit 15 shows how the absence of specific actions results in ESI project outcomes that were considered less

Weishaar First Draft; submitted 12/5/08

32

successful in overcoming knowledge barriers.

Weishaar First Draft; submitted 12/5/08

33

Exhibit 14 Firms that were more successful in ESI outcomes (Robey, et al., 2000)

Weishaar First Draft; submitted 12/5/08

34

Exhibit 15 Firms that were less successful in ESI outcomes (Robey, et al., 2000)

Assuming one measure of success related to ESI project outcomes is the enhancement of an organizations ability to gain competitive advantage, Lengnick-Hall, et al. (2004) contend that ESs do not create a competitive advantage on their own. Rather, they can provide a platform for increasing social and intellectual capital which are sources of competitive advantage. According to Lengnick-Hall (2004), success of an ESI project depends on how well the system is integrated with the organizations culture and learning capabilities. Key to realizing the promise of ESI projects, is recognizing the system as an enabling technology rather than as an IT solution.

Weishaar First Draft; submitted 12/5/08

35

To achieve long-term competitive advantage, organizations must be able to do four things: (1) leverage ERP connections to enhance the structural, relational and cognitive dimensions of their social capital; (2) use their social capital to build their intellectual capital and thereby have a superior base of knowledge with which to compete; (3) transform their ERP systems to conform to new insights and subsequent changes in the social system; and (4) change their information systems to accommodate new avenues for competitive value that originate beyond operations activities.

Referencing Markus, the source of competitive advantage resides not in knowledge itself, but in the application of the knowledge (use/re-use), (2001) and Dixon, effective reuse of knowledge is arguably a more frequent organizational concern and one that is clearly related to ES-success (2000), Sedera (2007) analyzed a total of 310 responses from 27 organizations, using a four-phased Knowledge Management process to identify both common and specific requirements of stakeholders, (i.e., strategic, management, operational and technical). Through a in-depth review of the literature and rigorous statistical analysis, Sedera establishes the value of stakeholder input during all phases of the ESI project lifecycle and reveals how all stakeholders consider knowledge retention strategies and end-user training critical to ESI project success.

Recognizing that its not enough to deploy an ES and expect automatic improvement, Mathrani and Viehland, (2007) focused their attention on the essential task of transforming ES data into knowledge by applying analytic and decision-making processes. Referencing a model (Exhibit 16) originally presented by Davenport (2000, p. 222), the researchers consider the findings of a case study to

Weishaar First Draft; submitted 12/5/08

36

develop a model that managers can use to evaluate their own situation and identify areas that need special attention to make sure their organizations ES investment enables improvement in how the business is managed. (Mathrani, et al., 2007, p. 5).

Exhibit 16 Model of How ES Data are Transformed into Knowledge and Results (adapted from Davenport, 2000) The Mathrani and Viehland study highlights how, in order to succeed in todays competitive world, it is imperative that businesses shift focus from improving efficiencies to increasing effectiveness. Integrated access to pertinent information captured by ES must be available so that effective decisions can be made towards successfully implementing strategies, optimizing business performance, and adding value for customers. Knowledge, assert the authors, is a key factor in this process. Among many other processes, the transformation of knowledge through lessons learned occurs within the context of project management, another CSF of ESI project success. 2.4 Project Management Context

Weishaar First Draft; submitted 12/5/08

37

There can be little doubt that effective project management is significantly critical to ESI project success. As indicated in Section 2.2, in a December, 2008 study, Chetcuti concluded that Project Management was among the top five factors most critical to ESI project success. Chetcuti also observed, The overall positive results for H3, [Hypothesis - Project Management is significantly critical to ESI success], also include the agreement that formal project management techniques are necessary for a successful implementation and that project teams need to be motivated, balanced, empowered and most of all highly competent (p. 4). Cited in Chetcutis study are Nah and Laus findings (2001) with regard to the significance of project management as a CSF, (as shown in Exhibit 17).

Appro priate Busine ss & IT Legacy System s

Extremelycritical& importantforsuccess Critical&importantfor success Somewhatcritical& importantforsuccess Importantbutnotcritical/ necessaryforsuccess Neithercriticalnor importantforsuccess

Busine ss Plan & Vision

BPR

Chang e Manag ement Culture & Progra m

Comm unicati on

ERP Teamw ork & Compo sition

Monito ring & Evalua tion of Perfor mance

Project Champ ion

Project Manag ement

Softwar e Develo pment, Testing & Trouble shootin g

Top Manag ement Suppor t

20

23

31

21

33

20

36

30

15

37

20

24

14

11

24

14

20

18

27

11

11

Exhibit 17 Survey Results from CIOs (Nah and Lau, 2001) Not cited in the Chetcuti study, Bhatti (2005) reported similar findings, stating, Project Management [as a] critical success factor is strongly correlated with the other success factors As indicated in Sections 2.2 and 2.3, while identified as one of the more critical factors, Project Management is just one of at least 11 other critical

Weishaar First Draft; submitted 12/5/08

38

success factors. Because of their complexity, particularly in terms of organization context, ESI projects differ significantly from typical projects. According to Davenport, since ES projects are more extensive than other systems projects and involve higher levels of technical and business risk than most systems, its not really a good idea to view ES projects in the same light as other IT initiatives (2000, p. 170). Davenport quotes Harvard professors Rob Austin and Dick Nolan, who observed, its a mistake to view ESI as just another systems or capital expenditure projectthe nature of ES activity is not well-suited to traditional project management techniques (as cited in Davenport, 2000, p. 170). In their 2004 study, Wognum, et al. observe, Such projects encounter many uncertainties, which cannot all be predicted or prevented from the start of the project. An enterprise project, therefore, would need a rather organic structure.

The challenge of uncertainty seems to be a recurring theme in ESI literature. On this subject, Davenport again quotes Austin and Nolan. IT systems management, with its emphasis on exhaustive requirements definition and detailed planning, has never worked that well on the large IT projects it was designed for and is simply not a realistic basis for managing the high and multi-dimensional uncertainty involved (2000, p. 170 171). Gebauer and Lee (2007) cite Gebauer and Schober in identifying process uncertainty as one of three characteristics [the other two being variability and time-criticality] of a business process that can impact ESI success. According to Gebauer and Schober, Process uncertainty refers to the difficulty to predict the tasks and resources that are required to perform the business process in a particular instance (2006). Gebauer and Lee also cite Kumar (2004), who stated, Uncertainty is the result of both a lack of structure of business processes that is ambiguous and

Weishaar First Draft; submitted 12/5/08

39

requires considerable judgment from a decision maker (e.g., one-off situations and of the difficulty to predict process requirements in dynamic business environments). Based on Eisenhardt, (1985) Hwang characterized uncertainty as one of the central dimensions in the organizational management framework (2005), and because organizational factors are an integral part of ESI (as discussed in Section 2.3), management of uncertainty is a key concern of the ESI project manager.

The ESI literature includes a number of approaches to deal with the considerable uncertainty associated with ESI projects. Somers and Nelson (2003) focus on the importance of understanding path dependence of the ES experience and of the players and activities that should dominate each of the implementation phases. Hwang (2005) focuses on informal control mechanisms, (i.e., cultural control and selfcontrol). Gebauer and Lee (2007) assert the importance of flexibility as a decision factor during system design and implementation. Kennet and Raphaeli (2008) focus on risk and change management methodologies (correspondence analysis and partial order mapping), which ...help to characterize and compare ESI readiness across different parts of a company and compare risk profiles of different ESI components. Monks, (2008) points to the importance of implementing software capability consistent with the organizations Project Management Maturity level; investing time and energy up front to define and decompose vision into quantifiable measures; re-engineering processes and defining impacts as part of the project plan; hiring outside help; and assigning dedicated internal resources to the implementation project.

3.0

PROBLEM ANALYSIS

Weishaar First Draft; submitted 12/5/08

40

Implementing an ES, is a complex and dynamic process, often characterized by occurrences of unplanned problems and events (Davenport, 2000). As with ESI efforts in many organizations, there is an overarching sense that outcomes of a number of AAG ESI projects have fallen short of expectations. Is this sense of ESI project outcomes valid? If so, what are the key factors that hinder AAG ESI project success, and what are some best practices that might be incorporated into existing project management processes to improve ESI project outcomes?

3.1.

Research Design

To empirically explore the key factors impeding the success of AAG ESI projects, a mixed method approach, (consisting of both qualitative and quantitative data collection methodologies) was developed to collect input from AAG project participants and stakeholders. A summary of the data collection methodology and sampling schedule is presented in Exhibit 18.

Step

Objective

Population

Target R e s p o n s e

Approach

Schedule

#1

#2

#2b

Assess leader perspectives re: ESI project outcomes Assess expected/ desired outcomes /CSFs of recent AAG ESI projects Assess actual outcomes/CSFs of recent AAG ESI projects

AAG ESI project sponsors and PMO leaders AAG ESI project stakeholders, sponsors & team members AAG ESI project stakeholders, sponsors & team members

10 key leaders 30 respondents (30%) 30 respondents (30%)

Interviews Online surveys based on project notebook audit Online survey (combined w/ Item 2a)

Oct 22 Nov 9, 2008 Dec 26, 08 Jan 2, 09 Dec 26, 08 Jan 5, 09

Weishaar First Draft; submitted 12/5/08

41

#3

Validate survey findings (gaps between 1a & #1b)

AAG Executive, IC & PMO leaders

10 key leaders

Interviews

Jan 5, 09 Jan 7, 09

Exhibit 18 Summary of Data Collection Instrument/Sampling Schedule 3.1.1 Study Population The study population consisted of approximately 100 AAG employees who have recently been, and/or currently are, involved in one or more AAG ESI projects either as stakeholders, sponsors, IC members, steering committee members, PMs, PMO leaders and/or team members, including, but not necessarily limited to:

TOTEM Single Source SOLAR AIMS TRAX

Selection of the study population was based on accessibility as well as generalizeability (proximal similarity model) to similar groups made up of the same and/or similar participants (theoretical population) who most likely would be involved in similar projects in the near future.

3.1.2 Sampling Methods A nonprobability, convenience sampling method was used for quantitative data collection, based on a proximal similarity framework consisting of time, setting, place and people as gradients of similarity.

To provide support, guidance, and evidence for the validity of the convenience sampling method based on proximal similarity model, a nonprobability, purposive, expert sampling method was used for qualitative data collection, based on known expertise and experience with ESI projects.
Weishaar First Draft; submitted 12/5/08

42

Exhibit 19 provides an illustration of the study sampling frame, which was developed from the Roles and Responsibilities Matrices (RAMs) of five AAG ESI project notebooks along with guidance from the 12 key AAG leaders.

Exhibit 19 Study Sampling Frame

3.2

Instrument Selection and Design

Critical elements of the research objectives described in Section 1.2 drove the selection and design of the quantitative and qualitative data collection instruments. It was first necessary to develop a profile of corporate leaders involved in enterprise system implementation projects. Selected leaders were then given an opportunity to share their assessment of critical issues associated with key ESI projects and the environments in which they function.

Because the study needed an analytical method to describe the quantitative nature of the relationships between critical issues and project outcomes, the foundation of

Weishaar First Draft; submitted 12/5/08

43

project team surveys and their analysis was based on common themes identified in leader assessments and literature review. The choices for this component were either custom development, use of standard instruments, or some combination of methods. Due to limitations of time, (i.e., narrow window of participant availability due to holiday vacations), use of a custom method was chosen and a suitable instrument was subsequently developed, using SurveyMonkey, an online survey design and delivery tool. As seen in the sample below in Exhibit 20, (and in the entire survey, Appendix A.26), questions were based on Likert-type scale. (Due to time constraints, the survey was distributed without testing.) A link to the survey was included in a request for participation, distributed to the survey population on Dec 26, 2009.

Weishaar First Draft; submitted 12/5/08

44

Exhibit 20 AAG ESI Project Survey: practice/effectiveness of CSFs 4.0 DATA COLLECTION

Data collection was subdivided into four steps corresponding with the study objectives described in Section 1.2.2. The steps are summarized in Exhibit 21.

Ste

Objective

Sampling Fra me

Target R es p o ns e

Actual R e s p o n s e s 13 key leaders

Approach

Assess leader perspectives regarding ESI project outcomes Assess expected/ desired outcomes/ CSFs of recent AAG ESI projects Assess actual outcomes/CSFs of recent AAG ESI projects Validate survey findings (gaps between #2a and #2b)

AAG ESI project sponsors and PMO leaders AAG ESI project stakeholders, sponsors & team members AAG ESI project stakeholders, sponsors & team members AAG Executive, IC & PMO leaders

12 key leaders

Interviews

2a

30 respondents (30%) 30 respondents (30%) 5 key leaders

33 AAG ESI project participants 33 AAG ESI project participants 5 key leaders

2b

Online surveys based on project notebook audit findings Online survey (combined w/ Item 2a) Interviews

Exhibit 21 Summary of Data Collection Steps 4.1 Step One: Key Leader Interviews

The primary objective of step one was to validate the thesis premise, namely that ESI project outcomes do, indeed, generally fall short of leader expectations. A secondary objective was to identify the specific projects that should be included in this paper to serve as appropriate case studies of AAG ESI projects. A third objective of Step One was to gather leader perspectives regarding expectations, possible frustrations, and

Weishaar First Draft; submitted 12/5/08

45

potential root causes of ESI project outcome shortfalls. Thirteen key leaders were selected based on their unique level of involvement with a number of ESI projects. These leaders represented twelve different functional areas from five of seven major divisions, including: Human Resources, Legal, Finance, and Sales & Marketing. Exhibit 22 Executive Leadership Realignment as of December 8,

Open-ended interview questions were developed to stimulate critical thinking and honest, forthright commentary based on the leaders first-hand experience. Participants included:

Managing Director, Strategy Management Director, AAG PMO Marketing, Sales & Customer Service Managing Director, Audit Programs Principal Consultant, IT & Strategy Managing Director, Talent Strategy & OD V.P, InFlight Services Manager, IT Systems Management Managing Director, CRM Organization Development Consultant Director, Lean Projects Sr. V.P. Customer Services

Following a brief overview of the thesis topic, objectives, and approach, within a time frame of between one half and one full hour, subjects were asked the following questions (in some cases, paraphrased): 1. What projects would you characterize as enterprise system implementation projects? 2. How would you characterize the outcomes of these projects? 3. What factors would you identify as impediments to ESI project success?

Weishaar First Draft; submitted 12/5/08

46

4. Of those factors, which would you identify as having the most significant impact on ESI project success? 5. What would be the value of addressing the most significant factors the companies face in the effective implementation of enterprise systems? Interviews were completed and responses recorded with permission of the participants.

4.2.

Step Two: AAG ESI Project Survey

The primary objective of steps 2a and 2b was to identify critical factors hindering AAG ESI project success from the viewpoint of individuals directly involved with these projects, through a quantitative methodology to provide statistical validity. A secondary objective was to validate the impression (shared by key leaders) that AAG ESI project outcomes generally fall short of expectations. To help determine the best approach for this aspect of the study, findings from leader interviews were kept in mind throughout the literature review. Following the lead of a number of researchers, (as discussed in Section 2.2 to 2.4), a survey utilizing a Likert-type scale was developed to describe the quantitative nature of the relationships between key issues (CSFs as established in Bhattis Critical Success Factors for the Implementation of Enterprise Resource Planning (ERP): Empirical Validation, 2005) and project outcomes.

The AAG ESI Survey, consisted of six sections (included in Appendix A.13), with the Section 1 entailing a brief description of the goals and approach of the research study, as well as a glossary of terms. In addition to self-identification questions, Section 2 provided participants with an opportunity to self-assess the outcome of the AAG ESI

Weishaar First Draft; submitted 12/5/08

47

project (TOTEM, TRAX, AIMS, Single Source, and/or SOLAR) in which they were recently (or currently) involved, either as project manager, executive sponsor, steering committee member, project sponsor, project team member, project stakeholder, or other (technical consultant, analyst, SME, etc.). Because the issue was raised by a number of leaders, other preliminary questions were designed to determine to what extent pre-planning or discovery sessions are employed prior to project kickoff, and to what extent utilization of the software package as an enterprise or cross-functional solution is discussed as part of the initial vision, scope, and/or justification for the investment/endeavor during these preliminary discussions. (Evidence of these types of discussions is not captured in documents contained in project notebooks). Section 3 of the AAG ESI Project survey consisted of twelve questions designed to determine the actual significance of specific CSFs in project outcomes. To ensure consistency in respondents understanding of the CSFs, each question included a description of the specific CSF per Bhatti. Survey participants were asked how they would characterize each CSF in terms of effectiveness, i.e., very effective, somewhat effective, neither effective nor ineffective, somewhat effective, or not effective. Intended to determine the theoretical or perceived value of each CSF in achieving desired ESI project outcomes, Section 4 presented participants with an opportunity to rate each CSF on a scale of 1 to 12 (with 1 representing the highest value). Section 5 provided participants an opportunity to add comments with regard to additional factors that might hinder or impeded ESI project success.

Weishaar First Draft; submitted 12/5/08

48

As indicated in Exhibit 9, (Section 3.2), surveys were sent via email to 100 AAG ESI project participants. The target response rate was 30% or 30 surveys (considered a generally sufficient sample size to establish statistical validity (based on normal distribution). Actual response rate was 39% or 39 respondents, however 6 respondents did not complete the survey after completing demographic information, leaving 33 valid responses for a response rate of 33%. (Subsequent analysis will be based on 33 completed surveys). Distribution of respondents based on project role is shown in Exhibit 23.
Survey Respondents by Project Role
45 40 35 30 Number 25 20 15 10 5 0
Other Project Stakeholder Project Team Member Steering Committee Member Project Sponsor Executive Sponsor Project Manager

Valid sample target = 30

Exhibit 23 Survey Respondents by Project Role Respondents 4.3 Step Three: Project Sponsor Interviews

Survey results were discussed with thesis project sponsors (previously identified as key leaders) as a way of validating the researchers interpretation of survey findings and narrowing the focus of the remaining sections (to ensure maximum value of study recommendations and conclusions to the organization). Preliminary answers to original research questions (based on findings from literature review, interviews and survey) were explored and refined, establishing a platform for best practices framework.

Weishaar First Draft; submitted 12/5/08

49

5.0

DATA ANALYSIS

Once data was logged and checked for accuracy, analysis was conducted utilizing general inductive approach (for qualitative data), and descriptive and inferential statistics (for quantitative data, where appropriate) to establish internal, construct, and conclusion validity.

5.1

Key Leader Interview Data

As described in Section 3.2.1, twelve company leaders where interviewed to validate the thesis premise; to identify specific AAG ESI projects to be included in this study; and to explore their perspectives regarding expectations, possible frustrations, and potential root causes of ESI project outcome shortfalls. The twelve key leaders were selected based on their unique level of influence and involvement with a number of ESI projects.

5.1.1 Key Leader Company Affiliation Exhibit 24 presents the distribution of leaders based on primary company affiliation, whether AS, QX, or what is referred to as shared services supporting both companies under the label of AAG.

Weishaar First Draft; submitted 12/5/08

50

L d C p n Affilia ion ea er om a y t s

1 5 Count 1 0 5 0 C pn om a y Q X A S AG A

Exhibit 24 Leader Company Affiliation

5.1.2 Key Leader Tenure As seen in Exhibit 25, leader tenure with AS and/or QX varied from 4 months to 27 years, with a mean of 13.08 years.

Leader Tenure Years

0
AA JB CH SJ WJ SL NM CM SM AS SS AU PW Me

10

15

20

25

30

Leaders

Exhibit 25 Tenure of Interviewed Leaders

Weishaar First Draft; submitted 12/5/08

51

Analytic strategies for qualitative data were based on general inductive approach. While the interviews were very informal and organic in nature, five basic questions were posed and discussed. Guided by interview questions (which encompass evaluation objectives) and responses, recurring themes were identified and key categories were developed into a framework consisting of category label, researchers description of category meaning, and a quotation from interview notes to elaborate the meaning of the category (Thomas, 2006).

5.1.3 AAG ESI Project Selection/Perceived Outcomes Questions 1 & 2: Fairly straight-forward (i.e., specific AAG projects considered ESI projects and whether outcome is considered successful or not). The majority of leaders asked how the term, enterprise system, would be defined for this study. Three leaders recommended narrowing focus to one or two specific types of ESs, based on: (1) scope, for example: integration of software applications combined with changes in business processes within a single business unit (BU); integration of software application combined with changes in business processes across multiple business units (XBU); or integration of software applications combined with change in businesses processes across more than one company (AAG); (2) approach, for example: phased rollout versus big-bang or complete cutover within a short period of time; and (3) whether software application was built or bought. Two leaders cautioned, There are no one-size-fits-all solutions. The extent of customization was also addressed as a distinguishing factor by a few leaders. A grid, (presented below in Exhibit 26), combining categories gleaned from answers to questions 1 and 2, summarizes these distinctions, and includes indicators of consensus among leaders

Weishaar First Draft; submitted 12/5/08

52

regarding project success (+), or failure (-). As one leader observed, many, if not most ESI project outcomes fail to fulfill the promise of an enterprise-wide solution.
ProjectName TOTEM SingleSource (ORCA) SOLAR QXAIMS ASTrax Approach phased phased big-bang big-bang big-bang Bought/Built Bought Bought Bought Bought Bought BU/XBU/AAG XBU XBU BU XBU AAG Vendor OLE/VTN Technologies XYEnterprises Siebel/Oracle AIMS TraxUSACorp -/+ + +

Exhibit 26 Summary of Categories from Leader Interview Questions 1 & 2 5.1.4 Factors Hindering AAG ESI Project Success Questions 3 & 4: Key leaders identified approximately twenty factors that are likely to impact AAG ESI project success. These factors are loosely categorized and summarized in Exhibit 13. A clear vision and/or value proposition was most often cited as a critical success factor, with one leader summing up the overall impression that, clearly defined ownership and value proposition (like SOLAR and ERP [accounting]) is essential to ESI success, though most of the time the vision is never truly realized which is normal. Note: An X indicating whether the factor was addressed during the course of the project (as determined through research of project charters and other documents found in project notebooks and Audit Programs Special Project AAG Harmonization Analysis 2Q 07) was subsequently added to appropriate cells in the grid presented in Exhibit 27 below to. Values were then attached to each factor based on the number of leaders who indicated the factor as an issue or concern during the course of each interview.
Issue/Project Vision/Value Prop Owner TOTEM X Single Source (ORCA) X SOLAR X X QX AIMS X AS Trax X X Value 12 12

Weishaar First Draft; submitted 12/5/08

53

Communication Buy-in Successor Change Mgmt Mandate Provisioned Phases Decision Rights Process Mapping AP/SCM/IT/Lean Authorization Path Vendor Consultant Bounded by BU PM processes not a function of time

X X X X X X X X X X X

X X X X

11 10 10 9 9 6 6 5 3 3 3 3 2

Exhibit 27 Summary of Categories from Leader Interview Questions 3 & 4 5.1.5 Value of Improving AAG ESI Project Outcomes Question 5: Without exception, each key leader indicated that development and utilization of standardized tools and/or techniques that address many of the identified factors would be considered significantly valuable and worthwhile. However, while there was consensus regarding the value and benefit of such standardized tools, one leader urged caution with regard to increasing the level of bureaucracy in decisionmaking, indicating you dont want to lose the entrepreneurial spirit and ability to learn through experimentation on small scale.

Weishaar First Draft; submitted 12/5/08

54

5.2

AAG ESI Project Survey Data

As indicated in Section 3.2.2, the AAG ESI Project Survey was divided into five sections. A summary of responses to questions from each section is provided in both narrative and chart form.

5.2.1 Section 1: Self Identification Question 1.1: On which AAG ESI project(s) were you involved? As seen in Exhibit 28, the majority of survey respondents were involved with the SOLAR project, followed by Single Source, then Trax, AIMS and TOTEM.

Survey Respondents by Projec t Name 26% 14 18% 12 10 8% 31% 8 6 18% 4 2 0

Tr ax LA R SO rc e AI M TE M TO S Si ng le

Number

AS

QX

So u

Weishaar First Draft; submitted 12/5/08

55

Exhibit 28 Survey Respondents Project Affiliation

Question 1.2: What role did you have on the project? As seen in Exhibit 11 in Section 3.2.2, the majority of respondents (11) self-identified as project team members, while 6 respondents self-identified as project managers, 4 as project sponsors, 3 as project stakeholders and 2 each as executive sponsor, steering committee member and other.

Question 1.3: How would you characterize your experience level with ESI projects? Exhibit 29 illustrates how respondents characterized their experience level in their role on the ESI project, with 2 out of 4 project managers and 7 out of 17 team members indicating a high level of experience; 1 out of 4 project managers and 4 out of 17 team members self-identifying as somewhat experienced. A number of project team members, as well as participants describing their role as other, self-identified as first time on ESI project, which seems reasonable as exposure to such large and complex projects often occurs first as a team member or SME/analyst.

Weishaar First Draft; submitted 12/5/08

56

Respondents' Experience Level by Role

8 7 6 5 Count 4 3 2 1 0
be r itt ee ld er r tM an Ex ag ec er ut iv e Sp on so r Ot he r ns o

Highly experienced Somewhat experienced First ESI Project

Me m

5.2.2 Section 2: Pre-planning and project outcomes As indicated in Section 3.2.2 of this study, Section 2 of the AAG ESI Project Survey was driven by research question 1, providing respondents with an opportunity to self-assess the outcome of their AAG ESI project, and, because the issue was raised by a number of leaders, included other preliminary questions designed to determine to what extent pre-planning or discovery sessions are employed prior to project kickoff, and to what extent utilization of the software package as an enterprise or cross-functional solution is discussed as part of the initial vision, scope, and/or justification for the investment/endeavor during these preliminary discussions.

Question 2.1: How would you characterize the pre-planning or discovery phase of your project? Almost half of respondents indicated that the pre-planning process for their project was somewhat extensive, with 27% indicating pre-planning was very

St e

Exhibit 29 Respondent experience level based on project role.

Pr oj

er in g

Te a

ec

St ak eh o

Sp o

Co m m

Weishaar First Draft; submitted 12/5/08

57

extensive. Three project managers commented that they inherited their project after pre-planning was over. A summary of responses for Question 2.1 is shown in Exhibit 30.

Extent of Pre-planning 16 14 12 Number 10 8 6 4 2 0

in im al ve Ve ry en si Ne i th en t er

45%

27%

12%

9%

6%

ha t

Exhibit 30 Extent of pre-planning prior to project kick-off

Question 2.2: If pre-planning/discovery took place, to what extent was the proposed software package considered an enterprise solution? Nearly three quarters of respondents indicated that original discussions during pre-planning included references to the proposed software package as an enterprise solution. Exhibit 31 presents a summary of responses to Question 2.2.

So m

So m ew

ew

No nex ist

ex t

ha t

Weishaar First Draft; submitted 12/5/08

58

Pre-planning included expectation of 'enterprise' solution 25 20 Number 15 10 5 0

el y De fi ni t ei th e ib l li k no t ol ut e ly Ab s el y y r

70%

24%

03%

0%

3%

Po ss

Exhibit 31 Level of expectation of the project as enterprise solution

Question 2.3: Did project scope, charter, and/or business case include a clear vision, expectations, and/or objectives? Responses were somewhat split with regard to having a clear vision as part of early project planning, with 42% of respondents indicating a very clear vision and 39% having a somewhat clear vision. Four respondents commented that while objectives for the project itself were very clear, an ongoing vision for enterprise implementation was non-existent. For example, according to one
Weishaar First Draft; submitted 12/5/08

ot

59

respondent, There was no discussion in either [TOTEM or Single Source] how this [solution] would be implemented system wide. Exhibit 32 provides a summary of responses to Question 2.3.

Clarity of vision 16 14 12 Number 10 8 6 4 2 0

cl ea r le ar cl ea r le ar en t on -e x N is t

42%

39%

06%

09% 03%

un c

Ve ry

ha t

ha t

So m

Exhibit 32 Clarity of initial vision, expectations, objectives

Question 2.4: If a vision, expectations, and/or objectives were stated, to what extent did the vision, expectations, and/or objectives refer to the project as an enterprisewide, cross-departmental, or cross-functional solution? Over two thirds of respondents indicated that where a clear vision/expectations/objectives were stated, enterprisewide/cross-department/cross-functional implementation/deployment was definitely included.

So m

ew

Ve ry

ew

un c

Weishaar First Draft; submitted 12/5/08

60

Just over a third of respondents were less than certain that the vision for the project included the expectation that the solution would be deployed or implemented across multiple departments or functional areas. With regard to TOTEM for example, two respondents provided vary similar comments, for example, very unclear in the beginning as to what our objectives/roles were as project team members, as well as the time commitment expected. Understood we were going to assist with deploying TOTEM within our dept/division at some point, but unclear on when/how that was to take place and who would be deciding that Another team member stated, Vision was included, but was never defined and there was not buy-in from the other divisions involved to make it truly an enterprise solution. No overall governing body was identified to make it actually enterprise wide. Much of the activity and governing resides with the Sponsoring division, which was never the intent of the vision. A summary of responses to Question 2.4 is presented in Exhibit 33.

Enterprise solution as part of vision 25 20 Number 15 10 5 0

el y De fi ni t Po ss ib l ei th e li k no t ly Ab so lu te el y y r

64%

21% 09% 3% 3%

Exhibit 33 Extent to which enterprise solution was included in project vision

Weishaar First Draft; submitted 12/5/08

ot

61

Question 2.5a: Because project success can be defined in a number of ways, the survey question specifically intended to answer research question #1 was asked in two different ways. Question 2.5a focuses on success in terms of the triple constraints of scope, cost, and time. A third of the responses indicated that project participants considered their project outcome to be highly successful, while 42% indicated a perception that their project was moderately successful. Over one quarter of respondents indicated that their project outcomes were less than successful. Two participants indicated that the outcome of their project (TOTEM) had yet to be determined. A summary of responses to Question 2.5a is presented in Exhibit 34.

Outcome in terms of triple constraints 16 14 12 Number 10 8 6 4 2 0

sf ul er ly od er at e Ne i th ss fu l sf ul ce s

42% 30%

15% 9% 3%

es

su cc

uc ce Hi gh

Hi gh

un s M od er at e ly

Exhibit 34 Perception of AAG ESI project outcome in terms of triple constraint

ly

un su c

ly

Weishaar First Draft; submitted 12/5/08

62

Question 2.5b: The focus of the final question in Section 2 of the AAG ESI Project Survey was on outcome success in terms of the original vision. Responses to this part of question 2.5 were nearly identical to responses to the first part of the same question, with one third of participants indicating that their project was highly successful and 42% describing the outcome of their project as moderately successful. Almost one third of respondents indicated a perception that the outcome of their project was less than successful. A summary of responses to Question 2.5b is provided in Exhibit 35.
Outc ome in terms of original vision

16 14 12 10 8 6 4 2 0
sf ul

42% 27% 21% 09%

Number

er

ly

od er at e

Ne i th

es

su cc

Hi gh ly

Exhibit 35 Perception of AAG ESI project outcomes in terms of original vision

M

To test the generalizeability of findings from Section 2, the researcher hypothesized that, if survey responses were filtered to reflect results from individual AAG ESI projects, less variability between filtered responses would indicate more agreement between respondents on critical issues, and thus validate generalizeability. Conversely, more variability between filtered responses would indicate less agreement, indicating less valid generalizeability. While sample sizes for each specific project were admittedly small, (with 7 respondents each from SOLAR and Single
Weishaar First Draft; submitted 12/5/08

od er at e

ly

un s

uc c

es

sf ul

63

Source, and an average of 13.5 respondents from TOTEM and AS Trax and QX AIMS disregarded altogether due to having only 3 respondents identifying themselves strictly with the Horizon project), the test was nevertheless conducted to determine if results from one or more projects caused skewing of summary findings and to gain additional insight into the impact of pre-planning and clear vision on the outcome of each AAG ESI project. (Detailed analysis available in supplemental Excel documents). Question 2.1: As seen in Exhibit 36, the extent of pre-planning differed between projects, with 67% of respondents on the Single Source project indicating very extensive pre-planning efforts, and an average of 51% of respondents on TOTEM, SOLAR, and AS Trax indicating somewhat extensive pre-planning. It should be noted that two Single Source participants skipped this question, indicating that they joined the project after pre-planning had taken place. Almost one third of TOTEM project participants reported that pre-planning was either somewhat minimal or non-existent on their project.
Extent of Pre-planning - Single Source 5 4 Number 33%
Number Extent of Pre-planning - TOTEM 5 4 3 2 1 0
m in im nal ex is te nt No ns iv e Ve ry he r

67%

40%

3 2 1 0

20% 10% 10%

20%

0%

0%

0%

Ve ry ex te ns iv e So Ne m ew it he ha r t m in No im nal ex is te nt

ex te

ha t

ha t

So m ew

So m ew

So m ew

ha t

Ne it

Weishaar First Draft; submitted 12/5/08

64

Extent of Pre-planning - SOLA R 6 5 4 3 2 1 0

Ve ry

Extent of Pre-planning - A S Trax 7 6 5 4 3 2 1 0

y Ve r

63%

50% 33% 17 0% 0%

Number

25% 13% 0% 0%

er ha tm in im No al nex i st en t

Number

So m ew

ha te

ha t

So m ew

Exhibit 36 Extent of pre-planning prior to each project kick-off

Question 2.2: As seen in Exhibit 37, an average of more than three quarters of Single Source, SOLAR, and AS Trax participants indicated pre-planning discussions included the expectation that the software packages involved in their projects would definitely be considered enterprise solutions. TOTEM participants expressed less agreement about their software solution. This may well be correlated with lack of agreement that pre-planning took place on the TOTEM project. Nevertheless, it does not appear that TOTEM results significantly skewed summary results.

P r e- pl anni ng i nc l uded ' ent er pr i se' sol ut i on S i ngl e S o ur c e

So m ew

P r e- pl anni ng i nc l uded ' ent er pr i se' sol ut i on TOTEM

6 5 Number 4 3 2 1 0

71%

6 5 4 3 2 1 0

45% 36%

29% 0% 0% 0%

Number

So m ew

Ne it he r ha tm in im No al nex is te nt

ex te ns iv

Ne ith

xt e

ns iv e

9% 0%

9%

De fin ite ly Po ss ib ly Ne it h er No tl Ab ik el so y lu te ly no t

De fin ite ly Po ss ib ly Ne it h er No tl Ab ik el so y lu te ly no t

Weishaar First Draft; submitted 12/5/08

65

P r e- pl anni ng i nc l uded ' ent er pr ise' sol ut ion S OL A R

P r e- pl anni ng i nc luded ' ent er pr ise' sol ut ion - A S

8 7 6 5 4 3 2 1 0

88%

10 8 Number 6 4 2 0

75%

Tr ax

Number

12% 0% 0% 0%

17%

08%

0%

0%

Exhibit 37 - Level of expectation of the project as enterprise solution Question 2.3: Did project scope, charter, and/or business case include a clear vision, expectations, and/or objectives? As seen in Exhibit 38, almost 50% of SOLAR, TOTEM, and AS Trax project participants indicated their vision was only somewhat clear, and while 83% of Single Source project participants described the vision associated with their project as very clear, as indicated previously, four survey respondents commented that while objectives for the project itself may have been very clear, the ongoing vision for enterprise implementation was non-existent, (see example in Section 3.3.2).
Clarity of vision - Single Sourc e 6 5 Number 4 3 2 1 0
ha So t m cl ew ea r ha t un cl ea Ve r ry un cl ea No r nex is te nt cl ea r

De fin ite ly Po ss ib ly Ne it h er No tl Ab ik so el y lu te ly no t

83%
Number

De fin ite ly Po ss ib ly Ne it h er No tl Ab ik el so y lu te ly no t
Clarity of vision - TOTEM 6 5 4 3 2 1 0
ar ar r No r nex is te nt cl ea un cl ea un cl e cl e

45%

27% 9% 9% 09%

17% 0% 0% 0%

Ve r

Ve r

ha t

ha t

So m ew

So m ew

So m ew

Ve r

Weishaar First Draft; submitted 12/5/08

66

Clarity of vision - SOLA R 5 4 Number 3 2 1 0

ar ar r le ar No nex is te nt cl ea cl e cl e

7
50% 50%

Clarity o f visio n - A S Trax 46% 38%

6 5 Number 4 3 2 1 0

8%

8% 0%

0%

0%

0%

ar

ar nc le Ve r

So m ew

So m ew

Exhibit 38 Clarity of initial vision, expectations, objectives

Question 2.4: To what extent did the vision, expectations, and/or objectives refer to the project as an enterprise-wide, cross-departmental, or cross-functional solution? Exhibit 339 below shows that, for 86% of SOLAR project participants, widespread implementation was definitely a part of the project vision. Similarly, for 84% of AS Trax respondents, enterprise-wide deployment was either definitely or possibly expected. TOTEM and Single Source respondents expressed less agreement however, with three project participants describing system-wide deployment as either not likely or absolutely not. Summary findings for this question (36% responding either possibly, neither, not likely, or absolutely not) do not seem significantly skewed by SOLAR responses, as, based on average, 42% of other project participants indicated some uncertainty about the enterprise application of their project software packages.

So m ew

So m ew

le ar No nex i st en t

le a yc

ha tc le

ha t

un c

un

ha tu

Ve r

ha t

Ve r

Ve r

yu

nc

Weishaar First Draft; submitted 12/5/08

67

Enterprise solution as part of vision Single Source 5 4 Number 3 2 1 0

lik bs el ol y ut el y no t y bl y er D ef in it el N ei th Po ss i

Enterprise solution as part of vision TOTEM 6 5 4 3 2 1 0

y ef in it el

57% 29% 14% 0% 0%

Number

45%

36%

9% 0%
no t

9%

bl y

er

N ei th

lik e

ly

Po ss i

N ot

No t

Enterprise solution as part of vision SOLAR 7 6 5 4 3 2 1 0

y Po ss ib ly ef in it el N ei th er

Enterprise solution as part of vision AS Trax 10 Number 8 6 4 2 0 69% 15% 15% 0%

Po ss ib ly N ei th er N ot lik A el bs y ol ut el y no t
A

86%

Number

14% 0% 0% 0%

bs ol ut

el y

0%

lik el bs y ol ut el y no t

Exhibit 39 Extent to which enterprise solution was included in project vision Question 2.5a: Responses regarding project success in terms of triple constraints appeared to be somewhat dispersed. For example, while 42% of AS Trax participants reported their project outcome as highly successful, four participants (or 34%) considered project outcome either neither successful or moderately unsuccessful. An average of 46% of participants on the three other projects reported outcomes as moderately successful, with an average of 15% reporting outcomes as neither successful or unsuccessful. Histograms of project findings are shown in Exhibit 40. A summary of project outcomes (and standard deviations) in terms of triple constraints in the table shown in Exhibit 41.

ef in it el y

N ot

Weishaar First Draft; submitted 12/5/08

68

Outcome in terms of triple constraints Single Source 5 Number 4 3 2 1 0

sf ul od er at M el od y er Ne at el it y he un Hi r su gh cc ly e. un .. su cc es sf ul

Outc ome in terms of triple c onstraints - 6 5 4 3 2 1 0 TOTEM 36% 45% 18% 0% 0%

57% 29% 14% 0% 0%

Number
Hi gh ly

Outc o me in te rms o f trip le c o nstraints 5 Number 4 3 2 1 0

su cc es sfu Mo l de ra Mo te ly de ra Ne te ith ly er un Hi su gh cc ly e. un .. su cc es sfu l

Outc ome in terms of triple c onstraints 6 5 4 3 2 1 0 - AS Trax 42% 25% 17% 17% 0%

SOLA R
Number
Hi gh ly
% 0.29 0.57 0.14 0.00 0.00 7 % 0.36 0.45 0.18 0.00

38%

38% 13% 0% 13%

Exhibit 40 AAG ESI project outcomes in terms of triple constraints

AAG ESI Project Outcome/triple constraints Single Source Highly successful Moderately Neither Moderately unsuccessful Highly unsuccessful Total TOTEM Highly successful Moderately Neither Moderately unsuccessful Weight 5 4 3 2 1 Total 1.43 2.29 0.43 0.00 0.00 4.14 5 4 3 2 1.82 1.82 0.55 0.00 0.83 Mean

su cc es sf M ul od er at M od el y er at Ne el it y he un Hi r su gh cc ly e. un .. su cc es sf ul

Hi gh ly

su cc es sf M ul od er at M el od y er Ne at el it y he un Hi r su gh cc ly e. un .. su cc es sf ul

su c Hi gh ly

ce s

Weishaar First Draft; submitted 12/5/08

69

Highly unsuccessful Total SOLAR Highly successful Moderately Neither Moderately unsuccessful Highly unsuccessful Total AS Trax Highly successful Moderately Neither Moderately unsuccessful Highly unsuccessful Total SD

0.00 11 % 0.38 0.38 0.13 0.00 0.13 1 % 0.42 0.25 0.17 0.17 0.00 1 5 4 3 2 1 5 4 3 2 1

0.00 4.18 1.88 1.50 0.38 0.00 0.13 3.88 2.08 1.00 0.50 0.33 0.00 3.92 0.783 0.031 0.775 0.84

Exhibit 41 Summary of project outcomes in terms of triple constraints Question 2.5b: In describing project outcomes in terms of the original project vision, Single Source participants expressed more agreement, with 50% reporting outcomes as highly successful and 50% reporting outcomes as moderately successful. Responses regarding the three other projects were more dispersed, with an average of 27% of participants describing their project outcome as neither successful or unsuccessful or moderately unsuccessful. Three TOTEM participants indicated that because the second phase of the project was still in progress, the project outcome was still unknown. Histograms of findings for each project are shown in Exhibit 42, with a table summarizing project outcomes in terms of original vision in Exhibit 43.

Weishaar First Draft; submitted 12/5/08

70

Outc ome in terms o f original visio n 5 4 3 2 1 0

Mo de ra te ly ce ss fu l er

Outc o me in terms of o riginal vision TOTEM 8 7 6 5 4 3 2 1 0

sfu l

Single Sourc e 50% 50%

Number

64% 18%

Number

0%

0%

9%

9%

Ne it h

cc e. ..

Mo

Outc ome in terms of original vision SOLA R 5 4 3 2 1 0

Outc ome in terms of original vision - A S 7 6 5 4 3 2 1 0 50% Trax

Number

Number

33%

33% 17% 17%

Mo

25% 8%

Ne de it h ra er te ly un su c. ..

su cc es Hi gh ly

un su

Hi gh ly

de ra te ly

Mo

de ra te ly

su c

17%

Ne de it h ra er te ly un su c. ..

de ra te ly

ce ss fu l

Exhibit 42 AAG ESI project outcomes in terms of original project vision

Mo
Mo

AAG ESI Project Outcome in terms of original vision Single Source % Weight Total Mean Highly successful 0.50 5 2.50 Moderately 0.50 4 2.00 Neither 0.00 3 0.00 Moderately unsuccessful 0.00 2 0.00 Highly unsuccessful 0 1 0.00 Total 7 4.5 0.90 TOTEM %

Ne de it h ra er te ly un su c. ..

Hi gh ly

Hi gh ly

su cc es sfu l Mo de ra te ly

su c

Mo

Weishaar First Draft; submitted 12/5/08

71

Highly successful Moderately Neither Moderately unsuccessful Highly unsuccessful Total SOLAR Highly successful Moderately Neither Moderately unsuccessful Highly unsuccessful Total AS Trax Highly successful Moderately Neither Moderately unsuccessful Highly unsuccessful Total SD

0.09 0.64 0.18 0.09 0 11 % 0.33 0.33 0.17 0.17 1 2 % 0.50 0.25 0.08 0.17 0 1

5 4 3 2

0.45 2.55 0.55 0.18 0.00 3.73 1.67 1.33 0.50 0.33 1.00 4.83 2.50 1.00 0.25 0.33 0.00 4.08

0.75

5 4 3 2 1

0.97

5 4 3 2 1

0.82 0.097

Exhibit 43 Summary of project outcomes based in terms of original vision With no more 42% of AAG ESI project participants describing their project outcomes as highly successful in terms of triple constraints, and no more than 50% describing their project outcomes as highly successful in terms of the original project vision, combined with the consensus among key leaders regarding project outcomes the thesis premise that outcomes of AAG ESI projects are less than successful would appear to be validated.

5.2.3 Section 3: Practice/Effectiveness of CSFs Survey Section 3.0: As indicated in Section 3.2.2 above, questions from this section of the survey were intended to determine the actual effectiveness of twelve critical success factors (as identified through literature review) in achieving desired project outcomes. Participants were asked to rate effectiveness of each CSF as either very

Weishaar First Draft; submitted 12/5/08

72

effective, somewhat effective, neither effective or ineffective, somewhat effective, or not effective.

Note that Question 3.1: Project Management, was further divided into six subcategories to allow respondents to evaluate the effectiveness of project management practices depending on their role on the project. Based on the histograms, project managers tended to rate project management effectiveness higher than participants acting in other capacities on a project, with 39% rating project management as very effective. While an average of 19% of respondents in other project roles rated project management as very effective, an average of 54% rated project management as somewhat effective. Of particular note, over one third (34%) of project stakeholders perceived project outcomes to be less than successful. The frequency of responses from the perspective of each role for the CSF, Project Management are shown in Exhibit 44.

A c tu al e ffe c tiv e n e ss - PM fro m A c tu a l e ffe c tiv e n e s s - PM fr o m Ex e c S p o n s o r p e r s p e c tiv e PM p e r sp e c tiv e 10 8 6 4 2 0 39% 35% 13% 13% 0%

15 12 9 6 3 0

57% 19% 19%

Number

Number

5%

0%

Ve ry ef So fec. me . wh . at ... N So eith er me wh No at.. te . ffe c.. .

Ve ry ef So fe.. me . wh at ... Ne i th So er me wh No a t... te ffe c.. .

73

Weishaar First Draft; submitted 12/5/08

A c tu a l e ffe c tiv e n e s s - PM fr o m Pr o je c t S p o n s o r p e r s p e c tiv e 15 50%

Number

A c tu al e ffe c tive n e ss - PM fro m S C me mb e r p e rsp e c tiv e 15 53% 26% 11% 5% 5%

Number

10 5 0

23%

18%

10 5 0

5%

5%

Actual effectiveness - PM from Team Member perspective

57%

15

Number

10 5 0

18%

18% 07%

Number

Ve ry e So ffe. . m ew . ha ... Ne So ithe r m ew No ha. t e .. ffe c. ..

15 10 5 0 14% 0% 52% 29%
0%

Ve ry ef f .. . So me wh .. Ne ith er So me wh .. No te ffe ...

A c tu al e ffe c tive n e ss - PM fro m Stak e h o ld e r p e rsp e c tive

5%

fe ct iv e

So m ew

Exhibit 44 Project Management practice/effectiveness

To test the generalizeability of summarized findings from Section 3, responses were filtered to reflect results from individual AAG ESI projects, with the assumption that less variability between filtered responses would indicate more agreement between respondents on critical issues, and thus validate generalizeability. Conversely, more variability between filtered responses would indicate less agreement, indicating less valid generalizeability. As indicated previously, while sample sizes for each specific project were admittedly small, the test was nevertheless conducted to determine if results from one or more projects caused skewing of summary findings and to gain

So m

So m

. Ne So ith m ew er ha ti n No t e .. . ffe ct iv e
74

ct iv e

he r

ef fe ct iv e

ec ti ve

ive ef fe Ve ry ew ct

ef f

ef fe

Ne it

in ef

ha t

Ve r

Weishaar First Draft; submitted 12/5/08

ew

ha t

No t

ha t

ef ..

additional insight into the perception of PM effectiveness for each AAG ESI project. (Detailed analysis of filtered responses is available in supplemental Excel documents).

Generally speaking, with regard to project management effectiveness from the perspective of various project roles, a review of the histograms for each AAG ESI project yielded little additional insight due to very small or very disparate sample sizes, for example, while 3 Single Source and TOTEM project managers participated in the AAG ESI Project Survey, only 1 AS Trax PM participated and 0 PMs from the SOLAR project. Sample size of project team members were less disparate, with 6 participating from Single Source, 9 from TOTEM, 7 from SOLAR, and 9 from AS Trax.

As seen in Exhibit 45, team members seemed to be generally in agreement with regard to project management effectiveness, with from 11% (TOTEM) to 22% (AS Trax) describing project management as very effective, and from 50% (Single Source) to 71% (SOLAR) describing project management as somewhat effective. On all but the SOLAR project, 11% (AS Trax) to 33% (Single Source and TOTEM) described project management as either neither effective or ineffective or somewhat ineffective.

Actual effectiveness - PM from Team Member perspective

5 4 6 5 4 3 2 1 0

Actual effectiveness - PM from Team Member perspective

56% 22% 11% 11% 0%

Number

3 2 1 0 17%

33% 0%

0%

ec tiv e

Number

50%

ec tiv e

he r

ct iv e

ct iv e

he r

ve

ec ti ve

ef fe ct i

ec ti ve

in ef f

ha t

No t

ha t

in ef f

ef f

ef f

Ve r

ha t

Ve r

So m ew

So m ew

So m ew

So m ew

ha t

No t

ef fe ct i

Ne it

ef fe

ef fe

Ne it

Weishaar First Draft; submitted 12/5/08

ve

75

Actual effectiveness - PM from Team Member perspective

6 71% 8

Actual effectiveness - PM from Team Member perspective

67% 22% 11% 0% 0%

Number

4 2 0
tiv e ct iv e er ve ec ti ve Ne it h ct i

Number
0%

6 4 2 0
ct iv e

14% 0%

14%

tiv e ff ec No t

er

ec ti ve

Ne it h

ef fe

ff ec

ef fe

ef fe

ef f

in e

ef f

ha t

No t

Ve r

ha t

Ve r

ha t

So m ew

So m ew

So m ew

Exhibit 45 Project management effectiveness from team member perspective In terms of the eleven other critical success factors, histograms for each CSF is presented in Exhibit 46 below. Aggregation of histogram summaries, resulted in the following observations.

An average of 77% of respondents characterized the role of Clear Goals and Communication as either very effective or somewhat effective, and, conversely, either neither, somewhat ineffective, or not effective by 23% of respondents.

An average of 70% of respondents considered Top Management Support, Change Management, and Business Process Reengineering as very effective or somewhat effective and, conversely, either neither, somewhat ineffective, or not effective by 30% of respondents.

Training and User Involvement were considered either very effective or somewhat effective by 66% of respondents and, conversely either neither, somewhat ineffective, or not effective by 34% of respondents.

Vendor Consultants, Teamwork and Composition, and Risk Management were considered either very effective or somewhat effective by 60% of respondents and, conversely, either neither, somewhat ineffective, or not effective by 40% of respondents.

IT Infrastructure was considered very effective or somewhat effective by 51% of respondents and, conversely, either neither, somewhat ineffective, or not effective by 49% of respondents.

So m ew

ha t

in e

ef fe

ct i

Weishaar First Draft; submitted 12/5/08

ve

76

Ve ry
Number
Number

Ve ry

10

15

10

15

20

0
0 5

So m ew ha t tiv e
ef fe c e ha t tiv ew

ef fe c
So m ef fe c

12%

12%
tiv e

ef fe ct iv e
So m ha t Ne i th

39%

So m ew ha t in ef fe c er Ne i th
ew

15%

09%

Weishaar First Draft; submitted 12/5/08

Actual effectiveness - BPR 58%

Actual effectiveness - IT Infrastructure

No t e

tiv

12%

30%

ef fe ct i ve

er in ef fe ct iv No e t ef fe ct iv e

3%

9%
Number

So m

Ve ry

ew

ef fe

ha t ef fe

ct

ive

So m

Ve ry

Number

10

15

10

15

20

0 12%

ew

ef fe c ha t So m ef fe c

tiv tiv

e e

30%

ct ive er

So m ew ha t in ef fe Ne i th

48%

36%

ew

ha t

Ne i th

15% 12%

18% 9%

Actual effectiveness - Training

ct ive No te ffe ct iv e

er in ef fe ct iv No e t ef fe ct iv e

6%

12%

Actual effectiveness - Change Mgmt

77

Ve ry

Number

So m

ew

ef fe

ha t

ct

So m

Ve ry
Number

10

15

ive

ew ha t ef fe ive

ef fe ct

10

15

20

21%

30%

ef fe

ct iv

39%

So m

ct ive er

So m ew ha t in ef fe Ne ith

45%

ew

ha t

Ne i th

er

18%

3%

in ef fe

Weishaar First Draft; submitted 12/5/08

ct iv

12%

18%

No e te ffe ct iv e

ct iv No te e ffe ct iv e

9%

Actu al effectiven ess - R isk M g m t

3%
Number

Ve ry

Number

10

15

So m

ew

ef fe

10 0 5

15

20

ha t

ct iv

ef fe

33%

18%

ct

So m ew

ive

27%

5 2%

ha t

Ne i th

er

6%

S u p p ort

in ef fe

15 %

24%

1 2%

ct ive No te ffe ct iv e

Ve ry So ef me fe ct wh ive at ef fe ct ive So Ne m ew i th er ha ti ne ff No e.. . te ffe cti ve

9%

Actu al effec tiven ess - Top M g m t

Actual effectiveness - Communication Actual effectiveness - Team w ork and C om position

78

3%

Actual effectiveness - User Involvement

15
Number

A ctual effectiveness - V endor

15 Number 10 5 0 42% Consultants 27% 18% 9% 3%

33%

33% 15% 12%

10 5 0
er ive ct ive No te ffe ct iv e e Ne i th ct iv

6%

Ve ry

Ve ry

ha t

ha t

ha t

ew

ew

ew

So m

Actual effectiveness - Clear goals

20 15
Number

55%

10 5 0

24% 9% 9% 3%

Ve ry

ew

So m

Exhibit 46 Effectiveness of specific CSFs in achieving ESI project success To summarize CSF-effectiveness across all projects (shown in Exhibit 47), a weight of 1 to 5 was attached to each of the five nominal descriptions, with very effective, receiving a weight of 5, somewhat effective receiving a weight of 4, and so on. In summary, participants reported Clear Goals as most effective in practice, followed closely by Communication, User Involvement, and Project Management, while IT Infrastructure and Change Management were identified as least effective.

So m

ew

ha t

er in ef fe ct iv No e t ef fe ct iv e

tiv

ef fe ct iv

ef fe c

ha t

Ne i th

So m

So m

So m

ew

er in e No ffe. t e .. f fe ct iv e

ive ct ef fe

ef fe

ef fe

in ef fe

ef fe

ha t

Ne ith

ct

ct

ive

Weishaar First Draft; submitted 12/5/08

79

Summary of CSF effectiveness

1.00 0.90 0.80 0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00
ag em en t

.77

.73

.78 .67 .70

.76

.75

.79

.72

.78

.77

.80

Count

Pr oj ec

Exhibit 47 Aggregation of CSF practice/effectiveness CSF To check for generalizeability, variability of CSF-effectiveness findings between projects was determined through the same scoring method used to summarize CSFfindings for all projects. A table showing the CSF-effectiveness scores for each project, (and standard deviation for each CSF), is presented in Exhibit 48.

As indicated in the table, on average, SOLAR project participants reported the highest level of effectiveness for nine out of eleven CSFs, followed closely by AS Trax, then Single Source, and TOTEM. Highest level of variability between projects was identified with regard to Training, Top Management Support, Teamwork & Composition, and Use of Vendor Consultants, which was consistent with findings from review of survey comments, project notebooks, and key leader input.

CSF Project Management BPR

BP R In Tr ai fr a s ni t r ng uc C tu ha re ng e M gm To Ri p t sk M gm Mg m t t C om Sup po m un rt ic at U T io s e ea n r m In w Ve vo o rk nd l or vem C on ent su lt C an le t ar G oa ls IT

an

SOLAR 0.76 0.83

AS Trax 0.82 0.72

Single Source 0.77 0.71

TOTEM 0.73 0.68

SD 0.04 0.06

Mean 0.77 0.73

Weishaar First Draft; submitted 12/5/08

80

Training IT Infrastructure Change Mgmt Risk Mgmt Top Mgmt Support Communication Teamwork User Involvement Vendor Consultant Clear Goals Mean Score

0.97 0.60 0.77 0.80 0.86 0.83 0.80 0.86 0.91 0.83 0.82

0.75 0.73 0.73 0.82 0.78 0.78 0.77 0.73 0.85 0.78 0.77

0.74 0.74 0.60 0.69 0.77 0.77 0.63 0.74 0.60 0.83 0.71

0.66 0.62 0.68 0.72 0.60 0.76 0.70 0.80 0.71 0.74 0.70

0.13 0.07 0.07 0.06 0.11 0.03 0.08 0.06 0.14 0.04 0.06

0.78 0.67 0.70 0.76 0.75 0.79 0.72 0.78 0.77 0.80 0.75

Exhibit 48 Comparison of CSF practice/effectiveness by project 5.2.4 Section 4: CSF Value/Significance Section 4 provided respondents with an opportunity to rank the twelve CSFs on a scale of 1 to 12, 1 being the highest rating. Analysis of survey results entailed applying a weight of 1 to 12 to each rating level, with the highest rating receiving a weight of 12, the second highest rating receiving a weight of 11, and so on. Descriptive statistics were generated and the mean of each CSF were compared and ranked, as presented in Exhibit 49. Aggregation of the relative ranking of CSFs yielded the following observations: Survey respondents consider Top Management Support, Clear Goals, Teamwork and Composition, and Project Management as most critical to achieving desired ESI project outcomes. Survey respondents believe User Involvement, Training, Business Process Reengineering, and Technology Infrastructure to be nearly as critical to achieving AAG ESI project success. Survey respondents reported use of Vendor Consultants, and Risk and Change Management as least critical relative to other CSFs.

Weishaar First Draft; submitted 12/5/08

81

CSF - value/significance
0.80 0.70 0.60

Count

0.50 0.40 0.30 0.20 0.10 0.00

BP R Tr ai ni ra ng st ru ct Ch ur an e ge M gm To Ri t sk p M gm Mgm t Su t Co pp m or m un t ic at io Us Te am n er w In Ve or vo nd k lv em or Co en ns t ul ta nt Cl s ea rG oa ls IT In f
SOLAR 0.87 0.64 AS Trax 0.62 0.53 Single Source 0.68 0.75 TOTEM 0.63 0.57 SD 0.12 0.09 Mean 0.70 0.63

As indicated previously, to verify generalizeability, variability of findings between individual projects was checked. A summary of CSF value/significance ranking per project is shown in Exhibit 50. As indicated in the table, greatest degree of variability between projects was identified with regard to Risk Management, followed by Use of Vendor Consultants, Training, Project Management, Change Management, IT Infrastructure, and Top Management Support.

The high level of variability in CSF value/significance among and between project teams could well be due to ineffective structure of survey question. Additional study is needed to develop more valid inferences.

Pr oj ec

Exhibit 49 Relative ranking of value/significance of 12 CSFs CSF

CSF Project Management BPR

tM

an

ag em en

Weishaar First Draft; submitted 12/5/08

82

Training IT Infrastructure Change Mgmt Risk Mgmt Top Mgmt Support Communication Teamwork User Involvement Vendor Consultants Clear Goals Mean Score

0.88 0.72 0.57 0.79 0.90 0.76 0.67 0.70 0.68 0.90 0.76

0.60 0.50 0.37 0.42 0.67 0.58 0.67 0.54 0.42 0.65 0.55

0.66 0.58 0.40 0.50 0.67 0.63 0.77 0.60 0.33 0.72 0.61

0.59 0.45 0.60 0.61 0.63 0.49 0.75 0.66 0.55 0.68 0.60

0.14 0.12 0.12 0.16 0.12 0.11 0.05 0.07 0.15 0.11 0.09

0.68 0.56 0.49 0.58 0.72 0.62 0.71 0.62 0.50 0.74

Exhibit 50 Summary of CSF value/significance raking per project Exhibit 51 below a gap analysis, (comparing results from Section 3 with results from Section 4). Inferences were made through general inductive approach based on analysis of survey comments, interview notes, and project notebook documents and not statistically validated. Largest gap Use of Vendor Consultants is likely due to absence of consultants on Single Source project. Second largest gap - Top Management Support is likely due to perception of lack of top management support as reported by TOTEM project participants. Gap related to Change Management likely due to value ranking relative to a number of other CSFs, which were rated higher. Also, actual effectiveness score was relatively high in light of anecdotal comments. For example, while one AS Trax participant observed, There was resistance to the very end. It has resulted in some key departments severely struggling now, because they never accepted TRAX was going to happen. The entire division pays the price, the cumulative score for change management effectiveness among all AS Trax participants was nevertheless relatively high (0.73) Gap related to Teamwork and Composition likely due perception reported by multiple teams that not enough time is spent in pre-planning to ensure that all the right people are involved.

Weishaar First Draft; submitted 12/5/08

83

CSF Project Management BPR Training IT Infrastructure Change Mgmt Risk Mgmt Top Mgmt Support Communication Teamwork User Involvement Vendor Consultant Clear Goals MeanScore

Actual 0.77 0.73 0.78 0.67 0.70 0.76 0.75 0.79 0.72 0.78 0.77 0.80 0.75

Valued 0.86 0.74 0.69 0.70 0.58 0.70 0.89 0.76 0.84 0.75 0.62 0.90
0.76

SD 0.06 0.00 0.06 0.02 0.08 0.04 0.10 0.02 0.08 0.03 0.11 0.07
0.01

Value Difference 0.09 0.00 -0.09 0.02 -0.12 -0.05 0.14 -0.02 0.12 -0.04 -0.15 0.10 0.01

Exhibit 51 Summary of disparity between CSF practice/value 5.2.5 Section 5: Other Factors Hindering AAG ESI Project Success Section 5: The final question of the AAG ESI Project Survey provided respondents with the opportunity to indicate any other factors not previously addressed in the survey that might hinder project success. Eleven participants, (33%), elected to provide input, which included comments such as: Poor choice of product and lack of project management skills at top level and the ownership of the project seems to be the tricky one to navigate at times. The demonstration[of] Single Source [capabilities] was very impressive, however, we did not utilize any of the options to make it a more effective product to the end user. I am not sure if it was a decision made based on money, or whether we were under time constraints [that] we put it out as is. The concept of an IT FREEZE date being honored by the biz side. Vendor such as Trax giving us bad quality releases, obviously not testing them at all. Incompatibility of vendor package with ITs Quality Assurance software. Having 3 different PMs. M&E experienced a lot of difficulties during our project phase. We did not have Sponsor involvement; adequate resources in the form of people or IT support; and we had very ineffective training from the vendor. Our team consisted of a maximum of three people and we had to teach ourselves how the application worked in order to implement it fully for the division.

Weishaar First Draft; submitted 12/5/08

84

To complete the [AS Trax] project on time, more resources could have been used to ensure a complete understanding of the software was obtained prior to determining changes to the business model. Post-implementation follow through. The difficulty in getting the organization to understand the long-term benefits of CRM, which SOLAR is foundational for, continues to be an important challenge. Burning platform the case for change must be clear and compelling. there should be more full-time focals assigned to the Trax project, per the original business case. The idea to use less focals and have SMEs assist them did not work. Therefore the focals were overwhelmed and limited with the quality of work they could do per time constraints. This was a tough project. It was originally scheduled for 18 months but ended up 2 years long. We had three IT project managers, three QA analysts which made it hard to spool up everyone each time there was a change. Despite that, the project was a huge success. Our last PM from Point B, did and excellent job of pulling things together - holding people accountable for their pieces and it worked out fine. Above participant comments were analyzed through general inductive method and categorized along with CSFs based on relevance to critical issues as identified through key leader interviews. Shown in Exhibit 52, critical issues/CSFs were then ranked based on the sums of all related input.
Leader Issue as Function of CSF
Vision/Value Prop Owner

CSF
Clear Goals Top Mgmt Support User Involvement, Communicati on, Clear Goals Top Mgmt Support Top Mgmt Support Communicati on

Leade r Value 12 12

Difference (x10) 1 1.4

Project Participa nt Issue 3 2

Most Critica l Issue 16 16

Buy-in Successor Mandate Communication

10 10 9 11

1 1.4 1.4 0

15 12

12 11

Weishaar First Draft; submitted 12/5/08

85

Change Mgmt Top Mgmt Support Project Mgmt Project Mgmt BPR, Project Mgmt User Involvement, IT Infrastructure Project Mgmt Vendor Consultants Project Mgmt

Change Mgmt Provisioned Phases PM processes not a function of time Decision Rights Process Mapping

9 6 2 6 5

0 1.4 0.9 0.9 0.9

1 2 6 1 1

10 10 9 8 7

AP/SCM/IT/Lean Authorization Path Vendor Consultant Bounded by BU

3 3 3 3

0.2 0.9 0 0.9

3 1 1

6 5 4 4

5.3

Exhibit 52 Categorization/aggregation of leader issues, CSFs, and comments Project Sponsor Interviews

As indicated in Section 3.2.3 above, the purpose of Step 3 was to discuss AAG ESI Project survey with thesis project sponsors (5 key leaders a subset of 12 key leader sample set) to validate the researchers interpretation of survey findings and narrow the focus of the remaining sections. Preliminary conclusions (answers to original research questions based on findings from literature review, interviews, and survey results as summarized in Exhibit 53 below) and recommendations (preliminary platform for best practices framework) was explored and refined to develop final platform for best practices framework. Thesis project sponsor interview notes were captured (Appendix A.22). A summary of thesis project sponsor responses is presented below in Exhibit 53.
1. How do organization leaders and other key stakeholders characterize typical outcomes of AAG ESI projects. With no more 42% of AAG ESI project participants describing their project outcomes as highly successful in terms of triple constraints, and no more than 50% describing their project outcomes as highly successful in terms of the original project vision, combined with the consensus among key leaders regarding project outcomes the thesis premise that outcomes of AAG ESI projects are less than successful would appear to be validated.

Weishaar First Draft; submitted 12/5/08

86

2.

What are the typical factors that hinder ESI success? More than half non-technical, related to either organizational or project management context, i.e., poor change management, inadequate knowledge transfer, lack of top management support, lack of user involvement, insufficient risk management, and inability to align business processes.

3.

What are the primary typical factors that hinder AAG ESI success? Unclear vision/value proposition (lack of clear goals); lack or unclear of ownership; lack of buy-in; unclear or no 'successor' of subsequent phases; lack of mandate; insufficient or ineffective communication.

4.

What are best practices for improving EFI project outcomes? Relevant CSFs: organization context - develop/communicate clear goal/value proposition/mandate to all stakeholders; project management context - identify all stakeholders, standardize effective decision-making/risk management/business processes

5.

How can the organization incorporate best practices to improve ESI project outcomes? Develop/utilize ESI project process map incorporating/aligning identified CSF activities with project process groups/phases. Include as part of AAG Project Management "nonnegotiable" requirements/standards

Exhibit 53 Summary of study conclusions/answers to research questions

6.0

CONCLUSIONS/RECOMMENDATIONS

The findings of this study confirmed what many ESI project participants have experienced and reported ESI projects are both stressful and extremely challenging. Even companies operating at the highest level of project management maturity report struggles and setbacks when it comes to ESI projects. Project participants can go through great pains to make sure every precaution is taken to utilize every available standard project management tool and technique during initiating, planning, executing, and monitoring & controlling of an ESI project, and the outcome may still fall short of expectations.

Findings from this study revealed that AAG ESI project participants have a lot to proud of. While some disparities between practice/effectiveness and value/significance of critical success factors were identified, given the relatively low level of project and organizational change management maturity just two years ago (when many of these

Weishaar First Draft; submitted 12/5/08

87

projects were initiated), findings of 42% highly successful and 50% moderately successful is truly remarkable. While opportunities for improvement were discovered in areas such as top management support and communication, a number of respondents included comments that expressed appreciation for the dedication, commitment, resourcefulness, and tenacity of project managers, team members and other project participants.

A key objective of this project was to identify the key factors that hinder AAG ESI project success so remedies can be created to ensure that participants on future ESI projects will experience the pride and satisfaction that comes with knowing, without any doubt, that their project was considered an unequivocal success. 6.1 CSF Gaps

A gap analysis was conducted to identify specific opportunities for improvement of CSF effectiveness. As seen in Exhibit 52, Section 5.2.5, the study revealed the largest gaps between the practice/effectiveness and value/significance of Clear Goals, Top Management Support, User Involvement, and Communication.

6.1.1 Clear Goals and Definition of Success In addition to the gap associated with Clear Goals, study findings suggested considerable ambivalence among project participants with regard to AAG ESI Project outcomes. A number of study respondents commented that sometime during the course of the project, original objectives relating to the enterprise aspect of the system implementation became obscured or lost altogether in the pursuit of achieving predetermined timelines. In other words, while a project outcome might be a successful system cutover on a planned day, the project may still not be considered

Weishaar First Draft; submitted 12/5/08

88

successful if hand-off to another department did not occur as described in early discussions of software capabilities and expectations.

During interviews, a few key leaders described other circumstances that often result in obscured expectations, goals, and/or definition of success. For example, sometimes, to sell an ESI project to upline decision-makers, it becomes necessary to scale back goals and objectives to make the project seem more feasible. While perhaps effective in obtaining necessary approvals and resources, without a plan for enterprise-wide deployment, the ROI on the purchase of the robust enterprise solution will certainly not be realized.

There is little doubt that clear goals and objectives are essential to guide any endeavor, large or small. But particularly when the project entails an ongoing organizational effort around ESI (since it usually exceeds the time, scope, and costs of typical business projects), a clear definition of success must be articulated and shared with the entire organization, along with an explanation of why the ES is being implemented and what critical business needs the system will address.

As indicated in Section 2.1, recent research findings indicate that organizations are realizing the benefit of a phased approach to ESI implementation. Such an approach would likely entail management as a program, consisting of multiple projects that are carefully coordinated and aligned by a program manager. Program management is a function of Project Management (under the auspices of the PMO), and, as such, based on Esteves and Pastors (2000) unified CSF model (Exhibit 54), falls within the

Weishaar First Draft; submitted 12/5/08

89

contextual domain of Organization/Strategy, which will be discussed later in this section.

Exhibit 54 Estevez and Pastors Unified CSF Model (2000) 6.1.2 Top Management Support Several issues raised during key leader interviews are considered functions of the CSF, Top Management Support, including clear owner, clear successor, provisioned phases, and clear mandate. AAG ESI Project Survey respondents also addressed the issue of top management support with their answers to multiple choice questions, including comments such as, IT management great; M&E management not so great. Various M&E departments still in denial up to very end (AS Trax). Many major initiatives were in work concurrently which limited support of top management earlier in the project. Top management couldn't make up their mind as to whether Totem was an enterprise solution or not. I think more could have been done to bring department Managers and Directors on board with the impending change and therefore provide better support to the TRAX team.

Weishaar First Draft; submitted 12/5/08

90

Widely recognized as a key success factors for almost any medium to large project, based on the literature, top management support is considered even more critical to the success of ESI projects. At the very least, top management support has two main facets: (1) providing leadership; and (2) providing necessary resources and authority power for project managers. If effective, top management support for ESI projects provides for all the concerns raised by leaders and project participants above.

Whether explicit as a mandate, or implicit in behaviors and actions, senior leaders are key to establishing cross-boundary commitment. They must act as advocate and evangelist for the ESI project to ensure acceptance and to help embed the project effort into the organizations culture. To demonstrate support and ensure project success, senior management should monitor the implementation progress, provide clear direction, and be willing to allow for a mindset change by accepting that a lot of learning has to be done at all levels, including themselves (Bhatti, 2005).

Ensuring top management support is a function of Project Management (under the auspices of the PMO), and, as such, based on Esteves and Pastors (2000) unified CSF model (Exhibit 54), falls within the contextual domain of Organization/Strategy, which will be discussed later in this section.

6.1.3 User Involvement/Communication User involvement refers to a psychological state of the individual and is defined as the importance and personal relevance of a system to a user. It is also defined as the users participation in the implementation process. There are two areas for user involvement when the company decides to implement an ES: (1) user involvement in

Weishaar First Draft; submitted 12/5/08

91

the stage of definition of the companys ES needs, and (2) user participation in the implementation of ES. The functions of the ES rely on the user to use the system after going live, but the user is also a significant factor in the implementation 1 System implementations often impose a threat of reduced control over a users work. Enhancement of control through involvement can be accomplished by offering choices to the employee, involving them with meaningful decisions during the systems process; laying the groundwork for predictability by painting a complete and accurate picture in advance of the users exposure to the system during and after implementation; allowing the employee to assume some measure of responsibility during the system design and implementation process making them accountable for the results of specific tasks integral to the implementation process and encouraging shared ownership of the project; offering opportunities to reduce or escape from the stress that is inherent in a system implementation project.

Increasing user participation in one or more of these dimensions enhances postdevelopment user involvement and attitude. The effectiveness of involvement as a success factor is also be enhanced if implementers recognize that: A user is involved when he or she considers a system to be both important and personally relevant. In addition, if an individual believes that the system is personally relevant, he will be more likely to form a positive attitude toward the system since attitudes are generally formed on the basis of beliefs. The strength of an individuals involvement is directly related to the extremity of his or her attitude toward the system. A high level of involvement could drive an extremely positive or extremely negative attitude. A low level of involvement, however, leaves a person susceptible to other influencers (e.g.,

Weishaar First Draft; submitted 12/5/08

92

persuasive forces, factual arguments). With increased user involvement and a positive attitude, users will have an increased desire to participate in development. 1 6.1.4 Communication Communication is one of most challenging and difficult tasks in any ESI project. It is considered a critical success factors for the implementation of ESs by many authors. It is essential for creating an understanding, an approval of the implementation, and sharing information between the project team and communicating the results and the goals in each implementation stage to the whole organization. In addition to gaining approval and user acceptance, the communication will allow the implementation to initiate the necessary final acceptance. The communication be consistent and continuous, starting early in the ESI project and include an overview of the system and the reason for implementing it.

Share Approach Document that provided a history of information systems implementation, the strategic importance of the initiative, the projects organizational structure, goals, roles, responsibilities of the organizational layers, obstacles and the project budget. Resting the responsibility of communication with the projects highest project team emphasizes the importance of this project component.

General success factors related to internal (project team) and external (project to organization) communication as reported by practitioners include: setting appropriate expectations; communicating with all affected parties; using multiple means of communication to reach multiple audiences (e.g., email is fast, efficient, self documenting and relatively non-intrusive; face-to-face communication is vital for

Weishaar First Draft; submitted 12/5/08

93

teamwork and collegiality); providing advance accurate notification of changes allowing time for users to move past an emotional reaction prior to initiation of the change; providing project members with immediate and full-time access to project managers; frequent, regular progress report meetings with directly impacted users; frequent, regular meetings with project managers to discuss developments and tactical concerns; offering opportunities for private communication as appropriate; project-wide meetings at important milestones; and regular unbiased reporting of project progress.

Articulation of a clear vision and definition of success is both an organizational and a project management concern, as is obtaining and maintaining top management support, ensuring user involvement and executing an effective communication plan.

6.2

Organizational Context

As Chetcuti states, Contextual factors exert different pressures on what is deemed central to success (2008, p. 5). Key contextual factors under consideration throughout this study related to organization characteristics and project management maturity, which have undergone significant change not just during the past two years but during the past two months! With the start of a new year comes the kick-off of a new set of key initiatives, within the context of an increasingly sophisticated organizational structure and what a rapidly maturing PMO.

Helper Groups include:

Change Management template to identify potential issues

Weishaar First Draft; submitted 12/5/08

94

LEAN Looking internally rather than externally; changing the way we do things; reducing waste; metrics; focus on improvement through process reengineering rather than technology more rapid, less costly; technology adds complexity; 20 workshops in 2009; generates projects and gain insights to improve project outcomes

Audit Programs - Risk Management; 2nd set of eyes; preventative and proactive as opposed to reactive

Legal Awareness and documentation; data and data flows to be mapped

6.2.1 Project Management Context Initiative projects starting out in 2009 will benefit from the leadership of a dedicated program manager (a.k.a. Principal Consultant, IT and Strategy Management). The role of the program manager is changing how we run our programs.

6.3

Implementation Strategy

Through utilization of SME from Helper Groups, many CSFs being addressed. Whats missing is a standardized framework integrating the roles of the Helper Groups.

6.3.1 ESI Framework Tying CSFs to PM process groups, this framework ensures that the various CSFs are considered at a very early stage of the project so that the integrated-enterprise system can be designed and implemented. The systematic consideration of the various CSFs in the life cycle of the implementation of the integrated-enterprise systems greatly reduces the risk of implementation failures. Risks and potential problem areas are identified and mitigation plans can be put in place.

Weishaar First Draft; submitted 12/5/08

95

Post project deliverables - Readiness assessment/organization fit and flexibility;

6.4

Project Contribution

This project has provided insight into the role of enterprise systems in helping organizations gain competitive advantage in todays highly competitive and rapidly changing world through continuous improvement of support and value-creation processes. Employing a mixed method approach, utilizing both qualitative and quantitative data collection methodologies, key impediments to successful implementation of AAG ESI projects were determined. A register of best practices was developed based on comprehensive literature survey. A unified CSF model (Esteves and Pastor, 2000) was offered as a platform for development of a CSF Framework which, along with an AAG ESI Project Orientation PowerPoint deck and a Risk Management Toolkit. A strategy for implementation was presented entailing completion of post-project deliverables (through collaboration with key stakeholders from the PMO and Helper Groups) for upload onto PMO server.

6.5

Lessons Learned

Lessons learned were primarily centered on timing of thesis project. The one-two punch of year-end project deliverables and holiday vacation plans was extremely challenging. Another significant challenge was associated with lack of familiarity with key components of a research paper, including, but not limited to: accepted research designs and methodologies, http://www.socialresearchmethods.net/kb/contents.php

Weishaar First Draft; submitted 12/5/08

96

A table of lessons learned is presented in Exhibit 55 below.

Process Group/Phase
Initiating

Lesson
Discuss possible topics with sponsors far ahead of time Select and receive UAA approval for topic early Avoid scheduling thesis writing during holidays Review statistical analysis tools and techniques Allow at least four months for thesis writing Allow at least two months for literature survey Allow one month to research and design quantitative data collection instrument Allow at least one month to refine analysis Include time to test survey instrument prior to distribution Shortcut literature review by starting with topic bibliographies Limit summary of individual studies to one or two sentences Assess individual project findings as thoroughly as aggregated results Collaborate more effectively with fellow cohorts

Planning

Executing

Monitoring & Controlling Closure

Check in more often with project sponsors Stop writing Exhibit 56 PM 685 Thesis Project Lessons Learned

Weishaar First Draft; submitted 12/5/08

97

6.6

Final Words

Future work in this area Got a process work the process! Spend more time in discovery; dont be afraid to wallow in unknown Lessons Learned transfer of knowledge more important than ever

Weishaar First Draft; submitted 12/5/08

98

REFERENCES
Bhatti, T. R., (2005). Critical Success Factors for the Implementation of Enterprise Resource Planning (ERP); Empirical Validation, The Second International Conference on Innovation in Information Technology, September, Dubai, UAE. Retrieved on November 11, 2008 from http://www.itinnovations.ae/iit005/proceedings/articles/F_4_IIT05_Bhatti.pdf Bolles, D. and Hubbard, D. (2007). The Power of Enterprise-Wide Project Management, AMACOM, a division of American Management Association, New York, NY. Carr, N., (2003). IT Doesnt Matter, Harvard Business Review, Harvard Business School Publishing. Retrieved on November 1, 2008 from UAA Library Consortium Cash, J., Earl, M. and Morison, R., (2008). Teaming Up to Crack Innovation & Enterprise Integration, Harvard Business Review, Harvard Business School Publishing, November. Retrieved on November 9, 2008 from UAA Library Consortium. Davenport, T., (1998). Putting the Enterprise into the Enterprise System, Harvard Business Review, Harvard Business School Publishing, July-August. Retrieved on November 9, 2008 from UAA Library Consortium. Davenport, T., (2000). Mission Critical: Realizing the Promise of Enterprise Systems, Harvard Business School Press, Boston, MA.

Weishaar First Draft; submitted 12/5/08

99

Evans, J. and Lindsay, W., (2008). Managing for Quality and Performance Excellence, Thomson Southwestern, Mason, OH. 7th Ed. Gack, Gary A. (2004, September 15). Applying Six Sigma to Software Implementation Projects. iSixSigma.com. Retrieved December 23, 2008, from http://software.isixsigma.com/library/content/c040915b.asp Gebauer, J. and Lee, F., (2008). Enterprise System Flexibility and Implementation Strategies: Aligning Theory with Evidence from a Case Study, Information Systems Management, Vol. 25, Issue 1, December, pp. 71-82. Retrieved on November 9, 2008 from UAA Library Consortium Hong, K. and Kim, Y., (2001). The critical success factors for ERP implementation: and organizational fit perspective, Information and Management, Vol. 40, Issue 1, October, pp. 25 40.

Hwang, Y., (2005). Investigating enterprise systems adoption, uncertainty avoidance, intrinsic motivation, and the technology acceptance model, European Journal of Information Systems. Basingstoke, June, Vol. 14, Issue. 2; pg. 150. Retrieved on December 23, 2008, from ProQuest through
UAA Library Consortium Kenett, R. and Raphaeli, O., (2008). Multivariate methods in enterprise system implementation, risk management and change management, International Journal of Risk Assessment and Management, Vol. 9, No. 3. Retrieved on November 22, 2008 from ScienceDirect at www.sciencedirect.com Law, C. and Ngai, E., (2007). ERP systems adoption: an exploratory study of the organizational factors and impacts of ERP success, Information and Management, Vol. 44, pp. 418-432. Retrieved on November 11, 2008 from ScienceDirect at www.sciencedirect.com Lengnick-Hall, C., Lengnick-Hall, L. and Abdinnour-Helm, S., (2004). The role of social and intellectual capital in achieving competitive advantage through enterprise resource planning (ERP) systems, Journal of Engineering and Technology Management, Vol. 21, pp. 307-330. Retrieved on November 9, 2008 from Science Direct at www.sciencedirect.com Lorincz, P., (2007). Evolution of Enterprise Systems, International Symposium on Logistics and Industrial Informatics, September 13-15, Wildau, Germany. Retrieved on November 29, 2008 from IEEE Xplore. Lusk, S., Paley, S., and Spanyi, A., (2005). The Evolution of Business Process Management as a Professional Discipline, BP Trends. Retrieved on November 30, 2008 from UAA Library Consortium. Mathrani, S. and Viehland, D., (2007) "Using Knowledge-based Processes to Improve Enterprise System Effectiveness, ACIS 2007 Proceedings. Paper 19. Retrieved on November 5, 2008 through Google Search from http://aisel.aisnet.org/acis2007/19

Weishaar First Draft; submitted 12/5/08

100

Mihailescu, D., Carlsson, S., and Mihailescu, M., (2007). Evaluating Enterprise Systems Implementation Methodologies in Action: Focusing Formalized and Situational Aspects, The Electronic Journal Information Systems Evaluation, Vol. 10, Iss. 1, pp 83 90. Retrieved on December 12, 2008 through Google search from http://www.ejise.com/volume-10/volume10-issue1/Mihailescu.pdf Monks, R., (2008). Secrets of Successful Enterprise Project Management Implementations 2008 World Championship Projects and Portfolios, Primavera 25th Annual Conference, Las Vegas, NV. Retrieved on December 5, 2008 through Google Search from http://www.primavera.com/conference/cd/tracks/best_practices/Secrets_Suc cessful.pdf Morton, N. and Hu, Q., (2008). Implications of the fit between organizational structure and ERP; A structural contingency theory perspective, International Journal of Information Management, Vol. 28, pp. 391-402. Retrieved on November 11, 2008 from ScienceDirect at www.sciencedirect.com Nicholas, J. and Steyn, H., (2008). Project Management for Business, Engineering, and Technology, Principles and Practice, Butterworth-Heinemann, New York, NY. 3rd Ed. Papows, J., (1998). Enterpise.com: Market Leadership in the Information Age, Perseus Books (member of the Perseus Book Group, available through HarperCollins Publishers, New York, NY. Parr, A. and Shanks, G., (2000). A Taxonomy of ERP Implementation Approaches, Proceedings of the 33rd Hawaii International Conference on System Sciences. Retrieved on December 12, 2008 from IEEE Xplore, http://ieeexplore.ieee.org/stamp/stamp.jsp? arnumber=926908&isnumber=20043 Project Management Insitute (2004). A Guide to the Project Management Body of Knowledge (PMBOK Guide), American National Standard (ANSI/PMI 99-0012004, Newtown Square, PA. 3rd Edition. Puterbaugh, J., (2003). Project Post-mortem, Multi Channel Merchant, October. Retrieved through Google search on 11/12/2008 from http://multichannelmerchant.com/opsandfulfillment/ship/fulfillment_project _postmortem/ Robey, D., Ross, J. and Boudreau, M., (2000). Learning to Implement Enterprise Systems: An Exploratory Study of the Dialectics of Change, MIT Center for Information Systems Research. Retrieved through Google Search on November 11, 2008 from http://www.cis.gsu.edu/~drobey/Cis8160/jmisv18.PDF Sedera, D., (2007). Stakeholder View of Enterprise System Knowledge Management Process, 11th Pacific-Asia Conference on Information Systems. Retrieved on November 9, 2008 from http://eprints.qut.edu.au

Weishaar First Draft; submitted 12/5/08

101

Smith, H. and Fingar, P., (2006). Business Process Management: The Third Wave The breakthrough that redefines competitive advantage for the next fifty years, Meghan Kiffer Press, Tampa, FL. Soliman, F. and Youssef, M., (1998). The role of SAP software in business process reengineering, International Journal of Operations & Production Management, Vol. 18, Iss. 9/10, pp. 886 895. Retrieved from UAA Library Consortium on November 10, 2008. Stephens, Todd R., (2004). The Future of Information Management, The Data Administration Newsletter. Retrieved on November 29, 2008, through Google, http://www.tdan.com/view-special-features/5406 Strong, D. and Volkoff, O. (2004). A Roadmap for Enterprise System Implementation, Computer, IEEE Computer Society, June, pp. 22 29. Retrieved on December 12, 2008 from IEEE Xplore, http://ieeexplore.ieee.org/stamp/stamp.jsp? arnumber=01306380&isnumber=28995 Thomas, D. R., A General Inductive Approach for Analyzing Qualitative Evaluation Data, The American Journal of Evaluation, Volume 27, Issue 2, pp. 237-246. Retrieved on November 17, 2008 through UAA Library Consortium Viehland, D. and Shakir, M., (2005). Making sense of enterprise systems implementation, University of Auckland Business Review, Vol. 7, No. 2. Retrieved on November 9, 2008 from Science Direct

Weishaar First Draft; submitted 12/5/08

102

Weishaar First Draft; submitted 12/5/08

103