BYU Dining Services Online Training

ADDIE Instructional Design Model

A Structured Training Methodology
The ADDIE instructional design model provides a step-by-step process that helps SAS Online Training plan and create online training. The ADDIE design model revolves around the following five components: 1. 2. 3. 4. 5. Analysis Design Development Implementation Evaluation

These five stages of the ADDIE model encompass the entire training development process; from the time someone first asks, "What do people need to learn?" all the way to the point where someone actually measures, "Did people learn what they needed?"

The ADDIE Process

The ADDIE instructional design model forms a roadmap for the entire training project. SAS Online Training uses this popular instructional design model to help our clients analyze their training needs, design and develop training materials, implement training, and evaluate its effectiveness. Sometimes, SAS Online Training works directly with a client's training specialists, who have studied the ADDIE model. However, we're often contacted by directors and managers who know their department has a training need, but they don't know much about the instructional design process. SAS Online Training specialists share a flowchart of the ADDIE instructional design model with our clients. It helps our clients locate where they are in the training project lifecycle, and it provides a common language for us to discuss the project.

The ADDIE Analysis Phase

The First Steps to Quality Training The first phase in the ADDIE instructional design model is the analysis phase. Great training programs don't come together by accident. They require planning and analysis. You'll produce the best training if you first analyze three important areas: The business goals you want to achieve The material that must be taught The learners' current capabilities

The Value of a Needs Analysis SAS training is contacted by departments that have important and urgent training projects. Sometimes, a client will ask SAS Online Training to skip the analysis phase and jump straight to training development. They'll say, "Let's get people writing training materials now!" However, that can be a risky and very costly approach. Carpenters utilize the old adage, "measure twice; cut once." Even though carpenters are talking about wood, and we're talking about training, we share a common goal do it right the first time. So, we could change the carpenter's old adage to fit the ADDIE methodology. "Analyze fully; design once." The ADDIE analysis phase serves a major role in the quality assurance process. It defines the project's needs and ways to measure its success. If you skip the ADDIE analysis phase, you can easily introduce mistaken assumptions into the project. Wrong focusthe course content may not address the department's business needs Too easy or too hardthe course could bore or frustrate the learners Incomplete, redundant, or inaccurate contentthe course might not teach the correct material If you rush to development, you may not catch those errors until you post the module to the BYU Learning Management System (LMS). At that point, it can be costly to fix or redesign the course. In essence, the training needs analysis is time well-spent. Who Guides the Needs Analysis? During the needs analysis phase, the training specialist may speak with many people to learn about the project and its overall goals. Here are just a few examples of individuals who can provide information: Project sponsors (executives or senior leadership)who can discuss the business goals and objectives Subject matter expertswho can describe undocumented knowledge Representative members of the target audiencewho can demonstrate their current skills and behaviors It is often critical to work with anyone who will be impacted by or have influence on the final training product. Questions that Drive the Analysis When you start your project with a training needs analysis, you collect critical information about business needs, learners' capabilities, and course content. Here are some of the questions that a training specialist may ask during the ADDIE analysis phase: What are the business needs driving this training project?

What are the goals and objectives for this training project? How will you define success for both the learner and the project? How will you measure that success? Who is the intended training audience? What do the members of the learning audience already know? What do they need to learn? What resources are already available? The training specialist uses the answers to these, and any possible combination of other questions, to write the course's performance objectives. Steps in the Needs Analysis In this section, you can learn about the five steps that SAS Online Training's training specialists perform during the ADDIE analysis phase: Discover any existing materials Define measurable business goals Conduct an instructional analysis Analyze learners and contexts Write learning objectives Some of these steps can happen concurrently, but generally our training specialists begin with the discovery process. Begin with the Discovery Process When SAS Online Training's training specialists conduct a training needs analysis, they begin with the discovery process. We ask our clients to share with us any materials or documents that would be relevant to the training project. In terms of the ADDIE methodology, the training specialist uses these documents to understand the client's current situation and the training project's context. So, the discovery process serves as the foundation for the entire ADDIE model. Collect the Information We believe that the discovery process should happen at the beginning of the ADDIE analysis phase. When our training specialists start on a project, they ask the client to assemble the relevant documents for the project. Each training project leads to a unique discovery process. Usually, the training specialist asks the client a series of open-ended questions about existing resources and documents, so that the discovery process becomes a discussion between the training specialist and the client. SAS Online Training collects as much information as possible at the beginning of the project. We believe that it's better to have plenty of data than not enough.

Analyze the Resources Our training specialists carefully analyze the documents that they collect. They take notes and prepare their questions for subject matter experts. This self-study process helps our training specialists quickly learn about the project and its needs without placing a large time demand on the client's subject matter experts (SMEs). Our training specialists will take these questions to subject matter experts later in the ADDIE analysis process. We know that SMEs are busy people, and we acknowledge that by making efficient use of their time. Confirm the Research At the end of the discovery phase, the training specialists contact the client and present a list of the documents they have received and reviewed. We ask our clients to confirm that we have collected and reviewed the proper resources for the project. The Benefits of the Discovery Process The discovery process collects key information at the start of the project. Ensures the training specialists have access to key documents and resources at the start of the project Places learning and knowledge gathering at the front of the project Brings the training specialist up-to-speed on the project Reduces the time commitment required from the client's subject matter experts (SMEs) Limits the risk that training specialists might move forward without critical information Saves time and backtracking later in the project Serves as the project's first major quality checkpoint

ADDIE Analysis: Align with Business Goals

Define Measurable Business Goals If a training project is to have long-lasting value, then it should be connected to specific business results that you can measure. In this step of the ADDIE analysis phase, SAS Online Training helps clients identify the key business metrics they want to improve through the project. Training projects can't solve every business problem, but a good training project should be able to articulate its goals in ways that can be measured.

Envision the Successful Project We believe that its important to align each training project with your department's business goals. When you define what success will look like, youre more likely to achieve those results. So, if the training project will be successful, how will it affect people and how will it impact the department's bottom line? Vague business goalwe want to teach a new process to our team members Specific business goalwe want to improve team member productivity by encouraging them to use best practices Highly specific business goalwe want to improve team member productivity by 5% through increased adoption of these three best-practice procedures It's easy to write training around vague business goals, but it's difficult to measure their impact on your department. It's also nearly impossible to measure their return on investment. When you map the project to specific business goals, you ensure that the project focuses on measurable results. You'll also be able to present your project in a way that will appeal to SAS leadership. Choose Achievable Goals SAS Online Training's training specialists help clients select achievable goals for their training projects. We believe that clients should set the project's business goals, since they know their needs best. We encourage a dialogue between the following groups: The project's manager Senior leadership or executives who are supporting the project Others involved in the project's success (BYU Risk Management, Human Resource Development, etc.) During the discussion, our training specialists share their knowledge and their experience with training projects. We guide the discussion and encourage the participants to address the tough questions: Is the proposed goal realistic?neither too high nor too low Will the project have enough time for development and implementation? What internal elements will need to support the project? Have those elements committed to the project? The project's goals must match the client's commitment to them. We've seen departments set high training project goals that became unrealisticbecause the project didn't receive enough time, resources, or cultural support.

Be Willing to Scale Some projects are unrealistic in their scope. Let clients choose between options when projects demand a larger commitment than currently is feasible. Offer advice that help clients make informed decisions about their training projects. Allow more time for course development Add additional training specialists to the project Reduce the scope Set a lower performance metric goal What Do Learners Already Know? During a learner analysis, the training specialist examines the learners as a group. Sometimes this step is called a training audience analysis or even just an audience analysis. In this step, the training specialist examines the learners' current knowledge and capabilities. What do the learners already know and have the ability to do? The training specialist uses the information from the learner analysis to create a course that focuses on your learners' actual needs. If you don't conduct the learner analysis, you'll have to make assumptions about the learners' current capabilities. Sometimes, if you are very familiar with your audience, you can make informed guesses. However, some assumptions can lead to unexpected surprises when you launch the training project. Keep Learners Involved You want a course that challenges but doesn't overwhelm your learners. If you don't take time to study the learners and their contexts, you could make a course that bores learners because it's too basic. You could also create a course that's impossibly difficult for a group of learners because it might assume that learners know more than they really do. It is not only important to know what material you're going to teach, but also what your learners need to be taught. Give Learners Clear Directions Have you ever used an online map service to create a set of driving directions? When you enter your starting point and your destination, the service provides a series of step-by-step instructions that will guide you to your goal. Training is no different; you need a roadmap and directions. Your learner analysis provides your project's starting location Your organizations business goals provide the destination Your instructional analysis provides the step-by-step instructions that take learners from start to finish When SAS Online Training's training specialists look at a training project, they identify what people will need to learn to achieve the department's business goals. The learners might need new knowledge, skills, or behaviors. SAS Online Training's training specialists conduct an instructional analysis to determine how to guide learners from their current capabilities to the course's goals.

Create an Instructional Analysis During the instructional analysis step, a training specialist might conduct a task analysis and create a competency map for learners. These tools help the training specialist define what learners must be able to do once they have completed the course. Think back for a moment about the online roadmap. You can't just walk out of your front door and instinctively know how to travel to a new place. Worse yet, imagine if you printed out a set of online directions that were missing an important turn. You might get lost and frustrated; you might even give up and never arrive at your destination. For learners, training is a journey; they rely on you to provide them with an accurate set of directions. An instructional analysis ensures that the course will: Cover all information and steps that learners will need to know Exclude information that learners already know Exclude information that learners don't need to know The more accurate the instructional analysis, the easier the journey will be for the learners. Think from a Learner's Perspective If you ask an expert to create a list of steps for a task, they'll probably omit many steps they instinctively perform. An expert can take the right actions without consciously thinking about each step. When a training specialist conducts an instructional analysis, they watch the process with fresh eyes. They look for "unstated" knowledge and steps that the expert never consciously thinks about. Imagine you want to teach someone how to write and send a letter. You probably learned this skill when you were in grade school, so you don't consciously think about all of the mundane details it takes to mail a letter. You're an expert now; you're intuitively capable of those tasks. You'd actually have to stop and think about each step that you perform. Write the letter, including the introduction, body, and closing Address an envelope properly and legibly Affix proper postage to the envelope Deliver the envelope to the post office That's a basic task analysis, but there are some assumptions here that could cause problems for someone just learning how to send a letter: The list never tells the learner to put the letter in the envelope. The list never tells the learner to seal the envelope. How does the learner determine proper postage?

Where should the postage stamp be placed? Now, consider all of the complex tasks involved in writing a proposal, navigating your department's inhouse proprietary software, or meeting compliance requirements. If a learner doesn't know about a step, it could mean the difference between success and failure. The instructional analysis makes sure that the course content exactly matches what learners need to know.

Write the Learning Objectives

What is a Learning Objective? At the end of the analysis phase, all of the data collection and analysis comes together into a cohesive, concise document that describes the course's learning objectives. There are several different styles of learning objectives, and each uses a specific linguistic formula. However, learning objectives generally contain the following elements: An observable task that a learner will be able to perform at the end of the course The conditions utilized to perform the task The criteria that will be used to measure a learner's success Here's an example of a learning objective: "At the end of the course, the learner will be able to process three sample customer orders within fifteen minutes without any errors." Check the Course's Destination The learning objectives serve as a major quality assurance checkpoint in the ADDIE instructional design methodology. In many ways, the learning objectives are like the "destination check" that you hear when you board a commercial airline flight. The flight attendant might announce, "This is Flight 5371 to Austin, Texas." If you're not heading to Austin, you've got a chance to get off the plane before they close the doors and push away from the gate. In the same way, the learning objectives announce the course's destination. The training specialist will use these learning objectives to build the course's instructional designincluding content, activities and tests. After SAS Online Training's training specialists write the learning objectives for a course, we present them to the client. We ask the client to review the learning objectives and assess whether their training needs are properly understood and represented. If your course doesn't have a clear set of learning objectives, you will run into two risks. First, the instructional designer will have to guess the course's goals and learning objectives. That's like telling an airline pilot to fly to California. A competent pilot could fly the plane safely to any of a dozen major airports, but what's the best destination? Vague objectives lead to imprecise results. Second, when you don't have clear learning objectives, your learners are like passengers who board a plane and just hope that it will take them to the right city. Will the course actually meet their needs and help them achieve the department's business goals?

Next Steps Once we've confirmed the courses learning objectives, the training specialists will be ready to proceed to the next phase of the ADDIE modelinstructional design.

ADDIE Instructional Design Phase

The Role of Instructional Design Once a training specialist has written the course's learning objectives and confirmed them with the client, it's time to begin the instructional design phase. During the design phase, the training specialist plans what the course should look like when it's complete. At the end of the instructional design phase, the training specialist produces an instructional design document for the course. In many ways, this document is similar to an architect's blueprints or a software engineer's design document. The instructional design document describes the course's content, but it doesn't contain the course contentjust like a blueprint isn't a house and a software design document isn't the actual software. Create an Instructional Strategy At the start of the instructional design phase, the training specialist should have a pretty good idea of what the learners will already know when they start the course (through a learner analysis). The training specialist should also know what learners will need to learn during the course (as stated in the learning objectives). How do you create a course that helps people move from what they already know and gain mastery of the new material? That's the question that the instructional design process answers. During the instructional design phase, the training specialist reviews the course's learning objectives and considers the following questions: How should content be organized? How should ideas be presented to learners? What delivery format should be used? What types of activities and exercises will best help learners? How should the course measure learners' accomplishments? The answers to these questions help the training specialist produce the instructional design document. This document describes the course structure and its instructional strategies. During the instructional design phase, the training specialist does not create course content. The actual course content and training materials will be created during the training development phase.

Steps in the Instructional Design Phase There are basically three steps in the instructional design phase: Plan the instructional strategy Select the course format Write the instructional design document Developing an Instructional Strategy At this point in the instructional design process, the training specialist makes important choices about the course's structure and its methods. Overall, these choices combine to form a comprehensive instructional strategy to help people achieve the course's learning objectives. When instructional designers create instructional strategies for courses, they draw upon theoretical knowledge and practical experience. There are many different ways to sequence and present content to learners. It's the instructional designer's responsibility to choose the correct instructional strategies for the course and the learners. On this page, we'll take a look at three issues that instructional designers consider when they devise an instructional strategy: How will course material be grouped and sequenced? What instructional methods and tactics will be used to present material? How will assessments measure a learner's success? These three issues often overlap with each other; a choice in one area may affect the other areas. Grouping and Sequencing Content The training specialists must decide if any of the course's learning objectives should be grouped together. You can't teach everything at once, but sometimes it makes sense to put related topics together for the learners. These related topics can form the basis for a course module. Once topics have been grouped together, the training specialist has to organize the content into a course structure. The content inside of each group needs to be sequenced and then the groups themselves need to be sequenced together to form the course structure. Here are just a few of the many possible sequencing options: Step-by-step Part-to-whole Whole-to-part Known-to-unknown

10

General-to-specific As you can see, there are many different ways to organize and present course material. The instructional designer chooses the structure that makes the most sense for the learners and the course content. Choosing Methods and Tactics In the instructional design phase, the training specialist has to decide how the course material will be presented to the learners. Specifically, we're looking at the types of activities and exercises that will be in the course. Here are just a few examples of different types of learning activities: Individual reflection Modeling Scenarios Mnemonics Drills Applied practice If you want to teach someone how to type on a keyboard, you might recommend rote drills and applied practice. However, if you want learners to develop interpersonal skills, rote drills offer limited value. Role play scenarios and group discussions would probably be more effective learning activities. Generally, the course's activities and exercises must fit with the type of learning people will be asked to do. Designing Assessments At the end of the needs analysis phase, the training specialist created learning objectives that defined measurable tasks and criteria for success. Now, in the instructional design phase, the training specialist creates assessment tools that will measure the learners progress. If you have a driver's license, you probably completed two types of tests before you received your license. You completed a written test that measured your understanding of street signs, laws, and procedures. You probably also performed an on-the-road test where someone observed your driving skills. The two tests measure different capabilities. You could be very knowledgeable about traffic laws and procedures but a poor driver behind the wheel. Similarly, you might be good at driving the car but poor at recognizing street signs and safety procedures. You have to pass both tests before you can obtain a driver's license. The course's assessments should measure a learner's progress towards each of the learning objectives. The types of assessment must fit the learning objective.

11

Selecting the Course Format

Choose the Class Type In the ADDIE model, the training specialist chooses the course's delivery method during the instructional design phase. This seemingly simple choice will affect almost every aspect of the design document and the final course content. The course delivery format should mesh with the learners' needs, the content, and also the client's business goals. Some types of course material can be easily presented through job aids and self-study materials, but other courses work best when learners come together as a class under the direct guidance of an expert instructor. The course delivery method impacts how learners experience the course and its content. If the training specialist chooses the right delivery method, it will make the learning process easier for the learners. However, if the delivery format doesn't fit the content and learner's needs, then the course will have very limited success. Online Training There are many different course delivery options for the instructional designer to choose from. Here's a list of some of the choices and links to other pages where we discuss these formats in more depth: Instructor-led courses Paper-based self-study materials Synchronous e-learning Asynchronous e-learning Job aids Ideally, the training specialist selects the class type during the instructional design phase. At this critical step SAS Online Training needs to be very careful to select the correct delivery format meaning that if the training or blended training does not best suit online training this is the time to honestly recommend another delivery option.

Instructional Design Documents

The Role of the Design Document At the end of the instructional design phase, the training specialist writes an instructional design document. This document provides more than just a simple course outline; it provides a high-level overview of the entire training solution. A training specialist's instructional design document provides detailed instructions on how to build the course, but it doesn't contain any actual course content; it's similar to an architect's blueprint or a software engineer's design document.

12

Generally, an instructional design document will perform the following tasks: Describe the overall learning approach Identify instructional media choices if not online training then stop the process and recommend another venue for the client Cluster and sequence objectives Describe course exercises, activities, and assessments Together these five elements create the overall instructional strategy for the course. A short course might have a very simple design document, but complex and lengthy courses can have very detailed design documents. The instructional design serves as a major quality assurance checkpoint. The training specialist and the client discuss and agree to the design before development begins. It's a lot easier to adjust the design than redevelop materials later in the project. Benefits of the Design Document SAS Online Training's training specialists use the instructional design document for four main purposes: Check that the design concepts are cohesive and complete Present the proposed training solution to the client Invite feedback about the design Provide instructions to other training specialists who may work on the development phase of the project Instructional design documents may also contain additional project-specific elements. For example, in online courses, the instructional designer might describe the interface's appearance and functionality. Check the Design Concepts and Content The course's instructional strategy should allow the learners to achieve the course's learning objectives. Once the instructional design document has been written, the training specialist can take a step back and look at the whole design, not just individual pieces. The design document makes it easier to spot areas that have unresolved questions or need additional information. Present the Proposed Solution SAS Online Training's training specialists will present the instructional design document to the client. Often, we'll schedule a meeting or conference call to walk through the course design and explain our choices. Our instructional designers base their choices on adult learning theories and methodologies, but we ask them to explain their choices in language that makes sense to non-specialists.

13

If you're not a training specialist, your eyes might glaze over if someone told you how each learning objective links to Robert Gagn's Nine Events of Instruction or Keller's ARCS Model of Motivation. You just want to make sure that the instructional design choices will actually fulfill the learning objectives. We use clear and comprehensible language to explain the choice. Invite Feedback about the Design After our training specialists present the instructional design, we ask for feedback from the client. We often collect suggestions from project leaders, leadership champions, subject matter experts, and other stakeholders. The instructional design document specifies what the final course will be like. It's important to build consensus and agreement before starting course development. Sometimes, when we walk through the instructional design with the client, we hear someone say, "oh, we really should add . . ." or even "that's been changed . . ." However, that's exactly the feedback we're looking for. It's much easier to revise the instructional design than to revise a fully-developed course. Provide Instructions to Other Developers Large training projects often require more than one training specialist. For example, an online module requires a full team of training specialistsinstructional designers, graphic artists, storyboard writers, editors, programmers, and voice talent. The instructional design document guides the complex project and allows everyone to be involved with the project's goals and structure. In SAS Online Training staffing makes this impossible unless each team member takes on multiple roles.

ADDIE Development Phase

Successful Training Development On the surface, training development seems simpletraining specialists create the course materials, yet what separates a great course that fulfils its objectives from a weak course that misses its mark and puts people to sleep? Here's what our training specialists have learned through experience. A successful development phase draws upon the information collected in the needs analysis phase and the decisions made in the instructional design phase. If the team has done solid work during the first two phases of the ADDIE methodology, then the training development phase should proceed smoothly and quickly. The training specialists and client have agreed on the course's purpose, structure, and content. Now it's easy to focus on writing the materials. In contrast, if there are unresolved issues from the first two ADDIE phases, then problems usually start to appear in the training development phase. You might see missed deadlines, weak and off-target materials, and even substantial cost overruns.

14

Steps in Training Development In this section, we'll look at the ADDIE model's training development phase. We'll focus our discussion on the high-level steps that are common to most training projects. If you're looking for specific advice on how to format a leader's guide or how to create an e-learning template, this section probably won't help you. Instead, we look at the strategic processes that SAS Online Training's training specialists use to create training materials for our clients: Create a prototype Develop the course materials Conduct a tabletop review Run a pilot session Since there are many types of training projects, the development phase often adapts to fit the project and the client's needs. One project might devote a lot of time to prototyping, while another session may devote more time to tabletop review and pilot testing. In many situations, it's a matter of matching the right quality assurance steps to the project. Our training development in the workplace page explores these choices in greater detail.

Developing a Prototype
The Value of Training Prototypes A training prototype provides a preview. It shows what the final course will look like when it is complete. Both training specialists and clients love prototypes. Until this point, people have been envisioning the course materials in their minds. In this step, the training specialist builds a tangible sample that everyone can see and discuss. Training prototypes often vary in scale and complexity. For some courses, the prototype might be just a few template pages. Other courses might need detailed step-by-step storyboards. The course's format often influences the type of prototype the training specialist will create. If this is the first time you deal with a client, SAS Online Training specialists can use the Behind the Scenes training module to set the stage and then show a similar module. If the development is new, it is worth the time to voice over a prototype before the presentation to allow the client to see what the finished product will resemble. Marketing the prototype will help you with the management buy-in you need to successfully complete the module and post it to the LMS. SAS Online Training's training specialists build prototypes that fit the type of course they're developing. Simple courses don't need massive prototypes. However, when a project grows in size and complexity, prototypes help people envision the final deliverable project.

15

Course Templates and Prototypes Some departments on campus, like Physical Faciliites, have created standardized templates for all their online courses. These templates provide a consistent look for the department's training message and reinforce the department's branding. With these templates, people don't have to reinvent the wheel for each new course. However, the training templates can become limiting and restrictive. Our training specialists have seen departments whose in-house brand guidelines actually inhibit learning. If an organization uses standardized templates, they need to be flexible enough to allow courses to deliver their content successfully. SAS uses non-standardized templates for many new projects but also standardizes design for different parts of the organization, such as BYU Catering or The Commons. If the course is related to Safety BYU Risk Management requires that a standardized book-end template be used to identify the material as safety/hazard training. There is also a special safety symbol used that will emphasize safety policies/procedures:

Developing Training Materials

The Course Development Process If the training specialist has followed the ADDIE instructional design model, then the prior steps will provide solid preparation for the course development process. The course developers will have access to the following information: Any prior course materials and other documents found in the discovery process The course's learning objectives The instructional design document A prototype or template that models the course's layout and appearance

16

The Role of the Course Developer A good course developer understands both instructional design and training delivery. When the course developer creates content, two questions are paramount: Does this material meet the learning objectives? The Course Development Team The course development team can include writers, editors, graphic designers, online training programmers, usability experts, and project managers. Some people may be needed for the entire course development process, while other people may be called in to accomplish just a few specific tasks.

The Tabletop Review

The Goal of Tabletop Review After the course has been developed, the training specialist delivers a first draft of the course materials to the client. It's now time for a tabletop review of the course. During the tabletop review, the training specialist and client check the content's accuracy and completeness. They walk through the course materials as experts looking for errors rather than as learners interacting with the course. The tabletop review serves as a quality assurance step. The training specialist and client check the course's content before any learners interact with the course. How the Tabletop Review Works The tabletop review works best when the course developers and the client's review team can actually speak with each other. A tabletop review could occur during a face-to-face meeting or a phone call. It's much more difficult to conduct a tabletop review through e-mail or reading reviewer's notes in electronic files. During the tabletop review, the training specialist asks the client's reviewers to focus on two important questions: Is the content accurate? Is the content complete? Is the material communicated effectively? If a specific section seems vague or unclear, the training specialist might ask a third question how can we communicate this idea better? It's important to gather ideas that will strengthen the course materials, but the tabletop review shouldn't turn into "what if" discussions that last for hours.

17

The tabletop review can focus very tightly on these issues because so many questions have already been asked and answered during the needs analysis and instructional design phases. Tabletop Review Participants The training specialist usually will ask the client's project leader and key subject matter experts to attend the tabletop review. These participants will best be able to address the questions of accuracy and completeness. Some clients may also want other internal people to attend the tabletop review. Additionally, the client may require additional internal review cycles before the course begins pilot testing (beta test).

Pilot Testing a Course

Pilot Testing with Actual Learners In the tabletop review step, project members reviewed the course content for completeness and accuracy. Now, it's time to put the course in front of the learners and measure how they interact with the materials. In most cases, the pilot test will be the first time actual learners experience the course. The pilot test of the course takes place before the official course implementation. It provides the training specialists and the clients a final chance to review the course prior to its official launch. The Benefits of Pilot Testing In many ways, a course's pilot session is similar to a software beta test. Whenever you put a piece of software or a course in front of actual learners, they'll interact with it very differently than trained experts will. Experts and learners will be sensitive to different types of issues within the course. Subject matter experts can make sure that the course materials are accurate and complete, but they might not catch that a learning activity's instructions should be worded with more clarity. People who are actually trying to learn the course material provide a very rigorous test for two reasons. They're often very willing to provide candid feedback. Additionally, at the end of the course, you can measure how well the pilot course's learners have achieved the course's learning objectives. That's why SAS Online Training's representation of the ADDIE model includes a two-stage review process after course development. Both tabletop review and pilot testing provide important quality assurance checks, and there's really no way for one process to replace the other. Pilot Testing Methodologies Learners may be asked to fill out a survey after they finish the course. Then, the training specialist reviews the data and perhaps conducts follow-up interviews.

18

Issues Identified in Pilot Testing Before the pilot test, the training specialist builds a checklist of issues. Some of these issues are standard review items, but others will be specific to the course's content and its delivery format. Here's a very brief list of some issues that the training specialist might measure during the course pilot: Measure the amount of time learners need for each module and activity Check learners' engagement with the material Detect points where material may be too easy/too difficult Confirm that learners understand the instructions for activities and exercises Evaluate the flow and balance of the course Test how well learners achieve the course's stated learning objectives by the end of the course Validate the course assessment tools Collect feedback from learners about the course Locate points where the course should be revised During the course pilot, it's important to let learners interact with the course rather than try to correct things on-the-fly. When you spot something wrong, it might be tempting to jump in and "add one thing" but that can create a cascade effect throughout the course. After the course pilot, the training specialist and the client meet and decide what revisions should occur before the course launches.

ADDIE Implementation Phase

Launching the Course The ADDIE model provides a systematic methodology to plan, develop, and test the course before it launches. If you follow the ADDIE model, you'll have a high degree of confidence about the course when it's ready to launch: The course meets important business/organizational goals The course covers content that learners need to know The course reflects the learners existing capabilities Additionally, you'll have reviewed the course's content for accuracy and completeness. You'll also have conducted a pilot test to ensure that learners will actually master the skills they need to achieve the course's learning objectives. It's possible for someone to write and launch a course without following the ADDIE instructional design methodology, but there's a much higher degree of risk. The course could have the wrong focus, confuse or frustrate the learners, or even lack critical content. So, if the course has been developed without planning or testing, then all you can do is hope that the course will go well.

19

Course Delivery Issues There are plenty of issues to address during the ADDIE implementation phase. It's important to make sure that the course gets delivered smoothly and effectively to the learners. Of course, these delivery issues will substantially depend on the course's delivery format. SAS Online Training uses the BYU Learning Management System (LMS) as the course delivery method. It provides documentation and easy accessibility to records.

ADDIE Evaluation Phase

Evaluating the Course The ADDIE model stresses the concept that good training programs require planning, review, and revision. Each of the five ADDIE phases provides review checkpoints that allow the training specialist and the client to evaluate the work that has been produced so far. The ADDIE evaluation phase can produce pretty graphs and metrics, but that's not its main purpose. The evaluation phase measures the course's efficacy and locates opportunities to improve learners' on-the-job performance. Evaluation Questions When a course launches, it's not the end of the process. The ADDIE evaluation phase provides a final review checkpoint for the project. During the evaluation phase, the training specialist measures how well the project achieved its goals. Here are just some of the questions that might be explored during the evaluation phase. Do learners like the course? Do learners achieve the learning objectives at the end of the course? Do the learners change their behaviors in the workplace? Does the course help the department achieve its business goals? For some questions, it's fairly easy to collect information. You can find out learners' opinions of the course through a short survey immediately after the course. A pre-test and post-test can measure how well learners achieved the learning objectives. However, it takes more time and effort to measure changes in workplace behaviors and improvement towards business goals. In both cases, you can't measure these results immediately. You want to measure the long-term improvements rather than the immediate results. The evaluation phase can extend for months. Effective training helps learners make lasting changes to their workplace behaviors. The changes shouldn't just last for a few days or a few weeks, but they should remain with the learner months after the training course. A training specialist might follow-up with a sample group of learners several months after the course to see what the learners currently do. While the training specialist might identify people who need

20

refresher training, the study's purpose is to measure the course's long-term effectiveness. If many of the learners quickly fall back into their old habits, then that's a course-level issue that needs the training specialist's attention. Similarly, the course should produce measurable business results. During the needs analysis phase, the training specialist asked the department's leadership to identify business metrics that they want to improve through the training. Some courses may have an immediate effect on a metric that's measured daily or weekly, but many courses affect metrics that take longer to measure and detect a change. Sometimes the department has to wait an entire quarter or longer before it can measure the course's impact on its business results.

ADDIE in the Workplace

Real World The corporate world lives by project deadlines. You probably have several projects on your desk right now, and you're already anticipating others that are on their way to you. If you want to stay current, you have to keep moving forward and meeting deadlines. The ADDIE model describes an ideal-world methodology. It assumes that training specialists will have plenty of time to create a great training program. Unfortunately, that's not always the case. With an urgent training project, you have to balance quality and timeliness. The ADDIE instructional design model represents a complete instructional design workflow. It has built-in planning activities, quality assurance checkpoints, and feedback cycles. Sometimes, it's not possible to delay the training project's go-live date. Then the question becomes, "What can we do in the time that we have?" Balance Risks and Costs Because the ADDIE model builds each subsequent step from previous steps, skipping one step can create ripple effects throughout the entire process. While it takes more time to properly plan and execute a training project, it will substantially reduce the risk and the cost of errors. Like every business decision, you have to weigh risks and make a choice. However, it's best to be informed and make an educated choice. If you've got a training project with a tight timeline, SAS Online Training's training specialists can help you assess your options and the associated risks. With that information, you'll be able to make an informed decision. In this section, SAS Online Training looks at the ADDIE instructional design model through a real-world corporate training perspective. While we can't cover every factor that might affect a training program's development, we've tried to present some of the common choices in each of the five ADDIE phases. We begin our exploration of ADDIE and the Workplace by looking at the Analysis phase.

21

ADDIE Analysis in the Workplace

Real World Choices If your department has an immediate training need, it's tempting to skip over the training needs analysis phase and just start writing the actual training materials. You could save production time, but will you actually save money? Many companies wrestle with this question, because they want to make the right choice. Analysis and Quality Assurance The ADDIE analysis phase serves as a formal planning and quality assurance process. It defines the project's objectives using the language of instructional design, and it validates that the course will meet the department's needs and the learners' needs. If you don't perform the needs analysis, you will increase the project's risk. In many ways, a training project is very similar to a software development project. If you were leading a software project, you could start the project by asking engineers to analyze the project's needs and plan the application. However, you could also start the project by asking programmers to start writing code. If the programmers write code without a plan and a clear goal, they're likely to produce a software application that doesn't quite do what it needs to do. The code may also have a lot of bugs that may not be discovered until late in the development cycle or even post-launch. In the software industry, one standard metric states that QA planning and review activities produce a 10:1 return on investment. It's less costly to prevent software bugs through planning than by fixing them line by line in the code. The same principle applies to training projects. If you can identify mistaken assumptions during the assessment phase, it'll be easier and less expensive to correct them. If you wait until the training materials have been written, it'll be more expensive to go back and make changes. Timeliness vs. Quality Sometimes, a department has to choose between getting the training project done (timeliness) and getting the training project right (quality). Here are some reasons that companies make their choices:

Reasons to Choose The project will make only slight modifications to an existing, well-written course

Reasons to Choose

Course will be entirely new Business goals are unclear or have changed

The department's business needs indicate that it's better to deliver a partial training solution on time than miss the deadline Course is a one-shot event for a very small group of learners

The training program must achieve measurable results Course covers compliance issues or critical business

22

processes Course will be used for a long period of time or delivered to a large audience An existing course will be rewritten for a new learning audience with different needs

Some Potential Risks If You Skip This Step Here Are Possible Risks Training specialists may not know about (or use) important information when designing the course. Training specialists might not talk to the right subject matter experts. Discovery The course may not be written to suport the business' goals. It may be difficult to measure the course's effectiveness or results. Business Goals If the course content is too easy for learners, they may become bored. If the course content is too difficult for learners, they may become frustrated. Learner Analysis The course may omit critical steps and information. The course may become bogged down with lessimportant information.

Instructional Analysis

23

The project omits a major QA checkpoint that allows you to review and confirm the course's objectives. Mistaken assumptions may not be caught until much later in the project. Learning Objectives These mistakes may be more costly to correct.

Implementation Makes the Difference

During this phase, SAS Online Training must successfully deliver the course to learners. Each course represents a significant investment of department resources and time. It's very important for the course to make a significant and meaningful impact on the learners. On the strategic level, companies rely on training programs to reduce costs and improve profitability by improving people's performance in many different ways: Increase sales and customer satisfaction Improve efficiency and productivity Ensure legal compliance and reduce liability Guide people through new and changed processes Introduce new people to the departments methods and culture Companies that want to achieve these goals need well-written training programs that are launched successfully. However, it's important to remember that great course content doesn't guarantee a successful launch. In this section, we'll assume that the department has done its homework and created an excellent training course. Instead, we'll focus on the challenges that companies face during the ADDIE implementation phase.

24

Launching an E-learning Course An e-learning course oftern requires significant systems integration tasks. Here are some sample issues for an e-learning course that will be delivered online to learners.

Course Delivery Aspect

Some Key Questions Where will the course be hosted? How much storage space will be required for the elearning files? How many learners will need to access the course Hosting total? How many learners will access the course at any time? How much bandwidth will be needed (peak use and monthly)? Will this course need to integrate with an existing learning management system (LMS)? Will the LMS track course enrollment and course completion data? Integration Will this course output test scores and other data to an LMS? Is this course SCORM and AICC compliant?

How will learners enroll for the course? Will learners be able to access the course through the web or will they need to connect to an intranet? Access

25

Will any users connect to the course via dial-up? Will any users connect to the course via a VPN? Can the course recognize the learner's connection Learners' Connections speed and optimize course delivery?

Will learners have all necessary applications loaded onto their computers? Will learners need to download any applications or Learners' Computers plug-ins?

Who will be responsible for security issues related to the course files?

Security Who will help learners who have difficulties accessing the online course? Who will answer technical questions? Who will answer content questions?

Help Desk

How will learners be enrolled for the course? Will they enroll themselves or will someone enroll them? How will course rosters be tracked?

Registration

26

Who will manage training administration? Who will manage training logistics? How will course statistics be tracked? Who will be responsible for collecting and Logistics communicating these statistics?

There are many ways that an e-learning course can implode during the delivery process. An online course can be so popular that the hosting site crashes when everyone tries to access it at the same time. The course might not integrate properly with the department's learning management system. The file might have a broken link so learners receive the dreaded "file not found" error. ADDIE Evaluation in the Workplace Evaluating a Course's Success The ADDIE evaluation phase helps companies measure the course's impact on their learners and their business/organization. When SAS creates a custom course, it makes a substantial investment in time and resources. Many organizations seem to skip over the evaluation phase because they're thinking tactically instead of strategically. Once the course has launched, it is natural to shift gears and focus on the next training project but to be successful SAS Online Training needs to measure learning and its impact on the department. Dont allow the course to run until it becomes obsolete and an obvious source of pain. Types of Evaluation In 1959, Donald Kirkpatrick identified four levels of training evaluation: 1. 2. 3. 4. ResponseDo learners like the course? LearningDo learners actually learn the material? BehaviorDo learners change their workplace behaviors? ResultsDoes the course acheive the department's business goals?

In the corporate world, departments measure response through quick post-course surveys (often called "smile sheets"). These surveys often ask learners to answer simple, subjective questions about the course. Because these surveys are easy to conduct, many companies use them. However, these surveys can't measure complex learning or long-term behavior changes. Some companies use post-course assessments to measure how much people have learned before they return to the workplace. We'd like to think that these post-course assessments are part of every course,

27

but we've seen a lot of training courses that just present material without ensuring that learners have understood it. In the corporate world, courses with rigorous post-course tests are often called certification courses. Learners need to pass the certification test before they can be qualified to perform certain tasks or jobs. In order to measure behavior changes and business results, training specialists have to wait until learners return to the workplace. Sometimes, these studies take place months after the learners complete the course. That way, training specialists can measure what behaviors actually retained in the workplace. The Costs of Evaluation In most cases, a robust evaluation phase doesn't add much additional cost to the training project. Sometimes the entire evaluation phase can be conducted for less than five percent of the total project's budget. The other cost associated with the evaluation phase is time. To properly conduct an evaluation, training specialists will need to dedicate some time immediately after the course launches to measure how people respond to the course and what they learn. Then, a few months later, they will need to measure whether the course has actually led to long-term behavior changes. SAS Online Training's training specialists have observed two patterns when it comes to the training evaluation phase: Departments that clearly identify business goals during the needs analysis phase are more likely to conduct a rigorous evaluation phasethe department wants to measure (and improve) its degree of success. Departments that follow the ADDIE model carefully tend to see substantially better results for their training courses than companies that adopt a hit-and-miss approach to training development and delivery.

28