A guide to measuring the effectiveness of your online courses and training programs.

Ask yourself whether you consider the following statements to be true or false:

  1. The existence of a new training course or program where none had previously existed indicates improved learning.
  2. The number of learners who have successfully completed or passed a course is evidence of the course’s success.
  3. Learning can be objectively measured.

Now let’s examine the premise behind each statement more carefully.

All too often, training courses and programs in the workplace are celebrated simply for existing. Imagine an organization that had never offered leadership development before, then created a new program that led to high praise from executives, promotion across the organization’s communication channels, maybe even press from their industry journals, all before the program even launched. Its mere existence promised employee development, simply because nothing had existed before.

A year later, the organization’s chief talent officer receives a report indicating one hundred learners had successfully completed the program, so he makes the assumption that his organization now employs one hundred more effective leaders than it did a year ago. However, as time goes on, the organization continues to suffer the same challenges as in previous years, so he then decides that the program has failed and that concepts such as leadership cannot be taught or objectively measured.

What’s wrong with this picture? This common scenario depicts a lack of understanding on how to properly design learning programs and then measure their effectiveness. The following strategies will enable you to overcome common challenges associated with program evaluation as well as to make improvements to your existing programs.

Identify the Business Goal That Your Content is Intended to Address

In his book and TED Talk, thought-leader Simon Sinek advised us to “Start With Why.” In instructional design, as in business, it is critical to have the desired outcome in mind when beginning any new project. Simply put, if we don’t know where we’re going, and why, we’re a lot less likely to get there. This concept has been expanded upon by instructional designer Cathy Moore, who developed the process of action mapping to create effective training content in corporate settings. Moore’s action mapping process guides designers through identifying the business goal, creating solutions, and ultimately improving performance of both the learners and the business as a whole by creating meaningful learning experiences. This process demonstrates the return on investment of high-quality training, because as Moore describes, “We’re not held accountable for what we do [as instructional designers] because almost nothing we do is measured. As a result, our work isn’t seen as vital to the organization. By putting a measurable business goal — a high-level evaluation — first and making it the center of everything we do, we publicly commit to improving our organization’s performance and demonstrate our value.”

Write and Follow Clear, Measurable Learning Objectives

Once the business goal has been identified, the next step is to write learning objectives for the training content. Learning objectives serve as guideposts for designers and learners alike. They present learners with their goals, they ensure course content is relevant to these goals, and in well-designed courses, objectives are aligned with how learners are assessed.  To be effective, learning objectives must be specific and measurable, and often include performance-based verbs that indicate ideal learner performance upon course completion. When success has been clearly defined, instructional designers are empowered to create content that is more likely to hit the mark. When the definition of success is communicated to learners, they know what they are working towards, and how to demonstrate proficiency and even mastery. Ultimately, the definition of success allows learning professionals and learners’ managers to objectively measure knowledge, performance, and the return on investment of the training course or program. For more information on learning objectives, read “How to Write Meaningful Learning Objectives.”

Focus on the Metrics that Matter

Think back to the definition of success that was described in your learning objectives. What tangible or observable actions demonstrate learners’ success? Are they now able to overcome customers’ objections, explain the benefits of your product or service, or safely operate tools or equipment? These are the metrics that matter. The number of course completions, the amount of time spent either in the online course or  a classroom, even the number of voluntary registrations do not indicate success or effectiveness. Use your learning objectives and business goal to determine what metrics matter for you. Maybe it’s an increase in profit, a reduction in loss, lower turnover, or reduced time for the average learner to demonstrate proficiency. These are the metrics that connect to the original business goal that led to the course creation. The ubiquitous Kirkpatrick Model, as explored in our article on Creating More Memorable Learning Content, offers a guide to determining the results of training and culminates in analyzing the return on the training investment. Focusing on the metrics that matter will allow you to determine whether or not your program is achieving your desired results.