The What, When, Why and How of Program Evaluation

The first step in demonstrating compliance with learning outcomes assessment standards is to self-evaluate your academic program. This self-evaluation allows you to identify strengths and weaknesses, implement evidence-based interventions to improve program and educational quality, and validate the effectiveness of you learning outcome assessment process.

The program evaluation process focuses on answering the following questions:

• What do we want to instill?
• How do we measure it?
• When do we measure it?
• Why are we doing this?

Let’s answer and explore how each of these questions can make accreditation and meaningful student learning a reality for you and your institution.

What Do We Want to Instill?

This is directly related to the content of your program’s learning goals. Learning goals impact and determine the following two aspects of the learning outcome assessment process:

• The topics and skills being taught to students
• How students meet and accomplish the goals

Let’s take a look at an example learning goal and identify both aspects.

Example learning goal:

• Demonstrate the ability to lead and work effectively with others to accomplish collective goals and tasks.

First, the example learning goal determines your students should be taught teamwork and leadership skills. Furthermore, as pointed by the highlighted verb, the learning goal establishes that students need to demonstrate such skills in order to meet this specific goal.

Before you can assess learning, it’s important for you identify the type of outcomes that result from the learning process. According to program evaluation literature, the three outcomes that result from the learning process are the following:

• Affective Outcomes: attitudes
• Behavioral Outcomes: skills
• Cognitive Outcomes: knowledge

To learn more about the three learning outcomes and their importance is assessing the learning process, read our recent post on understanding learning goals and outcomes.

How Do We Measure it?

There are multiple assessment methods being implemented by higher education institutions to measure learning outcomes. Examples include:

• Essay Exams
• Homework Assignments
• Multiple-choice exams

However, it’s important to recognize that assessment methods have a key limitation impacting their reliability in learning outcomes assessment. The vast majority of assessment methods are not interchangeable in how well they assess the aforementioned three different types of learning outcomes.

In addition, the flaws in learning outcomes assessment are also impacted by the ‘Criterion Problem,’ or what program evaluation defines as the difficulty in measuring multidimensional concepts – the three learning outcomes. This problems translates to a flawed learning assessment process that results in deficiencies, things we should be measuring but aren’t, and contamination, things we measure that aren’t related to true learning.

To learn more about the ‘Criterion Problem’ and to discover the link between the three learning outcomes and their optimal assessment methods, read our recent post on increasing valid assessment.

When Do We Measure It?

The timing of learning outcome assessment is determined by the kind of conclusions you want to draw and where you expect the learning outcomes to occur. First, you must determine whether you’re assessing for proficiency or assessing for change.

• Assess for Proficiency: implement a learning assessment at the end of program or course.
• Assess for Change: implement a learning assessment at the beginning and end of program or course.

Furthermore, you must determine where the topics and skills being assessed are most likely to be taught by instructors and demonstrated by students. For example, if students are likely to demonstrate skill proficiency in the last course of the program, then the learning assessment would be implemented at the end of the program.

Why are we doing this?

The main reason for implementing learning outcomes assessment processes is to ensure educational quality for students. It’s your responsibility, as an educator, to arm your students with the knowledge and skills that translate to a successful career post-graduation.

Ideally, the improvement of educational quality occurs through a continuous feedback loop driven by program mission, program learning goals, student learning outcomes, and interventions.

Now that you have the tools to evaluate your learning outcome assessment process, how do you prove to accrediting bodies the process is effective and serves to improve student learning? To learn how to present your assessment data to accrediting bodies, read our recent blog post on the ideal structure and content of learning outcome assessment reports or access your free eBook – The Ultimate Guide to Learning Outcomes Assessment,