Blended learning can be implemented using diverse approaches, harnessing a wide variety of tools and activities to achieve specific learning outcomes. When it comes to evaluating blended learning approaches, it follows that strategies should also be very different.
In this article, we define some guiding principles in evaluating blended learning and present five sets of evaluation questions based on your original objectives for implementing blended learning. We finish with a popular framework that you might also find useful.
Guiding principles in evaluating blended learning
In evaluating blended learning, an important starting point is to formulate questions. These should relate to your original aims when designing blended learning. Put simply: How far has your blended learning innovation succeeded in achieving your objectives? Further questions could explore barriers or challenges you experienced in implementation, and how these might be overcome in a future implementation. You might also want to explore factors that made your intervention successful, with a view to making recommendations for teachers who would like to use a similar approach. Whatever the precise questions, your findings should help you to reflect on what you did and consider how it could be further improved.
Once you have formulated some evaluation questions, the next step is to design evaluation methods to help answer them. What types of data will you collect, and how will you analyse it?
Common blended learning objectives and example strategies for evaluation
This section lists five common objectives for using blended learning. Each objective has example questions and suggested methods for collecting and analysing data.
1. Objective: Enhancing student engagement
Evaluation questions:
- Did students spend more time on their learning because of the blended approach?
- How much time did students spend using the online resources and activities?
- How often did students access online resources and activities?
- How engaging did students find the online resources and activities?
- How engaging did students find the in-person activities?
- Why did students find these resources and activities engaging?
- What barriers were there to student engagement?
- How can I make the blended approach even more engaging
Possible evaluation strategies:
- Read the activity logs from online platforms you used, to measure how often students accessed them, and for how long
- Conduct a survey for students to self-report usage and perceived engagement
- Hold a focus group with some of the students to explore why they found specific resources and activities engaging, or less so, and to gather their suggestions for further enhancement
2. Objective: Enhancing understanding of concepts and knowledge
Evaluation questions:
- Did the blended approach help students in understanding concepts and knowledge?
- Why did students find these resources and activities useful?
- What barriers were there to student learning?
- How can I improve the blended approach to further enhance learning?
Possible evaluation strategies:
- Use grade data to compare students’ performance with those in previous cohorts, or use an experimental design to compare an experimental group with a control group
- Conduct a survey for students to self-report how much they have understood
- Hold a focus group with some of the students to explore why they found specific resources and activities useful, or less so, and to gather their suggestions for further enhancement
3. Objective: Enhancing the application of concepts and knowledge
Evaluation questions:
- Did students have opportunities to apply concepts and knowledge because of the blended approach?
- How did students apply concepts and knowledge?
- Why did students find these resources and activities useful?
- What barriers were there to students applying concepts and knowledge?
- How can I improve the approach to further enhance students’ application of concepts and knowledge?
Possible evaluation strategies:
- Use grade data to compare students’ performance with those in previous cohorts, or use an experimental design to compare an experimental group with a control group
- Conduct a survey for students to self-report which concepts and knowledge they have applied
- Hold a focus group with some of the students to explore why they found specific resources and activities useful, or less so, and to gather their suggestions for further enhancement
4. Objective: Increasing student motivation
Evaluation questions:
- Did students feel motivated to learn?
- How motivating were specific activities and resources?
- Why did students find these resources and activities motivating?
- What factors negatively affected student motivation?
- How can I improve the blended approach to further increase motivation?
Possible evaluation strategies:
- Conduct a survey, such as the Study Process Questionnaire (SPQ) or the Motivated Strategies for Learning Questionnaire (MSLQ), to understand students’ motivation and learning approach
- Hold a focus group with some of the students to explore why they found specific resources and activities motivating, or less so, and to gather their suggestions for further enhancement
5. Objective: Fostering reflection and metacognition
Evaluation questions:
- Did students reflect upon their own learning because of the blended approach?
- In what ways do students better understand their own strengths and weaknesses?
- In what ways are students better able to plan what and how they should learn?
- How can I improve the blended approach to further promote reflection and metacognition?
Possible evaluation strategies:
- Invite students to write reflective essays or blog posts, or record video presentations, in which they reflect on their learning, and analyse these to measure their level of reflection
- Invite students to write reflective essays or blog posts, or record video presentations, in which they explain how specific online activities or resources helped them to reflect
- Hold a focus group with some of the students to discuss how specific online activities or resources helped them to reflect
The Kirkpatrick Model
The Kirkpatrick Model, created by Dr. Donald L. Kirkpatrick in the 1950s, is a globally recognised method of evaluating the effectiveness of training and learning programmes. Use the model to assess your blended learning intervention and rate it against the following four criteria:
