A couple weeks ago I wrote a post about three types of assessment common in the Ontario school system: Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL). That post described all three and explained their importance. If you haven’t read it yet, I suggest you start there.
Over three weeks, I will break down each of the different types of assessment, as it pertains to instructional design and learning experience design. Last week I began with AoL. Now I’ll move on to AfL. Before I delve into AfL as it is used in instructional design and learning experience design, let’s review where the term came from as it relates to teaching in the primary and secondary school system.
Description of AfL from the school system
AfL is an informal assessment of what a learner has learned from an individual topic or task. This type of assessment is either unscored or a mark is given simply for the learner’s own knowledge (but it does not count toward a final grade score). In either case, the emphasis is on descriptive feedback.
The focus of AfL is on the process of learning. The AfL step helps learners identify their strengths and weaknesses, and it ultimately helps the learners focus on what they need to know to succeed.
AfL in instructional design
When it comes to the instructional design environment, quizzes, in-class activities, and interactive eLearning activities are commonly used for AfL. Let’s look at some considerations for AfL in instructional design.
1. Assessment must be based on the learning outcomes
It is essential to create assessments that are strongly based on the learning outcomes. At the start of the course design process, you should have written some learning outcomes for the course. These outcomes, the instruction, and the assessment should be closely aligned. For more information on this topic, check out the AoL post from last week.
AfL is all about feedback. In some cases, the assessment may be marked, but whether it is or not, the most informative feedback comes in the form of comments. In an instructor-led course, this may mean debriefing after each activity to try to understand why the learners made certain decisions. In an eLearning course, this may mean providing written feedback – ideally feedback that is both immediate and personalized.
Consider the two examples below. Try getting the answer right and wrong for each question. Which example made you feel more satisfied with your learning? [note that the try again button is there so you can try both a correct and an incorrect response, but it would be removed in an actual assessment, particularly in example 2]Example 1 Example 2
As you can see from the second example, the feedback is in fact part of the learning process. It is particularly helpful when the feedback is specific to the choice made. In this example, there were 3 different ‘incorrect’ feedback layers, each one was based on a different incorrect answer.
You may wonder about the learners who decide not to read the feedback. They see the checkmark or X and move on to the next question. Although those learners do indeed exist, not all learners will skip reading the feedback. Therefore, by providing feedback to everyone, we are allowing learners to make their own choice.
In the post of AoL, I skipped gamification because it really works best in a formative assessment capacity. Any time that there are countdown timers, score cards, leaderboards, etc. the stress levels of the learners can increase. Additionally, the learners may become distracted with these extra features and spend less time pondering each question. This is not a conducive environment for summative assessments. However, when learners are still in the process of learning and the assessment is AfL, games can be a nice, light, fun way to perform assessment.
I recently completed a workshop by Clint Clarkson, who is a master of game thinking in learning experiences. One of his takeaways was that almost everything can be turned into some form of a game (although as an ID or LXD, some discretion is necessary depending on the subject matter).
My personal preference for gamification is all-encompassing games with a storyline and, most importantly, a purpose. Let me explain my preference with an example of what doesn’t work well, and then compare that to what I would do to fix it.
When workshopping ideas for a learning-based virtual escape room, someone suggested that learners should answer multiple choice questions about the subject matter each time they tried escaping the room.
Now, in my opinion, this makes the learning and the game very disjointed. I don’t see the value of the game itself in this instance. The decisions that the learners make related to how to escape are not at all related to the decisions they need to make regarding the subject matter. You may as well just give them the multiple-choice questions and skip the game.
My idea for learning-based virtual escape rooms is that the learners must use their content-based knowledge to actually escape the room. For example, in an escape room based on ladder safety, the learners would have to select appropriate ladders, inspect them, safely carry them and use them properly in order to get out of the escape room (for example, through a high window). The learning content is 100% related to the escape room game. In these types of instances, the escape room is essentially a massive scenario-based assessment.
Regardless of the format of the game, without a direct relationship to the content, gamification can seem superfluous or inauthentic. Remember that the point of the game is to assess learning – which means that the content should be upfront and centre.
In my post on AoL, I discussed the value of scenarios for summative assessment. Almost every point there would apply to formative (AfL) as well. So, I will avoid repeating myself here (although check out this post to see what I had to say about scenarios in AoL) and simply bring in one additional point: consider how you will provide the feedback.
Scenarios can be built in a variety of ways. One way involves setting up a decision point, which allows learners to branch out to make their choice, but then after giving feedback, the scenario contracts back and continues in the same way, regardless of the choice made by the learners.
This strategy is simpler and faster to create, and also provides excellent opportunities for immediate and personalized feedback.
However, in some cases you may want the scenario to be more immersive. In which case each decision leads the learners on a different path, with dozens (or more) possible pathways through the scenario.
While I generally prefer these types of scenarios, they make it much more difficult to provide immediate feedback unless each node of the branching scenario starts with a very obvious indication of whether the last choice was correct. In these instances, it’s important to remember that feedback is one of the essential components of AfL. Therefore, a thorough debrief should be presented at the conclusion of the scenario, based on the actual choices made by the learner.
5. Tracking results
One final consideration for instructional designers is whether or not the AfL will be tracked. Obviously AoL would need to be tracked to determine if learners have achieved the learning outcomes. However, AfL does not have this same requirement.
While tracking AfL results does add in a layer of complexity in both the in-class or eLearning environment, it can be beneficial to know if learners are struggling long before they reach the AoL. This could provide opportunities for support and coaching. Additionally, when it comes to evaluating the success of the program, it can be useful to know which outcomes the learners struggled with throughout the course (not just on the AoL).
AfL in learning experience design
Learning experience design takes instructional design one step further by considering the learner experience. Since AfL occurs within the learning experience, let’s look at a few ways that can we ensure the learner is having a positive experience on the AfL.
1. Clear purpose and plan
Ensure that your learners know why you are doing the AfL. Let them know what will be assessed and how the assessment will work – particularly for gamified or scenario-based assessments. Additionally, let them know how they will be receiving feedback. Also, and importantly, ensure that they know if and how they are being marked.
2. Clear instructions
There are almost an infinite number of ways to do AfL. Your only limit is your imagination. However, what seems clear and straightforward to you, might not seem so clear to your learner. Let them know how to complete the AfL and what your expectations are. This is true for both instructor-led and eLearning courses. Again, this is particularly important for gamified and scenario-based assessments.
3. Next steps
Ensure that your learners understand what their options are after completing the AfL. Considering that AfL is not a final assessment, but rather than internal assessment to verify that the learners are on track, it is worth having a variety of options for your learners.
For example, learners that successfully complete the AfL may continue on with the course. Learners who just barely miss the passing mark might be able to return to certain sections of the course for review. Learners who are far from passing might need to review everything before moving on.
Ensure that the learners know what the options are and how to access the different options. Additionally, for learners who decide to complete some review before moving on, make a clear statement about whether and how the AfL will be redone.
The final point related to learning experience design is to ensure that the course is accessible (particularly with eLearning courses). Since the sky is the limit in terms of AfL choice (gamification, scenario, drag and drop, multiple choice, etc.), you might want to use accessibility standards as a helpful tool for deciding which methods to use.
Of particular concern are any type of drag and drop, image matching, and any style of AfL that incorporates audio or video.
Consider your learner analysis and/or learner personas, as well as any company- or government-mandated accessibility standards. All of these can help you pick the best AfL for your learners from the start (and reduce time spent redesigning non-accessible courses).
Well, like last week, this post also turned into a long one! If there is one thing to take away from this novel, it’s that AfL is all about the feedback – make sure you know how and when you will provide learners with feedback, and do you best to make it personalized.
Stay tuned next week for the upcoming post on Assessment as Learning (AaL).