Writing and assessing unit level learning outcomes

Writing assessable learning outcomes

Your first step is to consider what it is that your students should be able to do as a result of completing your unit. At the same time, you need to be thinking about how they would be able to demonstrate this to you.  In other words, how would you assess them?

In order to assess something it must be observable rather than something inside their heads like "knowing about xxx". Note: this is not to disparage invisible activity but only observable actions are assessable.

The verbs in our learning outcomes describe the behaviour we are looking for in our students.  Some verbs describe simple behaviour - for example, "to describe". Other verbs are more complex - for example, "to compare". A learner can only effectively "compare" if they first "describe" both things that are being compared. "Comparing" as such is a more complex verb than "describing". In effect, the more complex verb encapsulates less complex verbs.  This is a principle that may be very useful to us where the Undergraduate Curriculum Framework limits the number of learning outcomes we are permitted to set.

How then do we set about choosing appropriate verbs for our learning outcomes?

It helps to have some form of framework to think about the learning we wish to engender in our students.  One option that has commonly used in this context is Bloom's taxonomy (Bloom, 1956).  This describes six increasingly complex levels of understanding that we may wish to encourage in our students.  The premise is that students build on former learning to achieve the higher levels of understanding.  Bloom's original work has been usefully revised by Anderson and Krathwohl's (2000).  The following table shows the expectation of each level of Bloom's revised cognitive domain and some examples of action verbs that match to these expectations. This provides a useful tool to aid you in the selection of appropriate verbs.  It is not an exhaustive list and depending on what you specifically require of your students the relative position of some verbs might be debated.

Table 1 - Taxonomies of learning (Cognitive Domain). Anderson and Krathwohl's (2000) revision of Bloom's


Level

Cognitive Domain

Expectation

Applicable action verbs for learning outcomes

1

Remembering:

Retrieving, recalling, or recognizing knowledge from memory. Remembering is when memory is used to produce definitions, facts, or lists, or recite or retrieve material.

Define, describe, identify, label, list, match, name, outline, reproduce, select, state, recall, record, recognise, repeat, draw on, or recount.

2

Understanding

The students shows understanding of something; showing they have grasped the meaning. Students could show understanding by translating what they learned in a book into actual practice or by interpreting what is known in one context when used in another context.

Students demonstrate understanding if they are able to:
Convert, defend, distinguish, estimate, explain, extend, generalise, give examples, infer, paraphrase, predict, rewrite, summarise, clarify, restate, locate, recognise, express, review, or discuss, locate, report, express, identify, describe how, infer, illustrate, interpret, draw, represent, differentiate.

3

Applying

Carrying out or using a procedure through executing, or implementing. Applying related and refers to situations where learned material is used through products like models, presentations, interviews or simulations.

Apply, change, compute, calculate, demonstrate, discover, manipulate, modify, operate, predict, prepare, produce, relate, show, solve, use, schedule, employ, sketch, intervene, practise, or illustrate.

4

Analyzing

Breaking material or concepts into parts, determining how the parts relate or interrelate to one another or to an overall structure or purpose. Mental actions included in this function are differentiating, organizing, and attributing, as well as being able to distinguish between the components or parts. When one is analyzing he/she can illustrate this mental function by creating spreadsheets, surveys, charts, or diagrams, or graphic representations.

Analyse, break down, make a diagram, classify, contrast, categorise, differentiate, discriminate, distinguish, appraise, test, inspect, illustrate, infer, outline, relate, select, survey, investigate, make an inventory, calculate, question, contrast, debate, compare, or criticise.

5

Evaluating

Making judgments based on criteria and standards through checking and critiquing. Critiques, recommendations, and reports are some of the products that can be created to demonstrate the processes of evaluation.  In the newer taxonomy evaluation comes before creating as it is often a necessary part of the precursory behavior before creating something.

Appraise, assess, argue, compare, conclude, contrast, criticise, discriminate, judge, evaluate, choose, rate, revise, select, estimate, measure, justify, interpret, relate, value, measure the extent, validate, summarise.

6

Creating

Putting elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure through generating, planning, or producing. Creating requires users to put parts together in a new way or synthesize parts into something new and different a new form or product.  This process is the most difficult mental function in the new taxonomy. 

Compose, design, plan, assemble, prepare, construct, propose, formulate, set up, predict, deriver, elaborate, invent, develop, devise, rearrange, summarise, tell, revise, rewrite, write, modify, organise, produce, or synthesise.

Download a printable version of Anderson and Krathwohl's (2000) revision of Bloom's

In addition, to revising and refining the levels of Bloom's Anderson and Krathwohl's re-emphasise the relationship between the taxonomies and Bloom's original related concept of types and levels of knowledge.

These knowledge dimensions are defined as follows:  

  • Factual Knowledge is knowledge that is basic to specific disciplines. This dimension refers to essential facts, terminology, details or elements students must know or be familiar with in order to understand a discipline or solve a problem in it. 
  • Conceptual Knowledge is knowledge of classifications, principles, generalizations, theories, models, or structures pertinent to a particular disciplinary area.   
  • Procedural Knowledge refers to information or knowledge that helps students to do something specific to a discipline, subject, area of study. It also refers to methods of inquiry, very specific or finite skills, algorithms, techniques, and particular methodologies. 
  • Metacognitive Knowledge is the awareness of one's own cognition and particular cognitive processes. It is strategic or reflective knowledge about how to go about solving problems, cognitive tasks, to include contextual and conditional knowledge and knowledge of self. 

Start with partial learning outcomes

At this point we are in a position to articulate partial learning outcomes based on the verbs we have chosen.  The following table provides several examples of partial learning outcomes.

Table 2 - examples of partial learning outcomes


Level

Cognitive Domain

Example partial learning outcome

1

Remembering:

List the criteria to be taken into account when caring for a patient with emphysema
Define what behaviours constitute unprofessional practice
Describe the processes used in creating a design brief for a client

2

Understanding

Give examples of good financial management
Locate management strategies observed on placement within a continuum of good and poor management as described in the textbook
Explain the forces that led to a [named] change in a [stated] historical context

3

Applying

Show how changes in the criminal law affected incarceration of women in Scotland in the mid 19th century
Modify guidelines in a case study of a small manufacturing firm to enable tighter quality control of production
Use a range of communication skills in [situation x]
Solve problems in x

4

Analyzing

Assess the impact of a new training programme for newly appointed social workers
Relate how students perform in placement to their classroom teaching
Compare the practice of a newly qualified restaurant manager with that of someone who has been employed for 10 years

5

Evaluating

Justify a decision to do x
Reflect upon your experience of doing x
Summarise the advantages and disadvantages of doing x

6

Creating

Prepare a 10 minute presentation on topic x
Design a new product
Organise a patient education programme

Adapted from Carroll (2001)

Something to avoid

Using phrases like "have an understanding", "show awareness of", "have an appreciation", and "have a knowledge of" in learning outcomes is often debated. The problem is that the verbs contained have at degree of ambiguity associated with them and do not really make it clear to the student what they are expected to do.  These forms of words are more suited to programme or unit level aims rather than learning outcomes.

This was touched on in the earlier video comparing aims and learning outcomes:

 

 

Here is a quote from the HEA guidelines at:

https://www.heacademy.ac.uk/resource/writing-learning-outcomes.

"Our aims in teaching a particular module may be to engender 'understanding' or 'appreciation' of a particular topic. Learning outcomes should not use terms such as 'understand' or 'appreciate' as it is not immediately obvious to a student what they have to do in order to demonstrate that they 'understand' or 'appreciate' something. Tutors should think about how they ask their students to demonstrate their understanding. They may ask students, during an examination, to describe a process, to discuss a concept, to evaluate some data or to derive an equation. These are the tasks that the student actually does in order to demonstrate understanding, so these terms can be used to express the leaning out-comes”

The solution is to ask the question what would a student be able to do if they had the level of knowledge, understanding or appreciation I am looking for.  You can them return to Table 1 and select an appropriate verb to express this as an assessable learning outcome.

Completing your learning outcomes

Once you, as the teacher, have decided what knowledge and skills your students should demonstrate, you need to add the context in which students will demonstrate their learning and how well they must do so. Here are some examples of how to complete learning outcomes, building on the statements showing synthesis above:
After the course, the student will be able to:

  • Prepare a 10 minute presentation on topic x suitable for a student seminar. The seminar should enable fellow students to tackle the questions relevant to the topic on the final exam.
  • Design a product from a brief using the materials listed in the brief and appropriate to xxx market. The product should be innovative in that it should be different from anything currently on the market.
  • Organise a patient education programme, using the resources provided to you, that will teach topic x to y number of people over z time.

The following short video looks again at the learning outcomes for this resource to demonstrate this:

 

 

A summary: writing learning outcomes

All learning outcomes must have

  • A verb to describe the behaviour which demonstrates the student's learning
  • Information about the context for the demonstration
  • And finally: Learning outcomes must not all come from the lower levels of Bloom's taxonomy (i.e. knowledge and understanding). Claims for credit in Higher Education must include synthesis and analysis, evaluation and application of knowledge.

The higher the level of credit, the more autonomy, unpredictability, novelty and decision-making a student would expect to show to the assessor. Although a five year old and a Masters level student can both critically analyse, the context, information and outcomes would be different! To some extent, learning outcomes capture that difference although assessment criteria will be needed in addition to fully differentiate the two groups' performance. but that is another story.

 

next: Evaluating unit learning outcomes »