Learning and Teaching in Action: Assessment

Student in language lab

 

Deploying formative assessment during field work: using a classroom response system (CRS) outdoors on an ecology fieldtrip

Rod Cullen, Mark Langan, Robin Sen, Paul Chipman

Abstract

In bioscience subjects fieldwork can provide an interactive, rich and challenging learning environment in which our students gain experience of many practical skills that will be vital to them as graduates. However, they can be logistically difficult to manage and tutors can find themselves too busy organising the fieldwork activities to engage deeply with students, test their understanding and provide meaningful feedback.

The use of classroom response systems (CRS) is increasing in UK Higher Education. The AID4A project at MMU is currently evaluating Promethean’s CRS, known as ActivExpression (AE). As part of this project we were interested to see if the strengths that are often attributed to CRS (e.g. increased interaction with learning tasks, ability to elicit responses from a whole class and consequently to provide rapid feedback) might allow us to address some of the problems that tutors encounter in running fieldtrips. As far as we are aware a CRS has not been deployed in this way before.

We were able to successfully deploy the CRS outdoors (handsets operated to a distance of 20 metres) despite adverse weather conditions and some minor technical difficulties. As a proof of principle we were able to deliver and utilise the results of activities using the CRS that tested background knowledge, demonstrated some basic principles in making estimates and measurements in the field, encouraged reflection on the fieldwork and linked to prior learning.

Background

Fieldwork in biosciences

Fieldwork is generally regarded as an essential component of most bioscience degree programmes and is held in high regard by practitioners in many disciplines (Fuller et al. 2006). Fieldwork can take many different forms. Based on the definition of Lonergan & Andreson, (1988 p. 64) that fieldwork may be “any arena or zone within a subject where, outside the constraints of the four walls classroom setting, supervised learning can take place via first hand experience”, Gold et al. (1991) described five categories:

  1. Short field excursion: limited travel in limited time
  2. ‘Cook’s tour’: limited activity and extended travel
  3. Residential course
  4. Study tour: multiple locations/activities
  5. Project work (i) learner-driven; (ii) observational (instructor-driven)

There is a great deal of evidence that learners often enjoy their experiences of fieldwork whilst learning effectively (Fuller et al.
2006; Scott et al. 2006). The opportunity to experience (outdoor) complex systems and processes provides opportunities that cannot be duplicated in the classroom. In terms of the delivery of fieldwork, there has been a great deal of movement towards designing learning to be active, rather than passive. Such approaches are considered more effective, particularly with small-group approaches to problem-solving. There is also evidence that small group exercises can be more effective at promoting learning than whole class exercises (Simm & David, 2002). In our own experience, fieldwork provides an interactive, rich and challenging learning environment in which our students gain experience of many practical skills that will be vital to them as graduates.

Despite the obvious benefits, fieldwork is not without its problems and issues. For many reasons (too complex to go into here) opportunities for fieldwork within programmes are limited and can be expensive, making it essential to maximise the learning benefits associated with them. Logistically, fieldtrips can be difficult for tutors to manage. Time restrictions mean that tutors often need to concentrate their efforts on organising students and ensuring that all of the required tasks are completed. This can make it difficult to allow students to experiment with different techniques and approaches. In addition students can become very task focused, working through methods and instructions without really thinking deeply about the techniques they are employing. In short tutors can find themselves too busy organising the fieldwork activities to engage deeply with students, test their understanding and provide meaningful feedback.

Classroom response systems

Classroom response systems are a relatively new technological addition to classrooms in UK Higher Education, although they are becoming increasingly common in many educational settings (Fies & Marshall, 2006). The CRS normally include a set of individual hand-held or desktop keypads that transmit student votes/responses (usually via infrared or other wireless technology) to a central device that collates, analyses and displays results to a classroom, normally via a whiteboard system.

Fies & Marshall (2006) report that there is great agreement that CRS promote learning when coupled with appropriate pedagogical methodologies while others have reported that their use significantly improves interactivity in the classroom (Siau et al. 2006).

One of the main reasons that tutors make use of CRS is to increase the ease with which frequent formative assessment of students can be made (Roschelle et al. 2004). With CRS such feedback is elicited from everyone in the class (rather than just from individuals who choose to put up their hands in response to questions) and allows tutors to evaluate the understanding and address the needs of a whole class (Caldwell, 2007).

ActivExpression Interactive Devices for Assessment (AID4A) Project at Manchester Metropolitan University

The AID4A project has been initiated by the Transformative Learning Centre (formerly the Promethean Centre of Excellence) at MMU in partnership with Promethean Ltd and is supported by the Dean of the Institute of Education (IoE) and the Head of the MMU Centre for Learning and Teaching (CeLT).

ActivExpression is a unique new CRS with the ability to text complete sentences and numerical responses in addition to the more usual selection of options from multiple choice questions. The software that supports ActivExpression is an extension to Promethean’s Active Studio software used with the Promethean interactive whiteboards that are installed in many of the universities lecture theatres and classrooms. Promethean have supplied the project with 416 individual devices (13 sets of 32) as well as the licences for ActivExpression software upgrade for the Interactive whiteboards.

The project provides an opportunity for tutors, during 08/09, from the IoE and across MMU, to research and evaluate the use of cutting edge, innovative electronic assessment devices (ActivExpression) whilst responding to the university’s “Challenging Assessment” initiative that was launched at the beginning of April, 2008. This has given rise to a range of projects researching and evaluating the use of this CRS across the university. Further details of the AID4A project are available from Maureen Haldane (m.haldane@mmu.ac.uk).

Rationale for using a CRS on a fieldtrip

As part of the AID4A project we were interested to explore the potential of a CRS to address some of the problems and issues that we have identified for tutors running field courses. It seemed to us that the strengths that are often attributed to CRS (e.g. increased interaction with learning tasks, ability to elicit responses from a whole class and consequently to provide rapid feedback) might allow us to address some of these problems. We were also very interested in trying to use a CRS outdoors on a field course setting and, as far as we are aware from our literature review a CRS systems has never been deployed in this way before.

Lindow common fieldtrip

The Lindow common fieldtrip introduces key ecological techniques and data collection methods used in vegetation surveys as part of a summative assignment “Collection and Analysis of Vegetation Survey Data” for level 5 (2nd year undergraduate n = 34; 3 staff) students. This assignment contributes 40% to the final mark of the unit EG2206 Landscape and Ecology, which is part of BSc (Hons) Degrees in Ecology and Environmental Sciences/Management within the Department of Environmental and Geographical Sciences.

The fieldtrip aims to get students to engage critically with the field environment, key ecological methodologies and analytical techniques that underpin the learning objectives for a written assignment. As a result students should gain an understanding of the strengths and weaknesses of the techniques they have used. This field trip provided the perfect opportunity to deploy and test the CRS.

Evaluation questions

In this preliminary evaluation we wanted to explore the following questions in relation to the CRS.

  1. Can the technology be used outdoors and what are the practical limitations to this use?
  2. Can we devise and deploy activities that will give us useful and immediate feedback about students’ basic knowledge and background of moorland and woodland ecology?
  3. Can we devise and deploy activities that demonstrate some of the main problems of the data collection techniques employed in the field?
  4. Can we devise and deploy activities that will encourage students to reflect on:
    1. their understanding of the limitations of the field techniques and data collection methods?
    2. appropriate methods of statistical analysis (linking into prior learning from a core level 4 unit in statistics)?
  5. Can we provide student feedback via WebCT on the same day?

Activities

As a proof of principle we devised a set of six questions/activities involving the students’ use of the CRS throughout the field trip (Table 1)

Table 1: Active Expression deployment through the field course
Question/Activity Delivery/Location

Basic knowledge and background of moorland ecology

Q1. Which three deciduous hardwood tree species are native to the UK?
  A.Ash B.Beech C.Scots pine D. Oak E. Larch

Q2. Name two common moorland plant species

In the car park during the introductory talk.

Results allow the tutors to address errors in knowledge base.

Exemplify the main problems of the data collection techniques

Q3. Estimate percentage cover of heather (Calluna vulgaris) in the quadrat.

Q4. At a distance of 10m, what is the angle between the horizontal and the top of the tree (tied with a yellow ribbon)?

During the fieldwork and data collection activities.

Results will alert the students to the inherent variability between operators in making estimations and taking measurements.

Reflection and links with prior learning

Q5. What is the missing tree biomass that does not appear in the calculation of standing biomass (hint: the hidden half)?

Q6. What two statistical tests could you apply to confirm greater biomass of purple moor grass in the heathland versus woodland sites at Lindow common?

 

On the bus before leaving on the return journey.

Results will give an indication to tutors level of reflection of the methods and ability to link to relevant prior learning at level 4 (1st year undergraduate).


Findings

1. Can the technology be used outdoors and what are the practical limitations to this use?

Set up
The ActiveExpression software for the CRS was installed on a light-weight tablet laptop that could be used in a hand held fashion and easily carried while in the field (Figure 1). Thirty students were each issued with a CRS handset and four students shared one between two (32 handsets in total).


Figure 1

Weather conditions
On the day, the weather proved to be typically English and it was raining heavily with a blustery wind from the time that we left the coach until the end of the 3 hour field trip. Obviously this is not the ideal environment for using electrical equipment and a touch sensitive computer screen.
We adopted a simple low tech solution of putting the laptop inside a clear plastic bag. Surprisingly the tablet operated perfectly using the pen mouse from outside the bags even in heavy rain (Figure 2).


Figure 2

There was a slight technical problem with the hibernate setting of the tablet being too short (3 minutes) which closed the CRS session down before all of the submissions for the first question were complete. This was rectified by setting the hibernate timing for the tablet to 60 minutes.To protect the CRS handsets from the rain they were also placed inside protective plastic bags (Figure 3). None of the students reported that this impaired or hampered the operation of the CRS handsets.


Figure 3

The weather conditions did prove problematic when presenting some of our questions to the students. As we were in the field we could not present the options for the multiple choice questions via a white board or other screen so we simply wrote them on a paper flip chat (Figure 4). This rapidly became wet and difficult to read. For some of our questions this was not an issue, however, we would need to give some consideration to a better way of presenting the options for multiple choice questions to our students in such conditions.


Figure 4

 

Data transmission outdoors
The students and their CRS handsets were normally in close proximity to the tablet (2-10 metres) when inputting their answers (Figure 5). Despite the wet and windy conditions data transfer between the pods and the tablet was very fast and we did not experience any connection problems.


Figure 5

We were able to pick up input from the pods at a distance over 20 metres with no problems (Figure 6). We did not have time to test at greater distances.


Figure 6

2. Can we devise and deploy activities that will give us useful and immediate feedback about students’ basic knowledge and background of moorland ecology?

As a proof of principle we delivered two questions to test our students’ basic knowledge and background of moorland ecology during the introductory talk in the car park at Lindow Common (see Table 1 above). Although the majority of the students responded correctly to the multiple response required for Q1 (Figure 7) we were able to highlight misconceptions in a third of the students.


Figure 7

The results from the poll were very easy to interpret and despite the conditions (there was quite a bit of glare on the screen) were easy to read. The results led into a short, tutor-directed discussion of native British species, differences between coniferous and deciduous species and hard and softwoods.

Although this is a very simple example it does demonstrate a basic point. If we had simply asked the question of the class and taken the answer from the first person to raise their hand the chances are that they would have got the correct answer. Students with a misunderstanding may be reluctant to admit that they either do not know the answer or that their thinking was incorrect. Consequently the tutor gets a false impression of the knowledge base of the students. The CRS poll equips the tutor with more information on the knowledge and understanding of the whole class.

Question 2 was delivered as a text entry question as we wanted students to think of more than one response without being presented with options that could act as prompts. The basic text entry questions format in AE assumes a single answer as a response so we improvised and asked the students to enter their two answers as a comma-separated list. This worked reasonably well and it was possible to feed back to the group on the correctness of the species they had listed as a group. However, each of the 32 responses of two species was unique and it was difficult to get a picture of which were the most commonly listed individual species while in the field.

It was relatively simple to manipulate and use the information submitted on returning to the university to produce Figure 8 as a summary. This information was then available to be used in a short follow-up sessions linked to the field trip the following week.


Figure 8: Output from Q2 required some manipulation from the exported Excel spreadsheet to be more easily interpreted

 

3. Can we devise and deploy activities that demonstrate some of the main problems of the data collection techniques employed in the field?

As part of the field trip, students were required to employ data collection techniques that involved estimation and measurement.
In particular the students were asked to:

(i). Estimate the % cover of different plant species within quadrats.

The technique is highly subjective and there can be a great deal of variation between individuals making estimations of percentage cover of a species within the same quadrat (a quadrat is a device used to delimit square sampling areas in surveys of living things - see Figure 9). This has implications when making comparisons between the estimations of different individuals or when data are combined from the surveys of different individuals to increase sample sizes. It is important that students are aware of such variation and give consideration of ways in which such variation can be reduced or eliminated.


Figure 9

In order to demonstrate this, a 50cm x 50cm quadrat was placed over some moorland vegetation (Figure 9) and the students were asked to estimate individually the percentage cover of heather (Calluna vulgaris) within it. This was done in two groups (Group 1: n=14, Group 2 n=16 students).

A numerical input question was used to record the individual estimations made by each group of students. The results showed considerable variation in the individual estimations for both groups (Figure 10).

Figure 10: Variation in students' estimations of percentage cover of heather in the quadrat for groups 1 & 2

We were able to present these results immediately to each group and to have some discussion of the reasons for such wide variation (45-90%) and ways in which the variation could be reduced. These discussions were followed up in subsequent data analysis and review sessions linked to the fieldtrip.

(ii). Measure the angle between the horizontal and the top of a tree at a known distance from the base of the tree in order to calculate the height of the tree.

Calculating the height of trees is an essential part of calculation of above-ground biomass which was one part of the fieldtrip. Students were required to measure the angle using a hand-held clinometer (Figure 11). Taking measurements with this sort of equipment can be tricky and, as with estimating percentage cover of vegetation, there can be a great deal of variation between individual operators making the same measurement. It is important that students are aware that variation occurs as a result of operator error and that they consider ways in which such errors can be reduced or eliminated.

Figure 11

In order to demonstrate this, students were asked to measure individually the angle between the horizontal and the top of the same tree at a known distance from the base of the tree. This was also done in two groups (Group 1: n=14, Group 2: n=16 students).
A numerical input question was again used to record the individual angle measurements made by students in each group. The results showed considerable variation in the individual measurements made for both groups (31°-65°; see Figure 12).

Figure 12: Variation in students' tree angle measurements for groups 1 & 2

The variation in the angle measurements was surprisingly high and when presented to the students led to discussion of the reasons for the variation (operator height, taking measurements at different distances from the tree, inexperience in use of the clinometer etc) and ways of increasing the consistency and reliability of the measurements. Again these discussions were followed up in later sessions linked to the field trip.

 

4. Can we devise and deploy activities that will encourage students to reflect on:

  1. their understanding of the limitations of the field techniques and data collection methods?

  2. appropriate methods of statistical analysis (linking into prior learning from a core level 4 unit in statistics)?

On completion of the required fieldwork activities the students returned to the coach where the CRS was used to deliver two reflective questions addressing (i) and (ii) respectively. Question 5 is quite a simple question, to which the vast majority of the students responded correctly (Figure 13), we were able to determine that the students had given some thought to the calculation of biomass.


Figure 13: AE results for Q5

Question 6 results were useful in follow-up sessions (e.g. plant roots and productivity) linked to the field trip where the issue of complexity in the calculation of biomass was raised (Figure 14).

Figure 14: AE responses to Q6

Question 6 is a more difficult question in that it is approximately six months since the students completed their core level 4 unit in statistics. As with question 2, we asked the students to list two answers in a simple text entry question. We had the same problem with the presentation of the results as there were 30 more or less unique responses. It was immediately clear however that a lot of the students had listed incorrect statistical techniques for making the required comparison. Although it was not appropriate to go into great detail about the correct statistical approach on the coach the tutor was made aware that this needed to be revisited. This issue was subsequently given proper consideration in the follow-up session to the field trip where the analysis of the field data began and the requirements for the written assignment were discussed.

 

5. Can we provide student feedback via WebCT on the same day?

The Activexpression (Professional Edition) CRS records student responses to questions in a variety of graphical formats to an electronic flipchart. It was disappointing to find that, unlike in Promethean’s ActiveStudio, it is not currently possible to export flipcharts to MS Word, MS PowerPoint or as HTML as the software is currently still in development. In order to make the results available to students in WebCT the only options were to:

  1. use screen capture software and a graphics editing package to copy the figures into a PowerPoint document.
  2. export the session data into MS Excel spreadsheet and manipulate this to produce an appropriate figure (see Figure 8).

This proved to be more time consuming than anticipated and it was not possible to make the results along with some photographs and comments available to the students until the following day.

 

Student perceptions

We did not formally evaluate the fieldtrip or the deployment of the CRS from the student perspective as we were mainly testing that the system would work in this setting. What we can say is that all of the students participated and gave reasonably considered responses to all of the questions delivered to them. The activities delivered and the responses received generated discussion both in the field and in follow-up sessions associated with them.

We used the CRS to asked one simple question about the use of the CRS (Figure 15).


Figure 15: Students' thoughts on the use of CRS in lectures

The majority (85%) of the students on this field trip were in favour of CRS being used in their lectures. Although this is a very positive response it is based on our students’ very limited experience of a CRS in quite an unusual setting. We plan to investigate student attitudes and the impact in task engagement and learning in future work.

Conclusions

We successfully used the CRS technology outdoors to deliver all of the six questions/activities that we built into the fieldtrip. This was despite the adverse weather conditions including heavy rainfall and high winds. The hardware needed to be protected from the elements but worked well nevertheless and data transmission, despite the wind and rain, was effective up to the max distance we tested (20 metres).

We were able to deploy simple proof of principle activities that:

  • tested our students’ basic knowledge and background of moorland and woodland ecology. The leaders of the fieldtrip were able to respond immediately, based on the strengths and weaknesses which were identified.
  • generated data sets that demonstrated, in the field, the inherent variability in making estimations and measurements (of percentage vegetation cover and angles for the calculation of tree heights respectively). This provided a relevant stimulus for discussion during the fieldtrip and in follow-up sessions associated with the fieldtrip.
  • encouraged students to reflect upon the methods they had used in the field and make links to prior learning and experience in statistical analysis. This allowed the tutors to pick up on any misunderstandings in the follow-up sessions.

Despite the fact that it was not possible to export the electronic flipcharts into a format that could be made available online directly after the fieldtrip, it was possible to extract the relevant figures using screen capture software and make this available to students, via WebCT, within 48 hours. This will not be a problem in future version of ActivExpression as it will have the same export functions that are available in Promethean’s Active Studio software.

Based on our limited experience of using the ActivExpression CRS outdoors in a fieldtrip setting we have made the following recommendations for product development.

  1. More sophisticated text entry questions to allow input of lists where list items can be analysed as separate responses.
  2. More options for basic MCQ questions (six is not enough).
  3. Many of the students commented that the handsets would be a great way of recording field data. This would require some form of asynchronous use and perhaps a way of managing the handsets in groups.
  4. The range of output options for the AE professional flipcharts needs to be equivalent to those in Active Studio to enable results to be made available more quickly online.
  5. Although our plastic bag solution worked well to keep the laptop dry in the wet conditions there was a lot of glare on the screen. This would be a much bigger problem in bright sunlight. Some form of cover or shade to fit to the laptop would be helpful although we recognise that this may not be something Promethean has the capacity to develop.

References

Caldwell, J. E., (2007) Clickers in the Large Classroom: Current Research and Best-Practice Tips, CBE Life Sciences Education 6(1): 9-20.

Fies C. & Marshall, J. (2006) Classroom Response Systems: A Review of the Literature Journal of Science Education and Technology, 15 (1), 101-109.

Fuller, I. C., Edmondson, S, France, D., Higgit, D. & Ratinen I. (2006) International perceptions on the effectiveness of geography fieldwork for learning. Journal of Geography in Higher Education, 30(1), 89–101.

Gold, J. R., Jenkins, A., Lee, R., Monk, J., Riley, J., Shepherd, I. D. H. & Unwin, D. J. (1991) Teaching Geography in Higher Education. Oxford: Blackwell.

Lonergan, N. & Andreson, L. W. (1988) Field-based education: some theoretical considerations, Higher Education Research and Development, 7, pp. 63–77.

Roschelle, J., Penuel, W. R., and Abrahamson, L. (2004). Classroom Response and Communication Systems: Research Review and Theory. In: Annual Meeting of the American Educational Research Association. San Diego, CA, 2004.

Scott, I., Fuller, I. C. & Gaskin, S. (2006) Life without fieldwork: some staff perceptions of geography and environmental science fieldwork. Journal of Geography in Higher Education, 30(1), 161–171.

Siau, K., Sheng, H., & Fui-Hoon Nah, F. (2006) Use of a Classroom Response System to Enhance Classroom Interactivity, IEEE Transactions On Education, 49(3), 398-403.

Simm, D. J. & David, C. A. (2002) Effective teaching of research design in physical geography. Journal of Geography in Higher Education, 26(2), 169–180.

about the author

Rod Cullen
Centre for Learning and Teaching

e-mail: r.cullen@mmu.ac.uk
telephone: 0161 247 3356

Mark Langan
Dept of Environmental and Geographical Sciences

e-mail: m.langan@mmu.ac.uk
telephone: 0161 247 1583

Robin Sen
Dept of Environmental and Geographical Sciences

e-mail: r.sen@mmu.ac.uk
telephone: 0161 247 6208

Paul Chipman
Dept of Environmental and Geographical Sciences

e-mail: p.chipman@mmu.ac.uk
telephone: 0161 247 1575

Download this article as a .pdf file

Autumn 2008
ISSN 1477-1241