Editorial
Rachel Forsyth

The Concept of Plagiarism
Bill Johnston

Plagiarism Detection Software - a new JISC service
Rachel Forsyth

Can students assess students effectively? Some insights into peer-assessment
A. Mark Langan and and C. Philip Wheater

Exploring the potential of Multiple-Choice Questions in Assessment
Edwina Higgins and Laura Tatham

Developing a new assessment strategy
Gaynor Lea-Greenwood

Assessing the Un-assessable
Tim Dunbar

How to assess disabled students without breaking the law
Mike Wray

Returning Feedback to Students via Email Using Electronic Feedback 9
Phil Denton

Tools for Computer-Aided Assessment
Alan Fielding and Enid Bingham

Faculty Learning and Teaching Reports

Learning and Teaching News from the Library

| View this article as a .pdf file |

Philip Denton
School of Pharmacy and Chemistry, Liverpool JMU

Returning Feedback to Students via Email Using Electronic Feedback 9

The software discussed in this article is available for free download at http://cwis.livjm.ac.uk/cis/download/xlfeedback/welcome.htm
Alternatively, please email the author to receive the program as an attachment.

Abstract

Electronic Feedback is an MS Office marking assistant that has been developed at Liverpool John Moores University over the last 3 years. The software enables assessors to readily generate and return feedback to students, in the form of MS Word processed reports and email messages. Research indicates that the previous version of the software was used by 22 UK academics across a range of disciplines. They agreed that the program enabled them to return more feedback, of higher quality, in a shorter space of time. The latest software, Electronic Feedback 9, is considered to be more user-friendly and incorporates a novel collusion detection facility. The operation of this program is described in this article.

Introduction

As the Computer Assisted Assessment (CAA) unit at Loughborough University have noted, computerised testing can ease difficulties associated with large class sizes and diverse student capabilities (1). They comment that, “CAA is essentially automated assessment through objective testing.” Although the terms CAA and objective testing are often used interchangeably, electronically aided assessment has a much broader scope and a more comprehensive description of CAA would include:

• Objective Testing
  This could be multiple choice or text match type questions, delivered via the Web.
   
• Electronic Submission
  At a simple level, this could be students presenting work to their tutor via email. Furthermore, some Virtual Learning Environments (VLEs) can support threaded discussions that enable students to contribute to an on-line debate.
   
• Free Text Analysis
  There is a range of plagiarism detection tools that can be used to check for similarities between electronic text files. Some programs use search engine technology to look for matching text on the Web (2). Software capable of automatically grading free text is available (3), but is still in development and is not suited to high stakes assessment at present.
   
• Marking Assistants
  Packages that can aid in the computation of student marks are used regularly by academic staff, e.g. MS Excel. Advanced marking assistants can generate written reports (4).

As Brown (5) and Gibbs (6) have discussed, returning feedback to students on their work plays an important role in the learning process. Accordingly, providing undergraduates with comprehensive and timely information on their progress is considered to be one of the fundamental responsibilities of a HE tutor (7). Although electronic marking assistants have the potential to improve the return of feedback to students, they are not used routinely by academic staff. This is perhaps surprising, given that a recent summary of HE inspection reports concluded that UK Universities do not supply students with sufficient information on their academic development (8).

Electronic Feedback (9,10) is an MS Office marking assistant, developed at Liverpool JMU over the last three years. The program enables tutors to readily generate MS Word processed feedback reports that may be automatically emailed to individual students as messages or attachments. Previous research indicates this method is viewed positively by students (9). Version 8 of the program was used by 22 lecturers across a range of academic disciplines during the academic year 2001/2002. Results of a recent emailed questionnaire (average Likert scale responses, 1 = strongly agree, 5 = strongly disagree) indicated that tutors agree that by using the software there were able to return more feedback (2.1), of higher quality (1.6), in a shorter space of time (2.1). Tutors were neutral, however, as to whether their colleagues could use the program without formal training (2.9). To remedy this, the program has been rewritten from scratch, with new features included to ensure a more user-friendly interface. Version 9 of the program was released in July 2002 and its operation will now be considered.

Method

Electronic Feedback 9 consists of three files:

• Feedback9.xls, an Excel file into which data relating to a particular assessed is inputted.
   
• Guide9.xls, an interactive guide to the software. Users can navigate through this workbook using a pair of drop-down picking lists that show the contents of the guide.
   
• Fb9.doc, a Word document that automatically formats and emails feedback reports.

When ready to use the software, the tutor opens Feedback9.xls. This file is composed of a number of sheets, accessible from a main menu, Figure 1. After configuring the software so that it is appropriate for their personal and institutional use, tutors input data relating to particular assessment event into these sheets, following the route indicated by the arrows on the Main Menu sheet. When this procedure is complete, tutors can use the software to create, email and print feedback reports.

Figure 1 Main menu of Electronic Feedback 9

The system allows the following raw data to be inputted:

• Student Details
  This is a list of student names and email address. Registration numbers and salutations may also be entered, the default salutation being the student’s first name. During configuration of the software, it is possible to organise this sheet so that it matches the format preferred by the user’s institution. In this way, existing electronic data can be copied and directly pasted into the program.
   
• Grade Comments
  These are feedback statements activated by the student’s overall % mark, e.g. “First class work” for all students attaining 70% or above.
   
• General Comments
  These are included on all feedback reports generated by the software. Three separate general comments can be entered; top, middle, and bottom, these designations indicating their position on the report.
   
• Personal Comments
  This is a feedback statement for an individual student.
   
• Standard Comments
  These are those remarks that are anticipated to be required repeatedly during marking. They can be written in advance, or gradually composed as marking proceeds. Standard comments can be entered as a list, ‘normal mode’, or organised under discrete headings, ‘criterion mode’. This latter arrangement is of particular use when marking essays. In either mode, each standard comment is assigned a unique reference number. Some users may prefer to annotate the student’s work with these numbers, so that students can see exactly where the tutor would want a particular comment to appear.

Comments may include the special character ~, which is automatically replaced with the student’s salutation when feedback is created. In this way, it is possible to personalise even standard and general comments. Other special characters, , ^, { and }, allow tab-spaces, line breaks, superscripts and subscripts to be incorporated into feedback remarks.

It is recognised that the inputting of raw data is the most time consuming aspect of using the software, although Version 9 has a new facility that can make this process less onerous. It is possible to save standard, grade, and general comments, and student details in separate files, using the export procedure. The data in these files may then be readily retrieved using the import facility. This is useful feature when the same assignment, e.g. a laboratory exercise, is set year after year. In such cases, last year’s standard comments can be imported and used again. Work is currently underway to create a dedicated website where users of the software would be able to upload exported files of standard comments, for download and import by other users.

The software has a number of additional features that give the Electronic Feedback method the edge over traditional ‘red pen’ marking:

• Allocate Comments
  This is an interface that allows tutors to rapidly assign the standard comments to individual students. After selecting the student to be considered, tutors input the reference numbers of the standard comments they would wish to award that class member. Alternatively, the user can select the statement itself from a drop-down picking list. This latter approach has the advantage in that the tutor is not required to memorise the reference numbers.
   
• Automark
  Instead of entering student scores manually, % marks can be calculated automatically by activating the Automark feature. In this way, a student’s final mark is determined by the feedback statements allocated to them. In criterion mode, for example, it is possible to assign percentage weightings to individual criteria, e.g. introduction, and use these to determine the final mark.
   
• Allocation Statistics
  This sheet gives the tutor an idea of how the standard comments have been allocated across the group. It may reveal, for example, that a particular feedback statement was allocated to a large proportion of the class. This information is of clear value to the assessor. It can, for example, be used to inform future teaching strategies by directing the tutor’s attention to those aspects of an assignment that caused the greatest difficulty.
   
• Allocation Check
  This feature allows tutors to check for pairs of students allocated similar standard comments. Once two students who match the search criteria are found, their original scripts can be inspected to confirm if any similarities are coincidental or suspicious. Initial work has indicated that this approach can be used to detect plagiarism (11), even in hand written scripts marked a few days apart.

When all standard comments have been allocated, and personal comments have been inputted, the user may generate feedback reports. The options sheet allows the user to control their appearance. Thus, the tutor can select to hide or show the standard comment reference numbers, for example. Users may also decide to incorporate the allocation statistics on the feedback reports. The create feedback button on the main menu activates a dialogue box that gives the tutor the option to generate, print and email the feedback reports for all students awarded a mark, allocated a standard comment, or assigned a personal comment. If they prefer, tutors can first generate the reports, view them within MS Word to confirm they are acceptable, before emailing. Emails can be sent either as an attachment or as a message and users can input their preferred subject line. If emails are sent as a message, any URLs contained within the feedback will be ‘clickable’.

In principle, it is possible to configure virtual learning environments, such as WebCT, to automatically extract information from files created using Electronic Feedback. In this way, student names and marks can be rapidly uploaded and added to institutional records.

Results

A specimen feedback sheet produced in criterion mode is shown, Figure 2. The report also includes the allocation statistics.

Figure 2 Example Feedback Report Produced in Criterion Mode.

Conclusions

The software has been found to run smoothly and without major difficulty on Office 97 and 2000. Recent refinements (Version 9.3) mean that the software also works on Office XP. The automatic activation of MS Word from MS Excel runs more smoothly in Version 9 than in previous versions. Moreover, Feedback 9 files take up significantly less file space.

It is considered that Version 9 is far more user-friendly than Version 8. The previous software evolved over 2 years, with additional features being ‘bolted-on’ to the original template. This developmental process did not lend itself to user-friendly design! Version 9 makes extensive use of drop-down picking lists. These clearly indicate the restrictions on data entry, where these apply, and facilitate rapid data entry.

The principle of checking feedback for similarities can be used as a means to identify those students who have colluded on their assignments, although further research is required to establish the scope of this approach. Indeed, it is the intention of the author to survey existing users of the software in May/June 2003 to ascertain overall impressions of this new program. Informal feedback, however, gathered during a series of recent presentations to UK University staff, indicate that staff are enthusiastic about Electronic Feedback 9.

Further Information

The software is available for free download at http://cwis.livjm.ac.uk/cis/download/xlfeedback/welcome.htm
Alternatively, please email the author to receive the program as an attachment.

Acknowledgements

This work has been supported by a Liverpool JMU Teaching-Related Research Awarded.

References

(1) http://www.lboro.ac.uk/service/ltd/flicaa/why_caa.html

(2) Bull J, Coughlin E, Collins C and Sharp D, Technical Review of Plagiarism Detection Software Report [online], Joint Information Systems Committee (2001). Available from http://www.jisc.ac.uk/pub01/luton.pdf

(3) Mason O, and Grove-Stephenson I, Automated Free Text Marking With Paperless School, Proceedings of the 6th International Computer Aided Assessment Conference (2002) pp213-219. Available online via: http://www.caaconference.com/

(4) For a summary of available packages see:
http://www-staff.lboro.ac.uk/%7Elsdps/index.html

(5) Brown G, Bull J and Pendlebury, Assessing Student Learning in Higher Education (1997) p 7, Routledge, London

(6) Gibbs G, Improving the Quality of Student Learning (1992), p 17, Technical and Educational Services, Bristol

(7) For example, see the ‘Competency Framework for Effective Teaching’ devised by the Murdoch University, Perth, Australia, at http://wwwadmin.murdoch.edu.au/planning/docs/acfet/index.html#et4.

(8) http://www.guardian.co.uk/Archive/Article/0,4273,4150416,00.html.

(9) Denton P, Generating Coursework Feedback for Large Groups of Students Using MS Excel and MS Word, U. Chem. Ed. 5 (2001) pp1-8. Available online at:
http://www.rsc.org/pdf/uchemed/papers/2001/p1_denton.pdf

(10) Denton P, Generating and e-Mailing Feedback to Students Using MS Office, Proceedings of the. 5th International Computer Aided Assessment Conference (2001) pp157-173. Available online via: http://www.caaconference.com/

(11) Denton P, Detection of Plagiarism in Students' Written Work Using a Novel Electronic Marking Assistant, Proceedings of the International Conference on ICTs in Education (2002) pp 445-449.

Philip Denton
School of Pharmacy and Chemistry
Liverpool John Moores University
Liverpool L3 3AF
p.denton@livjm.ac.uk

February 2003
ISSN 1477-1241