Editorial
Rachel Forsyth

Blended Learning: One Small Step
Mark Stubbs and Ian Martin

Real-time Lesson Observation: using technology in teacher training to learn about technology in the classroom
Maureen Haldane

"Telestia AB Multi Media CD Rom - Pattern Technology
Clare Culliney

Using e-learning technology for student support - an early adopter’s experiences
Pete Cuthbert

An evaluation of the perceived importance and success of a Department-level key skills scheme
Nuala Byrne

Trends in Faculty Use and Perceptions of E-Learning
Som Naidu

Faculty Learning and Teaching Reports

Learning and Teaching News from the Library

| View this article as a .pdf file |

Mark Stubbs and Ian Martin
MMU Business School

Blended Learning: One Small Step

Introduction

This article describes a blended learning project delivered over 9 weeks to 230 full-time first year undergraduates studying a unit about emerging information technologies. The unit was originally planned on a weekly 1+1 model of 1 hour lecture reinforced by 1 hour tutorial, running for 9 weeks. Summative assessment was designed as individual, face-to-face demonstrations of web pages produced by the students. Holistic redesign of the unit, using web-based learning materials and computer-supported assessment, enabled leaner delivery and assessment, better reinforcing the intended learning outcomes and saving 85 hours of staff time. Principles and assumptions underpinning redesign, delivery and assessment are presented and reviewed from three perspectives: a behavioural perspective, an IT perspective and a resource perspective.

History

The Emerging Technologies unit had its roots in units delivered in previous years, but 2002/3 was scheduled to be its first formal year of operation following the quinquennial review of a suite of undergraduate programmes on which it appeared. The unit had been included in the portfolio of Business IT courses to increase first year students’ exposure to new information technologies and new ways of working. Like other first year units it was designed to set students’ expectations of academic endeavour and lay foundations for more advanced study. A team of three academic staff with advanced IT skills and a track-record of innovation in learning and teaching were allocated to run the unit. Whilst the team intended to make use of the latest technologies in lectures and provide further exposure through tutorials, the unit had been planned on the basis of a traditional delivery model. It was only the last-minute secondment of a key member of the teaching team that made lean delivery an acute priority. Some quick thinking, an entrepreneurial proposal and a supportive Dean transformed this problem into a pilot blended-learning project. Resources for “back-filling” the secondment were redirected toward off-setting the up-front development costs associated with redesigning the unit and developing web-based tools to support lean delivery and assessment.

Key project deliverables, quality monitoring, evaluation and dissemination arrangements were agreed to ensure (A) that the student experience was not compromised by the pilot project and (B) that other staff could learn from it. Re-direction of “back-fill” resources was conditional on both criteria being met. Deliverables and terms were agreed mid December. The redesigned unit, assessment and supporting e-learning materials had to be available in January when the students returned. Christmas was busy.

A Behavioural Perspective

Despite the time pressures, the two remaining unit tutors were committed to holistic redesign of the unit, not bolt-on substitution of e-learning materials for tutor contact. Our intention was to create a coherent system of input, activities and assessment that covered the approved learning outcomes and reinforced intended academic behaviour and values. In particular, we wanted to establish the following principles so that later years of study might build upon the norms of focused independent study that they engender:

P1: Tutor as expert of last resort
P2: Attention to detail
P3: Regular engagement
P4: Demonstrate learning outcomes

We were conscious that consistency and routine would play an important role in establishing these desired norms of behaviour. Although notoriously difficult to operationalise (Walsham 1993), we found Giddens’ “axes of structuration” offered a useful checklist for ensuring that our words and deeds were mutually reinforcing around these principles.

Giddens’ (1984) notion of “structuration” portrays social structure and individual action as mutually producing. He describes how this so-called “duality of structure” leads to self-organizing patterns that become more solid as they are reproduced over time and across space. For analytical convenience he highlights three axes or dimensions along which this mutual production takes places. He associates:

• interactions involving communication with social structures that dictate what is significant.
• interactions involving power with social structures of domination.
• interactions involving sanctions with social structures that dictate what is legitimate.

Our attention to these axes of communication, power and sanction is highlighted in the following accounts of our efforts to establish the four behaviour principles required to support the unit. When planning our unit structure and agreeing our policies for interacting with the students, our attention to the three dimensions was not always conscious. We had used the ideas in research and consultancy contexts (see for instance Stubbs and Lemon, 2001), so the axes provided us with a sometimes tacit, sometimes explicit, checklist. Nonetheless, they provided us with more than a convenient narrative device for post-hoc rationalisation of events.

P1: Tutor as Expert of Last Resort

Our “Tutor as expert of last resort” message was communicated regularly in lectures and reinforced in tutorials. Students were told that help would be prioritised for those who were making a genuine attempt to engage. They were shown snapshots of the data lecturers held about use of the e-learning environment. As learning curves were quite steep, students valued “tutor support” in tutorials. Tutors exercised power over this precious resource, rewarding with their time those students who had engaged with the e-learning environment and imposing sanctions against those who had not by denying them time until they had had a go for themselves.

Many students found the approach unfamiliar at first – “you’re a tutor, you’re meant to help us”. However, a slow but consistent prioritisation of support meant that students were gradually weaned off this expectation and within three weeks the “tutor as expert of last resort” principle had become established as a cultural norm amongst those attending tutorials – “don’t even bother asking, he knows you haven’t had a go yet!”

P2: Attention to Detail

It is almost impossible to underestimate the importance of “Attention to detail” to successful implementation of business IT solutions. Its particular significance for the unit was communicated via the assignment brief as a very specific set of naming criteria with which students’ submitted web pages must comply. Tutors exercised power over the definition of a valid submission to reinforce this. Students were informed in the assignment brief and in lectures and tutorials that submissions would be ignored if web page naming criteria were not followed to the letter. To reinforce the principle of “tutor as expert of last resort”, tools were made available in the e-learning environment so that students could check for themselves that submissions complied. These tools were demonstrated in lectures to reinforce the principle of “regular engagement” (see below).

At the appointed assignment hand-in time, the sanction for non-compliance with naming standards was applied consistently. We exercised our power over interpretation to classify work that failed to use the designated names for the web pages as failing to submit. The Common Undergraduate Regulations (CUR) sanction of a capped mark of 40% for late submission provided automatic sanction to reinforce the importance of “attention to detail”. Each student who had failed to name their files correctly was emailed personally to inform them that they must change the names of their files to enable valid re-submission before the absolute deadline. Students who were still missing correctly-named files were emailed on several occasions in the run-up to the absolute deadline with a clear warning that their continued failure to comply would result in an automatic 0% when the absolute deadline passed.

Despite the checking tools, 46% of students studying the unit failed to submit the two correctly-named files for the appointed hand-in time. This figure dropped to 23% at the absolute deadline, which included several not-yet-official withdrawals. After the first deadline many students were shocked to discover that they had fallen foul of the strict naming standard - “I couldn’t believe it, I just assumed it was right, I’m not going to make that mistake again”. The fact that they could still pass encouraged most to make the necessary changes and put their frustration down to experience; however a minority ignored the emails about changing the names and one became aggressive when he discovered that he had received 0% for his efforts. The failure of his appeal and his subsequent disciplinary provided important Faculty-level reinforcement for the principle of “attention to detail” and was welcome back-up for our clear and consistent communication about desired behaviour and the penalty for non-compliance.

P3: Regular Engagement

The importance of “regular engagement” for climbing the learning curves associated with web development and undertaking the research necessary to produce appropriate content was communicated as a key message in lectures. We were keen to reinforce this message with tutorials despite resource pressures.

Aytoun computer laboratories can handle a maximum of 25 students at a time (or less to allow for machine failure), so the 230 students on the unit would require at least 10 hours of computer laboratory time for hands-on reinforcement of learning outcomes. Large numbers were leading many first year colleagues to a fortnightly model for tutorials. The loss of our third team member meant that we could not cover 10 hours of tutorials per week for this unit alongside our other commitments. However, rather than see the students once every two weeks, we felt that routine played a key role in defining students’ time at university and set expectations about the nature of academic work and support. We felt that this was particularly important for setting first year expectations, so we exercised power over student routines by booking 10 hours of computer laboratories each week and scheduling students to regular one hour slots. To work within the resource constraints we made ourselves available for 30 minutes in each hour-long tutorial to answer questions. The availability of self-paced e-learning materials and the principle of “tutor as last resort” made this approach workable and reinforced our message that this assignment was best addressed through regular weekly efforts, rather than irregular spurts.

Lecture slides were made available on the Business School intranet so students who missed a lecture could get at them, but regular attendees were rewarded by assignment hints and tips, such as repeated demonstrations of the name checking tools, which were over and above the lecture slides. In other words we exercised power over what was communicated when and how to reinforce the message that “regular engagement” was advantageous. Not only was regular attendance rewarded, but students were also informed that their use of the e-learning environment in scheduled tutorial times provided attendance data that would be passed on to the first year leader for use in the first year Progress Board, which considers achievement and attendance across all units of study. The Progress Board reinforces desired norms of engagement through the sanction of inviting students falling below the required standard to explain their reasons to the year leader to see if preventative action could avoid them failing the year.

P4: Demonstrate Learning Outcomes

We were conscious that the language of learning outcomes often works better for educationalists than for the students who are supposed to be demonstrating them. To ensure that the approved unit learning outcomes translated into specific student performances, we highlighted which of those in the definitive document were being assessed in this particular assignment brief and used a grid structure to communicate exactly what level of performance in each of the different learning outcomes would be rewarded by what mark (click here to see the assignment brief in pdf format).

The structure and operation of the grid was explained in an assignment briefing lecture, and students were advised to fulfil all the criteria for gaining 40%+ in a particular learning outcome, before going on to attempt the 50%+ level of achievement, and so on.

The grid provided the interpretative scheme for shaping the way we exercised our power as tutors to allocate marks, ensuring that achievement against the specified learning outcomes was rewarded appropriately. Feedback was written in terms of assignment learning outcomes and marks were justified in terms of the criteria specified for each. This personal feedback was emailed to each student, reinforcing the message that performance against specified learning outcomes was the key to high marks.

An IT Perspective

Giddens’ original work on the notion of structuration offered little explicit guidance on the role Information Technology (IT) might play in social contexts. Although he has given more attention to the issue in later writing (Giddens, 2001), several scholars of Information Systems have picked up the challenge of offering an account of the role of IT in structural change that is consistent with Giddens’ work (see for instance Orlikowski and Robey 1991, Walsham and Han 1991, Rose and Lewis, 2001). A discussion of the relative merits of these approaches is outside the scope of this paper, but it is useful to note their shared interest in the potential of IT to support the spread of behavioural patterns across time and space, and their cautionary note about the inevitability of unanticipated and unintended consequences.

As IT developers we were conscious that IT has a distorting effect on communication. Different IT solutions create different possibilities for the richness, nature, synchronicity and symmetry of communication, and the memory of communication that can be maintained (Sproul and Kiesler, 1991). We were keen to use these distortion and memory effects to reinforce our four unit principles. The secondment of our colleague meant that efficiency gains from automating routine tasks were critical for running the unit in the way we wished. With these factors in mind we set about developing IT to support two groups: students and tutors.

Students at MMU Business School are familiar with using web browsers to access course materials. The web-based Business School intranet is used to relay all manner of information from staff to students: course messages, lecture notes, assignments, and the like. Students are used to accessing it inside the building and from home. The intranet holds over 8,000 documents and receives over 10,000 page hits most days, excluding document downloads. As received wisdom indicates that students interpret new technologies in terms of those with which they are already familiar (Orlikowski and Gash, 1994), we chose to build on familiarity with intranet/web browser interfaces to create the e-learning environment to support the first years taking the emerging technologies unit. Not only was a web environment most likely to be accepted, it was also demonstrating the kind of technologies the students were studying in the unit.

IT support for the blended-learning project involved two distinct areas:

1. A monitored e-learning environment for introducing HTML and JavaScript
2. An e-submission tool that provided computer-supported assessment and feedback

The first system (and e-submission) could have been developed in Web-CT but we chose to adapt an existing e-learning solution, already available within the Faculty. The e-learning solution, known as theWebDevRes (short for Web Development Resource), had been developed some years earlier to support final year undergraduates and Masters students who were learning advanced web development skills. It integrates e-learning materials with its role as an internal Internet Service Provider (ISP) for the students. This ISP role could not be handled in Web-CT so it had been developed outside. The WebDevRes environment offers sophisticated student tracking and access monitoring features and was already being used by over 600 students a year, so increasing its coverage to handle introductory HTML materials and JavaScript was regarded as a bonus that was achievable within the tight timeframe for the project.

The second system was more complex but had a longer development window - it needed to be robust in time for the first e-submission. Our intention was to create a system that:

1. enabled assignments to be picked up electronically at a designated time;
2. provided tutors with fast navigation around the 230 assignments;
3. detected automatically any work similar to that submitted by other students;
4. highlighted breaches of assignment requirements; and
5. captured and communicated personalised feedback.

The aim was not to replace the role of the tutor in assessment, but to assist and inform our marking through simple pattern recognition. We wanted to build a system for experts, not an expert system (Hirscheim and Klein, 1989).

A suite of tools to fulfil these requirements was designed and implemented on an old Personal Computer that had been configured to run a LAMP environment (Linux Apache MySQL PHP). The LAMP software has a zero purchase price and is highly customisable (for someone with sufficient skill and expertise). As the server had been designated as visible only inside MMU, within limits of professionalism we had total freedom over and responsibility for what was hosted and executed on the server. Assuming the role of web master and system administrator enabled rapid development of tools to address our requirements. These tools were tested and refined prior to the assignment submission deadline. Automated analysis of each student’s work was presented as a secure web page, which had hyperlinks for reviewing the submission and boxes for capturing marks and comments. The system was integrated with the Web Server’s email system so that our marks and comments could be sent as a personal email to each student.

Figure 1 – Marker’s Page Screen Shot shows a screen shot of the marking screen web page.

If we review the two systems from a “support for processes of structuration” perspective, the student e-learning environment can be seen as a mechanism for providing students with routine contact with codified tutor expertise. The asymmetric nature of the communication from tutor to student reinforced the role of tutor as expert. The 24x7 availability of codified expertise and the system’s on-going monitoring of student engagement with that expertise enabled actions that reinforced our principles of “tutor as last resort” (P1) and “regular engagement” (P3).

The second system provided support for the role of tutor as meticulous expert despite over-whelming student numbers. Automated name checking and pattern recognition in student work enabled quick turn-around of carefully-analysed work. In this way, IT provided valuable support for the “attention to detail” principle (P2). Tutors’ routine use of the screens to frame personal feedback in terms of performance against desired learning outcomes provided cultural reinforcement for that principle (P4).

Orlikowski and Gash (1994) describe how patterns of computer system use can emerge that are generally regarded as desirable but were not planned for in the original IT design. They describe how developers can exercise power over the development process to support and consolidate such behaviours in later software versions and “design out” undesirable patterns of use that have emerged. Our repeated typing of the learning outcome categories in the free-format text box used to hold feedback comments came from our desire to reinforce the “demonstrate learning outcomes” principle (P4), but it created extra work. Modifications to the screen design in the shape of a clearer pro-forma is therefore planned for next year to provide further reinforcement for P4 by making it an inescapable routine for all tutors.

Our use of IT produced another pattern of use that we did not anticipate. This was not quite so desirable. Personalised email feedback meant that students could respond with a one-click, knee-jerk moan. We will be reviewing our feedback strategy in the light of this for next year.

We will reflect further on our IT development at the end of the article, but it is useful to summarise here that we were able to combine IT and pedagogical design to establish our intended principles. Our IT produced some patterns of behaviour that we did not anticipate - one positive and deserving of reinforcement, the other worth reviewing.

A Resource Perspective

Our original plan was to deliver this element of the unit through a traditional 1 hour lecture plus 1 hour tutorial model over 9 weeks. We planned to use our team of three to assess two students in a viva/ demonstration every 15 minutes – one member of staff being lead assessor for each student, with the remaining member acting as a floating moderator.

Through the blended-learning project we varied the delivery model and used e-learning materials to reduce our contact to 30 minutes per week question and answer sessions in tutorials. The computer-supported assessment tool enabled us to remove the face-to-face element of assessment from all students. Instead we chose to viva only those whose work was suspiciously similar or who had breached key assignment requirements. We found that it took on average about 8 minutes to assess each assignment and comment our findings according to the grid criteria. The comparability brought by the explicit criteria meant that we were able to perform sample moderation in a 2 hour session involving three assessors (our seconded colleague generously offered help with some of the assessment). 40 students fell foul of our viva strategy and we found that the computer-supported assessment tool enabled each focused discussion to be concluded within 10 minutes. The comparison reveals that our blended learning model produced a staff saving of almost 85 hours.

Staff time required for the two approaches is compared below:

Discussion

The table above does not allow for the development costs associated with the project. The sense of being part of something creative and rewarding masked the hours that went into unit redesign and supporting software development. It is our intention to continue with this blended mode of delivery for future years and thereby recover the up-front development costs.

We found that by sticking to our rules for prioritising student support, more of our time was spent discussing interesting problems with interested students. “Just-enough, just-in-time” students struggled with the unit principles; they found our prioritised support unsatisfactory and regarded us as downright unhelpful. Performance was similarly polarized with some outstanding work and some work that scored very little against the strict assignment grading criteria. Levels of achievement were still well in line with first year norms and, allowing for some of the peculiarities of this particular first year cohort, the higher pole appears more representative of the potential for blended learning than the lower pole.

The success of the blended learning project in resource terms raised an issue for the Business School’s normal mechanism for ensuring equity of workload: the staff timetable. As this is constructed and compared on the basis of contact hours, savings generated through this kind of project translate into more work the following year for the innovators! The up-front development costs of this kind of project are intimidating enough without the prospect of hours saved being filled by work from elsewhere in the following year. Moves to a better load metric, based on number of Full-Time-Equivalent students and the nature of the unit, are on-going but are clearly urgently needed if lean delivery is to be encouraged.

This article is one of several attempts to explain the principles and practice of this pilot project to a wider audience. However, the extent to which others can follow our footsteps is questionable. Our use of Giddens’ checklist may be of interest but it was our technical skills that enabled us to set up a highly responsive interaction between IT and unit design. Our conversations with students and our observations of their behaviour and work in tutorials enabled us to make instant changes to the IT, for which we could see the effects in the hit logs and subsequent tutorials. Without our IT skills we would have needed outside help to realise our ambitions. Out-sourcing development to a third party would have inevitably introduced delays and communication overheads as we translated our reflection on student behaviour into formalised requests for changes to software code and then waited for those changes to be made. Going outside the teaching team for IT skill also raises questions of funding. This project was implemented on a ‘shoe-string’ budget. We modified existing software for the learning materials and used the free LAMP environment to develop the other tools we needed. The IT systems we produced were robust, reliable and worked for us (and for our colleague who re-joined us to help with the marking), but they were not fully-engineered solutions. There was (and still is) no comprehensive help or maintenance guide.

Literature on the nature of innovation would see little of surprise in this account (Rogers 1983). Early adopters of innovations (such as blended learning) tend to have the vision and technical expertise to make the ideas work. The majority need to be convinced by results and need more fully engineered solutions before they will follow. Discontinuity between the fundamentally-different nature of early and more mature adopter communities creates a “chasm” that several technical innovations have failed to cross. Early take-up appears promising as the techno-savvy visionaries experiment but the expected take-off fails to happen as the majority are waiting to be convinced by results that are slow to emerge (Moore, 1999). We suspect that these cautionary tales from techno-markets are potentially enlightening for understanding patterns in the take-up of blended and distance-learning in our institution and elsewhere. Perhaps advice about targeting 100% support for a small number of demonstration projects, rather than attempting (but failing) to meet the support expectations of all those who express an interest, has equal resonance.

Conclusion

This blended-learning project came about by accident. Our emphasis on redesigning the whole unit, not just replacing tutor time with some e-learning materials, ensured that it succeeded. The project might be hard to replicate. It needed a crisis, a Dean willing to recognise up-front e-learning development costs, and technically-proficient enthusiasts with a track-record of innovating learning and teaching through IT. It was stimulating and rewarding. The checklist derived from Giddens’ work proved useful and experience from the project has prompted us to reconsider the way we deliver our other units. We are convinced that web technology offers the possibility of treating learners as individuals, working at their own pace through materials, following hyperlinked support materials as and when they need it. We do not believe that the best role for IT is to replace face-to-face contact; it was much better at enhancing the increasingly pressured but nonetheless pleasurable opportunities we have for face-to-face contact with our students. Long live blended learning.

References

Giddens, A. (1984). The Constitution of Society, Outline of the Theory of Structuration (1993 reprint). Polity Press, Cambridge.

Giddens, A. (2001). Sociology (4th Ed). Polity Press, Cambridge.

Hirscheim, R. and Klein, H. K. (1989). “Four Paradigms of Information Systems Development”, Communications of the ACM, 32, 10, pp. 1199-1216.

Moore, G. (1999). Crossing the Chasm (2 nd Ed). Capstone, Oxford.

Orlikowski, W.J. and Gash, D.C. (1994). ‘Technological Frames: Making Sense of Information Technology in Organizations’. ACM Transactions on Information Systems, vol. 12, no. 2, April, 174-207.

Orlikowski, W.J. and Hofman, J.D. (1997). ‘An Improvisational Model for Change Management: The Case of Groupware Technologies’. Sloan Management Review, Winter, 11-21.

Orlikowski, W.J. and Robey, D. (1991). ‘Information Technology and the Structuring of Organisations’. Information Systems Research, vol. 2, no. 2, 143-169.

Rogers, E. (1983). Diffusion of Innovations (3 rd Ed). Free Press, London.

Rose, J. and Lewis, P. (2001). Structuration Theory, Action Research and Information Systems Development. In: Russo, L.; Fitzgerald, B. and DeGross, J.I. (eds) Realigning research and practice in information systems. Kluwer Academic Publishers, Boston, MA.

Sproull, L. and Kiesler, S. (1991). Connections – New Ways of Working in the Networked Organization. MIT Press, Cambridge, MA.

Stubbs, M; and Lemon, M. (2001). ‘Learning to Network and Networking to Learn: Facilitating the Process of Adaptive Management in a Local Response to the UK’s National Air Quality Strategy’, Environmental Management, 27(8), pp. 321-334.

Walsham, G. (1993). Interpreting Information Systems in Organisations. John Wiley & Sons Ltd., Chichester.

Walsham, G. and Han, C-K. (1991). ‘Structuration Theory and Information Systems Research’. Journal of Applied Systems Analysis, vol. 17, 77-85.

Mark Stubbs
e-mail: m.stubbs@mmu.ac.uk
telephone: 0161 247 3739

Ian Martin
e-mail: i.martin@mmu.ac.uk
telephone: 0161 247 3894

December 2003
ISSN 1477-1241