Sounds Good 2 (Audio Feedback Project)

Sounds Good presentation

Bob Rotheram — Fri, 04 Sep 2009 03:52:00 -0700

If you can spare 56 minutes, you might be interested to watch a video of my presentation on Sounds Good at a Lancaster University staff development event on 15 May 2009. Here's the link: http://www.lancs.ac.uk/celt/celtweb/02rotheram. Thanks to Lancaster University for recording the session and agreeing to a link from here.

I think most of the session is coherent but it includes a passage where I take people through a simulation of a book review exercise and receiving audio feedback. This bit is likely to be more understandable if you also study the book review exercise and then listen to the audio feedback. The materials for this are in the Downloads section of the Sounds Good website, at: http://sites.google.com/site/soundsgooduk/downloads.

Still breathing

Bob Rotheram — Wed, 01 Jul 2009 13:21:00 -0700

There’s life in the old dog yet. The trigger for this posting is the publication of JISC’s ‘Effective Practice in a Digital Age’ www.jisc.ac.uk/practice, which includes two versions of a case study on Sounds Good. Both are entitled ‘Making assessment a learning experience’. The shorter version is on pages 26-7 of the PDF file available here. The longer version, in Word format, is here.

Whilst I’m online, it’s worth saying that Sounds Good continues to arouse interest. Since the end of March I’ve given a further six presentations, making a total of 30 so far. Several more are lined up in the next few months. If you’d like me to run a staff development event on audio feedback, or speak at a conference, please get in touch and let’s see if we can arrange something.

Thanks, team!

Bob Rotheram — Wed, 25 Mar 2009 06:04:00 -0700

Now that the formal part of the project is over and the final report is pretty much ready to go, it's time to acknowledge that it's been a team effort and thank everyone.

Sounds Good has involved many people, staff and students, all of whom helped the project to succeed in one way or another. Thanks are particularly due to JISC, for the funding and also for consistent encouragement and support, most obviously from Lawrie Phipps, Programme Manager for the Users and Innovation Programme. My line manager, Prof Sally Brown, Pro-Vice-Chancellor for Assessment, Learning and Teaching at Leeds Met, deserves an accolade for allowing me to run the project as I wished and to give it more time than budgeted for. Simon Thomson, Sounds Good’s Deputy Project Manager, has been a valuable ally, sounding board and source of advice. I am also grateful to my main contacts at the three partner institutions for Sounds Good 2: Bob Ridge-Stearn at Newman University College, Caroline Stainton and Katie Jackson at the University of Northampton and Simon Sweeney at York St John University. Peter Chatterton, the project’s ‘critical friend’, provided reassurance and an extra forum for discussion as well as provoking productive thought. Isobel Falconer, our external evaluator, negotiated sensitively on how to review the project and then worked with colleagues before producing her helpful insights and perspectives. It has also been fun to get to know Will Stewart, leader of the ASEL project at the University of Bradford, and to share experiences with him. To all these people, and to others too numerous to mention, I much appreciate your contributions.

What a team!

Thanks, team!

Wed, 25 Mar 2009 06:04:00 -0700

Now that the formal part of the project is over and the final report is pretty much ready to go, it's time to acknowledge that it's been a team effort and thank everyone.Sounds Good has involved many people, staff and students, all of whom helped the project to succeed in one way or another. Thanks are particularly due to JISC, for the funding and also for consistent encouragement and support, most obviously from Lawrie Phipps, Programme Manager for the Users and Innovation Programme. My line manager, Prof Sally Brown, Pro-Vice-Chancellor for Assessment, Learning and Teaching at Leeds Met, deserves an accolade for allowing me to run the project as I wished and to give it more time than budgeted for. Simon Thomson, Sounds Good’s Deputy Project Manager, has been a valuable ally, sounding board and source of advice. I am also grateful to my main contacts at the three partner institutions for Sounds Good 2: Bob Ridge-Stearn at Newman University College, Caroline Stainton and Katie Jackson at the University of Northampton and Simon Sweeney at York St John University. Peter Chatterton, the project’s ‘critical friend’, provided reassurance and an extra forum for discussion as well as provoking productive thought. Isobel Falconer, our external evaluator, negotiated sensitively on how to review the project and then worked with colleagues before producing her helpful insights and perspectives. It has also been fun to get to know Will Stewart, leader of the ASEL project at the University of Bradford, and to share experiences with him. To all these people, and to others too numerous to mention, I much appreciate your contributions.What a team!

So where are we?

Bob Rotheram — Mon, 23 Mar 2009 05:15:00 -0700

At the end of February I called ‘time’ on Sounds Good: audio feedback after that date would not be taken into account when writing up the project. Since then, I’ve been gathering and analysing information and drafting the reports for JISC. Several months ago I asked on the blog ‘Are we nearly there yet?’ Only now, after a journey of a year, can we answer ‘yes’.

So where are we? Regular readers will know the main aim of Sounds Good was to test the hypothesis that using digital audio for feedback can benefit staff and students by:

saving assessors’ time (speaking the feedback rather than writing it)

and

providing richer feedback to students (speech is a richer medium than written text).

Initially the project was funded for the period January to July 2008. During this time a team of 16 Leeds Met lecturers experimented with digital audio to give formative and summative feedback on students’ coursework. Later, funding was provided under JISC’s ‘benefits realisation’ initiative for a second stage, ‘Sounds Good 2’, which ran until February 2009. In this phase the design called for six Leeds Met staff from the first stage to mentor 12 colleagues joining the project and for audio feedback to be introduced to three other higher education institutions: Newman University College, Birmingham; University of Northampton; York St John University.

Overlapping with the second phase, two HE Academy subject centres – Engineering and Geography, Environmental and Earth Sciences (GEES) – were funded to introduce audio feedback to their constituencies as part of JISC’s ‘Widening Stakeholder Engagement’ initiative. I’ve been helping the subject centres with this work.

Sounds Good has mainly been a qualitative study. Even so, it has produced a few statistics. Taking the two phases of Sounds Good together, 38 teachers in four institutions have supplied audio feedback to at least 1,201 students at all educational levels from foundation degree and first-year undergraduate to doctoral. The staff were located as follows: Leeds Met 23, Newman University College 8, University of Northampton 4, York St John University 3. In the first phase the numbers on the various modules ranged from six to 151, with at least 463 students receiving one or more items of audio or video feedback. In Sounds Good 2 the numbers on modules ranged from three to 150 and at least 738 students received one or more items of audio feedback.

The project has operated in widely differing circumstances, which has been a mixed blessing. The main advantage of this diversity is that has enabled a worthwhile preliminary exploration of the potential of digital audio for assessment feedback. On the other hand, the differing circumstances have led to a suite of case studies rather than one large, standardised experiment.

Sounds Good has worked very well overall. In the first phase it ran almost entirely to plan. In the second phase it generally went well in all four institutions, but there were a few minor problems, including:

Only four of the six Leeds Met mentors managed to engage with mentees.
Only seven, rather than the planned 12, mentees were recruited at Leeds Met.
The extended communication channels between me and some team members led, occasionally, to staff not being entirely clear what was expected and me being less well-informed than previously as to what was happening.
The data returned were somewhat less complete and even more varied in nature than in the first phase.
I found it difficult to give the project sufficient time in January-February 2009.

The Sounds Good staff team is, on balance, strongly in favour of audio feedback. Even if they didn’t manage to save time, a high proportion of the team have commented that they were able to give more, and higher-quality, feedback using audio, which they felt was worthwhile. Their reservations about audio feedback were mainly about the practical difficulties they encountered. Most of these could be regarded as ‘teething problems’ which might reduce or disappear with further practice and the use of the practice tips which we’ve published. The majority of the team have clearly said they intend to continue using audio feedback, and almost all will probably do so.

No doubt some staff are encouraged by the fact that the great majority of students were positive about receiving audio feedback on their coursework. Students particularly appreciate the personal nature of individual audio feedback, as well as the detail they often received.

As for the central question tackled by Sounds Good:

Can digital audio be used to give students quicker, better feedback on their work?

the answer is ‘yes’, in some circumstances. The most favourable conditions seem to be:

The assessor is comfortable with the technology.
The assessor writes or types slowly but records his/her speech quickly.
A substantial amount of feedback is given.
A quick and easy method of delivering the audio file to the student is available.

At this stage it is fair to say that most UK academic staff assessing student coursework would probably find it worth giving audio feedback an extended trial with at least with some of their assessment work. For many it would be sensible to begin where the conditions are most favourable. For example, this might be with a small cohort or where the detail or personal quality of audio feedback are particularly important. In contrast, it would probably be inadvisable to start by attempting to give individual audio feedback to a big cohort, because of the problem of accurately providing large numbers of audio files to students. However, with a big cohort an early, efficient step might be a ‘one-to-many’ communication: group audio feedback.

There is much yet to explore in the field of audio feedback. There is plenty of scope for larger trials, attempting to tease out the variables and studying the effectiveness of audio feedback (i.e. whether it enables students to learn more). However, a particularly pressing problem – one which might be solved quickly by a programmer – is to automate the process of sending feedback to students. Audio feedback is already an attractive proposition, yet if assessors could be confident that – regardless of cohort size – it would take them little or no time to let students have their audio feedback, even more would probably find audio feedback worth adopting.

Sounds Good has broadly achieved what it set out to achieve. It has done some valuable exploration and produced useful practice guidelines. All in all, it has delivered an excellent return on JISC’s modest investment and, most of the time, it’s been fun.

Untitled

Mon, 23 Mar 2009 05:02:02 -0700

Spoken version of blog posting, which reviews Sounds Good.

Practice tips on audio feedback

Sun, 22 Mar 2009 11:06:31 -0700

Bob Rotheram draws on the experience of the Sounds Good project to offer some practice tips on the use of audio for assessment feedback.

Simon (York)

Simon Sweeney — Wed, 11 Feb 2009 03:13:00 -0800

HELLLLLLLOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!
Finally managed to get back in to the Blog. The trouble is if you use three computers, three email addresses and forty seven and three quarters passwords you make mistakes. However, with the help of my pet whippet I now have an ENTIRELY MEMORABLE blog password!!!!!!!!
OK, Sounds Good!!!???
This has been - as Bob said it would be - an exciting journey (although frustrating at times). I will be putting a full report together...but talk about a learning curve!!! I found that wherever there was a choice to make I made the wrong choice, so everything took hugely longer than I anticipated. The worst thing has been getting the feedback to the students and getting an acknowledgement from them and getting the questionnaires back. This has been marked by a truly gargantuan level of difficulty.
A friend who runs an educational support organisation for creative writing told me last year his network (NAWE) had 1000 members and usually about 100 manage to attend the annual conference. 1 in 10 struck me as very good. Likewise with a survey, 1 in 10 response isn't too bad (though that means to get a response from 50 needs 500 surveys posting out which is a lot of work. Anyway, on Sounds Good I am indeed running at about 1 in 10. Actually I think this is low, and feel sure Bob will be disappointed. The worst thing though is I cannot be sure that the students have even got the Audio feedback at all because I am not receiving acknowledgements!!

So what went wrong? It seems to me that unless you set up the Assignment delivery electronically inside the VLE you are pretty well unable to post out the feedback through the VLE - and in any case once the module is to all intents and purposes over, the students never go near the VLE for that module again anyway!!!

Mark and I set about sending files through the VLE but it was HUGELY timeconsuming for Mark (and he's a VLE expert). So instead I sent the files out using my private email and the students' private emails if I had them, if not University email accounts. Here there's always the problem of file size and not knowing whether they have opened their email account - even once in recent times!!! Or ever!!!?? IN ADDITION - and Mark will be surprised by this - it took hours to do this and therefore probably didn't save (me) time over the 'attaching to the VLE approach'....
So... and that....has been ONLY SOME of the problems....
So do I have anything positive to say??? YES. Both my colleague Bev Geesin and I have been delighted with the Audio Feedback in terms of what it can bring to the process and the students who HAVE responded have all been very positive (except one who wanted written feedback AS WELL!)...so yes, I think Audio Feedback is the future. I would always want to use it (even though I think with all the technical problems resolved it WILL NOT SAVE TIME. It is though a much more sophisticated, detailed and personal feedback service. OK...full report at the end.

For the moment I am still stuck in a quandary. What should I do for those students who have not responded at all? I may have to go through the laborious process of attaching the files into the VLE...but even then, will they visit Blackboard weeks after they handed in their work???

OK...that's it. More detail will be provided later....Happy Blogging!
Simon signing off (and taking the whippet for a walk).

The indirector

Bob Rotheram — Fri, 19 Dec 2008 02:19:00 -0800

Three years ago, beginning my exploration of audio feedback, I was a one-man band. I did the literature search, decided on the technology, taught myself how to use it, recorded the feedback on my students’ work, sent it to them, gathered their comments on the experience, analysed the data and published the results. In a very straightforward way, my actions (or lack of them) determined how things went. It was hard work, but I was in control and it felt comfortable. If I had been lusting after a title, I might accurately – but absurdly – have called myself ‘director’ of the one-man band.

Then came Sounds Good, a 17-person audio-feedback ensemble at Leeds Met. Again it was my baby: people looked to me as the person ‘in charge’ and I readily accepted the responsibility of shaping the project and moving it along. So, amongst other things, I decided on the technology, issued briefings to team members, gathered completed questionnaires, analysed the data and, as before, published the results. But it wasn’t as before. The important difference was that I was at one remove from the action: I had almost no contact with students and produced none of the audio feedback. I did deal directly with the producers of the audio feedback – teachers at Leeds Met – but I don’t think I ever ‘directed’ operations, in the sense of ordering people what to do. Instead I largely made requests, offered recommendations and suggestions. We’re not in the army and the informal style seemed appropriate. I called myself ‘project manager’ rather than ‘director’. Being at one remove, and being lousy at delegation, I was often anxious as to what would happen. See here and here, for example. I wasn’t as comfortable as when I was running my one-man band but, as it turned out, the team did a great job and Sounds Good worked well. Thanks, colleagues!

Now it’s Sounds Good 2, an even bigger audio feedback enterprise where, in many instances, I’m yet further removed from the action. Most of the original Leeds Met team are continuing to use audio feedback with their students, but new teachers have joined us and some of the ‘old lags’ act as their mentors. So in some instances the communication chain between me and students now has an extra link: me – mentor – mentee – student (and back again). There are indications that it doesn’t always work perfectly and I occasionally detect a bit of ‘Chinese whispers’. At this moment, I’m not confident that I have the full roster of Leeds Met student groups who are, or will be, receiving audio feedback this semester as part of Sounds Good 2.

And then there’s what’s happening elsewhere. As part of Sounds Good 2, audio feedback work is getting under way at three partner institutions: Newman University College, the University of Northampton and York St John University. What’s going on there? I’ve visited all three places, know my institutional contacts and have met quite a few of the staff intending to give audio feedback. Some of them have started to blog about their experience and information flows back and forth. Even so, I worry that I don’t have the full picture and I do feel far removed from their students.

Considering the project as a whole, it’s bigger than I could have imagined back in 2005-6. I believe it’s going well but, as of today, I don’t really know. What I’m sure of is that other people now have far more influence over the outcomes than I do. Emphatically, I’m not the director, in the sense of being in full control, able to pull levers and make the intended actions happen immediately.

So I’ve dubbed myself the ‘indirector’ of Sounds Good 2. ‘Indirector’ is, I think, a neat label for someone who is, sort of, in charge of something, but not with the expectation of control that a director may have. An indirector operates indirectly, through other people, along sometimes lengthy and rickety communication channels. An indirector can’t or won’t order people to do things and instead resorts to other strategies, including encouragement, facilitation and finger-crossing. Some indirectors seem to find this easy. Given my instincts for simplicity and do-it-yourself, I don’t. But the new badge helps, a bit.

Bob Rotheram
Indirector, Sounds Good 2

First impressions

Steve Dixon — Wed, 17 Dec 2008 03:39:00 -0800

It's a few weeks now since we did the "first run" of audio feedback with our first years - a few weeks to allow us to catch our breath, look back and reflect on what we did. So what can I say? Initial responses to audio feedback, from both staff and students, have been overwhelmingly positive. Students particularly liked the informal nature of the feedback (particularly pertinent when bearing in mind that they are first years and the importance of retention rates these days), with comments like "It felt like the tutor was in the room there with you" being not uncommon. Interestingly, the (very) few negative comments from students were regarding the quality of the feedback itself, and not the medium - definitely something to bear in mind.

Staff were also generally positive, and comments tended to follow the patterns predicted by Bob in his helpful advice. Virtually all staff felt that the process saved them time, and, obviously, those who were more familiar with the technology felt much more comfortable with the process. Staff training sessions on the equipment generally went very well, and I think we'll keep these ongoing, so that familiarity breeds ease of use. Our turnaround time for marking (something that our Exams and Assessments Office keeps close tabs on) was markedly reduced.

It would be interesting to see if the move to audio has led to any change in the levels of engagement that students have with feedback. A common perception here at Newman is that students are only interested in their marks, and often don't read or only skim the feedback given to them for assignments. A first reading of their comments would suggest so, and as our students are now clamouring for more audio feedback in other modules, this may be something to take further...

Simon Sweeney's first post

Simon Sweeney — Mon, 08 Dec 2008 06:19:00 -0800

Greetings from York....
Finally making a contribution to the Blog. I saw the Guardian piece last week and was pleased to see good publicity for this innovative project!
We have had some interesting discussions at York St John on various kinds of electronic feedback - including electronic marking - something I have experimented with in the last year or so. But audio feedback is much more exciting. Bob came to YSJ a month or so ago and clearly articulated the benefits observable so far from the SoundsGood project and it sounded uncomplicated and student friendly.
I have a number of modules lined up to provide electronic feedback on - first of all an M-Level module that I have already marked and am about to record the feedback and distribute to students. I am planning to ask students for an instant response to see what they make of it. Most of these are NNS (non-native speakers) which I think adds an extra dimension to the benefits of audio feedback.
Later this term (hopefully before Christmas) I will have added another larger cohort on an undergraduate module. A colleague is also planning audio feedback for another M-level module.
I have used the recorder already - and have so far found it OK to use, if a bit complicated to rename files (it's vital to name the files in an instantly recognisable way).
That's enough for an initial contribution.

Power of the spoken word

Bob Rotheram — Tue, 02 Dec 2008 07:45:00 -0800

Sounds Good is featured on p.5 of the 'Digital Student' supplement in today's Guardian, published in association with JISC. You can see the online version at: www.guardian.co.uk/digitalstudent/spoken-word.

About to start stage one feedback.

Simon Thomson — Wed, 12 Nov 2008 10:01:00 -0800

At the end of this week over 100 students (I hope) will be submitting their work for me to assess. I used audio feedback last year on about 25 students from this very module. This year I plan to do about 80. The students upload the assignments via our virtual learning environment (VLE). I then send the audio feedback files back to the students via the VLE.

For this assignment due Friday I will be giving audio to about 50% of the students from this cohort. I will be interested to see how much time that saves me and of course the feedback I get from the students.

I will then be giving audio feedback to a different group of students next semester, on a different level just to get an idea how the feedback is received at different levels of learning.

I have always enjoyed giving audio feedback and now the process is second nature so I do not have the task of learning the technologies. For those of you who want to try audio feedback but are worried about the technicalities of undertaking it I would just give it a go.

Bob and I have given an introductory workshop on audio feedback to staff at Leeds Met and many of those were first time users. By the end of a 1 hour session the technology was not the barrier but hearing your own voice played back for the first time was the hardest thing to overcome.

Here Goes....

Steve Dixon — Tue, 11 Nov 2008 06:28:00 -0800

Hi, this is my first post to the Sounds Good blog, and I only have 10 minutes or so. I'm Steve Dixon, Senior Lecturer in Education Studies at Newman University College, and we're just about to start audio feedback.

Literally.

As in about nine and a half minutes time.

We're trialling with our first year cohort, so we have about 80 students or so. All the students have submitted an essay via Moodle (our VLE), and we're adding in-text comments then providing a link to the audio feedback, again via Moodle. The setting up and administration of this has been fairly straightforward, although as we have a marking team for this (6 staff), I've also had to deliver training sessions and write guidelines (Bob's original documentation was incredibly useful in this respect). We're using Handy H4s for recording, although one or two of the marking team have hinted at a preference for digital microphones.

OK, I now have less than 5 minutes before we have a last meeting before we start marking and giving feedback, and coffee is important. Wish us luck...

Are we nearly there yet?

Bob Rotheram — Fri, 31 Oct 2008 04:27:00 -0700

As a dad and veteran of many car journeys from the midlands to the north, I’m used to people trying to get a handle on how much further there is to go. I recently had another experience of this when someone asked me about Sounds Good. They were under the impression that we were ‘nearly there’.

We’ve certainly come a long way. After setting off in January, we followed the map quite closely. By July we’d given about 15 Leeds Met staff and 460 students experience of audio feedback on coursework. Pleasingly, whichever side of the microphone they were on, the great majority liked it. We shouted out of the windows and encouraged passers-by to get started. We showered them with leaflets, to try to help.

Amid the din, there were messages from JISC. They wanted us to extend the journey, to do ‘Sounds Good 2’ and work with two Higher Education Academy subject centres. But before we embarked on that, we had a summer break.

Come September, we were back on the road. ‘Sounds Good 2’ is about introducing more Leeds Met staff and students to audio feedback, as well as helping three other places to begin using it. Hence the flurry of meetings on home territory and gigs at Newman University College, Birmingham, the University of Northampton and York St John University.

The engineering and GEES subject centres have also got going. They’ve been funded under JISC’s ‘Widening stakeholder engagement’ initiative to help engineers and geographers start using audio for assessment feedback. Their plans include input from several folks who work with audio and, for my part, I’ve already visited Cardiff and Loughborough universities.

So are we nearly there yet? No. The road map now stretches to next March and quite a few new colleagues will be following us in their vehicles. There’s plenty of noise, most of which sounds good. However, some are only just getting into first gear and – as ever – I worry they may stall. Dropping the metaphor for a moment, there’s much to do if several new groups of people are to be successfully introduced to audio feedback and if the knowledge base is to be expanded.

In terms my kids would understand, when we set off I said we were going to Blackpool to see Grandma. We’ve been there, done that, and it was great. However, there’s been a change of plan. When we get to the M6 we’ll turn left for the Lake District rather than right to go home directly. And all the while I’ve got to watch the rear-view mirror to be sure our friends are still with us. It’s become a longer, more complicated journey but there’s lots to see and do, and it could be more fun. Let’s enjoy the ride.

Now then, I spy with my little eye…

Mean time for Greenwich

Bob Rotheram — Sun, 26 Oct 2008 05:21:00 -0700

Last Thursday, 23 October, was a great night out in London for 10 from Leeds Met (and about 1,300 others). We were at the Grosvenor House Hotel in Park Lane for the Times Higher Awards. I’ve previously mentioned that Sounds Good was shortlisted for an award in the category for ‘outstanding contribution to innovation and technology’. I also wrote that my reaction to this news was “Wow, I’m amazed!” It’s not false modesty when I say that, although I’m very pleased with the way Sounds Good is going, I didn’t think for a moment it would win an award.

And so it proved. In the ‘innovation and technology’ category, the worthy winner was the University of Greenwich with a method of combining the greenhouse gas carbon dioxide with toxic waste to produce material which can be used in concrete blocks. That’s much more likely to ‘make a difference’ than Sounds Good. I didn’t meet any of the Greenwich folks, but I bet they had a mean time celebrating!

The Leeds Met contingent – which included several members of the Sounds Good team – enjoyed itself too. My greatest achievement on the night was to stay upright whilst ‘dancing’ to the Stones. Thanks to all who made the evening possible: JISC who fund the project; the team for their great work; my manager, Sally Brown, for nominating us; Leeds Met for paying for the trip.

Sounds Good Piped.

Bob Ridge-Stearn — Fri, 17 Oct 2008 09:31:00 -0700

I’ve been playing around recently with Yahoo Pipes, so when I decided to trawl the web for info about Sounds Good, it seemed an ideal opportunity to try it out for real. So, here’s a pipe that pipes in info about Sounds Good and audio feedback from various news and blog sites I like on the web. It’s not particularly sophisticated, so feel free to adapt it if you like. http://pipes.yahoo.com/bobjacqs/soundsgood2 .

Newman joins Sounds Good 2.

Bob Ridge-Stearn — Fri, 17 Oct 2008 09:25:00 -0700

I first came across the Sounds Good project when I attended a session that Will Stewart, Bob Rotheram and Martina Doolan ran at the Next Generation Environments Conference at Aston back in April. Newman is a small college which prides itself in the personal attention we are able to give to our students so, if Sounds Good was showing that students valued audio feedback, I was sure our staff would be interested. We are also, like all universities, concerned with providing timely feedback and often there is a tension between constructive and comprehensive feedback and the speed at which it can be delivered. Although the first phase of Sounds Good did not suggest that providing audio feedback would reduce the time involved in providing it, it did suggest that with practice and by observing certain ways of working, it could be provided quite quickly. I returned to Newman and spread the word and soon had several keen tutors who wanted to get involved. Bob Rotheram visited us at the end of September and we are now officially on board and I hope bring to the project a diverse range of subjects and students. Joining in the second phase of the project allows us to benefit from the experience gained in the first phase and we can’t wait to start!

Bob Ridge-Stearn
Head of e-Learning, Newman University College, Birmingham.

Burglars, teachers and windows

Bob Rotheram — Sun, 12 Oct 2008 02:42:00 -0700

Question: What’s the connection between burglars due in court and university teachers launching modules? Answer: Windows. If the link isn’t immediately obvious, it’s probably because your career has been different from mine. I used to be a probation officer and now I’m a university staff developer.

When they hear the word ‘windows’, many people involved with technology think first of the computer operating system. I’m very involved with technology: the core of my job is about its uses in assessment. What’s more, I’m producing this piece with the help of Windows. But my PC is behaving itself today and Microsoft’s money-spinner isn’t what’s on my mind. Instead, I’m reflecting on the last couple of weeks of Sounds Good activity and am keenly aware that I’m using windows as much as I can.

Burglars, teachers and windows? Mmm.

In my previous blog post, ‘Let’s not get carried away’, I noted that I’d spent September racing around leading sessions about Sounds Good. Since then I’ve done several more, including two last Friday. Some of the ‘gigs’ have been by invitation, others because I’ve taken the initiative. More than once, it has been difficult to fit the session in but I’ve made the effort. Why push myself? Windows.

Burglars, teachers and windows? Need clues? Anxiety. Dissatisfaction.

Right now, the beginning of the academic year in UK higher education, can be a pretty anxious time, and not just for new students. It’s when teaching staff make key decisions and establish patterns for the next chunk of time – the term, semester or year. Some will be worried or uneasy about how they are going to run their modules and assess the students. At times like these they may be more than usually receptive to snake-oil salesmen who offer the prospect of both making their lives easier and pleasing students. But soon, for better or worse, the teachers will have made their major decisions on assessment, learning and teaching. Anxiety levels will have fallen, along with the chances of them buying snake oil – until the cycle begins again.

The windows I have in mind are, of course, windows of opportunity. I used to try to exploit them with burglars up for sentence and anxious to show they were turning their lives around. Nowadays my target group is very different but the strategy is the same: get the timing right; use temporarily-raised anxiety and dissatisfaction to produce a public declaration of changed ways; hope for a better course of events. A gentle nudge may be all it takes.

Gotta go. Snake oil – sorry, audio feedback – to sell and I can hear the sound of windows closing.

Let's not get carried away

Bob Rotheram — Sun, 28 Sep 2008 15:08:00 -0700

August was relaxing but September is hectic. This is often true for those of us who work in UK higher education, but Sounds Good is making September 2008 even more of a whirl than usual for me. Not that I’m complaining.

This month I’ve led sessions about using audio for assessment feedback at:

Leeds Met’s Assessment, Learning and Teaching Conference;
the Association for Learning Technology conference (ALT-C) in Leeds;
a research seminar at the ExPERT Centre (the Centre of Excellence in Teaching and Learning – CETL), Portsmouth University;
an Engineering Subject Centre event at Cardiff University.

Everywhere, the reaction has been very positive; people are taken with the idea of audio feedback. At ALT-C, having given my presentation to an audience of perhaps 150, I was surprised and delighted by the number of folks who came up to me in the next 24 hours, wanting to chat about Sounds Good, asking for my contact details and saying they or their colleagues would like to try using audio feedback. Several have been in touch since. This doesn’t usually happen to me! Right now, Sounds Good feels good.

Let’s not get carried away though. All the project has done so far is to confirm that students like receiving audio feedback and that the Leeds Met staff team think it’s worth the effort of learning how to do it. Most colleagues aren’t yet saving time by speaking rather than writing their feedback, but they know they’re providing a better service. The most favourable circumstances for giving audio feedback seem to be where the assessor:

usually gives plenty of feedback;
writes slowly but speaks fluently;
is used to the technology.

So it’s probably not a case of ‘one size fits all’. Also, as with most new skills, practice is likely to help. Audio feedback is worth a try but, like anything else, it’s unlikely to be the magic bullet for all our assessment woes.

ALT-C reflections so far

Mark Dransfield — Wed, 10 Sep 2008 05:15:00 -0700

The ALT-C conference has been interesting so far but as always with large conferences like these, you often wonder whether you're getting the most relevant information, or whether you should have chosen a different group of presentations.

My experience on the whole so far, has been mixed. I've tried to attend a variety of presentation styles; short papers, syposiums and workshops and have found bits of relevant information in all of them, but also large clumps of irrelevant info too. My experience from networking with others is that they have found the same.

One of the main things I can take from this conference is that as a University, we are doing many of the same things other Universities are doing; electronic assignment submission, podcasting, video learning objects, audio feedback, development of richer content and so on. The main issue is that we do not seem to be doing this strategically in any way and I don't think we're alone here. I think we are attempting to address this though, by working strategically with faculties and with our plan to re-write the e-Learning strategy.

The niggling questions at the back of my mind though, which have been stimulated by the conference presentations, are more to do with our students. What are their expectations of technology? Do they want a more blended learning approach or are they happy with a document repository? Would they like to see richer media such as audio and video content available to them within the VLE and do they even know that this is possible - do our academic staff? How do we get this information from our students and correlate that with the current practices of our academic staff, so that they are aware of the needs and drivers? How can we evaluate and monitor the effectiveness and quality of our e-Learning within the institution?

ALT-C: Sounded good, but was it? Bob Rotheram and the Sounds Good project

Carol — Wed, 10 Sep 2008 07:18:00 -0700

Report from ALT-C 2008
Sounded good, but was it: a review of the 'Sounds Good' project
Bob Rotheram, Simon Thomson
abstract (on ALT-C 2008 website)
project web site

The Sounds Good project is funded by JISC. It aims to answer the question 'Can digital audio be used to give students quicker, better feedback on their work?'

The project has involved 15 teachers and around 460 students, ranging from 1st year undergraduates to post graduates.

What did students think?
Bob reported that students were overwhelmingly positive about the audio feedback. Some students reported that they found it more personal, and that they felt that the lecturer had actually engaged with them and their work. Some students stated that the audio made it possible to grasp what the lecturer felt the most important points in the feedback were. A few students requested written as well as audio feedback.
(see Sounds Good blog post What do students think?)

What were staff impressions?
Staff in the pilot group were generally in favour of giving audio feedback. Several staff members felt that the feedback they gave was higher quality and more detailed as a result of using audio. They liked the fact that it brought the human voice into the assessment process.

The only staff member who does not intend to continue to provide some feedback in this way was someone who used extremely short comments, and who was a very quick writer/typer.

Staff generally were able to operate the recorders they were given (Edirol R-09 hand held recorders) but did find some practical difficulties with actually making the recordings. Sometimes this was due to high levels of noise where they were trying to record, some staff found the re-naming of audio files to be time-consuming and sometimes there were problems with getting the audio files to students. There were concerns from 3 members around how easy it was to scale up this mode of feedback to larger groups.
(see Sounds Good blog post What's it like?)

Bob gave out a sheet giving his practice tips on using audio for feedback. I can't currently find this on line, but there are a number of documents available on the Sounds Good website that cover the topic:
http://web.mac.com/simonft/Sounds_Good/Documents_files/Giving_audio_feedback.doc
http://web.mac.com/simonft/Sounds_Good/Documents_files/Administration.doc
http://web.mac.com/simonft/Sounds_Good/Documents_files/Technical_tips.doc
http://web.mac.com/simonft/Sounds_Good/Documents_files/Saving_time.doc

I can definitely see how audio feedback could bring something rich into giving/receiving feedback, and could provide voice and presence for a process that can feel disappointingly perfunctory to some students.

My biggest question would be how feasible it would be to scale it up across an institution. There is quite a lot of potential for recordings to get muddled unless the audio is captured directly into a system that manages the mapping from audio to student record. Checking and double checking details might be more cumbersome where a 4 minute recording needs to be reviewed rather than skim reading a sheet of comments. This could have an impact on exam meetings. For summative assessments, the audio recordings presumably need to be retained until paper copies of scripts are destroyed, and then removed from the system. This could require records management. (Although the same can be said of any electronic artefact stored as part of a student record).

But perhaps focusing too hard on large scale provision misses the point. An initiative like this might have the potential to support individual lecturers to change the way they provide written feedback due to their reflection on the efficacy of audio feedback. And it might make students reflect on the way they use feedback too.

I wonder too whether it would bring something important to distance learning.