Learning outcomes for #edcmooc

From the discussion forums:  Example 1

…what are the learning outcomes for which this course was designed? Which outcomes are served by each of the segments? How will we assess our progress towards these outcomes?

Thomas Luxon (a month ago)

Just over a decade ago, I asked staff and students at the university where I worked (not the University of Edinburgh) What is a learning outcome?  I heard from 111 staff, mainly academics, and 260 students.  I received some fascinating responses, but unfortunately my institution didn’t want me to publish them.  I could identify two main broad categories:  a result and an anticipatory statement or intention.  A third category also emerged when I looked at student responses – an aspect of a course, particularly assessment.

What is a learning outcome?

Staff Students
Result 53% 23%
Intention 29% 30%
Assessment/element of course 8% 31%

From a study undertaken by Christine Sinclair in 2002  (NB:  actual comments were much more nuanced)

My study threw some light on some mismatches I could see in staff and student expectations.  I suspect there is still some ambiguity about this term, and our EDCMOOC has strengthened this feeling for me.  My own preferred emphasis tends to be on results, but I do understand the strength of adding the word ‘intended’ as well.  In terms of a result I think an artefact is a very clear outcome.  But look at the range of learning embodied in these artefacts: here’s a link to a fantastic student-created collation of some of them.  Could we have anticipated what people would have learned?  What statements of intended outcomes would have done justice to what has emerged?  And what about the learning outcomes achieved by people who haven’t submitted?

Stating intended learning outcomes can be very useful for many courses. It helps teachers with planning their course design and students with knowing what’s expected of them.  But what’s been happening on our course here does suggest that it’s worth revisiting the words of Lawrence Stenhouse, writing in 1975:  ‘Education as induction into knowledge is successful to the extent that it makes the behavioral outcomes of the students unpredictable’ (p. 82).  There are more recent challenges to learning outcomes too, including this call to resist them.  I’ve done my share of teaching academics – and even also our students on the MSc in Digital Education – about course design using learning outcomes, but that does include applying a critical lens.

From the discussion forums: Example 2

One of the challenges of MOOCs is measuring outcomes. Since most of your students are educators we should want to help measure the outcomes. I would recommend you send us students a survey, perhaps every six months for the next two years, to see how we are using the materials and how we evaluate the course over time.

Richard Dine (3 weeks ago)

This is an interesting idea.  I once had a ‘learning outcome’ eight years after the input: I remember suddenly being aware of what ‘that woman at the conference was getting at’.  Her presentation had clearly been memorable, but I did not have sufficient pre-requisite knowledge in my repertoire at the time to process her meaning.  This experience reinforced my view that simple statements of outcomes in relation to a course may encourage a distorted picture.  How often and what intervals is it meaningful to check on outcomes – and how can we be sure what to attribute them to?

I accept that we will have lost some students because we did not explicitly state learning outcomes and show step by step how they would be achieved.  Even those students, I would argue, are likely to have left the course with some important learning outcomes that are specific to themselves.

From a teacher’s perspective, there have been many learning outcomes from this MOOC.  I think I have experienced several just from looking at the Twitter feed and discussion forums this morning.  There have been adjustments to my understanding of, for example:  responses to automation, language and cultural perspectives in a globalised course, interpretation of messages, the role of peer assessment as a learning opportunity, what is (not) possible when assessing at scale.  And there will some I haven’t even begun to process yet.

To everyone involved: I hope that your own learning outcomes from EDCMOOC have been satisfying, but not to the extent that you want to stop pursuing the topics to achieve more.  I have certainly been energised by seeing the outcomes of the past five weeks – and I still have a lot more to look at.

Christine Sinclair

@CMSinclair

Stenhouse, L. (1975) An Introduction to Curriculum Research and Development.  London: Heinemann Educational Books Ltd.

Advertisements

Assessment, #edcmooc style

I guessed it would happen, because of our experiences on the MSc in Digital Education, but I am delighted to see what is happening around the edcmooc digital artefact assignments. There are three things in particular that stand out: the effort, the sharing, and the feedback. These observations are based only on what’s been shared in forums, twitter and blogs – I haven’t looked at any of the formal submissions yet.

1. Effort. Digital artefact creation seems to inspire great creativity and effort. Inviting people to go ‘beyond text’ to represent academic knowledge, and to create work in public, appears to be motivating in a way that, say, writing an essay that only three people will read, generally isn’t. I think this is about audience, and about the pleasure of sharing and expressing understanding in multimodal ways. We are seeing people drawing on their own personal and professional expertise (coding, writing, film-making, music, photography, expertise with web-based tools, knowledge of interesting bodies of literature, and so on) to really grapple with course themes. I love it and will have a terrible time tearing myself away.

2. Sharing. A number of MOOC participants are rejecting (sometimes explicitly) protectiveness and secrecy in favour of sharing their ideas, drafts, processes and final artefacts. We could claim that we have reached a utopian state of trust within the MOOC (of course!!), but the networked nature of the assignment makes it possible, too. In a public, web context, sharing doesn’t take anything away from the sharer, but rather stamps ideas as theirs, drives traffic to their artefacts, and gives a reputational boost. This is digital scholarship in action. Those who are sharing in this way are providing helpful examples for others to build on, too. (The merits of the ‘exemplar’ are sometimes contested, but in general I am a fan.)

3. Feedback. The generosity of those sharing is matched by the generosity of those responding with enthusiasm, suggestions, constructive advice and audience responses. Particularly impressive has been the references to the assessment criteria, which a number of people in the forum are making in their informal feedback. By bringing these criteria into play at this stage, discussing and debating what they mean, and trying to apply them in context, MOOC participants are involving themselves in the feedback process in ways that I think will be extremely helpful during the peer assessment activity, and for those still working on their artefacts.

To sum up: heaps of praise for EDCMOOC participants, and the work they are beginning to do on these final assignments.

We have #EDCMOOC liftoff…

We’ll be using this team blog to document the ‘E-learning and Digital Cultures’ MOOC from the University of Edinburgh in partnership with Coursera.  We wanted to write about our involvement with the EDCMOOC over the next five weeks, recording developmental issues and reflecting on the experience of teaching and participating in this experimental course.  We’ll be feeding these posts back into the EDCMOOC as way of incorporating our perspectives as teachers, and we hope that it will also serve as a useful compendium for those outside of the course interested in our daily experiences as MOOC educators.  This first post outlines some of the dilemmas and challenges we faced in developing the EDCMOOC, and our reflections on the course launch and first day.

For a team used to working with in-house technologies on the MSc in Digital Education, developing a course in an externally provided platform was in many cases a new experience.  Furthermore, being as concerned with pedagogical methods as are with content, pushing the boundaries of the Coursera platform was always going to be in our interests.  We wanted to use the Coursera platform creatively, however this often meant requesting features that were not present, or were in development.  Nevertheless, experimenting with the early stages of such a high-profile platform have been rewarding, and we hope to see the technology develop as Coursera and their partners continue to explore new ways of teaching at scale.

While we wanted course activity to be distributed on the social web (with the use of blogs and Twitter for example), we were also interested in aggregating content produced by students so that it might be easily accessible from a single location.  The connectivist-informed MOOCs had employed this technique well, and we wanted to explore a similar approach from within the Coursera platform.  Using a combination of Yahoo Pipes and WordPress, we managed to develop an aggregated newsletter which displays participant blog posts each day, as long as they are tagged with the course hashtag: #edcmooc.  However, it remains an experimental system, and we are unsure about its stability as the course progresses, or its value as a way to engage with course content given the potential number of posts that could be pulled together.  We’ll be watching the EDCMOOC News closely in the coming weeks.

The course site was opened on the evening of Sunday 27th, and we were excited to see hundreds, then thousands, of participants enter the site within half an hour of the announcement email.  The Twitter stream erupted, and it was great to see such enthusiasm for the launch.  We were keen to gauge the reaction of participants, and Twitter proved to be a helpful way to get immediate feedback about the experiences of entering the course for the first time.  Most tweets were very positive, while a few flagged up issues and queries, some of which we were able to catch.  The forums within the Coursera site proved to be a better way of engaging with initial questions.

Early forum posts highlighted some lack of understanding with regards to the assessment and grading of the course, technical matters relating to the Coursera platform, and some initial engagement with the resources and themes.  While working to provide early clarification or responses to these posts, we also found the spread of different concerns interesting.  Early queries may provide some measure of what our participants find most important about the course.  Some immediately began discussing the resources, perhaps indicative of a primary interest in the content we had chosen, while others appeared to head straight for the guidance about assessment, clearly interested in understanding what the outcomes of participation might be.  Courses with such high enrolment numbers will inevitably have diverse participants, seeking many different things from a MOOC.  We’re really interested in whether courses of such scale can be everything to everybody: can they provide rich opportunities to self-direct and form personal learning networks, as well as guide learners through predefined paths, and do such all-encompassing attempts dilute the potency of a less-inclusive approach?

As the first day of #edcmooc Tweets streamed by, we saw a wealth of conversation, mostly positive feedback or comments focussed on the course content, interspersed with a few pleas for help, admissions of confusion and the odd unenthusiastic remark.  Something feels right about that kind of variety.  While MOOCs are sometimes thought of as being clouded by the optimism of their committed populations, we welcome productive critique and resistance, and hope it becomes an integral part of our continuing MOOC experimentation.

Jeremy Knox
@j_k_knox