Digesting #EDCMOOC feedback

The dust is settling and we are beginning to get a sense of the overall impressions left by the EDCMOOC experience. We’ve been extremely grateful for the time that participants have taken to reflect on the course, in the discussion forums and in their blogs. We’ve also got some feedback via a survey we released a couple of weeks ago, from a range of people – from those who never logged in to those who formally completed the course, and everything in between. We’ll aim to share more details of this survey and its results as soon as we can.

Some very positive news for us is that a large majority  (82.8%) of those survey respondents who actively participated in EDCMOOC said that overall their experience was good, very good or excellent.

overall

The following feedback was pulled from blog and forum posts, and doesn’t yet include the survey response feedback. We’ll be continuing to work on drawing this together, but we wanted to get some of this information out as soon as possible.

The feedback we’ve gathered falls into five main categories: connections, content, course design, teaching and assessment.

Content. Those who ‘got’ it were, for the most part, happy with what was on offer and engaged and excited by the topics, discussion questions and resources. We pitched the course at what we felt was an introductory level, and some thought our approach was oversimplified, which highlights one of the primary challenges of developing an open course – there is no way the level can be right for everyone who might sign up. However, even setting this aside, there is still some work to be done to ensure that everyone who participates in the MOOC understands the purpose of the course, and knows what to expect in terms of the content and our approach to the subject area. There were a number of posts in the forum in particular that indicated that the course content wasn’t what some had expected or wanted, and this was reflected in objections to the cultural angle, the theoretical aspects of the course, the lack of practical advice about course design, and so on. We have tried to clarify in the course description what is, and is not, on offer in this course. One of the other main things we want to focus on for next time is better articulating what we see as the vital need for educators and e-learning practitioners to be critical consumers and producers of ‘stories’ about education and technology. This is the key purpose of this course, and we will be working to ensure that these critical perspectives are foregrounded, discussed and debated at every stage of the MOOC.

Connections. The pleasure and excitement of the ‘massive’ was experienced, for most people, alongside a sense that they were making satisfying connections – whether fleeting, or ongoing. Where this didn’t happen, the MOOC sometimes felt disorientating and overwhelming. As one participant put it, “those students who were fortunate enough to make meaningful connections with other students probably gained the most out of the course”. A key message we are getting from this first run is that more people wanted a sense of connection than were able to achieve it. This may be due in part to differences in familiarity with social media spaces (where a lot of the most meaningful connections were made), and to the point at which people began to engage with the MOOC (some started very early – well before the formal course launch date). Next time the MOOC runs, we will seek more ways to support those who want to make meaningful connections (while leaving ample room for groups to self-organise, for cross-fertilisation to take place, and for people to work independently – all of which we think is really important).

Course design. There were a number of aspects of our course design that drew comment. Strengths were seen in the pre-arrival information and encouragement to network; the value of having MSc in Digital Education students as teaching assistants; the flexibility of the course (in terms of content, environments and activities) and the ability to take many different kinds of approach. More mixed feedback came about the timings (a mix of responses about whether 5 week was just right or not enough, and some comments about the suggested workload of 5-7 hours per week being unrealistic); and the ‘massive’ nature of the course (exhilarating vs overwhelming).

Teaching. While some commentators appreciated that the course was designed to foreground the voices of participants, many comments related to a desire to hear more from the MOOC teachers. The desire for more of everything from the teachers – hangouts, structure, guidance, discussion participation – gives a strong indication that MOOC participants need and want teacher presence to be high-profile, ‘in your face’. We’re actively discussing ways we can make our presence more felt in the MOOC.

Assessment. The  final assignment – the digital artefact – supported a level of creativity and engagement that was really impressive, both during the preparation period and after submission. For many, the experience of creating their artefact was, in itself, of great value, and others found the peer assessment process challenging and fruitful. There was a sense that participants would have valued some practice with the peer assessment tool, and with creating and giving feedback on artefacts. Some very constructive suggestions were made about the markers’ process of interpreting the artefacts: that the assessment criteria may have rewarded ‘literal’ interpretations of the course material rather than more abstract/adventurous/creative ones; the possibility that a short statement/self-reflection about the artefact (an ‘exegesis’) might help markers engage more with the creator’s intentions; and that finding a way to allow people to set their own learning goals and be assessed on these might be in keeping with the ethos of EDCMOOC. Finally, questions were raised about the scale (0-2), the automated awarding of ‘distinctions’ for combined peer grades over a certain threshold, and the lack of opportunity to respond to peer feedback (a feedback on feedback mechanism). These are all things we are considering carefully, and talking to Coursera about.

In summary, we’re working out how to make EDCMOOC even better next time in response to the feedback participants have shared, and where platform issues arose, we are raising these with Coursera. Thank you very much, again, to all those who participated, and all those who have shared their insights about EDCMOOC.

Advertisements

#edcmooc departures

The start of EDCMOOC was hard to pin down, and it’s clear it will not end simply, either. There is still so much to do, and so much to say.

Online endings are something we have experience with, and something I am very interested in from a research and a teaching point of view. Online departures do not always seem like departures, if the members of an online group or an event are still where they were, materially connected to the same devices, in the same geographical locations, with ongoing links to one another. For that reason, I think those endings have to be handled with care to ensure that they can be experienced as sad, relieving, exhilarating, proud, transitional, or however else they may be felt. There has to be some kind of deliberate marker of an ending, to give people permission to go. (which doesn’t mean they won’t come back together in new ways.)

So, while I have every enthusiasm for the ideas people are sharing about how to stay in touch, how to keep the work of EDCMOOC going, how to sustain and grow the network,  I also believe that these things have to have a purpose and a driving force that is post-EDCMOOC.

Having said all that, I am more than content to appreciate what unfolds next, whether or not I see it as departure-like. But if others find this idea interesting, some questions might be: what work do you want to do next? What project can you start, what event can you plan, what course can you develop? What should your EDCMOOC become?

The answer will be different for different groups and individuals as you move on from this (partly) shared experience, which is as it should be. I hope that people will keep using the tag where relevant, sometimes, as a marker that ‘EDCMOOC was here’. But EDCMOOC itself is a happening, and happenings are of a time. They are openings, made to be passed through.

References:

Kazmer, M. (2007). Beyond C U L8R: disengaging from online social worlds. New Media & Society 9(1); 111-138.

Assessment, #edcmooc style

I guessed it would happen, because of our experiences on the MSc in Digital Education, but I am delighted to see what is happening around the edcmooc digital artefact assignments. There are three things in particular that stand out: the effort, the sharing, and the feedback. These observations are based only on what’s been shared in forums, twitter and blogs – I haven’t looked at any of the formal submissions yet.

1. Effort. Digital artefact creation seems to inspire great creativity and effort. Inviting people to go ‘beyond text’ to represent academic knowledge, and to create work in public, appears to be motivating in a way that, say, writing an essay that only three people will read, generally isn’t. I think this is about audience, and about the pleasure of sharing and expressing understanding in multimodal ways. We are seeing people drawing on their own personal and professional expertise (coding, writing, film-making, music, photography, expertise with web-based tools, knowledge of interesting bodies of literature, and so on) to really grapple with course themes. I love it and will have a terrible time tearing myself away.

2. Sharing. A number of MOOC participants are rejecting (sometimes explicitly) protectiveness and secrecy in favour of sharing their ideas, drafts, processes and final artefacts. We could claim that we have reached a utopian state of trust within the MOOC (of course!!), but the networked nature of the assignment makes it possible, too. In a public, web context, sharing doesn’t take anything away from the sharer, but rather stamps ideas as theirs, drives traffic to their artefacts, and gives a reputational boost. This is digital scholarship in action. Those who are sharing in this way are providing helpful examples for others to build on, too. (The merits of the ‘exemplar’ are sometimes contested, but in general I am a fan.)

3. Feedback. The generosity of those sharing is matched by the generosity of those responding with enthusiasm, suggestions, constructive advice and audience responses. Particularly impressive has been the references to the assessment criteria, which a number of people in the forum are making in their informal feedback. By bringing these criteria into play at this stage, discussing and debating what they mean, and trying to apply them in context, MOOC participants are involving themselves in the feedback process in ways that I think will be extremely helpful during the peer assessment activity, and for those still working on their artefacts.

To sum up: heaps of praise for EDCMOOC participants, and the work they are beginning to do on these final assignments.

“This river of learning is not unbounded” – educational openings

We have plunged into the flow of the opening of education. …[but] this river of learning is not unbounded… (Betha Gutsche, EDCMOOC forum)

Betha’s post has helped me to articulate the sense I have that an opening education (Campbell, 2012) is not limitless or unstructured. An opening is a way through to somewhere, but it’s also, in itself, a place. What Christine might call a liminal place. It is here rather than somewhere else. And, depending on where you are and where you need to go, it might or might not be a good starting point.

One reason that matters is because it suggests that an opening isn’t an opening (or not the right opening) for everyone. If this is true of MOOCs, it might help us think about the concept of ‘retention‘. Perhaps we should ask about who stays in a MOOC, not “who is motivated enough?”, or “are MOOCs overhyped?”, but “for whom is this course an opening?”.

To pose this question might highlight new responsibilities for teachers and researchers in this area. For instance, if it turns out that MOOCs tend to be openings mostly for those who are already educationally privileged, we need to consider carefully what this means for the mission of the MOOC, which is typically framed quite differently.

On the level of the individual, we might also help MOOC participants to consider if the opening is in the right place (or appearing at the right time) for them. It can be understandably hard for people to see themselves as ‘dropouts’, even when it is clear to them that the course is not working for them. Maybe we need a ‘trial period’ (a money-back guarantee? :-)) for the MOOC, where people can check it out before they make a psychological commitment that might feel hard to back down from. A big “loitering allowed” sign by the opening…

A few of my favourite EDCMOOC things (today!)

In no particular order, a few EDCMOOC things I’m enjoying today.

Conversations in the cloud – https://voicethread.com/#q.b4104932.i21006791 – ten EDCMOOCers have been exploring presence and connection through a series of Voicethread comments.

Digital Vikings “Kapsul” – a crowd-sourced gallery for representations of the Digital Vikings meme that’s emerging in the MOOC, in opposition to the natives/immigrants binary. http://www.kapsul.org/public/digitalvikings

The EDCMOOC tag on Flickr, where so far 140 people have tagged images. http://www.flickr.com/photos/tags/edcmooc/

The “metaphors for opening education” thread in the forums.

Rage Against the Machine

Rage Against the Machine – EDCMOOC image from heywayne. CC: by-nc-sa

Joy and conformity – two sides of the ‘massive’ coin?

The title of this post is a conflation of two things I want to think about more. It also follows on, down a slightly winding path, from one of Jeremy’s posts yesterday, where he described his concern about people seeking ‘correct’ strategies for dealing with the large amount of content flowing into and through the MOOC.

First is a very lovely description of what – at its best moments – EDCMOOC can feel like. This was written by Kathy Fitch in the discussion forum, where she described witnessing and finding a sense of

joyful engagement and absorption

in the MOOC experience. She explicitly frames the pleasure of the MOOC as something that is not – metaphorically or literally – a classroom. Its excitement comes from being ‘massive’, from being a happening.

Second is an observation from Desi Pedeva about sameness – the way that MOOC participants are responding, all across the web, to the same set of resources, at the same time, in (in her view) many of the same kinds of ways. She argues that this happens in smaller courses, too, for the same reasons – the way that teachers control “the choice of what to include and what to exclude in their courses” – but at scale it becomes oppressive:

I can’t escape the feeling that my unique internal ideas have been trivialized by being exposed and multiplied by thousands on the Internet.

She goes on to argue that smaller courses avoid the “homogenisation of knowledge” primarily because they are small: “The small classes create pockets of diverse knowledge and diverse learning communities”.

So, here we have the “massive” as both an engine of joyful engagement, and an engine of trivialisation and conformity. I don’t believe that these perspectives are mutually exclusive, but I do think we are urgently required by the emergence, and our adoption, of the MOOC to consider what the implications of the “massive” may be for both knowledge and learning.

When the EDCMOOC course team started talking about developing our course, we didn’t ask how to teach 40,000 people at once, but rather what 40,000 people could do together. That’s one of the reasons why we wanted much of the work of the course to happen in the open web. But it seems to me that the thrill of being part of something so big, so public, and so distributed – finding resonances, making unexpected connections – is not without its difficulties. I think Desi has indicated one way these difficulties might be understood. If MOOC teachers and designers value difference and diversity, how do we design for this, while also for massive happenings?

Jen (@jar)

Lessons about forum design from EDCMOOC

It seems clear that the Coursera forum tool was built to facilitate factual questions and answers, not conversations or connections. Our EDCMOOC participants are trying to push it much further, and the cracks are definitely showing. Here’s my shortlist of things that Coursera could do to make their forum tool more conducive to networking and conversation. We’ll be passing these on to the Coursera team. Many of these points are pulled from suggestions in the EDCMOOC ‘feedback to Coursera’ forum, and additional suggestions or comments are welcome. I’ll probably add to this post over the next few days, too.

Networking:

Implement a ‘follow’ button so people can keep track of others they want to read more from.

The profile page should show all of the posts from that person.

Need to be able to favourite and link back to particular posts and store those favourites.

Search:

Results need to indicate which subforum a thread is in.

Subforums should be searchable.

Need the ability to search within a particular thread.

Need to be able to search by member of the class including yourself and see all of someone’s posts.

Comments:

Should be collapsible for easier reading.

It should be possible to reply to a comment.

Subscribing:

As many threads are very long, subscribing to just a post and its comments would be useful.

Daily summary of forum post threads to which you subscribe.

When a new post notification arrives by email, the link should take you directly to that post (not just to the thread).

Browsing:

Sort across a forum or all forums to see posts in date order.

The ability to sort by top votes.

Groups:

an optional grouping mechanism, where randomised groups of any size (determined by the teachers?) can be either automatically created, or people can choose to be allocated to one.

a mechanism for people to create groups (geographical, topical and so on) and invite others to join.

“where are the teachers”? “where are the videos”?

These two very early questions from the #edcmooc discussion board seem to be related. There is clearly already a Coursera (or perhaps an xMOOC?) structure and aesthetic which means that the expectation is for these courses to be designed around video lectures. Because ours isn’t, it’s proving difficult for some experienced MOOC participants to “read” the presence of the teachers.

This issue – of online teacher presence – is one that has interested me for a number of years (see this paper that Hamish and I wrote for one take on this). Most recently, Amy Collier and I have been having regular conversations about all kinds of MOOC and online education type topics, and we keep returning to the question of online course design, and the way in which this does (and doesn’t) represent the metaphorical body of the online teacher. The way the teacher is usually represented in the xMOOC seems to be quite specific, and rather constrained (to the size of a video window, really). The teacher’s role, in my view, includes the teacher’s frustrations, discoveries and adventures in shaping curriculum, and in making sometimes rather incalcitrant VLE platforms (in which I’d include Coursera) express their “philosophy and belief in action“. But this may not be a widely shared understanding of the teacher in the MOOC.

A number of participants have commented on the networked, web-friendly and distributed nature of this MOOC, and are seeing this as a strength of our approach. I agree, and I am extremely impressed with the creativity and generosity emerging from the #edcmooc network. So, I wonder: in what ways might the activity of the network be understood as teaching? To consider that, we need to consider the history of the MOOC and its roots in connectivist approaches which locate learning in the network. So, is there teaching in the network, too? What is it that the network is teaching?

Jen Ross
@jar