Planning for assessment

I’ve been stuck on how to improve our current assessment approach – we offer a five week open online course, based on the ISW (instructional skills workshop)   model. We focus on reflective practice, encouraging personal reflection on learning, and participants are asked to provide constructive feedback to peers on shared learning and on the facilitation of a mini-session (each week features a mini-session activity on a specific topic ranging from adult learners online to facilitating teams effectively online).

We’ve had some problems with persistence, consistent participation, and we’ve received some feedback from participants that suggests we should integrate a learning contract, put more time into clarifying the expectations of performance (especially as your lack of participation can so greatly affect your peers) and offer grades as a motivation to participate more consistently. The weakness of the openness of our approach (the 5 week course is free to BC educators and there are no prerequisites) and the depth of learning and practice we try to offer and support, is that participants may not fully commit to completion.

I fully understand as I’ve found it challenging to keep up my commitment to this OOC because it doesn’t directly relate to my paid work currently and because, to do it well, requires a fairly significant investment of my time (not because the Storyboarding OOC is really demanding – but the analysis and contemplation required to fully engage with our course, while simultaneously trying to provide some useful feedback to other participants, is often challenging).

So, my proposed assessment plan – only a couple of amendments to the current approach –

1.  Our primary purpose (I believe) is to enable learners to experience the online learning environment from different perspectives (playing different roles) and to couple that experience with readings, advice from experienced and successful online instructors (through videos or writings) to develop and improve their ability to successfully facilitate online learning for adult learners. So we need to help them develop a framework to guide their ongoing development and improve their self-assessment of their practice.

2.  How do participants currently assess their practice?
Reflective Practice:   Each participant is asked to set up a journal to collect their thoughts insights and challenges each week (5 weeks). They are also asked to review some sample rubrics used to assess student participation in online learning and to construct a rubric to help them assess their own participation against criteria and standards they establish (and can modify) as they progress through the course.

Each week, participants will share “nuggets” from their journals (and, if they choose, from their self-evaluation using their rubric). They review and discuss in an online forum with their peers to encourage re-evaluation and, potentially, new insights.

Participants take turns working in pairs or small groups to facilitate a mini-session on a pre-established topic that must be completed within five days (some flexibility). Constructive feedback from peers is shared in a discussion forum and the facilitators for each week complete a detailed survey to draw out their perceptions of their performance and to identify area they liked or would like to improve. Targeted feedback is provided by the course facilitators (private to each individual). No structured assessment is provided.

3.  My proposed changes/additions to this formative assessment approach:
–  Ask participants to share their goals for participation and learning at the beginning of the course (not necessarily their personal rubrics but the general goals). The summary activity in Week 5 called “Looking Back, Looking Forward” should include meaningful reflection on their participation and learning goals and the identification of selected area for application of learning or future areas of growth. They will be encouraged to consider the relationship between their self-assessment of their participation and what it means – not only to their future work with online learners but to their own progress towards the achievement of their learning goals in the course.

– emphasize the development of online facilitation skills in general ways throughout the course and then include a final activity to encourage participants to develop their own personal rubric to self-assess and continue to develop their online facilitation skills and their ability to facilitate group work or team work online.

I’m going to propose we award some badges for performance (as we can’t award grades) – participants could select the level they believe they have achieved and explain why they chose a particular level for each badge? Minimum criteria would be active participation throughout the course??

Online Learning Facilitation

Community Building Online

Team Work Online

 

I’ve still got to get feedback from my co-facilitator but I’ll put it on our GoogleSheet for open discussion and post a link to this blog post in the discussion forum.

On to the next challenge!

 

Advertisement

Storyboarding OOC: exploring collaborative, open course re-design

Did you ever hear the story about the emperor’s new clothes? Well, there is a certain sense of being naked when you start to share your course outline and publicly engage in revising, improving, tearing it apart.

Thanks to Gabi Witthaus, Brenda Padilla, and Jeff Stanford and the Storyboarding OOC (open online course – not massive 😉 they are currently coordinating and facilitating so well, I`ve been sharing my attempts to re-envision, refine, revise and repair an online facilitation skills course I`ve been co-facilitating with Sylvia Currie of SCoPE over the past year. We`ve had the luck to inherit (share) an OER resource originally developed by the talented instructors at Royal Roads University, based on the principles of the well-respected ISW (Instructional Skills Workshop) model.

We`ve tweaked it on the run and tried to stay true to the original learning outcomes and vision but I had been starting to feel like we needed to revisit them to ensure that our updates were still accomplishing what we intended or hoped.

Step 1 was selecting our storyboarding tool. Gabi shared thre main options (and provided additional cloud-based, free, suggestions – Prezi – presentations; Gliffy – diagrams, flowcharts; Freemind – mind-mapping; Scrumblr – Canadian, ‘no login required’ whiteboared)

Lino – http://en.linoit.com/      a free sticky note and canvas service	Popplet – http://popplet.com/ – a tool for iPad or web to capture your ideas	Google Sheets – http://www.google.com/sheets/ – create, edit, collaborate everywhere

If you haven’t explored these tools, they’re all useful in different ways. We initially chose Popplet because we loved the visual approach, but I’ve since reverted to GoogleSheets for detailed editing of our overly-long learning outcomes. I’ll switch back to Popplet to think about our design visually – always get new insights that way.

So, despite my best efforts I’m about a week behind. Luckily I saw Gabi’s post encouraging us to continue working at it, even if we’re behind. Just going to finish up Week 3’s assessment plan today. I really want to get caught up so I can get the benefit of a broader (and deeper) look at our learning activities.

Join us…there’s still time!

 

Juggling MOOCs…

I’m learning to curb my inner magpie when it comes to all these juicy, flashy MOOCs and free learning opportunities that flow past my Twitter node or pop up in my RSS collection in Netvibes. I comfort myself by storing the ones I’ve resisted in a growing collection in Diigo (I should retitle it WhenIRetire)

But one that I couldn’t resist was the newly launched (European-developed) Handson MOOC
It has a range of titles:   “Designing ICT-based Learning Activities” or “Design Studio for ICT-based Learning Activities” or “Hands On MOOC pilot 3”.  They can be forgiven the fluctuating course title when you consider the task they have taken on:  facilitating a MOOC with 7 language groups over a period of 5 weeks using a model of design adapted for the needs of higher education.

I can’t give you a link to the MOOC cuz it’s one of the ones that really brings into question the limitation of the term “MOOC” as I’d say this may be a massive course but it’s certainly NOT open and is only “connected” in somewhat superficial ways and the core content is hidden in Canvas or within the “dream projects” platform – Integrated Learning Design Environment

I have experienced some serious discomfort with the amount of personal information they seem determined to collect and I dislike the legalese and sign-offs required to participate. I just get a vivid image of some academics rubbing their hands in glee as they watch waterfalls of data come flowing down the Internet pipeline. So much data, so many possibilities for future publications or edtech start-ups.

However, I’m still hoping to learn more about the European perspectives on online education and to tuck some new strategies for developing meaningful learning activities into my back pocket. So, I guess it’s worth becoming research fodder.

I’m impressed by the Canvas environment – nice clean design and lots of functionality that I could see liking if I were teaching in a “walled garden”. I particularly like the listing of activities which constantly updates to keep me on track and tell me if I’ve been successful at the various learning tasks in the first week (not always that easy to be sure you’ve done it right without a fair bit of clicking and scrolling).

I was determined to earn the first badge – not cuz I need the reward but because I wanted to see how well their badging system works. I like the flow of activities (although I think they got a little carried away on the warm-up activities) and I can see the relevance of each one in terms of having me engage with the concept of learning design, their methodology and to begin developing my new online learning community through connecting with the “dreams” of other participants. (Note: We’re asked to describe a project we want to pursue – seems much bigger than just designing a learning activity as per course title?)

Another “aha” moment for me was when I realized that they have integrated storytelling and scenarios into their design approach. I’m just heading into Week 2 and the focus is on “personas” and scenarios. Looking forward to trying this approach – we’ll see if it really helps me to design better learning activities online.

Now, to try and catch up on some of the readings and to find the time for peer reviews!

 

Permission to fail, to fall, to make mistakes…

What a great start to Unit 3: The World Wide Web – From Concept to Platform to Culture — a rich listing of resources to explore to learn more about the Web and to help us try to answer the Unit questions for ourselves:

1. What is this thing called the World Wide Web? What are the values and ambitions that gave rise to its design?
2. If “the medium is the message,” what is the message of the web?
3. What are some threshold concepts that help us to understand what is meant by “the web”?
4. How is it reframing learning and education?
5. What do we stand to lose or gain in pursuing the possibilities opened up by the web?.

And, I think, for the first time in this MOOC, I’m going to post one of the suggested Maker activities -they call them Nuggets. What a great way to experience what we ask our students to do in online learning – to explore concepts by applying them in some way that helps them make meaning. One of the central ideas that resonated for me in the Oct 13 Hangout “Web Imperatives” with the Four+ Web Makers (Laura Hilliger, Chris Mattia, Howard Rheingold, Gardner Campbell, and Kim Jaxon) was the discussion around Howard’s statement “If you’re not falling off it, you’re not really exploring the edge” It sparked several threads of related ideas:

• how to help people see that technology is not a barrier and can be mastered by taking small steps
• that , if people take on a leadership role in learning about the Web (or teaching others about the Web) we can ensure that the Web represents what WE want, not what government or industry wants
• how to convince people that exploring the edge is worth it and that if you’re not falling (failing) you’re not trying hard enough
• that failure needs to be accepted as part of learning
• that we need to model how to respond to failure and support people as they begin to explore creating the Web they want

Here’s the link to my Prezi with some sketches and quotations and questions that represents my exploration – I still want to process some more before I explore some of the dissonance I felt when I really thought about the enthusiastic and positive statements of the group.

Exploring the Edge

Sylvia

rootless but connected…

In catch-up mode again…

Oct 10th was the final webinar for

9/29-10/12 Unit 2: Trust and Network Fluency (Leveraging your “Why”)

of Connected Courses. I’d managed to fall behind on my reading/viewing so I wasn’t ready with good questions but I was able to listen to about half of the session before I got called away. I’ll be reviewing it over the Thanksgiving (Canada) weekend! The fascinating discussion was between (among?) –

• Kira Baker-Doyle, assistant professor of education at Arcadia University and author of “The Networked Teacher”;
• Meredith Broussard, assistant professor of journalism and Temple Univeristy in Philadelphia;
• Ulrich Boser, senior fellow at the Center for American Progress and author of “The Leap: The Science of Trust and Why It Matters” and “The Gardner Heist: The True Story of the World’s Largest Unsolved Art Theft”;
• Nishant Shah,  founder and director of research for the Bangalore-based Centre for Internet and Society; and
• Jonathan Worth, award-winning photographer, Coventry University professor and leading teacher of open-learning photography courses, including Phonar Nation.

I enjoyed the examples (ideas) presented as to how to explain to students the benefits and risks of participating in an open, connected learning environment. I had discussions with concerned teachers at a spring conference in Kamloops (CNIE2014) who wondered whether their students were ready to learn in the open. While they could see the value, they were concerned about exposing students to risks that they didn’t feel they could foresee or control. There was a lot of discussion around the need for learning in protected, safe spaces as a way to provide the freedom and confidence to “play” and experiment.

I still don’t fully understand what is meant by “leaky networks” ( a concept enthusiastically presented by Nishant Shah) and unintentional (unpredicted?) learning that occurs because of the exchanges you participate in as you participate in the network?

I’m in a quandary as to whether I should just go through the stuff I’ve missed in Section 2 (which I’m really interested in) or start out with a renewed focus and commitment to Section 3 (which I’m also interested in). I am taking this course for my own knowledge and learning and I don’t like leaving interesting resources unread or unviewed. On the other hand, part of learning in a connected environment is keeping up with the flow, the interaction between all these fascinating new people in my learning network. What to do…mmmmm?

Note to self:  Find a better way to keep on top of the fragments of connected courses. It took me several tries to find the link to the actual recording of the Google Hangout event from Oct 10. I had it in my Google Calendar before, but it disappeared once the event was over. When I went to the list of readings and resources for the section, it wasn’t embedded there (at least I couldn’t see it). I finally found it by looking in the Calendar. Implication for my development of a connected course – embed a touchstone area where a learner can go to find the important pieces – and include a navigation activity in the first section?

Exposed learning vs learning in safe spaces…

Did you catch the Google Hangout with Michael Wesch, Cathy Davidson and Randy Bass on Monday? The End of Higher Education

The topic was the purpose of higher education and the discussion ranged from shared stories to passionate declarations of how education should be repurposed. They tried to frame the discussion around three questions (Mike Wesch did a great job of keeping it fairly focused)

The discussion focused on turning the traditional approach to course design upside-down. Instead of thinking about what we want to teach and then looking for content to develop a course, perhaps we should begin from WHY? Why should we teach this course? Why should student learn with us?

An interesting shift in perspective for a teacher, potentially valuable to students, but I still question the potential for change in an increasingly outcomes-driven education system. Although dedicated, experienced teachers like these three panelists are open to exploring the “why”, the pressure on schools (and on teachers) from parents, government funders and the business community is to make education more “relevant” and “meaningful” in terms of work, not in terms of citizenship or creating a more enlightened and empowered individual.

Although I found many nuggets that I have squirrelled away for further exploration, contemplation and sharing, some of the most thought-provoking comments for me came from Michael Wesch and Randy Bass. Randy spoke about the need for providing safe spaces through balancing structure and freedom so that students could feel free to take risks and be creative. He described the intimate learning spaces of the atelier-style course he taught “where you can model what it looks like to give and get critique”. He acknowledged that students may not have the social capital or comfort with being critiqued right away and needed a chance to see how it worked, practice it in a protected way before venturing out into the “connected classroom” environment where they might face “criticism from all directions”.

That really resonated with me as I’ve found that many of the teachers I’ve taught have felt very exposed and vulnerable if they thought that what they said or did could be viewed and perhaps misconstrued from people who didn’t understand the context. A valid concern and one that is rarely addressed in the open education and open educational practices discussions I’ve been part of.

Michael Wesch embraced the power of openness and talked about the value of creating creative open learning places where the instructor could facilitate student creativity by “putting yourself out there”. He cited ds106 as an example and encouraged people to find “…your authentic self and take a few chances so other people can take chances”. He pointed out that a lot of learning in an open course can take place on the fringes, where we, as instructors, may not be able to see or monitor what is going on. But learning happens when people form new connections, whether orchestrated by a curriculum and a teacher or more naturally as people play and explore new ideas. I like the sound of that although I still have concerns about how that works in a outcomes-driven institution where students are trying to earn credits that will be recognized by potential employers.

I’ll end with another quote from Michael that might resonate with you too “the most beautiful aspects (of learning) can be when the center loses its hold…”

Now back to contemplating “why I teach” – hard to narrow down to a statement

Yes! I’ve done it again…

I keep hearing Grace Jones singing in my mind….”Yes, I’ve done it again, done it again, done it again…”  I couldn’t resist signing up for the most open course ever…”Connected Courses: Active Co-learning in Higher Education” sponsored by DML Hub and the opening salvos are from Jim Groom (Click), Alan Levine (Link) and Howard Rheingold (Embed)!

Click = Jim Groom

Embed = Howard Rheingold

Link = Alan Levine

Check out the topics and line-up of leaders – some of the edupunks I’ve been following for years – always interesting, often inspiring.

9/2-9/14 Orientation/Registration/ Move In

9/15-9/28 Unit 1: Why We Need a Why

9/29-10/12 Unit 2: Trust and Network Fluency (Leveraging your “Why”)

10/13-10/26 Unit 3: The World Wide Web – From Concept to Platform to Cultures

10/27-11/9 Unit 4: Diversity, Equity, Access

11/10-11/23 Unit 5: About Co-learning

12/1-12/14 Unit 6: Putting it all into practice. Planning the connected course

Way, way beyond OCW model…

I remember how thrilled I was to hear MIT’s announcement in 2001 regarding their OpenCourseWare initiative and their commitment to make nearly all course materials freely available on the Internet. I loved the idea of sharing and openness and thought that this might be a way to offer a “hand-up” not a “handout” to anyone unable to afford to participate in education. I recognized that access would still be an issue as not everyone can afford a computer and connectivity but I believed in the power of networking and hoped that those who didn’t have technology could find helpful people who did.

OCW has evolved and spread around the world; although the definition has broadened over time, at least 89 opencourseware projects were cited in a 2013 article from the Open Education Database site. In 2013, MIT’s OpenCourseWare blog, Open Matters, reported “Record traffic for OCW” with 22.3 million visits from 11.8 million visitors.

The criticisms regarding the limitations of the original provision of PDF files led to the addition of videos, image banks, audio recordings, etc. The search engines have improved immensely and the quality of the online learning experience expanded with the addition of OCW Scholar and similar initiatives.

A related movement, “open access“, has also been evolving. It’s all about providing access to the world’s scholarly research. After all, since so much of this kind of research is funded by public dollars, shouldn’t the results be accessible by the public? Huge strides have been made in this area and I’d be curious to find out if anyone is doing research on the impact of accessible research?

So what’s the link between OCW and open access?  OCW was one of the first major initiatives aimed at putting knowledge out there on the web in formats that would be accessible to just about everyone. PDF readers were ubiquitous even then and even older computers could access the OCW web sites. Open access sites followed a similar format – making research searchable, presenting it in formats that most people would be able to view, and putting it on an open web site. But that wasn’t enough.

Open Book Publishers in Cambridge is an open access monograph publisher, started by two academics, Dr. Rupert Gatti and Dr. Alessandra Tosi, in 2008.  Open Book Publishers has made an impressive number of academic works accessible to many (all of its books are free to read online and some can be downloaded). They’ve seen a significant growth in readers from India, Nigeria and Ethiopia (Matthews, 2014) and now they’ve started making their open books available to people who don’t have access to the Internet (and can’t read their open books online).

Since May 2013, Open Book Publishers  has made four of its books available to mobile phone readers through a non-profit organization called Worldreader. More than 8,000 people from 113 countries have downloaded and read these books using Worldreader’s app. The publisher is also developing a partnership with Paperight, a South African company that makes online books available through a network of shops that allow people to pay to print/photocopy parts or all of the books (for those that might not have e-readers or phones).

Open Book Publishers may not be purely “open” in their approach to disseminating scholarly works (i.e., a mixture of free and paid access) but they are evolving a sustainable model that combines fund-raising, grants and creative partnerships with small business or non-profits to improve access to high-quality knowledge materials internationally.

Further reading:

Jump, P. (2012, October 11). Unbound possibilities. Times Higher Education. Retrieved from http://www.timeshighereducation.co.uk/news/research/research-intelligence-unbound-possibilities/421432.article

Kwan, A. (2013, August 20). Open Access and Scholarly Monographs in Canada. Publishing @ SFU. Retrieved from http://www.ccsp.sfu.ca/2013/08/open-access-and-scholarly-monographs-in-canada/

Matthews, D. (2014, June 12) Open access publisher brings scholarship to developing world. Times Higher Education. Retrieved from http://www.timeshighereducation.co.uk/news/open-access-publisher-brings-scholarship-to-developing-world/2013833.article

 

Learning more about OERs…

My quest this week has been to find as many “how-to” open educational resources (OERs) as I can that are focused on helping people learn how to find, use, create, repurpose and reshare OERs. I had planned to build my own OER about OERs, based on the Lunch-and-Learn sessions I had done when I worked at the College, but realized that open content has moved on very quickly and expanded exponentially since then so it made more sense to scrape the Net.

The first resource I reviewed in detail, was an OER from “OSTRICH”, a JISC-funded project conducted by the University of Bath in the UK. Their OER   “Opening up the World of Learning” is intended for an academic audience and a face-to-face delivery format. There are four learning objectives and eight activities that are described in sufficient detail to allow for straightforward duplication of delivery OR make it easier to repurpose the approach and to reshare the edited structure. The resource includes: a workshop plan, a poster, a PowerPoint presentation, Getting Started handout, Finding and Evaluating OERs worksheet, Finding Open Content and Attributing Third Party Content handouts. A thorough coverage with a focus (of course) on UK projects and resources. I haven’t checked all the linked resources but I would probably use some elements of this if I were planning a face-to-face delivery.

I decided to refocus on Canadian OER workshops or tutorials and I was surprised at how many Canadian educational institutions participate nowadays (it wasn’t always the case).

Side Learning:  Even after the most cursory Google search (“OER workshop*” and “OER tutorial*”), I found lots to explore. I played around a little with Google filters to narrow things down to a more Canadian perspective and to keep it current. I found some interesting inconsistencies within Google search – if I went to Advanced search from the icon for Options, I couldn’t find a way to set a time frame. I could select four different options other than “any time” for the “Last updated” but not a time span I wanted.
On the other hand, if I looked at the menu options across the top of the search page, I found that I could click on “Any time” and found six other options including a “Custom” option that allowed me to narrow my search more effectively. Weird inconsistency. I’ll have to go back and do a side-by-side comparison as it seems the menu options across the top of the page provided far more granularity than the Advanced search page.

I found an interesting online OER Tutorial created and shared by Tessa Foster and Brad Fougere of Algonquin College that I’ve been working my way through and really like. There is lots of depth and possibilities for side explorations but the main body of the tutorial provides a solid grounding in the history of OER, issues of Canadian copyright and Creative Commons licensing, plus they’ve integrated interesting activities (social bulletin board, search-and-share) and summative unit quizzes in their screencasting tool . The layout is easy to jump around in and easy to read. Love the screencast tutorials for the different repositories. 

I’d encourage you to go and try out either or both – lots to learn and share.

More OER learning next week…Sylvia

Open Online Environments: Focus on P2PU

It’s been a while since I’ve focused on what’s up in open education. My interest was re-awakened through some of the discussions online over the last year or so from ETUG and SCOPE and the BC Campus supported Open Textbook project. Perhaps the government got stimulated by all the interest in MOOCs, or maybe the unproven idea that OERs can be used to reduce costs of developing and offering a greater diversity of educational opportunities?

I was also curious about a recent CIDER session that promised a review of research about open online environments with a focus on what has been learned by the P2PU (Peer-to-Peer University). As per usual, thanks to the efficient folks at CIDER, the recording and slides are available for viewing now! CIDER session recordings

Open Education: New Developments, Needs and Opportunities for Research
June Ahn, Asst Professor, UofMaryland, College Park, College of Information Studies, College of Education

While I love the idea of user-generated and controlled learning (which I’ve experienced in online communities like SCOPE), I was a little suspicious of the structure and intent of P2PU. When I first heard about them I went and signed up and poked into a couple of courses; they all seemed to be yet another iteration of online discussion. I’m happy to engage in that kind of learning, as long as I have some level of knowledge about the subject; otherwise, online discussions can seem like meaningless chatter or continual confusion. I had hoped P2PU would offer something more creative. From the sounds of things I will need to revisit to see what has grown.

Dr. Ahn is part of a group of educational technology researchers who worked closely with P2PU to try and understand what worked for the people who signed up and explored and what didn’t. They were given access to the backend of P2PU and have spent some time and thought in analyzing the data collected. The really cool thing is that they will share their work and the analytics and the data with other researchers – openly.

So, just some highlights of their findings that I found interesting (I’d encourage you to listen to the presentation yourself.

P2PU has been exploring what works in collaborative or cooperative online learning in a number of ways:
– set up a School of Webcraft with Mozilla (series of courses on coding)
– explored different aspects of open badges in terms of credentialing or personal formative feedback for guiding learner efforts
– governance of open education through Google Hangouts to facilitate open discussion of design to help learners design their own experiences
– Mechanical MOOCs – a gentle introduction to Python

While previous research on face-to-face cooperative learning found small groups to be most productive, online learning is a different situation. Dr. Ahn reported that the “sweet spot” was 30 to 40 learners. Due to the high dropout rate, you need critical mass of participants who contributre something to keep other learners engaged.

What does data tell us about how we should design to keep learners engaged? For new members, design was critical. New members wanted clear page prompts, clear instructions and a clear sense of what they should do.
To keep members engaged, page prompts were not as important as interactions. Although Dr. Ahn’s research did not differentiate between interactions between members or interactions between members and the coordinator, people kept coming back if there was something going on each time they returned.

An interesting question for future research: If you interact more with the organizer would your learning outcomes be different? (Dr. Ahn suggested in response to Dr. Anderson’s comment)

My questions are: how often will members return and not see any activity before they give up and drop away? What’s the tolerance for this? And an even bigger question, is there a way that design or technology can support the organizer in keeping activity going? How meaningful does the activity have to be? Gaming research seems to indicate that even simplistic rewards will keep a learner engaged. Wonder how long that effect holds in a non-sexy learning situation?

As I said, it’s worth viewing the recording and perhaps keeping track of Dr. Ahn and his colleagues as they continue researching.

Sylvia