Project Outcomes

We are nearing the end of this project. The project team is currently preparing the final project report for the ALTC, and we have been busy in recent weeks preparing a final version of the Web 2.0 Assessment Resources wiki. This wiki provides a suite of resources that have been derived from our project activities; we hope they will be useful for academics who are interested in learning more about what consitutes good practice in the assessment of students' Web 2.0 activities.

The resources wiki is the main outcome of this project. We have also published a number of papers in academic journals and conference proceedings. In addition, we have supported the lecturers who participated in our pilot projects to reflect on their assessment practices and engage in scholarship of teaching and learning activities. The outcomes from these pilot projects are presented in the case study reports in our resources wiki. Some of the case studies are also currently being written up for journal publication. Three of the case studies will be included in the special issue of the Australasian Journal of Educational Technology, on "Web 2.0 Assessment", which will be published in August 2011.

If you would like to know more about the outcomes from this project, please contact Jenny Waycott or Kathleen Gray: jwaycott @ unimelb.edu.au; kgray @ unimelb.edu.au.

Feedback on the Web 2.0 Assessment Resources Wiki

We have had some encouraging feedback from colleagues in the higher education sector in response to our Web 2.0 Assessment Resources Wiki.

We received one email saying:

I came across Web2AssessmentResources last week and was immediately impressed with the sophisticated use of the wiki especially in relation to navigation, and accessibility and searchability of content.   All significant signs of a well constructed learning design with intentional pedagogical-technology fit.  

And another colleague sent an email saying:

The website is very good in that nice and simple, clear, well structured with useful principles, case studies and resources.  I like the idea of including academic and student quotes!

We are very pleased to have received such positive feedback and hope this means our resources wiki is proving useful to the sector.

We are keen to hear more from others who have read the wiki. We intend to continue editing and improving this resource until July 2011. So if you have any suggestions for improvements please let us know by providing a comment on this blog or on the wiki itself, or emailing me: jwaycott @ unimelb.edu.au.

Web 2.0 Assessment Resources Now Available

A web site of resources for academics using Web 2.0 technologies in assessment activities is now available. These resources have been developed as part of the project “Web 2.0 Authoring Tools in Higher Education: Implications for Assessment and Academic Integrity,” funded by the Australian Learning and Teaching Council.

This is a soft release of the resources, so that interested staff can access them before teaching begins this year. The project will be continuing until July 2011, and the resources will be updated and modified until then. Feedback on the resources is welcome and can be provided by using the discussion tab on the web site, by commenting on this blog (below), or by sending an email to: jwaycott @ unimelb.edu.au

The website is available here:  http://web2assessmentresources.wikispaces.com/Overview

Ascilite 2010

Last week, three members of our project team attended the Ascilite 2010 conference in Sydney. We facilitated a workshop on "Students' use of web 2.0 technologies in higher education: Good practice in assessment and academic integrity". Eight people from a range of disciplines, backgrounds, and nationalities attended the workshop. We had some valuable discussions about the implications of web 2.0 assessment for good academic practice in higher education.

Here are some of the issues that participants raised in response to the question "what would good practice look like when university students are asked to demonstrate their learning using web 2.0 activities?":

• In web 2.0 activities there can be a shift of balance to the students. Students can have ownership and responsibility over their learning and content creation.
• Web 2.0 provides opportunities to develop students' abilities to self-evaluate and peer review. Web 2.0 activities enable the development of generic skills and graduate capabilities.Good practice means ensuring these skills are assessed: creating explicit assessment criteria that emphasise generic skills and graduate capabilities.
• There are good and bad aspects of making students' work publicly available. Students and staff need to be careful not to make personal information public.
• It is possible to create lifetime accounts in web 2.0 formats: e.g., alumni can keep in contact with universities and web 2.0 content can be used as eportfolios, enabling students to demonstrate lifelong learning.
• We need clear ground rules about behaviour in web 2.0 environments, particularly around netiquette. The facilitator/lecturer needs to model good web 2.0 behaviour.
• IP/copyright issues need to be addressed
• Policy is currently lagging behind technology and is not catching up quickly enough. It is difficult for educational developers to advise academics when policy is lagging.Within each university different faculties have different policies, which can be confusing for students who might be taking subjects from different faculties.
  

We also presented a paper at the Ascilite conference that gave a broad overview of our project. The paper emphasised the participatory approach we have taken, drawing on a range of experiences and perspectives to develop our project resources, which will be available soon. A full version of the paper is available in the conference proceedings.
 
It was a very enjoyable conference, with many interesting presentations, including several papers about the use of web 2.0 technologies in higher education.

AJET Special Issue

A call for papers has been released for a special issue of the Australasian Journal of Educational Technology, which will be published in 2011.

The special issue, entitled "Assessing students' Web 2.0 activities in higher education" will bring together papers that identify opportunities, issues, challenges, and examples of good practice in the assessment of students' Web 2.0 activities in higher education settings. Areas of interest include, but are not limited to:

• Rich descriptions of case studies demonstrating how Web 2.0 technologies have been used in assessment activities in university teaching and learning settings.
• Pedagogical rationales for assessing creative, personal, or unique student work in Web 2.0 activities.
• Methods of assessment of students' collaborative writing or social web activities, including metrics, criteria, and rubrics.
• ICT management strategies for formative feedback, peer review processes, and student/staff workload in regard to Web 2.0 assessment activities.
• Policy and practice issues in protecting student (and staff) moral rights, copyright, personal identity, and safety in Web 2.0 activities.
• The student experience of assessment in a social web environment.

For more details about submitting a paper to this special issue, please see the invitation for expressions of interest on the AJET web site.

Assessment 2.0 Resources: What Does the Higher Education Sector Need?

We have been speaking to the lecturers who took part in our pilot projects to find out what resources they think would be useful to support academics who are using Web 2.0 technologies in assessment tasks. We have had a variety of suggestions, including:

• The resources should be short, easy to read, and accessible.
• Vignettes, case studies, and examples showing how other lecturers have used Web 2.0 technologies in assessment would be useful.
• It would be helpful to have a glossary of policy terminology.
• Include a description of the types of Web 2.0 tools available and how they might be used in learning and assessment activities.
• Create a space and support for an online community of academics who have used - or are going to use - Web 2.0 tools in their teaching. This would be a space for lecturers to discuss the issues about assessment and Web 2.0 and support each other when designing and implementing Web 2.0 assignments.
• Include some interactive tools - e.g., creating a rubric, designing assessment criteria.

We are now in the process of sifting through the large volumes of data we have collected from our 17 case studies and staff and student focus groups. These findings will inform and enrich the resources we develop. One of the main deliverables from our project will be a handbook with a framework of good practice principles, case studies, examples, and materials to support staff development and student learning.

One of the challenges we are facing at the moment is how to create a resource that is accessible and easy to read but also does justice to the rich and varied examples of practice we have examined in our case studies. We are aiming, therefore, to create a large resource that can be divided into smaller stand-alone sections.

Does anybody have any other suggestions about what resources would be useful for supporting Australian academics who are using Web 2.0 technologies in their teaching and assessment tasks?

Busy times

We have almost completed data collection for our 17 pilot projects. For each pilot we have been examining how lecturers have designed, implemented and assessed a specific assignment in which students were asked to use social web technologies. During this phase of the project we aimed to: field-test our draft framework of good-practice guidelines, identify what more could be added to these guidelines, and develop rich case studies of assessment practices involving the use of social web technologies.

The pilots covered a range of university disciplines, levels of study, and various social web activities:

Blogging in Criminal Law, Cultural Studies, Cinema Studies, and Media Studies
Social bookmarking in Education
Social networking in Languages
Photo-sharing in Communication Design
Video-sharing in Economics and Business
Virtual worlds in Language Studies
Wiki writing in Accounting, Education, Science, Information Technology, and Languages
Combined Web 2.0 tools in Information Management and Information Technology

For each pilot we met with lecturers several times over the course of the semester. In some cases we also conducted classroom observations and collected assessment artefacts and examples of marked work. In addition, we recently ran focus group sessions with participating lecturers. The focus groups provided an opportunity for lecturers to talk about what they learned from taking part in our field-testing, to reflect on their experiences of assessing students' Web 2.0 authoring this semester, and to provide advice about what they would like to see included in the resources and outcomes from this project. The focus group discussions yielded valuable information about these issues and, overall, we have amassed an abundance of data from the pilot projects. In the next few months we will be closely examining our findings, writing up case study reports of each pilot project, and developing rich evidence-based resources to share with the higher education sector at the end of this year.

During July we also invited students to take part in a series of focus group discussions. We wanted to hear students' perspectives on what they consider to be good practice in the use of social web technologies for assessment activities at university. Students were eager to share their experiences and provided valuable information that sometimes differed substantially from lecturers' perspectives. The discussions were fascinating and we hope to be able to report on the findings later in the year.

If you would like to hear more about the findings from the survey and interviews we conducted last year, the Webinar presentation we gave in May provides an overview. The presentation is available here or on YouTube.

Transforming Assessment Webinar Series

Professor Geoff Crisp and his team at the University of Adelaide are running a series of "webinars" on the topic of transforming assessment through digital technologies. The first webinar, presented by Geoff Crisp, took place last week. An archive of the session is available here. The talk included some interesting examples of student learning activities that involve the use of Web 2.0 tools and virtual worlds.

We will be presenting at the next webinar on Wednesday 26th May (5pm EST). The talk will describe our project activities to date and provide an overview of our findings from surveys and interviews with Australian lecturers who assess students' Web 2.0 authoring. We will also review the draft guidelines for good practice that we have developed and are currently field-testing. Audience members will be encouraged to participate and contribute their experiences. The seminar is hosted by the University of Adelaide, Australia using the Wimba virtual classroom.
To join the webinar go to:
http://192.160.71.198/launcher.cgi?room=transformingassessment (and run the set-up wizard).

Further information on this and future events are available on the Transforming Assessment web site.

Publications and Publicity Update

Recently, members of the project team had a paper accepted in the Australasian Journal of Educational Technology. The paper has now been published and is available online. The paper reviews current literature and published examples of the assessment of Web 2.0 activities in higher education, identifying key challenges for academic assessment. The authors "make recommendations for practice, research and understanding to strengthen educational quality and academic integrity in the use of Web 2.0 authoring forms for assessable student learning."

We have also written an article about the project for the next edition of the newsletter of the Higher Education Research and Development Society of Australasia, HERDSA News. In addition, we will be presenting a workshop on our good practice guidelines at the next HERDSA Conference, which will be held in Melbourne in early July. We hope to follow this up with several further presentations and workshops at Australian conferences later this year.

Web 2.0 and learning management systems

Recently I met with staff from our university's IT Services who are undertaking a review of the university's enterprise systems, including the learning management system. We talked about the value of incorporating Web 2.0 tools within the LMS.

This got me thinking about what the lecturers we've spoken to during this project have had to say about Web 2.0 tools within learning management systems. Those who have used LMS wikis, blogs, etc., have talked about the poor usability of the tools - the word "clunky" comes up time and again. What is perhaps more interesting is that there are some who feel strongly that students would resist using social software that is packaged within an LMS. They believe that branding social technologies with a university logo would make them unappealing to students. There have also been suggestions that it would be inappropriate to contain social software within the virtual walls of the university: that it goes against the ideals of Web 2.0 and open publishing. On the other hand there are clearly administrative advantages to using LMS tools.

The connection between Web 2.0 software and LMS services is clearly of interest to the higher education community at the moment. An online seminar dedicated to the topic will be held in the US in late March. It will be interesting to see how Web 2.0 and LMS tools continue to converge in the near future and what impact this might have on the way Web 2.0 activities are incorporated into university education. Will it make it easier for lecturers to use Web 2.0 tools? Or will the question remain: do Web 2.0 tools really belong in learning management systems?

Pilot Projects

We are currently embarking on the pilot phase of our project, which involves field-testing our draft good practice guidelines on the assessment of student Web 2.0 activities.

There will be 18 pilot projects in total, drawn from different learning and teaching settings in five Australian universities (The University of Melbourne, Monash, RMIT, Swinburne, and Victoria University). The university teaching staff who have agreed to take part in our pilots have expressed great enthusiasm for the project, and we have been impressed with the range of innovative learning activities that our participants have implemented in their courses. The 18 pilot projects come from a wide range of subject areas, and include a variety of Web 2.0 tools and different learning activities. These include: wiki writing in accounting, social networking in languages, vodcasting in business, role-playing blogs in legal studies, reflective blogs in cultural studies, virtual worlds in business studies, virtual worlds in language studies, along with many others.

Each pilot project will focus on a particular assignment in which students are assessed for their Web 2.0 activities. We will be examining the processes lecturers and other teaching staff follow when designing, implementing, marking and reviewing the assignment. The main aim of this stage of the research is to assess the usefulness of our draft guidelines, to determine how the guidelines could be improved, and to provide detailed examples of case studies of the assessment of student Web 2.0 activities in higher education. These examples will be incorporated into the final good practice guidelines document, which will be disseminated to Australian universities at the end of this year.

Publications

An academic paper based on research that preceded this project will be published in a coming issue of the Australasian Journal of Educational Technology. The paper, written by project team members, has been accepted for publication and will be available soon.

Kathleen Gray has written a guest column for the monthly newsletter “Melbourne University Student/Staff E-news” (MUSSE). In this column, Kathleen cautions that while there are many arguments that support the use of Web 2.0 tools or social software in higher education, a “cool and clear” approach is needed to ensure that social web-based learning activities are implemented and assessed appropriately. Guidelines based on evidence and examples of good practice are needed to make this possible. Kathleen argues that while there are individual examples of “innovative and exemplary work” in which academics and students have used Web 2.0 tools in HE, we are yet to see “collective efforts in Australia or internationally to develop and share appropriate practices.” Our project aims to address this gap.

Assessment of Student Web 2.0 Activities

Our research project aims to develop a set of guidelines formalising academic practices, standards and reporting relating to the use of "web 2.0" or social technologies for assessing student learning in higher education.

Web 2.0 forms of authoring or content creation activities include blogging/microblogging, photo and video sharing, podcasting, social bookmarking, social networking, virtual worlds and wiki writing. These activities can be undertaken using well-known tools such as WordPress, Blogger, Twitter, Facebook, Flickr, YouTube, Second Life, Delicious, MediaWiki, and so on.

Students are increasingly being asked to engage in these social web activities as part of their formal education. However, little has been reported about the challenges involved in assessing these activities and the implications there may be for academic integrity and assessment standards and reporting. While there are various guidelines available pointing to good assessment practices in general, web 2.0 activities are sufficiently different from anything that has preceded them that such guides may not cover all of the issues raised by the assessment of student web 2.0 activities (abbreviated in this blog as ASW2A). Our project aims to address this gap by developing a set of guidelines based on academics' experiences in using Web 2.0 tools in a range of teaching and learning settings in the Australian higher education system.

Some existing resources about Web 2.0 and assessment include:

Bobby Elliott's (2008) report "Assessment 2.0: Modernising assessment in the age of Web 2.0"

Franklin and van Harmelen's (2007) JISC report on Web 2.0 in higher education

Boud and associate's (2010) report "Assessment 2020"