Using an e-portfolio to assess language learning skills development. A case study. T.

MacKinnon Senior tutor, Language Centre, University of Warwick. Our context. The Language Centre delivers an Institution-wide language programme in language study covering a range of languages and levels for Warwick students from any discipline. Language learning skills are particularly important to UK students who need to prove they can work effectively in intercultural contexts and may find their best job opportunities coming from foreign owned companies. Every year around 3,000 students take up the opportunity to add language study to their study profile, some for credit towards their degree (academic programme) and others for pleasure (lifelong language learning programme). This e-portfolio assessment project affected students on the academic programme at levels 5 and above (post A level) involves around 100 students a year and has been in place for 2 years. The students concerned were learning French, Japanese or Chinese. This paper will present the experience and evaluate the success of our implementation of an eportfolio for accreditation. Why implement an e-portfolio? The Language Centre recently embarked on a complete overhaul of assessment processes in order to achieve greater comparability of experience between languages for students and to address a need for greater assessment literacy for teaching. The engagement in this process of standardisation of assessment across languages brought us to a point where summative assessment in all four skill areas is largely carried out at the end of the language course to provide assessment of learning. However, we accept that the nature of language learning is an ongoing process, one in which the learner has to maintain engagement, take risks and strive to constantly improve through trial and error. It was important therefore to include and embed a strong formative element in our courses to support student reflection and self development . There was a need for students to understand and share the strategies that impact upon their performance in their summative assessments. This desire to embed assessment for learning within the teaching of the Centre prompted my research into the use of e-portfolios and a small working group was created to investigate implementation. Framing the e-portfolio assessment. For successful implementation, the purpose behind the use of the e-portfolios must be aligned to our context . According to JISC reports on the use of e-portfolios, we had to work within settings that require and are seeking some curriculum change so that the e-portfolio activities integrate well within the curriculum. In our context, e-portfolios for assessment provided an opportunity: to support and reward student engagement in the language learning process. to collaborate as teaching professionals across language groups and make explicit the skills we need students to develop.

1 T.MacKinnon April 2013

It was clear that the introduction of an e-portfolio could help to ensure that all tutors include opportunities which will stimulate dialogue between learner and teacher in their teaching programmes and could bring benefits to all engaged in teaching and learning. Even if a student does not achieve a first class pass in their chosen language tutors can reward: the steps taken by students to increase understanding of the learning process. the analytical skills acquired in diagnosing a problem area and taking relevant steps to address it. the acquisition of communication skills which are relevant to employment in a modern age.

The e-portfolio could help to establish a platform for future language learning as the student is better informed about the challenges s/he may face. Clearly there were likely to be significant opportunities through this project to foster professional growth if we followed JISC s advice to: Systematically share effective e-portfolio practice within your institution. It was decided that the e-portfolio would contribute 20% of the overall assessed total for the affected modules. We adopted a community of practice approach (Wenger, 2000) with the pilot group of staff and courses and encouraged student input which would feed forward into our teaching and assessment design for the future. After extensive research into assessment criteria, a narrative model was chosen, based upon work by Moon (2006) which would reward essential communication skills similar to those required in the preparation and presentation of a CV for employment (appendix 1). Students would need to establish an appropriate presentation of their language learning journey in order to make explicit how they addressed the challenges of reaching the required level of competence in their course. They would also need to show the ability to reflect on their learning and modify their strategies accordingly. A short document called the rationale document (appendix 2) would allow students to present the summary of their e-portfolio in order to give a coherence to their narrative, allowing assessors to evaluate student analytical skills. Implementing our e-portfolio. The technical solution we chose was an integration of mahara (an open source e-portfolio tool) within our moodle site Languages@Warwick. This is a flexible tool which gives students a private curation and creation area in which to reflect and experiment. Three key functions of mahara support these processes: a collection area: allowing direct export of moodle activity, upload of external resources a page creation tool: widget based display of collected resources, management of views a social networking tool: participation in and creation of social groups to share files and discussions.

In order to provide sufficient technical support for users a range of video tutorials and screenshot documents were created and made available through a help group in mahara itself through self enrolment and via a moodle e-portfolio support course which was added as a course for all students on the relevant languages and levels. The approach here combined the provision of general resources with access to responsive networks for communication in order to encourage the development of the self help skills which were tailored to individual needs. Several face to face dropin sessions were also put in place and advertised through the support course. 2 T.MacKinnon April 2013

Induction of staff into the concept of e-portfolio use was somewhat more challenging. At the start of the project significant discussion was undertaken in order to ensure that we shared a concept of the e-portfolio. The most widely held understanding of a portfolio was that of a collection of best artefacts which evidence a checklist of aptitudes such as an ideal piece of written work or oral recording. This is not at all compatible with the marking criteria which had been developed and tended to lead to expectations of the stipulation of the amount of evidence which students should submit. Such an approach would run counter to the original aim of developing student insight into autonomous selection and presentation of their learning journey. Furthermore, the physical portfolio is a static document when complete, it is difficult to share and is structurally rigid. The eportfolio format is more flexible and allows for a wide range of content and navigation. As for familiarisation with the technology, the project group was invited to explore the mahara interface prior to the start of the project. Most did not feel that they had the time to do this at the outset of the project. A shared group in mahara called the Assessment of e-portfolio group was set up through which all vital documents were shared, forcing staff familiarisation with the interface. This gradually resulted in greater staff familiarity and now, in our second year of the project, staff are quite experienced in the use of mahara. The timeframe for the e-portfolio work was established as: Term 1: staff present the e-portfolio concept to students in week 3, support groups available with student guide documentation and marking criteria. students identify their desired learning outcomes for their language learning course and start to collect artefacts. students are encouraged to play with the mahara technology in order to see what it can do for them, use the journal function to blog as they learn. At the end of term students are invited to share a mahara page with their tutor in order to show that they can use mahara.

Term 2: Students continue to collect and reflect on examples of their work (e.g. interactions through our virtual exchange, assignment submissions, useful websites) face to face meetings arranged to deal with queries and clarify expectations, support group published details of the submission process. Submission of finished page/collection of pages through mahara assignment in moodle course.

Term 3: students focus on summative assessment. staff meet to review and compare final grades for e-portfolios to ensure standardisation.

3 T.MacKinnon April 2013

Outcomes. In the pilot year 100 e-portfolios were submitted for assessment, producing a normal distribution of marks. This year 81 e-portfolios have been submitted by a slightly smaller cohort, they are currently being assessed. For staff, the assessment process of e-portfolios is time consuming and requires a team effort in order to ensure consistency. After the first year we established that each portfolio takes approximately 30 minutes to assess. The term three assessment meeting was a very productive session, informed by the experience of reading the student work. All agreed that the process was useful and contributed to our professional development as well as confirming a shared concept of the skills we value as language educators. A welcome side effect of engaging in this additional technological solution was the increased digital literacy of those staff involved in the pilot project. All have a greater understanding of the affordances of moodle and mahara and the complimentarity of the two virtual spaces. For students, many expressed concerns at the outset of the project. These centred upon whether they would be assessed of the basis of their digital skills and on misunderstandings of the criteria for assessment. These were addressed during the project in face to face meetings. Students were encouraged to express their experiences of using the technology as part of their e-portfolio. On submission we had very positive responses both about the process of completing an e-portfolio and the improved understanding of digital literacies. Here are a few of the many contributions from students who have completed e-portfolios for assessment: This e-portfolio has been an eye opening and inspiring project that gave me creative space to improve my written French This is one of the most enjoyable coursework projects I have completed - Yuan Chen, French 5. I am very pleased with the work I have done over the year and am proud of the progress that this eportfolio displays. I feel that in tracking my progress and in really thinking about how I learn and the best techniques for me I have learnt a lot about myself, which will be invaluable for all of my language learning in the future. - Sarah Copping, French 5. The creation of this e-portfolio is, undeniably, a new experience for me. Having become accustomed over years of schooling to a system in which academic prowess and hard work took precedence over my own personal response to my studies, I find myself unaccustomed to the emphasis on the personal reflection that this page represents. Having learned to use it, however, I find Mahara to be a rewarding endeavour that enriches my learning experience by reminding me of my overall aims as a French student. Sam Carter, French 5. I believe all the skills I have acquired during the course are very valuable for my future and this eportfolio is the perfect opportunity for me to show you how I have further developed those skills and what I have learnt from the module over the past 6 months. Alvin Au, Chinese for Business. I have chose to structure my e-portfolio monthly because I can see what I have done over time, and it also allows me adjust my goals or plansOne challenge I encountered was the frustration associated with learning, however I overcame this by allocating my time effectively in order to maximise my gain from my plan. For example, if I felt that I was not able to focus because of the frustration occurred in reading an article. I would let myself take a break or move on to something 4 T.MacKinnon April 2013

else and then revisit the article later. So, every time I could start learning with a fresh mind. Hao Li, Chinese 5. Conclusions and future plans. The benefits of the e-portfolio project both for students and for staff involved have far outweighed the time involved in the development and implementation. This was a student -focussed initiative which also helped support staff development and increased learning dialogue from student to tutor in a non-threatening way. Staff from the first pilot year remained involved in the second years implementation and form a core of experienced practitioners with a clear understanding of the aims and objectives of this type of assessed activity. It is clear that this format of e-portfolio is suitable for our current target group. It may be possible to also develop something more suitable for lower level language learners in order to support the development of autonomy and independence in language learning. The creation of an e-portfolio has the added advantage of providing explicit examples of successful language learning strategies which can be shared with other learners, providing useful research data for language teaching. Where we have done this we have of course asked for student permission to do so. From a student perspective, it is possible to take the pages created and provide them as a link within a CV to demonstrate ability to work internationally. It is also possible to export your data in HTML or LEAP 2 compliant form. It is also worth considering offering continued mahara access as part of an alumni package. Clearly it is not desirable to implement e-portfolios in multiple subjects at the same time as a widespread panacea. However when applied as a means of addressing a specific curricular need they can be very helpful. Feedback from last years external examiner for French encouraged us to develop a further variant for higher levels which focuses upon intercultural encounters. The success of this development is yet to be evaluated but the same technological framework has been used without problem. Colleagues in Spanish and German are considering joining the project in 2013-14.

5 T.MacKinnon April 2013

Appendices: 1. Assessment criteria. This is the e-portfolio assessment form. It will be completed and returned to you by your tutor when the assessment process is complete. Each aspect assessed carries equal weighting. Audience awareness, focus Appropriateness to task student has clearly selected relevant materials for the purpose of assessment, shows understanding of criteria. Page has a suitable tone, student has articulated and evidenced their language learning persona. Assessment view presents a coherent narrative of the learners journey. good use of mahara, some creativity, enhanced with good use of range of media. Highly authentic, personal, engaging and thought provoking, gives the right amount of detail, assessor can see students personal engagement with language learning process and efforts made to address difficulties. 1 2.1 2.2 3 F little/no evidence of selection, poor understanding of the purpose of page created for assessment, page does not have suitable tone for task, no real insights into students language learning persona. Assessment view does not present a coherent narrative of the learners journey. minimal use of technical possibilities, evidence and presentation do not enhance the narrative, difficult for assessor to follow, seems contrived/ superficial. Not very interesting and has little impact, story rambles and seems too long/too short or disjointed. No evidence of the application of learning strategies little analysis and reflection, mostly descriptive, lacks explicit personal awareness to apply to future language learning.

Effective use of eportfolio assessment view for conveying the learning process

2.1 2.2 3 F

Analysis & Reflection (Why?, So what? Now what?)

extensive reflection and critical analysis, develops a clear personal narrative, shows understanding of experiences and gives plans for future language learning.

2.1 2.2 3 F

6 T.MacKinnon April 2013

2. Rationale document for e-portfolio assessment view. Purpose of document: Students are asked to put together a short document making explicit their rationale for the decisions they have made about their assessment view. This will help the assessors to understand your approach. Furthermore this will ensure that you have made explicit the reason/s for including your chosen content. Nature of document. The document should be no longer than 300 words and should be produced in English. It must be your own work. It must be submitted in text form. (e.g. word doc/.pdf/journal entry) What the document should include: The document is to provide an overview of the e-portfolio assessment view that you are submitting. It should include: how the page meets the assessment criteria. important decisions made about selection of material for inclusion. a reflection upon the process of language learning from an individual perspective.

Timeframe: Students may wish to get started on this document in draft form during the second term. There should not be input from your language tutor but you may discuss with each other. Review the document and include it in the e-portfolio assessment view to ensure that it is a coherent summary, when you are ready you will then upload your e-portfolio assessment view in your moodle course as an assignment.

7 T.MacKinnon April 2013

References: Gray, L. Whats so difficult about implementing e-portfolios? A threshold concepts perspective arising from research into JISC projects. JISC. JISC e-Portfolio main page, including information on policy context, key resources, JISC projects: www.jisc.ac.uk/eportfolio (last accessed 1/04/13) JISC. Effective Practice with e-Portfolios www.jisc.ac.uk/effectivepracticeeportfolios (last accessed 1/04/13)

Moon, J. 2006. Learning Journals: A handbook for reflective practice and professional development ,
2nd edition, Routledge,. Ministre de lEducation de Nouvelle-Zlande. Digital Portfolio - Guidelines for beginners - Feb 2011 downloaded http://www.minedu.govt.nz/~/media/MinEdu/Files/EducationSectors/PrimarySecondary/Initiatives/ ITAdminSystems/DigitalPortfoliosGuidelinesforbeginners.pdf (last accessed 1/04/13)
Wenger, E. 2000, 'Community of practice and social learning systems', Organization, 7 (2), 225246

8 T.MacKinnon April 2013