AfL - A Practical Guide
AfL - A Practical Guide
It was not a particularly memorable day as days go, but the impact of our decision that afternoon had far-reaching and hugely positive consequences. It was the day in 1997 that my colleagues and I in the Assessment Reform Group (ARG) made the decision to commission Paul Black and Dylan Wiliam to review existing research on formative assessment. We had been trying for almost 10 years to stem the growing tendency across the UK to use external tests for creating school league tables, target setting and appraising teachers, all of which more or less ignored the huge potential of classroom assessment to support learning. However, the accountability juggernaut was not for turning, even though the Blair dynastys clarion call of Education! Education! Education! had not yet begun to ring hollow. It was the day we decided to up the ante and make it our target to resurrect assessment by teachers, not to abandon testing, but to refocus assessment activities where they should be, on pupil learning. The Kings College pair had been long-term advocates of formative assessment and among the best researchers in the international assessment eld. We knew the Nufeld-funded commission would be in good hands, but the 10 years since have demonstrated that their work was to spark a sea-change in thinking about assessment. Two publications did it: the scholarly review itself all 70+ pages of it and a little pamphlet: Inside the Black Box. With these Paul and Dylan lit the blue touch paper and we followed suit by stoking the re. We set out the key features of using assessment to support learning and promoted the concept and processes of Assessment for Learning (AfL), as it became widely known. The power of AfL to improve pupil learning and raise standards was plain for all to see, and soon everyone was sitting up and taking notice. And why? The reasons are probably legion, but at the heart of them is the fact that teachers instinctively knew that assessment should not be a separate process that is simply done to students. It should be fundamentally integrated with learning and teaching. Once teachers reected on the AfL philosophy and tried out the key elements (including sharing learning intentions, effective questioning and peer assessment), they quickly took ownership of it. As David Hargreaves put it in 2005: Assessment for learning is spreading rapidly, in part because teachers nd that it works the scientic evidence and the practice evidence are aligned and mutually supportive. But that is only part of the story. How did teachers nd out about AfL? How did they begin to appreciate that those many, short interactions with pupils in every lesson had so much potential to improve learning? Certainly the work of the ARG helped with research outputs and projects such as Beyond the Black Box, Assessment for Learning: Ten Principles, Testing, Motivation and Learning, and Assessment Systems of the Future. Excellent communicators of the message, such as Ruth Sutton, Shirley Clarke and our own Brian Yeats, Sharon Cousins and Sandra Hayes, among others too numerous to mention, also inspired many teacher gatherings.
Arguably, however, just as this practical guide illustrates, it was agencies such as CCEA and the ve Education and Library Boards (ELBs) that really made it happen through investing time, funds and the expertise of the aforementioned colleagues. Much of the groundwork took place in the early 2000s before the large-scale AfL Action Research Project got underway in 2004, with second and third expansion phases formally lasting until 2007. However, the project very much lives on in the various schools involved and at a systems level in all schools through AfLs comprehensive endorsement in the Northern Ireland Curriculum. In particular, it made substantial contributions to contemporary activities such as Ruth Leitchs project Consulting Pupils on the Assessment of their Learning (CPAL), which was funded by the Economic and Social Research Council (ESRC), and the ARGs Nufeld-funded Analysis and Review of Innovation in Assessment (ARIA). At the centre of all of these activities, one observation is indisputable. Just as it is only learners who can learn (teachers cannot do it for them), it is only teachers who can change their own practices by careful self-reection on what they currently do and how it might be improved. As teachers, we must all be learners too and ensure that AfL does not become a set of teaching tips that ultimately fall into unthinking routines of trafc lights, no hands up, WALT boards and so on. We must also be wary of what Mary James considered was happening in policy-making circles in 2004 in England (not, it should be emphasised, in Northern Ireland, Wales or Scotland!): assessment for learning is becoming a catch-all phrase, used to refer to a range of practices. In some versions it has been turned into a series of ritualized procedures. In others it is taken to be more concerned with monitoring and recordkeeping than with using information to help learning. This is why this resource is so important. The evaluations of the AfL Action Research Project showed that there is still work to be done. We need to keep visible and accessible the processes and benets of assessment that support learning. We need to understand why such assessment is important and have the evidence to back it up. And most importantly, we need the testimony of teachers, illustrated in this handbook, to keep the momentum going. The ARG was rst constituted in 1989 as across the UK it began to become obvious that the multi-layered national curriculum assessments were disrupting learning and teaching (the oldies will remember the hundreds of statements of attainment and their umbrella attainment targets!). I joined in 1994 only as the teacher voice against the overuse and misuse of external key stage testing was beginning to be heard (for example their dubious linking to the accountability of schools), and we gave vent to that voice also. Now, 15 years later, as we approach the winding up of ARGs activities in 2010, we have witnessed across the UK the wholesale endorsement of AfL and the growth of assessment by teachers for summative purposes. There is already talk of an ARG Mark II, but whatever our past contribution may have been, or may be in the future, the key development is secured a savvy and dynamic community of assessment practice that is the partnership of teachers and support professionals in Northern Ireland. I hope that this practical guide will prove to be another key element in supporting the partnership and promoting the most effective uses of assessment in Northern Irelands classrooms.
John Gardner
ii II
Contents
Foreword Contents Introduction i iii v
Section
1 2 3 4
Section
Elements of Practice
Focus on Learning Effective Questioning Formative Feedback Scaffolding Reection 19 25 29 33
Section Resources
Resource 1: Overlapping Strategies of AfL and Thinking Skills and Personal Capabilities (TSPC) Resource 2: General Guidelines for a Feedback Policy Resource 3: Outline Structure for Parents Advice Literature Resource 4: Sample Parent Leaet Resource 5: Raising Awareness PowerPoint Resource 6: Consulting Pupils on the Assessment of their Learning (CPAL) Resource 7: Assessment for Learning (AfL) Overview Poster Resource 8: The 4Rs Posters Resource 9: Learning to Learn Wall (Foundation Stage)
These resources are available for download from www.nicurriculum.org.uk under Assessment within both Key Stages 1&2 and Key Stage 3.
59 60 61 63 65 67 68 69 74
Acknowledgements
iii III
Introduction
Assessment which is explicitly designed to promote learning is the simple, most powerful tool we have for both raising standards and empowering lifelong learners. (Beyond the Black Box, Assessment Reform Group, 1999) In 2004, a major Assessment for Learning (AfL) project titled the AfL Action Research Project began in 25 primary schools. Supported by CCEA and the Education and Library Boards (ELBs), over the next three years this extended to 65 primary and post-primary schools. The AfL Action Research Project encouraged teachers to experiment with aspects of AfL methodology by adapting the theory and principles of formative assessment to suit their own teaching context, in particular the research of Dylan Wiliam and Paul Black (Inside the Black Box). The emphasis was on building condence and expertise. Teachers in each ELB had the support of a CASS ofcer and the opportunity to meet and exchange ideas within learning groups.
Project Highlights
The project promoted four key elements of practice: a focus on the learning (clarity about and sharing of learning intentions and what counts as success); effective questioning (significance of questioning to improve learning); formative feedback (feedback that moves learning forward); and scaffolding reflection (developing self-critical, reflective learners).
Eighty percent of teachers involved in the project chose to share learning intentions and success criteria as their starting point. This was a logical rst step; making the learning explicit and important by sharing it with pupils is a prerequisite for the other strategies. Within the rst few months, teachers were describing positive changes: Pupils were more confident and more willing and able to talk about their learning. Teachers felt they were more tuned into pupils learning needs and that they were developing a more sophisticated understanding of learning and teaching based on authentic evidence.
v V
It includes rst-hand accounts and pictorial examples of how teachers are implementing AfL in their classrooms. In addition, the outcomes of education, in terms of skills, dispositions and attitudes to learning, are emphasised throughout. The main messages are as follows: Assessment can be used not just to measure learning, but to promote learning. The principles underpinning AfL are important considerations; its not just about adopting and adapting the practical strategies. Approaches need to be personalised to suit the particular needs of your pupils. There needs to be time and opportunity to observe, reect and discuss in order to rene your ideas, beliefs and practices. Its important to note that the four key AfL elements of practice are interrelated and can be delivered through different classroom approaches. Investing time in reecting on and developing your knowledge and beliefs about learning will inuence this practice and ensure that the strategies do not become mechanistic. Also, we suggest that you continue to adapt the strategies, contextualising them so that this pedagogy has a positive impact on your pupils and their learning. Like pupils, you need to reect and learn from your experiences in systematic ways to build your knowledge base. You can advance your knowledge and practice by taking small steps and by learning from one another through ongoing whole-school dialogue. To support you in this, Section 1: Whats Behind It? goes beyond the actual elements of practice and describes important considerations that, if embraced, foster the supportive learning environments that the curriculum promotes. Among these considerations is Carol Dwecks work on Mindsets, which is presented in Section 1 to support you as you help nurture pupils independence, condence and positive and robust mindsets for learning. Likewise, the 4Rs Posters (Resource 8) have been designed to promote self-efcacy by making the repertoire of learning skills explicit; the posters emphasise to pupils that success in learning can be achieved through effort. The Assessment for Learning (AfL) Overview Poster (Resource 7) summarises the four key elements of practice and the theory underpinning each element. It is intended to be a useful reminder as you incorporate these into your classroom practice. It could also be shared with parents to help them understand the changing practices. In the context of the Northern Ireland Curriculum, AfL should not be viewed in isolation. Consequently, the table in Resource 1 describes how the four AfL elements of practice and the seven classroom strategies for Thinking Skills and Personal Capabilities overlap. Consideration of these classroom strategies can focus attention on what practical steps you can take throughout your school to ensure that your pupils become successful and condent learners. (More details on Thinking Skills strategies are available on pp.14-16 and pp.27-33 of the Thinking Skills and Personal Capabilities booklet in your schools Curriculum Support and Implementation Box.) In short, this guidance is about effective assessment practice, which is applicable to all age groups and Key Stages. We hope that you will engage with the messages so that AfL strategies become embedded in a meaningful way throughout your school.
The following symbols are used in Sections 2 and 4: Views of Teachers Views of Pupils
vi VI
Constructivism
Learning happens inside learners heads we cant learn for them by some special trick of teaching.
Whats this about?
Learners have to construct their own learning from what teachers give them. We cant put learning in their heads! Constructivism is a view of learning and teaching that has at its core a concept which might seem obvious: learning is something that happens inside the heads of learners. No matter how meticulously we plan or what marvellous strategies we use during teaching, we cant reach inside learners heads and put the learning there. There is a gap between learning and teaching that learners have to negotiate in order to construct new knowledge, skills and attitudes albeit with our skilled input and help. This view strongly underpins what all teachers and school leaders know lies at the heart of good classroom practice: knowledge of individual learners needs and quality teacherpupil relationships. It is via these that the gap between learning and teaching may be more successfully bridged.
Success criteria being negotiated with pupils to enable them to manage their learning activity.
How to go about it
The three components of successful classroom practice that encourage the development of independent learning skills are structure, skills and information. While the desire to promote independent learning is a vital startingpoint, it is these three elements of classroom practice that will help to ensure success.
Structure
Teachers who build structures, skills and information for their pupils help to foster independent learners! Consider how you could use components of classroom design to scaffold pupils self-management. Classrooms with signicant levels of independent learning tend to be much more structured. They have clear procedures for everything, from accessing and returning resources and pursuing learning activities, to using the teachers feedback to make improvements in learning. Structure enables pupils to make good choices, especially when they are not sure what to do next.
Skills
Invest time in helping pupils to develop their independent learning skills. Teach them how to ask and answer questions, how to use success criteria to negotiate learning activities and use feedback effectively, and how to manage the demanding skills of reecting about their own (and others) learning.
Information
Provide sets of information in child-friendly language that enable pupils to access support during their learning activities. This information may be in the form of success criteria or reminders of strategies and procedures, such as questioning and feedback. These play an important role in reducing the pressure on pupils memory to hold such information. This can often make the difference between independence and over-reliance on you or their peers.
2 Assessment for Learning A Practical Guide
Key Concepts
Learning is best served when pupils take responsibility for it and can transfer it across a range of contexts where it is both authentic and valuable. Whats this about?
Apart from the overarching principle of constructivism, key concepts that underpin AfL are transferable learning, transparency and responsibility.
Transferable learning
At the planning stage it is worth considering how transferable learning intentions are. It can be helpful to think of learning intentions that apply to many contexts as more valuable than those which are specic to only one concept. Life skills, such as working together and managing information, are good examples of highly transferable learning intentions.
Transparency
AfL practice involves sharing a great deal of what was once the sole preserve of the teacher. For example, we let pupils in on the secret of success criteria, working with them to create tips on how to be successful in their work. We also show them how to provide feedback that prompts next steps for improvement, both for their peers and for themselves.
Responsibility
Central to AfL is a commitment to enabling pupils to manage their own learning. It invests them with the skills and responsibility they need to make good choices as they negotiate learning challenges. We do not ask pupils to take on aspects of the traditional role of the teacher because it is fashionable, but because it is effective. It helps them raise the quality of their work, and it promotes their learning. One context where responsibility plays a central role is when pupils give feedback on their own and others work using success criteria. It all comes down to pupils knowing what good looks like in any given learning context!
How to go about it
Transferable learning
Ask yourself how much trust you can place in your pupils ability to manage their own learning. At the planning stage, ask yourself three sets of questions: What do my pupils most need to learn next? What is their most pressing need? How can I phrase this as a learning intention? Does this learning intention describe the new learning that pupils will achieve from their learning activity or the activity itself? If the learning intention does describe the learning, can I rephrase it to make it transferable, or is it too context-bound for this? See Focus on Learning in Section 2 for more details.
Transparency
Look for opportunities with the pupils in your class to model your skills of: setting success criteria; providing formative feedback; and asking critical questions and answering them. Then you will be able to take time to teach your pupils these skills and provide low-risk contexts in which they can develop them.
Responsibility
Think about how much you feel able to trust your pupils to manage their own learning. Giving responsibility to pupils to provide feedback to others and self-assess needs to be carefully staged by degrees. It is important to avoid damaging pupils condence and self-esteem to the extent that pupils feel unable to have a go.
Assessment for Learning A Practical Guide
The success and improvement culture of AfL makes a positive contribution to building mastery mindsets.
Pupils need to believe from a very early age that they can develop their intelligence by: making an effort; and using the right techniques or strategies in their learning. Formative feedback focused on success criteria ensures that they know making an effort can lead to success and improvement.
10
Refer to the ideas in the 4Rs Posters (Be Resourceful, Be Resilient, Be Reective and Be Reciprocal) as you consider how best to promote a growth mindset in your context. For example, pupils need to be resourceful and resilient in the face of difculties to avoid displaying helplessness. Strategies like compiling and using a stuck board can be useful. These provide support so that pupils know what to do when they dont know what to do (Piaget). Using others as a resource for learning (being reciprocal), for example by encouraging think-pair-share, talk-partners, peer assessment and group work, supports a growth mindset through sharing ideas and considering alternative ways of approaching learning tasks. Building in time for pupils to be reective through self-assessment and self-evaluation encourages them to know their strengths and to think of different ways to improve their learning. They can acknowledge what worked well for them and identify how they can move forward in their learning.
11
If you dont believe in your capacity to improve, why would you respond to feedback?
12
Consider how you could incorporate the idea of positive role models into your classroom. These can be a powerful inuence on learners. They might be people who have achieved success through a belief in themselves, or they might have used exceptional effort to cope with challenges and barriers in their lives. They could be famous personalities or people you meet every day. Good role models offer inspiration and support by what they say or do.
13
Motivation
Explains why we think and behave the way we do ... it is more important than ability in determining success in life.
Whats this about?
Motivation is generated from the inside while being heavily inuenced from the outside. Alan McLean, 2006 Motivation comes from within. We are motivated by beliefs about ourselves, that is, by how we value ourselves in terms of our selfconcept. This involves both self-worth (Im an OK person) and self-efcacy (I can do this and I can improve). Emotions motivate us, either in a positive, energising way or in a negative, energy draining way. Our emotions appraise situations and inuence how we respond. While we cant motivate others, we can have signicant inuence on how they develop their self-concepts and respond to learning challenges. If we want to nurture a positive emotional response to learning, we must help pupils to understand that their ability is changeable so that they adopt a self-improvement rather than a prove myself attitude. (See A Mindset for Learning in Section 1.)
15
Self-directed learning
Self-directed learning involves: self-management: the ability to be organised, stay focused and control behaviour/actions within different contexts; self-monitoring: the process of reecting, self-reviewing, evaluating and adjusting learning strategies; and self-regulating: the ability to control interest, attitude, behaviour and effort, to persist within a task or in pursuit of a goal. These skills and strategies are important because when pupils believe that they are in control of internal factors, such as effort or motivation, they are less likely to respond negatively to setbacks and challenges. (See the Overlapping Strategies of AfL and Thinking Skills and Personal Capabilities table in Resource 1 as well as Resource 8: 4Rs Posters. See also the self-management section of the From-To Progress Maps, which appears in Appendix 2 (page 35) of the Thinking Skills & Personal Capabilities booklet contained in your schools Curriculum Support and Implementation Box).
Self-directed learners are responsible owners of their own learning process. NWREL, 2004
16
Failure is one of the greatest arts in the world. One fails forward to success. Thomas Edison
17
When I was making a web in my group, I could feel the world of art at my hands. It was like my brain switched to automatic weaving in and out. Year 5 pupil
18
Elements of Practice
Focus on Learning
Focusing on the learning, rather than the activity.
Whats this about?
Learning intentions tell pupils what theyre going to learn. Success criteria tell pupils what to pay most attention to in their activity in order to get the desired learning out of it. Focus on learning involves making two essential elements very clear to pupils: 1. the learning intention: the new learning that pupils will get from the next stage in their learning programme; and 2. success criteria: those parts of the learning activity that are essential (in helping them to achieve the learning intention). These two elements are usually made very salient and visually accessible throughout the lesson or series of lessons. Making the learning focus clear, and scaffolding activity towards that learning, is something that all effective teachers do. However, it is often done orally, and there is evidence to suggest that making these vital elements visible throughout a lesson (or sequence of lessons) helps to focus pupils attention on them more effectively.
Its about making the learning the key focus, instead of over-focusing on making the activity successful in our classrooms.
Share the learning make it explicit. Try to keep learning intentions and success criteria noticeable and accessible in the ways that best suit your context.
19
Remember, you can present learning intentions and success criteria in pictorial form as well as through text, or use a combination of both!
Post-primary teachers often provide copies of subject specications to pupils pursuing GCSEs and other qualications; these, when they are written in pupil-friendly language and presented in a more accessible format, provide clear signposts for clarifying important outcomes and improving pupils performance.
Learning intentions
When planning, take care to ensure that the learning intention tells pupils what they are going to learn. It is not uncommon, however, to nd a description of the learning activity with the words We are learning to added. For example, We are learning to analyse the poem Night Train was a learning intention that a teacher changed on reection to We are learning to understand the power of metaphor. The former was a description of the activity, the latter a description of the new learning focus that the teacher had chosen. Keep in mind the value of transferable learning. Teachers often say that this reduces the amount of re-teaching they have to do, because pupils see the learning, rather than getting caught up in the context of the learning activity. This is not to say that all learning is transferable; there are many things that pupils need to know or be able to do that are highly specic to a single context. However, it is an important issue to consider when planning.
Learning intentions tell pupils what theyre going to learn, not what theyre going to do!
20
Foundation Stage
Learning Intention
We are learning to listen to each other.
Success Criteria
Look at the person who is speaking. Dont do anything else while they are speaking. Think about what they are saying.
In this setting, it can be useful to represent these visually. In the primary classroom, you may wish to develop learning intentions and success criteria twice a week with pupils, or more often if the learning merits it, particularly if pupils are learning something new.
Success Criteria
State the reason for your writing. Express your point-of-view. Use paragraphs. Use facts to back up your argument. Use emotive language.
It is difcult to give such a rule of thumb in post-primary classrooms. There, the frequency of use depends heavily on the nature of the subject and the amount of new learning present at any one time. It is important, however, to be selective. Keep in mind that the most important (or the most difcult) new concepts or skills being taught are the most appropriate ones to publish formally as learning intentions. That way, pupils wont feel swamped by them. The rule, above all, is to make learning both manageable and benecial for learners and teachers. For example:
Post-Primary
Learning Intention
We are learning how to draw a line graph.
Success Criteria
Draw the axes. Decide the biggest amount needed on both axes. Work out the intervals needed on both axes. Plot the points, and join them accurately.
21
Pupils know what they are aiming for and take responsibility for achieving personal targets.
22
Start of Lesson
Recap on prior learning (from previous lesson, or prior experience). Connect it to the new learning intention.
Pupils respond to the improvement prompt, and carry out the improvement.
Feedback is provided on successes and areas for improvement, all referenced against the success criteria.
Lesson Cycle
During the learning activity, keep supportive comments pinned to the success criteria and learning intention.
Set up the learning activity to activate the new learning. Brief the pupils for this as usual.
Just prior to commencing the learning activity, negotiate and write success criteria to scaffold it.
23
Elements of Practice
Effective Questioning
(Dylan Wiliam)
How to go about it
Make questioning more focused
Stop using the rhetorical questions often used to manage behaviour, such as Is everyone sitting down yet? or Has everyone brought their homework? These are frequently used to soften instructions, but they really teach pupils to ignore many of our questions. Deliver courteous instructions instead, such as Sit down now please, everyone.
25
Use no hands up
The wait time is so obvious we just wonder why we rushed children so much to give us an answer. This is a great way to extend participation to the whole class during questioning, rather than having the same few pupils answer most or all of your questions. Pupils do not indicate that they have an answer; instead, you choose somebody to provide one. This prevents the tendency for some pupils to stop thinking when others hands go up! Be selective about when to use no hands up, however, to ensure that it builds on your current practice rather than replacing it.
26
27
Elements of Practice
Formative Feedback
Suggestions for improvement should scaffold rather than tell pupils what to do to improve. They should offer clear guidance on how work can be improved, the next steps in learning and how pupils can take them.
Formative feedback focuses on success and improvement, not just measuring performance.
How to go about it
The vast majority of formative assessment is informal, with interactive and timely feedback and response.
I started quality marking with just one group of children at a time. Im now in the way of writing prompts, and children are getting used to feedback from their talk partner, so it doesnt take so long.
30
Timing is important
Providing only end-point feedback may come too late, leaving little opportunity for the pupil to make improvements. Think about giving feedback while the work is in progress. See Section 1: A Mindset for Learning for information on using praise.
31
Scaffolding Reflection
Elements of Practice
Scaffolding reection is about building in time for pupils to recognise what and how they are learning and to make changes as they go along. Whats this about?
Scaffolding reection is about building in time for pupils to recognise what and how they are learning and to make changes as they go along. Reection helps us recognise what and how we have learned and what we need to focus on in the future. Scaffolding reection in the classroom involves helping pupils to develop and use the skills of: peer and self-assessment reecting on what theyve learned; and self-evaluation reecting on how theyve learned it. This is about valuing and encouraging pupils involvement in making judgements about their own and others learning. The process of and language for reection should be modelled by the teacher, and pupils should come to view it as part of their learning experience. Pupils need to have opportunities to discuss their learning, to share their understanding and to see mistakes as learning opportunities.
33
Engaging in peer and self-assessment is much more than just checking for errors or weaknesses. Black & Wiliam: Working Inside the Black Box, 2002
Through group and peer assessment, pupils become teaching and learning resources for one another.
Good learners are exible. They watch how they are going and change things as they go along. Guy Claxton, 2005
Self-evaluation
Self-evaluation: helps pupils become more aware of and improve their learning strategies; and can lead to improved outcomes due to its emphasis on thinking and talking about success and improvement. Teacher modelling and working with peers help develop the quality of pupils dialogue and ability to talk about their learning. These collaborative processes encourage them to make insightful judgements about their own learning how and why theyve been
34 Assessment for Learning A Practical Guide
How to go about it
You can incorporate reection: at the beginning of learning to connect new learning and to elicit what pupils already know or can already do (using KWL: Know, Want to know, Learned); during learning to ensure that pupils stay focused and make improvements as they go along (ongoing plenaries using self, peer or teacher feedback); and/or after the learning to make pupils aware of successes and areas for improvement and to inform their learning targets (thinking about what and how they have learned, and giving, receiving and responding to feedback).
Asking themselves What did I do well and why? helps pupils understand success and how they can improve.
I found that in plenary sessions, my pupils were able to clarify uncertainties with me. It gave a forum for expressing difculties, and their condence has been enhanced. They know that their understanding is valued.
35
How to go about it
Refer to your Curriculum Support Implementation Box
This box of materials provided to your school in 2007 includes guidance on how to plan for AfL (and all other new areas of the curriculum) using the 4As model: Aspire, Audit, Adapt and Action.
37
38
39
Focus on Learning
Sharing Learning Intentions and Success Criteria with pupils
Learning intentions that describe learning rather than activity are planned. Learning intentions/ success criteria are published, perhaps via WALT or Learning Board, or are written into workbooks. Good learning intentions/success criteria are modelled to pupils. Introduction of learning intentions/success criteria is started with one subject/one teaching group.
Effective Questioning
to deepen learning
The point of questions is considered what do you want to get from each one? The risk of closed questions is lowered. Wait time is pushed to 3-5 seconds. No hands up is tried for some questions.
Response partners are changed frequently anyone can work with anyone else. Pupils are asking questions across the class. Pupils are aware of Juicy Questions those that prompt further questions rather than just providing an answer.
40
Formative Feedback
Comment-only feedback is provided. Feedback systems are explored for preference, for example bubble and box, highlighting, annotation. Increased emphasis is placed on feed forward (prompt for the next improvement) rather than just judging performance.
Scaffolding Reection
Peer and selfassessment/ selfevaluation
Self-assessment is driven by plenary review against success criteria. Self-evaluation is driven by plenary review using big questions. Other structured forms of plenary review are used to prompt pupils reection. Improvements are noted in pupils selfesteem, particularly lower performing pupils. Improvements are noted in pupils condence to attack tasks, particularly low performing pupils. Pupils are far more focused, not only on the task, but the learning intention too.
Self-assessment takes the form of giving feedback to oneself. Pupils are reflecting on how they personally learn best. Individual target-setting occurs over half-term/ term cycle with reviews.
Mastery orientation emerges pupils feel they can do. Performance is enhanced, perhaps most particularly among low-end of performance range, but all perform better against historical baseline.
41
One of the things that makes a big impression on many parents is the range of teachers feedback that they see in their childrens books and homework. If formative, comment only feedback is going to be used effectively, it is important to consider and anticipate parents reactions. It may be well worth the time invested in helping them to understand why comment only feedback is benecial to their childrens learning.
How to go about it
Your school will be aware of the best way to communicate effectively with your parents, whether through newsletters, curriculum workshops, parents meetings, regular informal discussions or during normal consultation meetings with teachers. However, Resource 3 contains a set of questions and answers that you can use to discuss communication strategies. Amend these to suit your own needs and your schools ethos.
43
Impact on Learning
Impact on Learning
Comments from the Northern Ireland AfL Action Research Project
Background
Throughout the AfL Action Research Project, teachers were asked to reect on the impact of their AfL practices. Individual ELB support ofcers led workshops and discussion forums that included a range of activities designed to: encourage the teachers to explore their own thinking; discuss successful strategies developed in other schools; reflect on findings from research; help the teachers plan their own approaches; and enable the teachers evaluate their own progress. Regional celebration and dissemination events provided teachers from Cohorts 13 with the opportunity to share ideas, experiences and practice and to showcase evidence of the impact in their schools. Primary and post-primary teachers shared stories of their experiences implementing AfL, reecting on: what worked for them; the challenges they experienced in developing their own practice; and in what ways the strategies they tried made a positive impact on learning and teaching.
Representatives from the wider educational community also participated in these events to add to discussion, learn from schools experiences and to see examples of AfL in practice (as evidenced in the displays of teachers resources and pupils work).
45
Impact on Learning
Teachers completed questionnaires, articulating their views and ideas and the comments made by pupils. These comments have been structured/clustered under the four elements of AfL practice: Focus on Learning Effective Questioning Formative Feedback Scaffolding Reflection.
Comments
The following is a sample of comments from participants across all Key Stages. They offer advice to teachers and describe the impact the four elements of AfL practice had on: pupils self-esteem; pupils self-confidence; pupils understanding; pupils ability to talk about their learning and how to improve; teacherpupil relationships; how they teach; and revealing new insights about ways to improve their current practice.
Focus on Learning
Views of Teachers
I thought it would be impossible to share learning intentions and success criteria with P1 children, but I used photographs instead of writing and its worked a treat. The visual timetable is very useful for providing the children with a broad outline of the day in advance. It is particularly helpful for those children who fall into the category of the autistic spectrum. Little techniques like using coloured pens to bubble round the learning in their books help the children to focus. Once I encouraged the children to contribute to the success criteria, this made the process more meaningful, as the pupils had ownership of what they were being asked to do.
46
Impact on Learning
Success criteria are compiled as a class using the pupils own language. They see the point of what they are doing, and because they have been developed by them, they focus on them throughout the project.
I decided to use mind mapping as a tool, as I wanted to use a method that would offer some pictorial representation as well as key facts. I needed my GCSE pupils to recognise how the syllabus theory content tted together as part of a jigsaw. I used it as a teaching tool to enable them to learn how to learn as well as for making notes. The method is also useful for assisting memory recall.
47
Impact on Learning
Clear instructions are set out relating to classwork and homework. Pupils play a crucial part in negotiating their own success criteria and they are aware of what they are being marked on. All the work is looked at together and pupils enjoy suggesting how improvements could be made. Teachers in our school have said sharing learning intentions with children has made them reect even more on the learning intentions in their planning. We have learned that children are capable of a greater level of understanding of their learning than we would give credit for, and for a lot of children it has unlocked the mystery that only the teacher knows what they should know! Its important to realise that you cant effectively utilise success criteria and the two stars and a wish approach if you try it with every lesson. Time is too constrained. Concentrate on one lesson a week to start with and do it really well, so that it is meaningful to you and your pupils, before gradually increasing its use.
I loved the idea of sharing the secret with the children and telling them what they are going to learn so that they can succeed, rather than vaguely expressing it and assuming and hoping that they can do the work. Ask one pupil to write up the learning intention this helps the class to settle and to focus on learning at the start of each lesson. This has worked particularly well with a difcult post-primary class, who were always very easily distracted. Get pupils to come up with the success criteria when they do this, they automatically break up the task and visualise it. In my SEN class, I have introduced a WALT board and used IEPs to produce focused plans. Clear learning intentions are shared at the beginning of the session, and Ive worked to provide clearer success criteria, with some input from the children. Ive displayed visual icons and referred to these during regulated learning. In addition, Ive differentiated activities to match learning intentions to match learning intentions and now plan brain breaks. I also now conduct focused plenaries.
48 Assessment for Learning A Practical Guide
Impact on Learning
Views of Pupils
It points you in the right direction.
The remember board is brilliant. I never get lost in my learning anymore and hardly ever forget to do things most of the time! 10-year-old 10-year-old
Effective Questioning
Views of Teachers
When asking more open-ended questions, we found that fewer questions are better because that allows time to seek better answers, for example: Can you tell me more? Can you think of another reason? Pupils seem more at ease with coming up with answers and explanations to my questions and are taking more time to think things through. They have become more settled, have a more mature approach and will ask good strong questions. When I used no-hands-up, I could actually see the young children trying to think! There was also more of an awareness of body language, eye contact, tone of voice and focus of attention between me and the pupils. All answers are used to develop understanding. The aim is thoughtful improvement, rather than getting it right rst time. When all children are addressed, one childs self-esteem is not affected.
49
Impact on Learning
A change to my questioning habits had an immediate effect on the answers. Wait time and millionaire style questions had a positive effect and included more of the class who would have sat back and not tried to answer. Planning is a key factor in good questioning. The teacher must model effective questioning in order to get effective questioners. Everyone in school is very positive about the strategies adopted for effective questioning. The children like to have time to think, and they also like the opportunity to share their ideas with someone else before sharing them with the whole class. This encourages everyone in class to be involved and boosts condence.
Views of Pupils
Everyone was putting their hands up and I felt left out. I felt I was slow. 9-year-old
We have think time its good if youre stuck. It lets you have peace to think about it. 8-year-old
Its ok if the answer isnt always right. Thats how we learn. 10-year-old
I liked Wait Time because it took the dumbstruck feeling off you when you didnt know the answer. Year 8 pupil
I liked Wait Time because it gave me a chance to talk my answer through instead of making a fool of myself if I got the answer wrong. Year 8 pupil
50
Impact on Learning
Formative Feedback
Views of Teachers
Ive always marked with the principal or parents or the inspectorate in mind now Im marking for the children! I have made an effort to get away from a grade culture and move to a comment type feedback, but I nd that pupils still ask but what grade is that? Gradually the pupils and I are looking more towards how can this be improved? but it is a challenge to change the grade mindset. I have found that formative assessment has helped my pupils to become more condent with what they can do. Knowing that this is the area where I will offer the most support has made it acceptable and comfortable for them to admit areas of weakness/areas they need help with. It has encouraged positive two way communication between me and the pupils in my class. Giving feedback has become more of a discussion between me and the pupils, and we refer back to what is on the board. When the children cant understand a task, it is much easier to help them by focusing them on the board and what Im looking for. They have noticed that even the brightest girl in the class has wishes in her book.
Although I was already writing comments, providing a scaffolding comment made me think carefully about the individual child and his or her specic area for improvement (differentiation). The children need plenty of oral feedback/improvement prompts. Written comments about pupils writing are not always accessible or as effective in Primary 2 and 3. It has made a massive improvement to the weaker children. They dont compare themselves with others anymore by discussing grades; they realise that they can be artists! When an improvement such as greater depth of shading is written as a prompt on one homework, then this improvement is readily transferred to the next piece of art work. They readily make the transfer... I dont think Ill ever go back to the marks ever!
Assessment for Learning A Practical Guide 51
Impact on Learning
By using two stars and a wish, the children are now more interested in reading the comments and understanding how they can make [their work] better. They pay more attention to the comments that they can read, and they like to make the improvements. Almost immediately they liked the no more marks (approach) and the everybody is the same you can improve yourself (ethos). I struggled with writing comments on hundreds of art homeworks every week! I believe in the impact of formative feedback because I have seen the improvement that it has made to the quality of art work in my classroom and to the childrens belief in themselves.
In my SEN class, I give oral feedback while pupils engage in a task. Feedback is instant and accessible, as these children cannot read comments. They assess their own work, with support, using visual success criteria. They are also involved in learning talk, such as What have you done well? One success is circled, and areas for improvement are identied. I provide waiting time for processing, and all achievements are recognised. We recognised the need to educate parents so that [when they see their childs work] they [understand] the focus for the marking. For example, a good idea is to write the learning intention at the top of the piece of work so that the parent can see the focus for the learning/ marking.
Views of Pupils
I like talking about my work with my talk partner. You can share each others ideas. 8-year-old
It really helps me to see what Ive done well and how I can make it better next time. Year 5 pupil
I love hearing what everyone thinks about my work. Year 5 pupil It really helps me see what Ive done well and how I can make it better next time. 9-year-old
52
Impact on Learning
Scaffolding Reection
Views of Teachers
I use plenaries halfway through the lesson. We go back to the success criteria, and my pupils think about how well theyre doing. Gallery sessions in art are really useful, and the children really enjoy and learn from them. I get the children to think about how well theyve worked together, and I use questions like What really made you think? The pupils in my Year 10 class have been engaging in selfassessment [and self-evaluation] by asking: What three things do I really understand about this topic? What am I not sure of? What questions would I need to ask to help me understand it better? This has helped my pupils to focus not only on what they have learned, but also how they are learning. It has also helped me as a teacher to assess my teaching.
Self-evaluation has promoted reection in my classes. I encourage my pupils to think about: What did I find difficult in that class? Do I understand it now? How did I figure it out? Being involved in the process of self-assessment has enabled a shift in responsibility in my classroom from teacher to pupils. They have a better understanding of the intended learning, of what needs to be done to improve their work and why it needs to be improved. This has encouraged a process of ongoing self-assessment, which has led to more detailed, mature end products.
53
Impact on Learning
We were already using a series of faces to evaluate how they felt about activities, so this was something already in place to build upon. My Year 6/7 class can now draw up collaborative success criteria and use these to assess each others work using the two stars and a wish approach. This peer-evaluation has worked very successfully in Literacy. The next challenge will be to develop their skills of selfassessment and self-evaluation. The children are more involved in taking responsibility for their own learning through self-assessment aids such as trafc lights and through critiquing each others work with response partners.
Views of Pupils
I think using the coloured traffic light cards to let the teacher know if you understand everything is a good idea because if your are embarrassed and dont want to say you are confused, you can just flip your card to red. Year 8 pupil
We use two stars and a wish for our weekly targets. It really helps me think about what Ive enjoyed and what Id like to do better. 10-year-old
We explain to each other how we get an answer. Harrys method is very clever and quick. 11-year-old
54
Impact on Learning
The Final Word from Teachers
The children in our school have discovered that if something makes you think, you are learning something new. Using the AfL strategies has had a denite impact on the childrens motivation, enjoyment and ability to focus on the task. Despite teething problems and the time needed, the benets to pupils far outweigh the difculties. At the start it was difcult to let go and to let the children take control I was afraid that if another teacher walked past and the class was talking, he/she might think that the class was not on task. But letting go can be very interesting. Often the children come up with things I would never have thought of. I feel that I am focusing more on what I want my Year 5 children to learn rather than on what I want them to do. They feel more secure in their interaction with me, and there is a reduced fear of giving the wrong answer. I felt that all the children, especially the less able, were more focused on working out an answer and engaged in active learning. The pace of the lesson was slowed down. There was less quantity but certainly more quality teaching. Overall, I feel the children have become more focused in their learning. I have particularly found this to be the case with those children with special needs. They can refer back to the WALT board to check their work, and this has greatly beneted them. Children dont worry about where they are at if they are aware that they are getting better. As a school, we have learned to accept where we are with our pupils and value the progress that children make related to their own past performance. Assessment for Learning has meant a change in my thinking as a teacher. Initially, I found it a bit daunting, but I have found it has beneted my understanding of creating a positive learning environment where everyone is encouraged to have a go. Successes are discussed but also failures. Its been challenging for me to reect on my own teaching and see my faults regarding meeting the needs of the pupils. Often driven by time, I nd we are afraid to take a step back and see how things can be achieved in a different way. Try out one of the strategies for a few lessons, take feedback from your pupils and record your own feelings about what you have tried. Then move on to another strategy. I personally feel that Assessment for Learning really gets to the core of successful learning and teaching, and my teaching has beneted from introducing it. Assessment for Learning has given me the opportunity to try out new ideas and evaluate their effectiveness. It has helped me to refocus my own teaching strategies and so has helped me improve on the quality of learning and teaching within my department.
Assessment for Learning A Practical Guide 55
For me, the highlight of using AfL is that it opens up a new area of worthwhile communication between teacher and child.
The childrens minds are focused on what they are learning, especially during play-based learning, when we pause between activities and ask Can anyone tell me something new theyve learned?
The person who has learned the most is ME! I have been enlightened as to how I can encourage children to take ownership for their own learning.
I have found that leaving out statements such as Arent you clever? and replacing them with You are learning a lot today! has indirectly changed how the children approach learning.
I feel condent that there is a learning culture in my classroom rather than a doing culture. There is a lot less busy work.
Impact on Learning
Final Word from Principals
Building up learning intentions, success criteria and peer-evaluation has been a very gradual process. It takes time to build the procedures and ethos of Assessment for Learning. Allow teachers time to embed each new approach in their classroom practice before introducing anything more. Its easy to fall into the trap of thinking youre doing Assessment for Learning if you do a crash course in it. But taking small steps means you are more likely to be effective in improving the childrens learning. Action research the strategies get the staff together, agree to try out a particular aspect over a certain period of time, carry it out in class, gather evidence to reect on the practice, observe each other and come back at a future meeting to share problems, ideas and good practice, and to agree the next steps forward. Our school organised a celebration day with posters and pupils work. Staff response was very positive. All agreed that they had beneted from the sharing of good practice.
56
57
Resources
Resource 1
Overlapping Strategies of AfL and Thinking Skills and Personal Capabilities (TSPC)
AfL Strategies
Focus on Learning Scaffolding Reection
TSPC Strategies
Making thinking important Making thinking explicit
These classroom strategies require a commitment to giving pupils time to talk about and reect on their thinking and learning. They are about having a classroom environment that promotes thinking and reection, for example through displays such as learning boards or stuck boards. They help pupils to understand the purpose of their learning and know the criteria for its assessment in advance. Reection helps them develop a deeper awareness of themselves as learners.
Scaffolding Reection
These classroom strategies emphasise the importance of providing opportunities for transfer of thinking and learning. They are about pupils being able to make connections, applying what they know and what works for them as they engage in different tasks. Using self-assessment and self-evaluation to develop awareness of their personal strengths helps them become more self-directed. They show greater independence through their ability to monitor their own learning and set goals for improvement.
These classroom strategies are about recognising the social nature of learning ~ that we learn with, from and through interacting with others. Collaborative learning provides opportunities for the sharing of knowledge and skills. Maximising the potential of collaborative learning has implications for classroom layout and the organisation of learning activities. In terms of moving learning forward, peer feedback benets both the giver and the recipient. It also contributes to pupils condence in giving and receiving praise and criticism.
Effective Questioning
These classroom strategies require a focus on the purpose and quality of both questioning and challenge. These are not just about checking knowledge/understanding. They are about encouraging children to think in different ways, for example to elaborate/justify/explain in order to deepen and extend their thinking and learning. It is important to consider why and how we ask questions and to build them into lesson planning. Open-ended questions and challenges give pupils the opportunity to respond creatively and to construct their own meaning rather than being instructed to advance learning.
59
Resources
Resource 2
Purposes
Principles
Good practice
Monitoring
Agreed Procedures
60
Resources
Resource 3
61
Resources
How will it affect pupils future chances of success?
There is much research that indicates several reasons for people not doing as well as they should at college, university or in their employment. These include: low self-esteem and confidence, leading them to avoid challenge; lack of resilience, leading them to give up when they encounter difculty; and lack of ability to assess their own performance accurately, which means they underestimate or overestimate themselves. Compare these with the benets noted above, and you can easily see the potential Assessment for Learning has for inuencing success.
How does this affect pupils performance in external examinations like GCSE?
Assessment for Learning approaches help pupils to be much clearer about what is expected of their performance at given levels, and this helps them to prepare for examinations with greater condence.
How will it inuence the quality of information that I will receive about my childs progress at school?
The Annual Report will provide information that is organised according to Assessment for Learning principles. This simply means that you will be clearly informed about your childs past performance in each area of learning as well as their next steps for improving their future performance.
62
Resource 4
Talk to your child about what they are learning your interest means a lot, even if it doesnt seem like it. Encourage your child to have a go, especially when they are uncertain. Praise and reward effort as much as you would performance or results. Read the comments on your childs work and talk to him/her about how they can use their comments to improve their work. Encourage your child to keep improving. When your child gets stuck, dont give him/ her the answer; instead, encourage them to try different ways to become unstuck. Encourage your child to ask questions.
Resources
63
Resources
64
Many think that assessment is all about tests and exams, grades, right and wrong, and pass and fail and it can be. This type of assessment is usually called summative assessment, as it takes place after a period of learning. Its purpose is to see how well the pupil can perform using their new learning.
In contrast, Assessment for Learning, (sometimes called formative assessment), happens during the period of learning and helps the pupil to know: where they are in their learning; where they need to go next; and how to get there.
Assessment for Learning informs the process of learning as it is happening, rather than simply measuring its outcome afterwards.
In implementing the Northern Ireland Curriculum (2007), teachers across Northern Ireland will make use of Assessment for Learning to raise their pupils achievement and to increase their motivation and condence.
We will: inform pupils about what they are going to learn and why it will be useful to them; share exactly how their work/learning will be assessed; use a comment-only marking strategy from time to time; mark their work by identifying areas of success and highlighting areas for improvement; encourage them to reflect on their own work, as well as the work of other children in their class, which builds and deepens their understanding of quality; use oral questioning in the classroom to stimulate their thinking and deepen their understanding; encourage them to have a go by giving them more time to think of answers to oral questions; value all answers, because making mistakes is part of the normal learning process; and encourage them to be more resourceful and independent in their learning by equipping them with helpful strategies that they can use when they get stuck in their learning.
Resources
Resource 5
Slide 1
Slide 2
Slide 3
65
Resources
Slide 4
Slide 5
Slide 6
Slide 7
66
Resources
Resource 6
67
68
Scaffolding Reflection
Understanding: Pupils develop deeper understanding through using success criteria in giving and receiving feedback.
Collaboration: Through group and peer assessment, pupils become teaching and learning resources for one another.
Self-Knowledge: Pupils develop a deeper awareness of the skills, capabilities and strategies that work best for them.
Clarity: Pupils know what they are meant to be learning. Focus: Learning is made explicit because it is shared in a simple, unambiguous form. Quality: Success criteria help pupils know what good looks like. Targets: Pupils know what they are aiming for and take responsibility for achieving personal targets.
Effective Questioning
Formative Feedback
Dialogue: Thinking and enquiry can be enhanced through pupils asking questions of each other and building on each others ideas.
Response: Pupils are more condent in giving an answer if they are given time and their responses are treated with respect.
Rationale: Feedback is formative when pupils are given time to act on it. Transparency: Pupils know the criteria for feedback as they engage in the learning task. Outcome: Feedback is focused on success and improvement, not just measuring performance. Timing: Feedback can advance learning throughout the task, not just at the end.
Resources
Resource 8
69
Imaginat r u io o y
Us e
70
Be Resourceful
esources R W e s
U
Make informed judgements and decisions.
Reasonin e g s
Work things out step by step.
l el
n
Use what you already know. Seek help when you need it. Keep a note of really good sources of information. Locate and select appropriate sources of information. Notice how others do things and build on their ideas.
Imagine what success in your learning looks like and feels like.
kG s A
s
ns o i t
Use different types of questions: What would happen if ? How is this the same? different? What are the options? Can you explain that in another way? What evidence do you have? What made this successful? Why do you think that?
ood Q ue
ke Links a M
Look for connections between ideas, information and experiences. Make predictions based on what you think, feel or already know. Apply what youve learned and how youve learned in different contexts.
71
72
73
Resource 9
Resources
74
Enjoy learning new things Ask good questions Give reasons for my ideas Work things out step-by-step
Think about what I can do when Im stuck Share my ideas Know what I say and do will affect others Say why when I agree or disagree
Listen carefully
Reective Resilient
Resourceful
Reciprocal
Enjoy challenges
Share my ideas
Listen carefully
Acknowledgements
CCEA would like to thank everyone who assisted in the development of this resource. In particular, we would like to thank: the principals and teachers involved in the Northern Ireland AfL Action Research Project; the CASS officers for their commitment in supporting the schools throughout the action research process; Professor John Gardner, Queens University Belfast, for his advice and guidance in assessment practices; and Dr Ruth Leitch, Queens University Belfast, for her support and collaboration during her study: Consulting Pupils on the Assessment of their Learning (CPAL). AfL ACTION RESEARCH SCHOOLS Acorn Integrated Primary School, Carrickfergus Alexander Dickson Primary School, Ballygowan All Saints Primary School, Ballymena Aquinas Grammar School, Belfast Ballee Primary School, Ballymena Ballinderry Primary School Blytheeld Primary School, Belfast Bunscoil an tSlibhe Dhuibh, Belfast Bunscoil Bheann Mhadagin, Belfast Bunscoil Phobal Feirste, Belfast Castledawson Primary School Cedar Lodge Special School, Belfast Clandeboye Primary School, Bangor Cookstown High School Currie Primary School, Belfast Donegall Road Primary School, Belfast Drumaghlis Primary School, Crossgar Edmund Rice Primary School, Belfast Fane Street Primary School, Belfast Forth River Primary School, Belfast Gaelscoil na bhFl, Belfast Glencraig Primary School, Holywood Greenwood Primary School, Belfast Harberton Special School, Belfast Harding Memorial Primary School, Belfast Hart Memorial Primary School, Portadown Hazelwood Primary School, Belfast Hezlett Primary School, Coleraine Holy Child Primary School, Belfast Holy Rosary Primary School, Belfast Holy Trinity Primary School, Enniskillen Kilkeel Primary School Killowen Primary School, Coleraine Knockbreda Primary School, Belfast La Salle Boys Secondary School, Belfast Loreto Convent Primary School, Omagh Malvern Primary School, Belfast Mercy Primary School, Belfast Millington Primary School, Portadown Newtownbreda High School, Belfast Orangeeld High School, Belfast St Aidans Christian Brothers Primary School, Belfast St Colms High School, Dunmurry St Columbs Primary School, Desertmartin, Magherafelt St Columbas Primary School, Clady, Strabane St Conors Primary School, Omagh St Josephs Primary School, Antrim St Josephs Primary School, Meigh, Newry St Kevins Primary School, Belfast St Malachys Primary School, Camlough, Newry St Marys High School, Lurgan St Patricks Primary School, Aghacommon St Patricks Primary School, Holywood St Patricks Primary School, Mayobridge St Patricks Primary School, Newry St Patricks Grammar School, Downpatrick St Brides Primary School, Belfast St Gerards Education Resource Centre, Belfast St Oliver Plunkett Primary School, Belfast St Peters Primary School, Belfast Saints and Scholars Primary School, Armagh Sacred Heart Primary School, Belfast Star of the Sea Primary School, Belfast Taughmonagh Primary School, Belfast The Armstrong Primary School, Armagh
Bein g
Re s
ful ce r u
Be
g in
flec Re
tive
g ein
Reciproc al
Be
in
nt esilie R g