Draft 1.0 – Feedback to: [email protected].

Livingstone Academies: Problem-Solving and Computational Thinking Rubric

Introduction

● This Problem-Solving and Computational Thinking Rubric has been developed through classroom observations, seeking advice from expert and experienced teachers, and
researching the 21st Century Skills and computational thinking skills domain. This rubric is a first draft effort by a group of educators on articulating learning behaviours and
dispositions of learners in the classroom.

● The learning behaviours and dispositions have not been grouped by age group or phase of education because there is insufficient academic research into this field. This document
should be considered in context, that is, both age and ability appropriate, and should be applied to school curriculum and subject being taught. Teachers are encouraged to adapt
the rubric based on their classroom experiences and share it with the wider community.

● The Problem-Solving and Computational Thinking Rubric is focused on learners and not the concepts underpinning problem solving, critical thinking, computational thinking or other
overlapping thinking skills from other subject disciplines. Instead, these concepts are implicit in the statements. The learning behaviours in the ‘realised’ column may be aspirational
for learners in primary (ages 5 – 11) and early secondary (ages 11-13) education. Teachers should apply professional judgement in how much to share and how to explain new or
complex vocabulary to some learners.

● This Problem-solving and Computational Thinking Rubric can be a valuable tool for teachers when planning and delivering lessons ensuring learners have opportunities to develop
positive attitudes to, for example, complexity or ambiguity. This is because the response of learners to learning opportunities (behaviourism) is dependent upon the activity, and
both the way in which the activities are presented to learners by the teacher (constructivism and constructionism) and how learner processes the newly acquired knowledge
(cognitivism). Therefore, there is a relationship between the ‘approaches’ by teachers and the behaviours exhibited by learners’ and as such ‘approaches’ will feature in the rubric to
provide some context.

● Teachers might also find the Problem-Solving and Computational Thinking Rubric useful in avoiding the use of passive verbs to describe learning outcomes (e.g. be aware of, be
familiar with, know, understand, etc.) because the associated behaviours are often internal or not public, that is, they are not observable and therefore we can never really
understand how the learner processes the newly acquired knowledge.

Created by: Mark Dorling and Thomas Stephens

Tested and edited by: Teachers form Aspirations Academies Trust, form the Digital Schoolhouse satellite schools, Phil Bagge, Andrew Csizmadia, Pete Marshman, Thomas Ng, Shahneila Saeed and Dr. Chris Stephenson
Feedback and review from the wider Initial Teacher Training and Computing Education community.
Copyright 2016 Livingstone Academies part of the Aspirations Academies Trust
 ● This rubric should be regarded as a tool to aid (self, peer and teacher) assessment of learners Problem-Solving in any subject. Problem solving in any subject like any skill requires
repeated exposure to a range of real world problems and how they are studied, solved and represented. Therefore, this should be used periodically in conjunction with a learner
portfolio of work to make an overall judgement on learners Problem-solving and Computational Thinking capabilities.

● Teachers may find that the rubric can be used as an aid during lesson observations alongside an updated lesson observation form (see CAS QuickStart Computing reference below)
containing computational thinking prompts because it can help mentors/coaches to consider if particular learning behaviours are or are not present in learners due to the design
and delivery of teaching materials or learner attitudes and motivations.

Learning Attributes* Hibernating Passive Active Realised

“Confidence in ● Makes no effort to attempt to ● Gathers some of the necessary ● Gathers all necessary information ● Confidently and systematically
understanding problems and grasp any aspects of the problem. information to be able to to understand the problem. breaks down problems and filters
dealing with complexity” ● Avoids problems/challenges that understand the problem. ● Understands all aspects of a out unnecessary information and
have more than one step or part ● Grasps some but not all aspects problem. is able to explain the processes
to solving them. of the problem often making ● Displays independence in breaking involved.
● May unintentionally overly educated guesses. down problems and filtering out ● Effectively organises all parts of a
complicate problems. ● Consistently seeks advice and unnecessary information. problem to develop coherent sub-
reassurance. ● With support, builds solutions in solutions.
● Requires support on how to parts (sub-solutions) to ● Sub-solutions are appropriately
approach each part of the recompose for a final solution. coupled (joined).
problem/challenge. ● Reduces the complexity of the
solution without affecting its
behavior.
“Persistence in working with ● Doesn’t engage with and avoids ● Reluctantly engages with difficult ● Responds positively to difficult ● Relishes difficult problems and
difficult problems” problems that are difficult to deal problems but doesn’t persevere problems and validates outcomes. finds fun in being persistent.
with and hard to solve. for long. ● Displays persistence at times of ● Able to explain how they have
difficulty. overcome difficult problems.
● Demonstrates resilience despite
setbacks.
Created by: Mark Dorling and Thomas Stephens
Tested and edited by: Teachers form Aspirations Academies Trust, form the Digital Schoolhouse satellite schools, Phil Bagge, Andrew Csizmadia, Pete Marshman, Thomas Ng, Shahneila Saeed and Dr. Chris Stephenson
Feedback and review from the wider Initial Teacher Training and Computing Education community.
Copyright 2016 Livingstone Academies part of the Aspirations Academies Trust
“Tolerance for ambiguity” ● Refuses to acknowledge or ● Shows a willingness to ● Shows an ability to tolerate ● Celebrates ambiguity and having
struggles to accept ambiguity in acknowledge and accept some ambiguity in both different interpretations
both problems/solutions. ambiguity exists in both problems/solutions. ● Compares the performance of
● Struggles to get started without a problems/solutions. interpretations that do the same
precise plan. thing.
“Deals with open ended ● Struggles to get started without a ● Does minimum expected but no ● Applies effort to independently ● Consistently displays curiosity to
problems” plan and clear more. explore an appropriate range of exhaustively investigate and
expectations/deliverables. ● Investigates a limited number of problems/solutions. analyse a broad range of
● Follows instructions and only does problems/solutions. appropriate problems/solutions.
what they are told.
“Adapts existing knowledge ● Shows reluctance to or actively ● Struggles to identify patterns that ● Tinkers with solutions to find new ● Independently identifies and acts
or solutions to solve new avoids learning from previous match a problem to a previously uses. on patterns in problems/solutions.
problems” solutions. learned solution. ● With some support identifies ● Independently seeks out pre-
● When directed and with patterns and trends in problems existing solutions (not directly
reassurance, will consider and solutions. within their existing sphere of
adapting pre-existing solutions to ● Chooses pre-existing solutions knowledge or understanding)
solve the current problem. they are already aware of to adapt transferring ideas and/or solutions
and solve the current problem. from one problem context to
another.
“Iteratively develops, tests, ● Struggles to express ideas as a ● Implements a solution using a ● With support designs and models ● Independently designs, models,
and debug solutions” solution. given (completed) design. solutions. tests, debugs and refines solutions
● Looks to submit the first working ● Prototypes solutions quickly and ● Requires occasional support when (using a test plan and data where
solution as the finished product. submits more than one iteration testing and debugging solutions. appropriate).
● Lacks an awareness of the need to of a solution for feedback. ● Uses logical reasoning to predict ● Independently chooses an
test, debug and refine solutions ● Begins to use logical reasoning to outcomes showing an awareness appropriate way to represent
iteratively. predict outcomes. of inputs. solutions.
● Has a preferred way of ● Shows an awareness of the need ● With support chooses an ● Tidies up their solution to increase
representing solutions and often to debug solutions but requires appropriate way to represent the comprehension of how it
chooses this method regardless of constant support and advice solutions. works.
the task. during this process. ● Carefully records the iterative ● Embeds comment and explanation
● When directed will consider process and makes appropriate of how a solution works to
different ways to represent refinements to the solution. improve understandability and
solutions. maintainability.
Created by: Mark Dorling and Thomas Stephens
Tested and edited by: Teachers form Aspirations Academies Trust, form the Digital Schoolhouse satellite schools, Phil Bagge, Andrew Csizmadia, Pete Marshman, Thomas Ng, Shahneila Saeed and Dr. Chris Stephenson
Feedback and review from the wider Initial Teacher Training and Computing Education community.
Copyright 2016 Livingstone Academies part of the Aspirations Academies Trust
“Evaluates and weighs-up ● Doesn’t consider evaluating the ● Shows an awareness of the need ● Independently evaluates the ● Considers when good is good
outcomes carefully” efficacy of a solution. to evaluate solutions against quality of solutions against given enough and uses this to design a
criteria but requires support and criteria. criteria to critically evaluate the
advice during this process. ● Considers if a solution is ‘fit for quality of a solution.
purpose’. ● Measures the efficiency of
● Views the work of others and solutions to improve it.
identifies transferable efficiencies. ● Makes trade-offs between
conflicting demands then makes
refinements and future solutions
based on prioritising.
“Communicates and works ● Has a negative impact on others. ● Does not have a negative impact ● Has a positive impact on the ● Makes consistently positive
with others to achieve a ● Communicates ineffectively on others. group. contributions to the group.
common goal or solution” ● Makes inappropriate ● Passively participates in the ● Communicates effectively with ● Groups working effectively in
contributions to the group. group, making no significant others. parallel on the same
● Uses incorrect subject vocabulary. contribution. ● Groups working (with support) in problem/solution.
● Has no sense of own strengths ● Uses the correct subject parallel on the same ● Balances autonomy and
and weaknesses or those of other. vocabulary. problem/solution. collaboration.
● With prompts can explain how a ● Makes positive contributions to ● Values others learning and
solution works to others. and supports others. teamwork styles to encourage
● Has a sense of own strengths and ● Explains how a solution works to contributions from others.
weaknesses but evidences little others. ● Where appropriate, constructively
strategy to deal with them. ● Understands own strengths and leads others.
weaknesses and solicits help from ● Explains how a solution works with
appropriate others. clarity.
● Uses team to effectively
compensate for own weaknesses
and uses own strengths to support
others.
* Column a used inspiration for, and adapted from, the CTSA Operational definition of Computational Thinking for K12 Education

Created by: Mark Dorling and Thomas Stephens

Tested and edited by: Teachers form Aspirations Academies Trust, form the Digital Schoolhouse satellite schools, Phil Bagge, Andrew Csizmadia, Pete Marshman, Thomas Ng, Shahneila Saeed and Dr. Chris Stephenson
Feedback and review from the wider Initial Teacher Training and Computing Education community.
Copyright 2016 Livingstone Academies part of the Aspirations Academies Trust
Bibliography
BBC Bitesized (2016), Computational Thinking
http://www.bbc.co.uk/education/topics/z7tp34j

Barr, V. & Stephenson, C. (2011), Bringing Computational Thinking to K12

http://www.amanyadav.org/CEP991A/wp-content/uploads/2014/08/Barr_Stephenson_2011.pdf

Bhattacharya, N., (2016), Tinkering (an introduction)

https://vimeo.com/182482442

Brennan, K., & Resnick, M. (2012), New frameworks for studying and assessing the development of computational thinking
http://web.media.mit.edu/~kbrennan/files/Brennan_Resnick_AERA2012_CT.pdf

Computing At School (2014), CAS Barefoot Computational Thinking

http://barefootcas.org.uk/barefoot-primary-computing-resources/concepts/computational-thinking/

Csizmadia, A., Cuzon, P., Dorling, M., Humphreys, S., Ng, T., Selby, C. and Woollard, J. (2015), Computational thinking: A guide for teachers
https://community.computingatschool.org.uk/resources/2324

Computing At School (2015), Lesson observation form with prompts

http://www.quickstartcomputing.org/secondary/section4.html

CSTA (2011), Operational definition of Computational Thinking for K12 Education

https://csta.acm.org/Curriculum/sub/CurrFiles/CompThinkingFlyer.pdf

CSTA Voice (2016), Irene Lee: Reclaiming the Roots of CT, Page 4
https://csta.acm.org/Communications/sub/CSTAVoice_Files/csta_voice_03_2016.pdf

Department for Education (2014), Computing Programmes of Study

https://www.gov.uk/government/publications/national-curriculum-in-england-computing-programmes-of-study
Created by: Mark Dorling and Thomas Stephens
Tested and edited by: Teachers form Aspirations Academies Trust, form the Digital Schoolhouse satellite schools, Phil Bagge, Andrew Csizmadia, Pete Marshman, Thomas Ng, Shahneila Saeed and Dr. Chris Stephenson
Feedback and review from the wider Initial Teacher Training and Computing Education community.
Copyright 2016 Livingstone Academies part of the Aspirations Academies Trust
Dorling, M. & Walker, M. (2014), Computing At School Progression Pathways
https://community.computingatschool.org.uk/resources/1692

Fuller et. Al. (2007), Developing a computer science-specific learning taxonomy

http://dl.acm.org/citation.cfm?id=1345438

Google (2016), Computational Thinking for Educators

https://computationalthinkingcourse.withgoogle.com/unit

Google (2016), Computational Thinking from a Dispositions Perspective

http://googleforeducation.blogspot.co.uk/2016/09/Computational-Thinking.html?m=1

Google (2016), Exploring Computational Thinking

https://www.google.com/edu/resources/programs/exploring-computational-thinking/index.html#!ct-overview

Potter. M, and Kustra (2012), A primer on learning outcomes and the SOLO Taxonmy
http://www1.uwindsor.ca/ctl/system/files/PRIMER-on-Learning-Outcomes.pdf

A focus on writing learning outcomes

https://afocusonlearningoutcomes.wordpress.com/resources/extras/

STEM Learning, Computational Thinking

https://www.stem.org.uk/elibrary/resource/35192

Created by: Mark Dorling and Thomas Stephens

Tested and edited by: Teachers form Aspirations Academies Trust, form the Digital Schoolhouse satellite schools, Phil Bagge, Andrew Csizmadia, Pete Marshman, Thomas Ng, Shahneila Saeed and Dr. Chris Stephenson
Feedback and review from the wider Initial Teacher Training and Computing Education community.
Copyright 2016 Livingstone Academies part of the Aspirations Academies Trust