A Professional Review of Assessment Practices

October 30, 2014
Professional Development provided for the Diocese of St. Petersburg by Catapult Learning
Bethany Center

Assessment - measurement of progress toward a standard / objective; the art of collecting evidence in order to determine students' depth of knowledge; data and evidence

Formative vs. Summative Assessment
Purpose
-Summative - measure student competency
-Formative - improve instruction and give students feedback

When
-SA - end
-FA - on-going

Teacher use
-SA - grading, promotion, programs
-FA - make changes to instruction

Student use
-SA - gauge progress toward goals (competing with each other)
-FA - self-monitoring

FA adds a purpose "adjusting ongoing teaching and learning to improve students' achievement of intended instructional outcomes" - W. James Popham (2008)

Formative Assessment types:
-lesson question asked before the lesson; essential question before teaching the unit; targeted practice; informal question; essential question asked daily

FA / SA:
-lesson question asked after a lesson, as a journal prompt; homework collected daily; quick quiz before class begins; essential question asked daily

Summative Assessment types:
-Essential Question asked after a unit, Major test, Performance Task, Essay, Project, Oral Report; Big Test (standardized)

When FA is graded for correctness, it becomes summative
*obviously, a difficulty is how do you hold students accountable for it?

FA grades should be for:
-completion / timeliness
-Motivation
-Effort
-Correctness in revising errors
-Metacognitive description of mistakes and relearning

SA grades should be for:
-Correctness
-"The primary purpose of grading is feedback to students to improve performance." -Doug Reeves

Academic behaviors, such as organization, responsibility, neatness, etc. could be evaluated in a different / separate grading category - outside of academic progress
-learning to learn skills
-behaviors that allow for learning - approaches to learning - conduct

It is possible to have a summative assessment transition into a formative one (or at least having an aspect of FA) - metacognitive description of mistakes and relearning

Proficiency towards standards as a year long process - keeping tabs and giving feedback to students and families and allowing for progress up until the end of the year

Measuring Depth of Understanding
Why is deep understanding so important?
-Deep understandings can lead to:
1. greater appreciation of knowledge
2. ability to apply learning
3. foundation for later learning
4. happier, more successful kids

Depth of Knowledge (Norman Webb)
Level 1 - recall or reproduction of process or skill - recollection of a fact, term, principle, or concept or performance of a procedure that can be learned through practice...basic understanding

Level 2 - basic application - a change of context requiring a choice of procedure, a simple decision, a straightforward problem, organization, or interpretation...enough understanding to apply

Level 3 - strategic thinking - both abstract, complex or non-routine contexts as well as a degree of reason, conclusion, argument, decision or planning...deeper understanding allowing flexibility

Level 4 - extended thinking - deep investigation, research, integration of knowledge, multiple conditions, a great deal of insight or problem solving...very deep understanding

Blending Webb's DOK with Bloom in light of Common Core State Standards: http://www.crecnow.info/blendedsolutions/docs/docs/Webbs_Depth_of_Knowledge.pdf

Level 4 DOK takes much time, therefore, it cannot be used very frequently

A difference from Bloom - verb usage does not dictate depth of understanding

Repeating a task tends to lower the DOK level, often all the way to DOK1

Simple strategies to increase complexity: Justify your answer, cite evidence in your summary, explain how an outcome would have changed given a different variable

Quality of Assessment Tasks
Alignment - does task require students to engage with the objective / standard
Complexity - what is the deepest DOK that could be revealed by this assessment
Evidence - must students provide enough work, reasons, interpretation, etc. to give ample evidence of student strengths and needs
Clarity - are directions as clear as possible

Other considerations for quality assessment tasks:
-necessity, tools, time, grading, context, stimuli, rubric

Assessment formats vs. types of learning
-Student Self-Assessment and Performance Tasks elicited the highest evidence of learning among the following: forced choice formats, short response, essay / oral reports, informal observations
-Types of learning included: informational topics, processes, thinking and reasoning, communication

Self-Assessment
Self-assessment is a valid form of assessment which can...
1. fill in gaps in your assessment of what students know and don't know
2. help teacher gauge your own assessments through comparison - what I measured in my students vs. what students thought about their own learning
3. promote metacognition - this generally takes a long time to develop and a great deal of time for students to master

Performance Task
Aligns to standards; engages students at appropriate DOK levels; elicits ample evidence of student learning; provides clear directions, including a rubric defining how work will be evaluated

Components of a performance task: http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/05/TaskItemSpecifications/PerformanceTasks/PerformanceTasksSpecifications.pdf

Feedback
Feedback is one of the most valuable ways to promote student learning

But, feedback must be timely, informative, positive and specific

It should align to the standards and be specific about the progress toward that goal

Feedback about student performance on an assessment should also supply feedback to the teacher

Key strategies for effective feedback:
1. Interpret the evidence - point out both strengths and weaknesses
2. Allow for a 2nd attempt
3. Prescriptive feedback will help dictate what needs to happen in order to improve
4. Use an effective media - written on the assessment or rubric may not always be the best format - and be sure to get students to understand the feedback
5. Learning / re-learning is the responsibility of the student - what does the student need to do?
6. Be clear in your feedback
7. Build trust

Analyzing Data
To do it quickly on HW or other FA:
-Look at an entire class at the same time - spread out all of the papers and look for common errors, one particular item that is indicative of the overall lesson
-Dry erase boards
-Tech based surveys / quizzes - www.surveymonkey.com
-Gallery walk - write work on chart paper - one piece of paper per item / concept
-Class debate
-Comparative presentations
-Table of data
-Signals - thumbs up / down / middle

The word "assess" means "to sit beside" (French and Latin origins)

-presented by: Dennis Desormier, Catapult Learning Consultant

Teaching and Assessing Depth of Understanding in Math

Notes from Grade 3 - 5 Math Teaching and Assessing for Mastery Workshop
Bethany Center, Diocese of St. Petersburg and Catapult Learning
October 16, 2014

Skill Fluency and Deep Understanding - balance between these through coherence (conceptual velcro)

"Understanding is not just about coverage of knowledge...but about 'uncoverage' - being introduced to new ideas and being asked to think more deeply and more carefully about facts, ideas, experiences, and theories previously encountered and learned" (Grant Wiggins).

Approaching 13 x 7 with deep understanding that recognizes place value, multiplication as repeated addition, multiplying 7 x 10, not 7 x 1 in the "tens column"

Multiplying is totaling equal groups, a number's digits represent different place values
-Key steps: multiply via place value

Teach with fidelity for deeper understandings

What does deep understanding really mean?
-ability to justify why a particular mathematical statement is true or where a mathematical rule comes from (from CCSS)

What happens to students without deep understanding?
-they memorize more than they need to
-students can't apply
-"jumping through hoops" to get the answer / receive the grade
-"fed" steps as opposed to discover how to do things themselves
-use approaches in situations where they don't actually apply
-lose sight of the beauty of mathematics

Deep Understanding can lead to:
-appreciation of beauty in the math world
-transfer to applications
-support for later learning

Conceptual velcro - making connections between and among knowledge and skills to deepen understandings
-instruction must support the connection of ideas and rooting knowledge to other pieces of information

Coherence - connect related ideas, connect facts and skills to key concepts, memorize for application

Direct Instruction (I do), Guided Practice (We do), Independent Work (You do)

Mathematical practices - by CCSS math lessons should have as many of the mathematical practices as possible:

1. Make sense of problems and persevere in solving them
2. Reason abstractly and quantitatively
3. Construct viable arguments and critique reasoning of others
4. Model with mathematics
5. Use appropriate tools strategically
6. Attend to precision
7. Look for and make use of structure
8. Look for and express regularity in repeated reasoning

-These Mathematical Practices describe what mathematicians do
-Get students to see themselves as mathematicians
-See ourselves in this way, too
-These MP standards usually "live" together and are intertwined
-Connection to Literacy standards about reading and writing (writers / authors)
-Valuable skills in math and life

Resource for Mathematical Practices Posters:
http://elemmath.jordandistrict.org/mathematical-practices-by-standard/

Take time throughout lessons to allow students to reflect on their learning
-slow down and afford them time to reflect on their understandings / misunderstandings / cloudy spots
-give students time to self-assess

How do you assess depth of understanding?
-it does not need to be a formal test
-Webb's Depth of Knowledge components (DOK)

1. Recollection and reproduction
*recollection of a fact, term, principle, concept OR performance of a routine or procedure that can be learned through practice

2. Basic application of skills / concepts
*Using information or concepts to choose a procedure, make a decision, solve a routine problem, organize and interpret information, or make a simple decision

3. Strategic thinking
*Use deeper understanding and analysis in abstract, complex, non-routine contexts to reason, draw conclusions, develop arguments, make decisions or develop a plan

4. Extended application and thinking
*An investigation or application of deep understanding requiring time to research, integrate knowledge, process multiple conditions, solve problems, draw conclusions

Webb's Depth of Knowledge does not correlate with Bloom's Taxonomy of Learning
-Webb's DOK have "Bloom" verbs in most of the four components of DOK - it is richer, more rigorous
-DOK is more concerned with the challenge behind an assessment, not just the verb being used - i.e. a DOK level 4 can use the verb explain or draw depending on the depth at which you are challenging students to demonstrate their understanding
-In DOK, even those application problems, if repeated with students enough, can become level 1 tasks

When using manipulatives, allow students time to "play" before asking them to use them to learn
-also ensure that policies and routines are clearly established and enforced

Florida has moved from FCAT to AIR testing to address CCSS (DOSP has not yet moved)
-Site for details about the new testing: http://www.fsassessments.org/resources/?section=1-students-and-parents

Visual Thinking Strategies

Visual Thinking Strategies,
the Common Core State Standards,
Critical Viewers 

Gospel Passage

A reading for the Gospel according to Matthew: Matthew 25:38-46(NRSV)

And when was it that we saw you a stranger and welcomed you, or naked and gave you clothing? And when was it that we saw you sick or in prison and visited you?’ And the king will answer them, ‘Truly I tell you, just as you did it to one of the least of these who are members of my family, you did it to me.’ Then he will say to those at his left hand, ‘You that are accursed, depart from me into the eternal fire prepared for the devil and his angels; for I was hungry and you gave me no food, I was thirsty and you gave me nothing to drink, I was a stranger and you did not welcome me, naked and you did not give me clothing, sick and in prison and you did not visit me.’ Then they also will answer, ‘Lord, when was it that we saw you hungry or thirsty or a stranger or naked or sick or in prison, and did not take care of you?’ Then he will answer them, ‘Truly I tell you, just as you did not do it to one of the least of these, you did not do it to me.’ And these will go away into eternal punishment, but the righteous into eternal life.”

Objectives:

I can describe what is Visual Thinking Strategies.

I can summarize how to implement it within my classroom.

I can identify at least one reason to incorporate VTS in my classroom 

Visual Thinking Strategies

Visual Thinking Strategies (VTS) is a student-centered curriculum in which the student examines and discusses works of art, prompted by questions selected to support careful, evidentiary looking.

VTS questions are:

1. What is going on in this picture?

2. What do you see that makes you say that?

3. What more can we find?

https://www.youtube.com/watch?feature=player_embedded&v=aVzcknOWpaE

Visual Thinking Strategies

3 Facilitation Techniques:

Paraphrase comments neutrally

Point at the area being discussed

Linking and framing student comments

Visual Thinking Strategies

Students are asked to:

Look carefully at works of art

Talk about what they observe

Back up their ideas with evidence

Listen to and consider the views of others

Discuss many possible interpretations

Visual Thinking Strategies and the Common Core State Standards

One of the major shifts in the Common Core State Standards for English Language Arts demands that students "produce and use evidence in a text to justify their views" (Santos, Darling-Hammond & Cheuk, 2012).

By using the VTS method on a regular basis, teachers can develop evidence-based reasoning habits in their students.

Visual Thinking Strategies and
CCSS ELA Anchor Standards

CCSS.ELA-Literacy.SL.3.1
Engage effectively in a range of collaborative discussions (one-on-one, in groups, and teacher-led) with diverse partners on grade 3 topics and texts, building on others' ideas and expressing their own clearly.

CCSS.ELA-Literacy.SL.3.1.a
Come to discussions prepared, having read or studied required material; explicitly draw on that preparation and other information known about the topic to explore ideas under discussion.

CCSS.ELA-Literacy.SL.3.1.b
Follow agreed-upon rules for discussions (e.g., gaining the floor in respectful ways, listening to others with care, speaking one at a time about the topics and texts under discussion).

CCSS.ELA-Literacy.SL.3.1.c
Ask questions to check understanding of information presented, stay on topic, and link their comments to the remarks of others.

CCSS.ELA-Literacy.SL.3.1.d
Explain their own ideas and understanding in light of the discussion.

CCSS.ELA-Literacy.SL.3.2
Determine the main ideas and supporting details of a text read aloud or information presented in diverse media and formats, including visually, quantitatively, and orally.

CCSS.ELA-Literacy.SL.3.3
Ask and answer questions about information from a speaker, offering appropriate elaboration and detail.

Visual Thinking Strategies and
English Language Learners

The CCSS guidelines for working with English Language Learners explicitly states that teachers should:

- Provide opportunities for classroom interactions (both listening and speaking) that develop concepts and academic language in the disciplines.

- Provide opportunities for ELLs to build on their strengths, prior experiences, and background knowledge.

Gospel Passage

A reading for the Gospel according to Matthew: Matthew 25:38-46(NRSV)

And when was it that we saw you a stranger and welcomed you, or naked and gave you clothing? And when was it that we saw you sick or in prison and visited you?’ And the king will answer them, ‘Truly I tell you, just as you did it to one of the least of these who are members of my family, you did it to me.’ Then he will say to those at his left hand, ‘You that are accursed, depart from me into the eternal fire prepared for the devil and his angels; for I was hungry and you gave me no food, I was thirsty and you gave me nothing to drink, I was a stranger and you did not welcome me, naked and you did not give me clothing, sick and in prison and you did not visit me.’ Then they also will answer, ‘Lord, when was it that we saw you hungry or thirsty or a stranger or naked or sick or in prison, and did not take care of you?’ Then he will answer them, ‘Truly I tell you, just as you did not do it to one of the least of these, you did not do it to me.’ And these will go away into eternal punishment, but the righteous into eternal life.”

What do you see?