Assessment for Learning

March 4, 2011

Creative Commons Attribution 2.0 Generic License  by  lloydcrew 

In March 2010 Professor Joseph Madaus from the Neag School of Education and Nicole McClure, a graduate student in our department and then member of our course, shared with us why and how we can implement Universal Design for Instruction in our teaching. About 25 minutes and 30 seconds into our recorded USTREAM session, Professor Madaus tells us that “UDI is the proactive design and use of inclusive instructional strategies that benefit a broad range of learners including students with disabilities.” But what does this look like in actual practice?

This Wednesday we will have the opportunity to learn about Universal Design for Learning from Ira Socol, who blogs at SpeEdChange, where he takes a critical look at our current educational systems and what we can and should do to create learning environments where all students can succeed. We’ll be meeting with Ira Socol on October 26, 2011 at 11:30 a.m. EST in LearnCentral and welcome others to join us. Prior to our session, we’ll submit our questions to Ira based on our readings for this week and our own experiences, interests and realities. During our face-to-face time we’ll take a look at some best practice examples of UDL in college-level courses.

Our materials for this week follow a parallel thread, in that they call into question traditional assessment models, from end-of-semester exams and teacher-driven assessments to the traditional grading structures found in classrooms from kindergarten to graduate school. We also have several examples in which educators have used socially mediated technologies to provide more personalized and meaningful learning, as well as teacher reflections on their efficacy. What do you think about these readings? Do traditional grading structures no longer have a place in our current and future learning environments? If so, what could take its place?

Advertisements

Assessment for Learning

Bloom's Revised Taxonomy

Photo Credit: Bloom’s Revised Taxonomy by dkuropatwa

This week’s materials call into question traditional assessment models, from end-of-semester exams and teacher-driven assessments to the traditional grading structures found in classrooms from kindergarten to graduate school. We also have several examples in which educators have used socially mediated technologies to provide more personalized and meaningful learning, as well as teacher reflections on their efficacy. What do you think about what you’ve read this week? Do traditional grading structures no longer have a place in our current and future learning environments? Is it even possible to move away from what Batson calls our “current legacy structure of testing and evaluation“?

As I’ve noted in a previous post on educational trends:

Well over a half a century ago, education psychologist Benjamin Bloom developed a way to frame learning goals by identifying the key intellectual skills necessary for learning to occur. Still in use today, the 1990’s saw its modification to reflect an emphasis on the active nature of learning.1 What you see above is the latest iteration of this classification system. As you’ll note, the visualization of this learning taxonomy shows the lowest cognitive level occupying the largest amount of real estate with the higher cognitive domains of analyzing, evaluating and creating occupying the smallest amount. Interestingly, Bloom found in the 1950’s that 95% of test questions only asked students to think at the lowest level of learning, information retrieval.2

It appears, however, that for enduring understandings to take place, we need to provide our students the opportunity to engage with concepts in multiple contexts:

Understanding the Concept

This week, as part of the process of brainstorming our individual classroom projects, we’ll consider the why, what and how of the assessment piece of our project. I’d like you to consider the following questions as you develop your learning experience:

  • Firstly, what key skills, understandings, and/or attitudes do you want your students to possess as the result of this project?
  • Secondly, what would serve as evidence of that mastery? This is where the assessment piece comes in. What performance tasks, projects, quizzes, self-assessments, observations or other evidence will show what your students know and are able to do?
  • And lastly, what learning experiences could provide your students with the opportunity to gain that competency? This, then, would be the web 2.0 environment you would use.

These questions reflect the core concepts of a ‘backwards design’ approach to creating effective learning environments as outlined by Grant Wiggins and Jay McTighe in their book, Understanding by Design. In this book, Wiggins and McTighe argue that only by first identifying the goals and assessments (the evidence of learning), and then working ‘backwards’ to design the syllabus and learning activities, can effective learning take place. And as evidenced-based learning outcomes become part of the higher educational landscape, it will be increasingly important for us to consider how, what and why we assess our students, our programs and our peers.

1Bloom’s Taxonomy by Heather Coffey.
2Kinetic connections: Bloom’s taxonomy in action by Bobby Hobgood, Melissa Thibault, and David Walbert.

Assessment

A group roped together climbing up the mountain

Photo Credit: The Climb by Jablan

This week, as part of the process of developing our individual classroom projects, we’ll focus on designing assessments. Grant Wiggins‘ and Jay McTighe‘s book, Understanding by Design (UdD), has had an enormous impact on curriculum development in the U.S. K-12 educational system. I’d like us to consider adopting some key UdD approaches as we work on our projects. Our first order of business, then, is to clarify our project goals.

  • What key skills, understandings, and/or attitudes do you want your students to possess as the result of this project?
  • Secondly, what would serve as evidence of that mastery? This is where the assessment piece comes in. What performance tasks, projects, quizzes, self-assessments, observations or other evidence will show what your students know and are able to do?
  • And lastly, what learning experiences could provide your students with the opportunity to gain that competency? This, then, would be the web 2.0 environment you would use.

As I’ve noted in a previous post:

Grant Wiggins and Jay McTighe (Understanding by Design) hold that the development of goals and assessments (the ‘backwards design’ model) prior to crafting syllabi and classroom activities is essential for effective learning to take place. In addition, they suggest collaborative design, sharing and regular feedback from peers as well as from students, in order to make appropriate adjustments to instruction and curricula. Their work has had a significant impact on K-12 education and as evidenced-based learning outcomes become part of the higher educational landscape, it will be increasingly important for us to consider how, what and why we assess our students, our programs and our peers.

In our required readings we have quite a few examples showing how these socially mediated technologies have been used to support student learning, as well as teacher reflections on their efficacy. What questions come to mind as you look at this week’s materials?

Over the next month I would like us to devote a portion of our class meetings to sharing progress on and questions about the design, implementation and evaluation of our projects so that together, we can reach our goals. We’ll talk in more detail about this on Friday. See you then!

And don’t forget, we will be participating in the Open Video Alliance’s Wireside Chat with Lawrence Lessig this week on the 25th from 6-7:30 p.m.

UCONN’s Institute for Teaching & Learning’s Director of Instructional Design and Development, Desmond McCaffrey has shared with us a chart that matches a variety of testing instruments and methods to Bloom’s original taxonomy so that you can have a sense of the various assessment options available to you. Thanks to Desmond for sharing this with us and to Catherine for passing it along.

Session Five—Assessment

Courtesy of Mark Baston from his Flickr web site
Report Card

Grant Wiggins and Jay McTighe (Understanding by Design) hold that the development of goals and assessments (the ‘backwards design’ model) prior to crafting syllabi and classroom activities is essential for effective learning to take place. In addition, they suggest collaborative design, sharing and regular feedback from peers as well as from students, in order to make appropriate adjustments to instruction and curricula. Their work has had a significant impact on K-12 education and as evidenced-based learning outcomes become part of the higher educational landscape, it will be increasingly important for us to consider how, what and why we assess our students, our programs and our peers. In this session we’ll look at some examples of ePortfolios: a holistic, summative approach exemplified by St. Olaf’s Center for Integrative Studies and a formative, longitudinal approach as reflected in UCONN’s ePortfolio system. We’ll also see what several authors have to say about the impact of digital repositories and just as importantly, how they predict scholarly activity will be assessed in the next decade and beyond. A decade during which information abundance will markedly change scholarly authority, in many ways reflecting an Understanding by Design approach.

Felice

This week’s readings included some very interesting articles. I was struck by Trent Batson’s definition of ePortfolios as a “cultural application,” capable of bridging the gap between informal and formal learning. If I am not mistaken, he considers it to be a sort of online journal in which students may trace and compare their own thoughts over a period a time. This definition, along with the emphasis on the mixed use – leisure and non-leisure – of the ePortfolios, led to the formation of a question in my mind: what exactly differentiates the ePortfolio from, on the one hand, an old-fashioned diary/journal and, on the other, from Facebook? While the latter part of my query remains unanswered (I would assume a certain sense of appropriateness), I understood former when I went to Katie Shaw’s site at St. Olaf’s College: the ability to display online her work on Sequential Art and Literary Narrative – two fields close to my heart – demonstrated to me the potential that the ePortfolio held for the circulation of one’s work, especially amongst those working in the same field. I found the very notion of a Center for Integrative Studies fascinating: if Uconn does not have such a center, I strongly suggest it create one, where students from a wide array of majors would be able to post the results of their studies and efforts.Jensen’s article was also very instructive although he does rely too much on the premise that engaged participation will filter out “crap” and convey a greater richness to online resources. His overview from Web 1.0 to 3.0 was particularly eye-opening, especially his statement that 2.0 is not YET a source of scholarly communication but 3.0 may one day substitute current methods of peer review and tenure. I do wonder about the diffusion of scholarly materials online: at the moment, Googlebooks allows one to read but not download or even print texts. Granted this may be still a relic of Web 1.0 but authors and publishers eventually must overcome their reservations regarding their the publication of online resources according to the criteria of Authority 3.0 that Jensen describes. The eventuality of the disappearance of scholarly publishers in two decades’ time has as its foreshadowing the pre-release of Radiohead’s most recent album ONLY online and for free. Lastly, Jensen’s description of Photosynth could not but call to my mind the futuristic holodecks of Star Trek fame.On the other hand, I must admit that Spurlin’s article left me less than enthused. She honestly admits the impossibility of determining the impact of technology on student learning, given the many variables involved, and she does bring up the extremely valid point of how technology should be employed for the general benefit of our students and how to incorporate technology into classroom. These points, however, are lost in a purely academic question concerning the distinction between assessment (student oriented) and evaluation (faculty oriented). Spurlin complicates matters with a series of odd statements, for instance that assessment is about aggregate information – i.e. “students as a group” – which perplexed me because it implicitly denies the status of group to faculty members. Moreover, she further confounds the reader in my opinion with the statement:“When you assess how well students learn at the course or academic program level, you don’t typically ask about the connection between how the information was taught and how well the students learned ….If faculty find that students have met the [course] outcomes, they don’t ask how the material was taught.” (p.7)

Clearly, if this were true, then there would be no student evaluations at the end of each semester. Lastly, her definition on page 6 of students’ “measurable components” which should help with learning practices and principles – the determination of their backgrounds, their technological skills, their demographics and their learning styles – are for all intents and purposes impossible to measure for any group larger than 30.

 

Fulvio

Just like Felice, I was struck by Trent Batson’s definition of ePortfolios. By saying that an ePortfolio is “not a higher education application” but rather “a cultural application” Batson challenges the classic idea one has of an ePortfolio and imagines a future when people will be using these tools beyond the field of higher education. As a matter of fact, Batson states that “the big market is going to be everyone having an ePortfolio, whether they are in college or not.” Although I agree with the fact that ePortfolios could bridge “the gap between informal learning and formal learning”, I am not sure I fully agree with the way Batson envisions the future of these tools. In fact, even though many people “like to collect things”, I just cannot imagine people using them “in their leisure time,” or “for fun”. By definition, I believe that an ePortfolio should be related to the work or the specialization of a person. If, as Batson seems to suggest, ePortfolios are to become like an online journal (as already pointed out by Felice) we will simply end up having another kind of tool, something that resembles more blogs, Facebook or other similar applications. Regarding Katie Shaw’s website at St. Olaf’s College, I once again agree with Felice, and I must say that I have also found the very notion of a Center for Integrative Studies fascinating, and very useful to allow students “to demonstrate the meaningful relations among the different kinds of work they do.”