Assessment for Learning

March 4, 2011

Creative Commons Attribution 2.0 Generic License  by  lloydcrew 

In March 2010 Professor Joseph Madaus from the Neag School of Education and Nicole McClure, a graduate student in our department and then member of our course, shared with us why and how we can implement Universal Design for Instruction in our teaching. About 25 minutes and 30 seconds into our recorded USTREAM session, Professor Madaus tells us that “UDI is the proactive design and use of inclusive instructional strategies that benefit a broad range of learners including students with disabilities.” But what does this look like in actual practice?

This Wednesday we will have the opportunity to learn about Universal Design for Learning from Ira Socol, who blogs at SpeEdChange, where he takes a critical look at our current educational systems and what we can and should do to create learning environments where all students can succeed. We’ll be meeting with Ira Socol on October 26, 2011 at 11:30 a.m. EST in LearnCentral and welcome others to join us. Prior to our session, we’ll submit our questions to Ira based on our readings for this week and our own experiences, interests and realities. During our face-to-face time we’ll take a look at some best practice examples of UDL in college-level courses.

Our materials for this week follow a parallel thread, in that they call into question traditional assessment models, from end-of-semester exams and teacher-driven assessments to the traditional grading structures found in classrooms from kindergarten to graduate school. We also have several examples in which educators have used socially mediated technologies to provide more personalized and meaningful learning, as well as teacher reflections on their efficacy. What do you think about these readings? Do traditional grading structures no longer have a place in our current and future learning environments? If so, what could take its place?

Advertisements

Assessment for Learning

Bloom's Revised Taxonomy

Photo Credit: Bloom’s Revised Taxonomy by dkuropatwa

This week’s materials call into question traditional assessment models, from end-of-semester exams and teacher-driven assessments to the traditional grading structures found in classrooms from kindergarten to graduate school. We also have several examples in which educators have used socially mediated technologies to provide more personalized and meaningful learning, as well as teacher reflections on their efficacy. What do you think about what you’ve read this week? Do traditional grading structures no longer have a place in our current and future learning environments? Is it even possible to move away from what Batson calls our “current legacy structure of testing and evaluation“?

As I’ve noted in a previous post on educational trends:

Well over a half a century ago, education psychologist Benjamin Bloom developed a way to frame learning goals by identifying the key intellectual skills necessary for learning to occur. Still in use today, the 1990’s saw its modification to reflect an emphasis on the active nature of learning.1 What you see above is the latest iteration of this classification system. As you’ll note, the visualization of this learning taxonomy shows the lowest cognitive level occupying the largest amount of real estate with the higher cognitive domains of analyzing, evaluating and creating occupying the smallest amount. Interestingly, Bloom found in the 1950’s that 95% of test questions only asked students to think at the lowest level of learning, information retrieval.2

It appears, however, that for enduring understandings to take place, we need to provide our students the opportunity to engage with concepts in multiple contexts:

Understanding the Concept

This week, as part of the process of brainstorming our individual classroom projects, we’ll consider the why, what and how of the assessment piece of our project. I’d like you to consider the following questions as you develop your learning experience:

  • Firstly, what key skills, understandings, and/or attitudes do you want your students to possess as the result of this project?
  • Secondly, what would serve as evidence of that mastery? This is where the assessment piece comes in. What performance tasks, projects, quizzes, self-assessments, observations or other evidence will show what your students know and are able to do?
  • And lastly, what learning experiences could provide your students with the opportunity to gain that competency? This, then, would be the web 2.0 environment you would use.

These questions reflect the core concepts of a ‘backwards design’ approach to creating effective learning environments as outlined by Grant Wiggins and Jay McTighe in their book, Understanding by Design. In this book, Wiggins and McTighe argue that only by first identifying the goals and assessments (the evidence of learning), and then working ‘backwards’ to design the syllabus and learning activities, can effective learning take place. And as evidenced-based learning outcomes become part of the higher educational landscape, it will be increasingly important for us to consider how, what and why we assess our students, our programs and our peers.

1Bloom’s Taxonomy by Heather Coffey.
2Kinetic connections: Bloom’s taxonomy in action by Bobby Hobgood, Melissa Thibault, and David Walbert.

Assessment

A group roped together climbing up the mountain

Photo Credit: The Climb by Jablan

This week, as part of the process of developing our individual classroom projects, we’ll focus on designing assessments. Grant Wiggins‘ and Jay McTighe‘s book, Understanding by Design (UdD), has had an enormous impact on curriculum development in the U.S. K-12 educational system. I’d like us to consider adopting some key UdD approaches as we work on our projects. Our first order of business, then, is to clarify our project goals.

  • What key skills, understandings, and/or attitudes do you want your students to possess as the result of this project?
  • Secondly, what would serve as evidence of that mastery? This is where the assessment piece comes in. What performance tasks, projects, quizzes, self-assessments, observations or other evidence will show what your students know and are able to do?
  • And lastly, what learning experiences could provide your students with the opportunity to gain that competency? This, then, would be the web 2.0 environment you would use.

As I’ve noted in a previous post:

Grant Wiggins and Jay McTighe (Understanding by Design) hold that the development of goals and assessments (the ‘backwards design’ model) prior to crafting syllabi and classroom activities is essential for effective learning to take place. In addition, they suggest collaborative design, sharing and regular feedback from peers as well as from students, in order to make appropriate adjustments to instruction and curricula. Their work has had a significant impact on K-12 education and as evidenced-based learning outcomes become part of the higher educational landscape, it will be increasingly important for us to consider how, what and why we assess our students, our programs and our peers.

In our required readings we have quite a few examples showing how these socially mediated technologies have been used to support student learning, as well as teacher reflections on their efficacy. What questions come to mind as you look at this week’s materials?

Over the next month I would like us to devote a portion of our class meetings to sharing progress on and questions about the design, implementation and evaluation of our projects so that together, we can reach our goals. We’ll talk in more detail about this on Friday. See you then!

And don’t forget, we will be participating in the Open Video Alliance’s Wireside Chat with Lawrence Lessig this week on the 25th from 6-7:30 p.m.

UCONN’s Institute for Teaching & Learning’s Director of Instructional Design and Development, Desmond McCaffrey has shared with us a chart that matches a variety of testing instruments and methods to Bloom’s original taxonomy so that you can have a sense of the various assessment options available to you. Thanks to Desmond for sharing this with us and to Catherine for passing it along.

Assessment

Assessment Cycle by Rich James

Assessment Cycle by Rich James

For this week’s discussion I thought it might be interesting to read and react to last year’s comments on these readings, one, as a way to connect across time and space with our former and current colleagues and two, as a way to initiate our discussion of these materials. What do you think about what they had to say and what they focused on?

At the beginning of class this week, Nathalie Ettzevoglou will lead a discussion on integrating social networking environments in undergraduate language and literature courses and share her experiences as both a teacher and learner. I’m looking forward to an informative and insightful exchange!

Session Thirteen—Mashups

Virtual Tour Mashup
A city tour via a mashup that integrates wikipedia, google maps and pictures to create a virtual tour of NYC. Photo courtesy of GISuser (Creative Commons License) from his flickr account.

In our penultimate session we focus on mashups, web applications that are hybrids of two distinct applications not originally intended to work with each other. An example would be the combination of google maps and flickr that enables flickr users to ‘geotag’ their photos. This all became possible when we could separate form from content as Professor Michael Wesch depicts in The Machine is Us/ing Us.

Please remember to read the Power of the Mashup by Suzie Boss and Jane Krauss, which you downloaded in August. (You’ll need to be a current ISTE member to download now.)

Our four articles this week take a look at mashups from multiple angles. Educause’s 7 Things You Should Know About Mapping Mashups provides a solid foundation for our understanding of mashups as well as suggested educational applications. In Dr. Mashup; or Why Educators Should Learn to Stop Worrying and Love the Remix, Brian Lamb artfully weaves together the evolution of mashups with multi-disciplinary perspectives on creativity and ownership. In so doing, he presents a compelling argument for a careful yet open evaluation of content and application remixes and mashups that could support student, teacher and institutional goals. Both Educause’s 7 Things You Should Know About Google Earth and Suzie Boss’ and Jane Krauss’ ISTE article, Power of the Mashup (must be a current ISTE member to download) focus on the integration of Google Earth within a student-centered instructional environment. Be sure to check out Jerome Burg’s Google Lit Trips and David Fagg’s iHistory Podcast Project for excellent resources and archived projects.

This will also be the session in which we will present and discuss our class projects—I look forward to learning from your experiences!

Martina

 The combination of maps and information is tempting as a tool in a foreign language classroom, particularly if you explore communities and countries with your students. I will definitely try it. So far I would like to give a short overview of my classroom project. Actually, I had two technically based projects this semester. First, in my Business German class the students were supposed to use voicethread in order to create and record a 3-5 minute telephone talk in an office. We had discussed these talks in class and read and heard many examples. The studnet worked in pairs of two. This tasks was a substitute for a test. Despite the early dificulties — your have to make sure that the students log in with the e-mail address you assigned them, e.g. the University’s address, otherwise, if they use another address, it is not working — it was very successful. I was amazed how creative and motivated the student’s were and how much fun they had creating different charecters using different voices, intonations, background noises, and expressions. The recording process seemed to be fun, e.g. when one recording was interupted by the dog of a student in the background, or when the topics itself were entertaining, e.g. a stubborn customer or the reclamation of a non working toaster — that was unplugged. I graded them based on creativity, correctness, vocabulary, and fluency.  There were only very successful contributions and we enjoyed the projects.

The other project dealt with collaborative writing in an interdisciplinary German discussion section. This discussion section was linked with the course “Contemporary Germany.” Since I used many visuals in class, e.g. the movie “Good-bye, Lenin”, the assignment was to comment on them. As a wiki I used googledocs and assigned there some questions. The technical difficulties in the beginning were the same as mentioned above: when the students used e-mail addresses other than those I had invited them, they could not sign up or log in. After we discussed this issue in class and after two outgoing students started the online conversation based on the questions I had written, the discussion was very active and fruitful. Students commented in the target language German using the new vocabulary and expressions they had learned in class. The gave insight into new aspects and ideas. I graded them based on the quality and amount of the contributions. This grade was part of the classroom participation and was, in addition, the opportunity for an extra-credit.

In my opinion both course appications were very successful additions to a regular language class since we could cover things we could not cover in the classroom. I will use them in the future, with the only change that I would describe the sign-up process in class and that I would like ot try wetpaint instead of googledocs since I like the idea to assign each student a color to write with and to make more clear who contributed what part of the discussion.

Session Five—Assessment

Courtesy of Mark Baston from his Flickr web site
Report Card

Grant Wiggins and Jay McTighe (Understanding by Design) hold that the development of goals and assessments (the ‘backwards design’ model) prior to crafting syllabi and classroom activities is essential for effective learning to take place. In addition, they suggest collaborative design, sharing and regular feedback from peers as well as from students, in order to make appropriate adjustments to instruction and curricula. Their work has had a significant impact on K-12 education and as evidenced-based learning outcomes become part of the higher educational landscape, it will be increasingly important for us to consider how, what and why we assess our students, our programs and our peers. In this session we’ll look at some examples of ePortfolios: a holistic, summative approach exemplified by St. Olaf’s Center for Integrative Studies and a formative, longitudinal approach as reflected in UCONN’s ePortfolio system. We’ll also see what several authors have to say about the impact of digital repositories and just as importantly, how they predict scholarly activity will be assessed in the next decade and beyond. A decade during which information abundance will markedly change scholarly authority, in many ways reflecting an Understanding by Design approach.

Felice

This week’s readings included some very interesting articles. I was struck by Trent Batson’s definition of ePortfolios as a “cultural application,” capable of bridging the gap between informal and formal learning. If I am not mistaken, he considers it to be a sort of online journal in which students may trace and compare their own thoughts over a period a time. This definition, along with the emphasis on the mixed use – leisure and non-leisure – of the ePortfolios, led to the formation of a question in my mind: what exactly differentiates the ePortfolio from, on the one hand, an old-fashioned diary/journal and, on the other, from Facebook? While the latter part of my query remains unanswered (I would assume a certain sense of appropriateness), I understood former when I went to Katie Shaw’s site at St. Olaf’s College: the ability to display online her work on Sequential Art and Literary Narrative – two fields close to my heart – demonstrated to me the potential that the ePortfolio held for the circulation of one’s work, especially amongst those working in the same field. I found the very notion of a Center for Integrative Studies fascinating: if Uconn does not have such a center, I strongly suggest it create one, where students from a wide array of majors would be able to post the results of their studies and efforts.Jensen’s article was also very instructive although he does rely too much on the premise that engaged participation will filter out “crap” and convey a greater richness to online resources. His overview from Web 1.0 to 3.0 was particularly eye-opening, especially his statement that 2.0 is not YET a source of scholarly communication but 3.0 may one day substitute current methods of peer review and tenure. I do wonder about the diffusion of scholarly materials online: at the moment, Googlebooks allows one to read but not download or even print texts. Granted this may be still a relic of Web 1.0 but authors and publishers eventually must overcome their reservations regarding their the publication of online resources according to the criteria of Authority 3.0 that Jensen describes. The eventuality of the disappearance of scholarly publishers in two decades’ time has as its foreshadowing the pre-release of Radiohead’s most recent album ONLY online and for free. Lastly, Jensen’s description of Photosynth could not but call to my mind the futuristic holodecks of Star Trek fame.On the other hand, I must admit that Spurlin’s article left me less than enthused. She honestly admits the impossibility of determining the impact of technology on student learning, given the many variables involved, and she does bring up the extremely valid point of how technology should be employed for the general benefit of our students and how to incorporate technology into classroom. These points, however, are lost in a purely academic question concerning the distinction between assessment (student oriented) and evaluation (faculty oriented). Spurlin complicates matters with a series of odd statements, for instance that assessment is about aggregate information – i.e. “students as a group” – which perplexed me because it implicitly denies the status of group to faculty members. Moreover, she further confounds the reader in my opinion with the statement:“When you assess how well students learn at the course or academic program level, you don’t typically ask about the connection between how the information was taught and how well the students learned ….If faculty find that students have met the [course] outcomes, they don’t ask how the material was taught.” (p.7)

Clearly, if this were true, then there would be no student evaluations at the end of each semester. Lastly, her definition on page 6 of students’ “measurable components” which should help with learning practices and principles – the determination of their backgrounds, their technological skills, their demographics and their learning styles – are for all intents and purposes impossible to measure for any group larger than 30.

 

Fulvio

Just like Felice, I was struck by Trent Batson’s definition of ePortfolios. By saying that an ePortfolio is “not a higher education application” but rather “a cultural application” Batson challenges the classic idea one has of an ePortfolio and imagines a future when people will be using these tools beyond the field of higher education. As a matter of fact, Batson states that “the big market is going to be everyone having an ePortfolio, whether they are in college or not.” Although I agree with the fact that ePortfolios could bridge “the gap between informal learning and formal learning”, I am not sure I fully agree with the way Batson envisions the future of these tools. In fact, even though many people “like to collect things”, I just cannot imagine people using them “in their leisure time,” or “for fun”. By definition, I believe that an ePortfolio should be related to the work or the specialization of a person. If, as Batson seems to suggest, ePortfolios are to become like an online journal (as already pointed out by Felice) we will simply end up having another kind of tool, something that resembles more blogs, Facebook or other similar applications. Regarding Katie Shaw’s website at St. Olaf’s College, I once again agree with Felice, and I must say that I have also found the very notion of a Center for Integrative Studies fascinating, and very useful to allow students “to demonstrate the meaningful relations among the different kinds of work they do.”