Miss Frizzle fails IMPACT evaluation
Ohanian Comment: I love this take on Rubrics. I also recommend Maja Wilson's Rethinking Rubrics in Writing Assessment and Tom Newkirk's Education Week piece Mania for Rubrics [subscription required, but if you write and ask me, I'll send it to you].
By Valerie Strauss
Last week a few hundred teachers were fired in the Washington D.C. school district based on a teacher evaluation system called IMPACT that was instituted under former chancellor Michelle Rhee.
There have been many complaints about the system, including charges that it is unfair to teachers who work in high-poverty schools, and that is chief assessment tool is five 30-minute observations by administrators and master educators of teachers each year as they work in the classroom. ThatĂ˘€™s a total of 2 1/2 hours a year of observation. Some teachers are also evaluated by the standardized test scores of their students, which many argue is an invalid and unfair method of evaluating a teacher.
This was written by Marni Barron, an instructional coach in the District of Columbia Public Schools, and Leigh Dingerson is a community organizer and writer on public education reform.
By Marni Barron and Leigh Dingerson
Recently, we were reflecting on the portrayal of teachers on screen these days. ThereĂ˘€™s the animated Ă˘€śdance of the lemons,Ă˘€ť and Michelle RheeĂ˘€™s teaching bashing in Ă˘€śWaiting for Superman.Ă˘€ť Now comes Cameron Diaz in Ă˘€śThe Bad Teacher.Ă˘€ť What happened to the teacher as guide? Or the teacher as inspiration? What happened to Miss Frizzle?
You remember Miss Frizzle. She was the uber-elementary science teacher of the public television series Ă˘€śThe Magic School Bus .Ă˘€ť The show was first broadcast in 1994, based on the books by Joanna Cole. Miss Frizzle is famous for the amazing field trips that she takes her students onĂ˘€”a fantastic demonstration of experiential learning where students donĂ˘€™t just learn about life on Mars or the workings of the heart and lungsĂ˘€Â¦they go there. Through the extraordinary power of the Magic School Bus, they shrink to size, and take off on educational adventures.
We remember watching episodes of The Magic School Bus with our children, hoping that our toddlers would someday have teachers as dynamic, quirky, creative and flamboyant as Miss Frizzle. But it seems like todayĂ˘€™s teachers are getting all the Miss Frizzle drilled out of them, both on-screen and off.
Which got us thinking about teacher evaluations and how, like everything else, what you get depends on what you measure.
We both live in Washington, D.C. The recently ended school year marked the second under the District of ColumbiaĂ˘€™s new evaluation system, called IMPACT. Just last week, the District announced that 206* teachers have been fired for flunking IMPACT this year.
IMPACT was launched in the fall of 2009 by former D.C. Chancellor Michelle Rhee, and was immediately lauded as a model for the rest of the nation. While much of the focus and reporting on IMPACT has been on its use of test scoresĂ˘€”so-called Value Added MeasuresĂ˘€”to judge teacher effectiveness, the majority of teachers in DC are not subject to the Value Added components of IMPACT. They teach in grade levels or subject areas that are not tested (yet). For these teachers, 50% of their evaluation is dependent on two, thirty-minute unannounced observations conducted by Ă˘€śMaster Educators,Ă˘€ť known as Ă˘€śMEs.Ă˘€ť Three additional observations are conducted by the schoolĂ˘€™s principal.
What are these evaluators looking for? What counts? IMPACT established a Ă˘€śTeaching and Learning FrameworkĂ˘€ť (TLF)Ă˘€”essentially a checklist of nine teaching practice areas that each teacher is expected to demonstrate during the course of their 30-minute, surprise evaluation. Within each practice area, there are a set of specific skills that must be demonstrated to qualify for an Ă˘€śeffectiveĂ˘€ť grade, and additional skills that must be present for the teacher to be considered Ă˘€śhighly effective.Ă˘€ť In all, to receive a perfect score on their observation, teachers must demonstrate over 60 strategies and skills over the course of 30 minutes.
Marni is an instructional coach in a DC elementary school. As we discussed teachers in the media, and DCĂ˘€™s Teaching and Learning Framework, she reflected that her role used to be helping teachers become better educators. Under IMPACT, her job is now defined as helping teachers pass their IMPACT observations. We thought about the effect of that change on teachers. And we thought of Miss Frizzle.
Rating Miss Frizzle
Could Miss Frizzle teach in D.C.? How would she fare on IMPACT?
We decided to find out, by conducting two formal observations using IMPACTĂ˘€™s nine-point rubric. Assessing teachersĂ˘€™ preparedness for their IMPACT observations is MarniĂ˘€™s job. She relished the chance to be an Ă˘€śMEĂ˘€ť for the day.
Our observation found Ă˘€śthe FrizzĂ˘€ť herding her students on to the Magic School Bus for a trip into the solar system. As her students traveled from Mercury to Jupiter to Saturn to Neptune, Miss. Frizzle allowed them to see, feel, and learn. They determined the gas, oxygen, hydrogen and water levels of each planet they visited. They collected rocks, and analyzed their composition. They worked collaboratively, sharing their knowledge with each other. At one point, the students gently prodded one disengaged student to rejoin the learning experience. Miss Frizzle helped guide the studentsĂ˘€”at one point by becoming Ă˘€ślostĂ˘€ť herself, and forcing her students to figure out which planet she was on based on scientific clues. They found her.
It was quite a lesson. But IMPACTĂ˘€™s rubric gave no credit to Miss Frizzle for the experiential and self-guided nature of this exploration to the solar system. She failed to announce an objective for the lesson at the beginning. She did not provide Ă˘€śscaffoldedĂ˘€ť prompts, or link their learning that day to previous lessons. While she had allowed her students to experience the solar system through a variety of senses and learning styles, she missed several requirements on the IMPACT checklist.
Under IMPACT, a teacher must be evaluated based on the strict rubric. Miss Frizzle scored only a 2.2 during our first observation. She was Ă˘€śminimally effective.Ă˘€ť No matter that her students had had the experience of a lifetime, and demonstrated extensive knowledge of the subject matter at hand. Under IMPACT a teacher could literally take her students to the Moon and still be minimally effective. We decided to give her another chance.
The next time we randomly popped in on Miss Frizzle, she had planned an extraordinary lesson on asteroids. For this, her students were required to intercept and re-direct an asteroid that was hurtling towards Earth, threatening a direct impact on the elementary school where she taught. The students launched into space, where they encountered several extraterrestrial objects (a comet, space junk). How could they determine whether each was the ominous asteroid? The kids realized they needed to analyze the objectĂ˘€™s composition, trajectory and speed. When they finally found the asteroid, they figured out that it was made of iron and therefore could be thrown off its course by a magnet. Mission accomplished!
Miss Frizzle had prepared well for the lesson, having all of the appropriate equipment available on the bus for the studentĂ˘€™s discovery process and eventual success. She did better on this evaluation. But she still fell short of Ă˘€śhighly effective.Ă˘€ť For example, the Frizz did not ask the students any questions. Rather, she provided them with opportunities to determine the relevant questions and then answer them themselves. This sinks her on IMPACT.
The overall average of our dear teacherĂ˘€™s two scores was 2.6Ă˘€”barely into the Ă˘€śeffectiveĂ˘€ť range. If we were to conduct three more IMPACT evaluations for a total of 5 (the number of times DCPS teachers are formally observed each year), the outcome for Ms. Frizzle could be dicey. If she were to drop to even a 2.59, she would be considered minimally effective, and subject to dismissal like so many teachers were, just last week.
SomethingĂ˘€™s Wrong Here
A teacher who is able to create a learning environment that is student-led and teacher facilitated is considered a master of their craft by the education community. But not by DCĂ˘€™s IMPACT rubric.
Of course, Miss Frizzle is fictional, and her extraordinary field trips arenĂ˘€™t really possible in todayĂ˘€™s under-resourced classrooms (no funds for magic school buses in most districts!). But our little exercise of conducting formal IMPACT observations of Miss Frizzle helped identify a troubling aspect of DCPSĂ˘€™ teacher evaluation system. ItĂ˘€™s not that the Teaching and Learning Framework is a bad thing. Particularly for new teachers, having a framework on good practices (stating objectives, checking with students for comprehensive throughout the lesson, etc.) is critical. In a strong professional growth system, teachers would not only be given such a framework, but would also be given carefully constructed supports and extensive professional development in the areas where they seemed to be struggling (IMPACT provides only rudimentary feedback from Master Educators, and little real professional support).
But for creative and dynamic teachers like Miss Frizzle, the IMPACT rubric is a death-knell. Teachers in DC now, according to several we have talked to, are changing their practice to conform to IMPACTĂ˘€™s checklist. Their salaries and their jobs depend on it. Some are tossing out their most creative lesson plans, knowing that if a Master Educator walked in on such a lesson, their job could be put at risk. WeĂ˘€™re forcing some of our best teachers to be less creative, to dumb-down their practiceĂ˘€Â¦or even to leave the classroom altogether. And yes, some of the cityĂ˘€™s dynamic and popular teachers have been fired because their lessons didnĂ˘€™t adhere to the IMPACT rubric.
Evaluation systems should be part of a building processĂ˘€”building great and creative and effective teachers. They shouldnĂ˘€™t be designed with the inflexibility of a mousetrap. Ă˘€śSnap! Gotcha!Ă˘€ť
We hope that our children will have teachers with the breadth of skills identified on the IMPACT checklist. But we also hope that our kids will be in classrooms with the many Miss Frizzles of Washington, DC Ă˘€“ those teachers who donĂ˘€™t just talk about the planets, but take their students to them. Without revisions, and without recognition that sometimes great teaching doesnĂ˘€™t conform to a checklist, we worry that Miss Frizzle, and teachers like her, may be getting thrown under the bus.
*number of teachers fired based on Post reporter Bill TurqueĂ˘€™s adjusted figures
Valerie Strauss, Marni Barron, & Leigh Dingerson
Washington Post Answer Sheet
INDEX OF NCLB OUTRAGES