Saturday, July 18, 2009

"Value Added" Test Score Method Probed

An article in this week's Education Week (July 15) points to controversy over whether "value added" assessment of teacher effectiveness might be built on "shaky assumptions." The "Value Added" method is based on judgements as to whether individual teachers are getting greater, lesser or on par student test score gains compared with those historically predicted for their students. While significantly better than the current method of comparing the scores of this year's students with last year's different students to judge schools or teachers, serious questions remain as to whether the value added method is ready for high stakes use. Researcher Jesse Rothstein, who authored the study that was the subject of the Ed Week article, and testing expert Dan Koretz who authored the recent book Measuring Up, don't think so.

Meanwhile, it looks like the long awaited new teacher evaluation system in DCPS, developed by special assistants to the chancellor Jason Kamras and Michael Moody, relies on a "value added" assessment of individual teacher's students DC CAS scores for 55% of a teacher's professional evaluation. Stay tuned for a fair amount of controversy as this gets rolled out. DCPS isn't just planning to use this method to justify a bonus or as the basis of a pilot study as in other school districts experimenting with value added -- eg. Prince Georges County, Denver, NYC. DCPS is going right to full implementation of this untested method for as much of the teacher workforce as they have value added scores.


Anonymous said...

This does not bode well for anyone. It appears that any real teaching may/will just fall by the wayside this year in order to really boost test scores. After all, who wants to fail as a DCPS teacher? Could make employment else where aa a teacher nearly impossible.

Anonymous said...

More teach to the test lessons will surely come from this kind of evaluation. It is really too bad because Rhee has a great opportunity to revamp the PPEP process successfully by implementing effective evaluation programs like the one in Montgomery County.

One question- what about teachers who don't teach testing subjects? How will they be measured?

Mark Simon said...

It's still a great opportunity. Everyone agrees that PPEP needs revamping. And you don't have to go far to see evaluation systems that are working. The proposed Draft is only a draft, with the date 5-27-09 stamped on it. There is still time for DCPS to see the error of their ways, to take a fresh and more serious approach with more serious consultants involved, or at least to scale this back to become a pilot in a few schools rather than the new system for the whole system. A lot depends on the reaction of the union, I think, even though Congress took away their right to have any say over teacher evaluation. The fact is that no rational employer implements a new evaluation plan that has no credibility with the teachers it is designed to evaluate. But its not over til the "draft" is marked final and gets implemented.

IndependentD said...

In my opinion added value evaluations will benefit our school system as a whole. This is because you can follow the students learing capabilities throughout their educational careers as well as get a solid understand of the quality of the teachers in the schools. But there are some variables that in my opinion can skew the data. One thing is that studnets may not really care to take the evaluation tests and may just bomb the tests without really trying. So in order to take this out I believe there should be some sort of incentive for the student to perform well on the test.

Thats all I have for now I hope you all have a good day.


Margot Berkey said...

If my high school age daughter decides she doesn't feel like studying, has a bad day, puts tons of energy into a few courses but not all, then her teachers' jobs may be at risk. Measuring teachers skill or effectiveness by her effort and success seems risky and unfair. I'll make sure she puts her best effort into all her subjects--not for the sake of her teachers' job evaluations, but for her own achievement. But I know lots of students who seem to lack the motivation to really work hard in all their classes. From what I've seen, it's not always the teacher who makes the difference. A teacher can be liked, respected and valued by a student but the student can still not have the motivation to be successful in their course. This means that the student's grades are not a true measure of the teacher's work.

Maybe a tool like this could be used more reliably at elementary school level when a student has a year-long relationship with a single educator for almost all of their instruction and the children are more willing to accept pressure from adults? I'm still not sure. I believe this approach will be shown to be a failure at the secondary level, but how many careers will be damaged to find that out?

AVParodi said...

Ms. Berkey's point is very important. Ultimately an evaluation that ties student test results to the teacher's rating misses the big picture. There are many reasons students do not do well on tests and it isn't always the teachers fault. If we only focus on scores as the end goal and the teacher as being the problem/solution to that goal then we miss putting things in place to help those failing students to succeed. Instead of finding ways to bring success to the students we are finding ways to blame people. This is the inherent problem with the teacher-as-miracle-worker scenario that so much of the "reform" movement works under. Under such a scenario we get a flurry of activity, i.e. firings and hirings, under the banner of "we are at least doing something". The only thing that this scenario produces is a circle game (cue in Joni Mitchell here).

We need a comprehensive evaluation system that doesn't just look at test scores and an occasional visit by "master teachers". The evaluation should include test scores but should also take into consideration a number of other factors. For example: how well can the teacher demonstrate a knowledge of their students' strengths and weaknesses; what are the outside the classroom obstacles that these students face (this is being dismissed by Rhee et al, but definitely needs to be considered); What has the teacher tried in the room - how many different ways of helping these students have they attempted; outside the room?; what support has the teacher received or not received; how well does this teacher work with their grade level colleagues and their other colleagues; what is the overall atmosphere of the school they work in - is it an atmosphere conducive to learning and success or the opposite (the latter is not necessarily on the wane given the climate in so many schools)?

There are so many other factors that need to be considered. There are teachers doing heroic work under impossible circumstances. To expect them to turn it around without proper support and the right resources is ludicrous.