"Teachers & Parents for Real Education Reform": New Study of Value-Added Modeling Reveals Problems

Thursday, January 27, 2011

New Study of Value-Added Modeling Reveals Problems

Jesse Rothstein

University of California at Berkeley researcher, Jesse Rothstein, has taken another look at a Gates Foundation study released in late 2010 that purports to show that value added scores are correlated with broader, higher level teaching and learning competencies. Rothstein's new analysis reveals that the data in the Gates research study actually shows the opposite of what their conclusions state. The fact that there is surprisingly little correlation between value added scores and broader teaching competencies measured by other tests means that teachers with low VAM scores might actually be excellent teachers when other, richer measures are used. The Rothstein correction calls into serious question the use of value added modeling in teacher evaluation, such as that used in the DCPS IMPACT system.

No comments:

What's Wrong With Waiting for Superman?

This month's The New York Review of Books has a powerful critique of WfS by Diane Ravitch titled "The Myth of Charter Schools" in which she puts the message of the film in a broader context.

An expose of where the funding for the film came from reveals its political agenda in Barbara Miner's article on the Not Waiting for Superman Web Site.

Here's a thoughtful review by Columbia University educational sociologist, Aaron Pallas... An Inconvenient Truthiness.

Read this careful, point-by-point reflection on the film by Rick Ayers, professor in teacher education at the University of San Francisco: "What 'Superman" Got Wrong, Point by Point."

The Nation magazine writes "Grading 'Waiting for Superman," an in-depth piece responding to the film and media coverage of public education in recent months.

Comment from the Education Law Center, "Superman Is from Another Planet."

Here are talking points,"Why We Can't Wait for Superman," from our partners at the National Opportunity to Learn Campaign.

Here are "The Real Facts about Waiting for Superman" according to the National Center for Fair and Open Testing, FairTest.

Education historian, Diane Ravitch, responds to the film in "Stop Trashing Teachers."

Here is a reflection by NY Times columnist, Gail Collins, "Waiting for Somebody."

Check out this reflection by Nicholas Lemann in the September 27 issue of the New Yorker magazine.

In this thoughtful piece, President of the American Federation of Teachers, Randi Weingarten answers her critics.

Mark Phillips, professor emeritus of secondary education at San Francisco State University, attempts to create what he considers a balanced view of the movie: "Strengths and Weaknesses of 'Superman'."

Our testimony October 28,2008 before the City Council on the Council Hearing Amendment Act

Chairman Gray and members of the Council,

We come before you today representing a growing group of DCPS teachers and parents we call “teachers and parents for real education reform” who very much care about improving the quality of teaching and learning in our schools. We support much needed education reform. We have a new blog at http://realeducationreformdc.blogspot.com/.

Most of those involved in this new initiative are teaching students this morning or working other jobs as DCPS parents. If you held this hearing in the evening or on Saturday, I would not be the one giving this testimony, and we would have a large group here adding emphasis to our words. We are all volunteers in this work. If you could consider evening or weekend hearings in the future, we would certainly appreciate it.

Our point today is really quite simple. In the past year transparency, accountability and the opportunity for citizen input into school system decision making has drastically diminished. There is no reason for this to be the case, even under mayoral control.

As you heard in the one budget hearing you held last year, principals have never before been so in the dark about the budgets for their schools, which makes local school planning very difficult. Even Mary Levy, who may know more about the DCPS budget than anyone in the school system, had trouble figuring out what was and was not in the budget. This isn’t right in a democracy.

At this time in the history of our school system, the opposite is needed. Even among those who advocate dramatic reform of our school system, which we do, there are very different philosophies and very different directions that reform can take. Those differences need to be debated out and the Chancellor and the Council need to hear from informed citizens now, like never before.

Some would have you believe that there are only those who favor reform and those who do not, and of course there is no point in hearing from those who oppose change. This is a distortion. In fact, there are many knowledgeable people in this town who believe in urgent change and are critical of aspects of the current reform effort.

We wrote, quite knowledgably we believe, in the pages of the Washington Post’s Close to Home section September 28th, that if the Chancellor is serious about improving the quality of teaching and learning, that there are five programmatic elements that need to be in her plans. None of these necessary components of a system that supports good teaching seem to be part of current DCPS plans. We need to see and be able to comment on the budget and the extent to which it does and does not support what most educators would consider essential for systemic improvement in the quality of teaching and learning.

The context here is important. There seems to be a stark philosophical contrast between the approach being taken by the Chancellor in DCPS and that being taken in all the surrounding jurisdictions – Montgomery, Prince Georges, and Fairfax; between Chancellor Rhee’s approach and that advocated by education researchers, educational testing experts, and that supported by those who for decades have advocated the creation of a professional teacher workforce. The approach being taken in DCPS is perceived as a radical outlier, nationally. At the very least, there is a need now, more than ever, to have an opportunity for some debate, some discussion and public input before the DCPS budget is finalized and decisions are made. That will not happen without Council action.

At any time, the dialogue between those empowered to make decisions about our schools, and those impacted by those decisions is essential, but it is more so now than ever before.

Please restore the hearing process that used to exist before mayoral control. We support the requirement in the bill before you that the Chancellor annually make public, on the DCPS web site, a detailed estimate of the projected overall budget request, and a school-by-school breakdown, before the request is final so that the public can really come to understand the vision and comment.

There Is Another Way

The proposed WTU contract does not address many of the problems teachers face and will not make DCPS a better place to teach or learn. Successful reform of education in DC will require more than what we’ve seen, and we believe you know that too.

We who chose teaching as a profession want support for teachers to improve their craft but see nothing but disrespect and blame in the proposed contract.

The proposed raises and bonuses rely on private funding, which is not guaranteed. The green and red salary tiers will result in a teacher workforce that is a revolving door of short-timers. We think this approach disrespects the profession of teaching.

We need dramatic improvement in teacher salaries and high standards for teaching quality, but this contract is not the answer. It won’t work and it may bring disrespect for teachers and teaching that cannot be reversed.

Arbitrary, unaccountable power in the hands of principals to make every teacher an “at will” employee does not lead to better teaching. DCPS hasn’t clearly defined what good teaching is, but wants teachers to put their fate in the hands of principals, or worse, unreliable student test results as the measure of school and teacher quality.

The AFT is a national leader in fixing teacher evaluation systems, support systems for new and veteran teachers, and accountability to standards of good teaching practice. Why weren’t they brought in to help develop the WTU’s reform proposals? It’s not too late for the AFT to work with the WTU and put forward a new plan.

How do we create a stronger contract?
1. Vote “no” on the tentative agreement! We can do better.
2. Demand that the WTU leadership and negotiating team bring in the expertise of the AFT to develop and bring our proposals to the table.
3. Don’t worry! Voting against the proposed contract means that the current contract remains in force. We do not lose if we work for a better approach.

Learn more and join with your colleagues who see another way to reform. Please fill out the index card with your name, email, school, and contact information. Drop it in the box as you leave.

To find out more fill out an index card and give it to: Liz Davis (lizday_1951@yahoo.com),
Kerry Sylvia (kerrysylvia@yahoo.com), or any of the teachers distributing this flyer.