y = Xβ + Zv + ε where β is a p-by-1 vector of fixed effects; X is an n-by-p matrix; v is a q-by-1 vector of random effects; Z is an n-by-q matrix; E(v) = 0, Var(v) = G; E(ε) = 0, Var(ε) = R; Cov(v,ε) = 0. V = Var(y) = Var(y – Xβ) = Var(Zv + ε) = ZGZT + R.
I know what you’re thinking. A huge bug just ran across my foot, sending me shrieking in horror away from the keyboard. Surely that can’t be an actual formula that is actually used for something?
Ah my friends, I wish that in fact a big bug had run across my foot. In which case I would be standing on a chair screaming and not typing, but still. Such is the depth of my despair about what professed “reformers” are doing–it’s enough to make me wish for bugs.
This formula, a “linear mixed model,” is used by the Houston school district to determine “value-added” teacher effectiveness. Boiled down, “value added” analysis looks at the past test scores of students to predict their future scores, and then depending on how well the student hits that prediction, a teacher is deemed effective…or otherwise.
As I’ve written about before, this system is a bit like blaming the dinner-party hostess for having bad guests. Is it her fault if the guests get drunk, spill the wine, pick a fight, insist they can only eat gluten-free, completely organic, locally sourced fish that was caught by a virgin on the night of a full moon, in a net made of spider silk?
Here’s a thought: if your dinner party did suck, would you want anyone to know about it? Would you want that fact blazoned across, say, the pages of your city newspaper? That’s what the Los(t) Angeles Times started doing this summer, stating that teacher ratings need to be made available “because they bear on the performance of public employees who provide an important service, and…parents and the public have a right to the information.”
So you can type in the name of your kid’s teacher and find out the “value-added” score. Nowhere in the database does it say if the classroom was over-crowded, if the teacher experienced a death in the family or some other personal transition, if the administration supported the teacher’s methods…there’s nothing. When you click one of the “most effective” teacher names, you get a set of graphs and the brief caveat that this score “captures only one aspect of a teacher’s work.” That sentence is in small print above the boldly colored line graph, which is labeled (in big black letters) LEAST EFFECTIVE and MOST EFFECTIVE.
After reading this (self-serving and self-righteous) article in the LA Times, I decided to ask someone “on the ground,” as it were, how to determine teacher effectiveness: my 5th grader. I asked him about grading teachers based on their students’ test scores. He thought for a minute, then shook his head no: “Because you know, what if someone is a bad test taker? Or just, you know, the teacher tries but the kid is just…if he just doesn’t get it, or if she has too many students?”
He thought a little harder: “maybe we should have the students grade the teachers?”
I suggested that students who get in trouble a lot might not want to give their teachers a good grade. He nodded: “well…we could ask the kids if they LIKED the teacher first and THEN ask if they think the person is good. And we could weight their answers like I do when I’m making the computer games and….” his voice trailed off. “That’s too complicated.”
Exactly, I said. And then I asked him to compare his second-grade teacher (sort of a disaster) with his fourth grade teacher (brilliant, amazing). According to Liam the fourth grade teacher “did more one-on-one stuff so she could really move people forward who wanted to and help the other kids, too. My 4th grade class didn’t have as many kids as my second grade class.”
Oh, I said. So a smaller class makes a difference?
“Duh, mommy,” he said. “You know, the government really should have like a department of kids or something. I mean, we have really good ideas.”
What do you think? Do you think maybe “X is an n-by-p matrix,” or perhaps “v is a q-by-1 vector of random effects” stands for “what the kids think?”
Somehow, I doubt it. Liam is right (as per usual, he would say). Figuring out teacher effectiveness is complicated. But somehow I think publishing inaccurate testing results won’t bring out the best in our public school teachers.
Maybe we ought to try the stocks? Rotten tomatoes? Heck, why not bring back flogging? It worked in Salem, didn’t it?
