The straw that broke the camel's back

This NYT article discusses the problems test makers face when their questions deal with the "real world."

The problem is exacerbated by educationist insistence on "authenticity" and "real-world" applications. In the math area, so-called "naked math" will no longer do. But the "real world" is messy and introduces complexity, ambiguity and variables that may surpass the level of sophistication reached at a particular grade level.

Beware the perils of ambiguity. It is a mantra that is increasingly pertinent to tests in mathematics and science. The two fields might seem immune from imprecision. But in mathematics, for example, today's tests assess more than a student's ability to do "naked computation," as Cathy Seeley, president of the National Council of Teachers of Mathematics, puts it. In many places, calculators have rendered meaningless the testing of basic computational tasks. Instead, more questions test students' comprehension in real-world contexts. A triangle is a corner garden bed. A rectangular object intersected by a line is a juice box, with a straw. A sloped line on a graph represents a year's worth of payments to the power company.

With these scenarios come variables, and mathematicians and scientists from British Columbia to Boston spend much time picking apart the questions, particularly in online discussion groups. If students are asked how many seeds can be planted in the surface area of a triangular garden, do you put seeds in the corners where there isn't room for plants to take root? What about relevant considerations like seasonality of utility bills or position of the planets? Multiple-choice questions, with no place to show your work and thinking, make such realities more vexing.

I love this problem that arises when you move from a line to an "authentic" straw:

Daniel Jaye, an assistant principal at Stuyvesant High School in Manhattan, served on the panel charged with investigating the troubled exam. He recalls a telling moment during the administration of the test at his school. "One of the kids raised his hand and the proctor called me into the room," Dr. Jaye says. The student was puzzled by a question about a straw that rested diagonally in a rectangular box, 3 by 4 by 8 inches. The question asked for the length of the straw to the nearest 10th of an inch. The answer, according to the Board of Regents, was 9.4 inches.

But, the student asserted, there was not enough information to answer the question correctly. If the question asked about the length of a line, he figured he could solve the problem. But because it asked about the length of a straw, he needed the radius of the straw to determine where it would touch the corner of the juice box.

"How am I expected to come up with an accurate answer?" the student asked.