Posted tagged ‘bad science’

One From the Road: Why Can’t David Brooks Behave When I’m On Vacation edition

August 20, 2008

So, I’m blissfully bagging (photographically) three of the Big Five up in KwaZulu Natal, and then equally blissfully chilling in perhaps the best location in the surreal beauty of Camps Bay, almost entirely free of the Intertubes, when I finally make landfall (runway fall has the wrong sound to it) in Johannesburg. There, I innocently sign on to my sister-in-law’s wireless, (just checking my email. Honest. And the Red Sox scores. And every single poll I’d missed, and….) and got smacked by this by-now-old-news story of David Brooks trying wax deep on his newish theme of the neuroscience of politics and culture . (Thanks, sort of, to James Fallows for leading me into this swamp).

The dog-bites-man headline, of course, is that Brooks essentially made up the critical facts of the research he cited. In a column trying to draw cognitive distinctions between the thought and perception of presumed collectivist Chinese and those stalwart individualist Americans, he got just about everything salient wrong. The study he cited did not claim to attempt a random sample, interviewing instead a captive audience of college students; the test image was not of an aquarium, but an underwater scene, the distinctions in results between the two populations were not as claimed – and none of these material errors was the big enchilada:

In a column purporting to probe crucial distinctions between Chinese and American psyches, Brooks cited a study whose Asian population was…wait for it…

…Japanese.

Now, if you want the full, devastating take down on Brooks and a very smart and just about as devastating critique on the body of research Brooks was alleging he had probed deeply enough to opine about, read this, by Penn’s and the Language Log’s Mark Liberman. I got nothing to add about the substance of Brook’s substancelessness beyond Liberman’s take down.

But what I do want to raise is the question of consequences. Brooks really screwed up here by the standards of his profession. He got several specific facts wrong, and those errors undermine the entire article. What is the appropriate response of his readers and, more important, his employers, those who provide him with one of the most significant bully pulpits in contemporary journalism.

First, please note that the observation that Brooks is an opinion-writer, not a news reporter does not buy him much mitigation. The old cliche – everyone is entitled to their own opinions, but not their own facts – applies here. His interpretation of the science he cited is his own; that I think it is wrongheaded, stupid, shallow, and betrays a lack of understanding of exactly the kinds of difficulties in the research that Liberman documents is of no consequence. People are allowed to be dumb, and other people can decide whether it’s Brooks or his critics who should don the dunce cap.

But the actual specific details of what he suggests is a growing scientific consensus are a different story. Those are actual events and results in the world. Mischaracterizing them to make a half baked (or even, in different and more careful hands, a fully baked) generalization is, in journalism, a kind of fraud, a pernicious betrayal (and disdain for) the trust of your readers.

That’s why in journalism in general and in the science journalism that I and my colleagues try to teach to our grad students, it gets repeated again and again that the first job is to get the facts right.

In science journalism, at least as I learned it and try, never quite perfectly, to practice and teach it, you need to take the next step. Just transcribing accurately what someone has told you or what you have read in a book, a paper or a press release ain’t enough. Actual understanding, and informed judgment matter too. If you are going to apply your own, non-expert interpretation to a result, you need to earn it – and you do so by mastering the background to that research first.

As I said above, others have done a much better job than I could demonstrating that Brooks failed this standard on every level.

So back to the question of what should happen to someone who so baldly screws up. A junior reporter, someone not so branded and “to bit too fail” as David Brooks would, if they demonstrated as much a disregard for facts as Brooks does here would be in serious trouble; if this were a third or fourth instance (and I invite folks to go back through blog reactions, including my own, to earlier Brooks fiascos) they would stand a good chance of being fired.

Now that’s not going to happen. If it mattered that much to the Times, then William Kristol, he of four published corrections since he started at the Grey Lady– would be out of a job. Dut the fact that Brooks made stuff up, in essence, to tell the story he already knew he was going to write (OK – so I’m inferring here, but this is a blog, and I get to) should matter to someone who might still cling to the idea that they worked for the “newspaper of record.”

Here’s what I would do. I wouldn’t fire Brook. That would just create another faux martyr for the bad guys. I would suspend him – say through November 8. I’d even suspend him with pay – and here I’m assuming that under his contract with the Times he’s constrained in what else he can do. And then I’d substitute for him on the next-to-last page of the dead tree edition with an intellectually honest, determined conservative. Get someone in their who can actually fight that corner. See what that feels like.

Just thinkin’ on the road, you know. Now its off to hear Pops Muhammad – much more fun than wallowing in the follies of the Lords of Journalism.

Don’t Know Nothing ‘Bout Causality (or science): Washington Post Edition…

March 5, 2008

…Or why you can’t infer from one truly awful writer that all writers are dumb as a box of rocks.

By now, pretty much everyone in the blog reading world (or at least those I imagine are the readers of this blog) has heard of, and maybe even read Charlotte Allen’s unwitting (witless?) self parody (immolation?) in last Sunday’s Washington Post.

 

You can find all the responses you want with a quick google search — here are a couple that catch the zeitgeist pretty well. (For a post with links to a lot more see this one.)

So, with all that out there, what more to add? Two things, actually, or maybe two and a half.

First, this blog has gone on, perhaps to the point of exhaustion, about the importance of even very simple quantitative reasoning as both the starting point for thinking within the scientific world view — and just for making sense of the everyday world of experience.

Allen’s inability to do this reaffirms how important it is — not least for keeping yourself from looking like a true idiot in front of a national audience. Here’s the problem: one of the early “arguments” (sic) that Allen uses to suggest that “several of the supposed myths about female inferiority are true,” is that “Women really are worse drivers than men” according to a ten year old study out of Johns Hopkins.

Jake Young whaled on this one here, proving once again (take a memo certain Post editors!) that it really, really helps to read and understand a paper before you glom onto its abstract. But that critique, useful as it is, misses the simplest stupidity that Young (and her editors) commit.

Young writes that the study

revealed that women clocked 5.7 auto accidents per million miles driven, in contrast to men’s 5.1, even though men drive about 74 percent more miles a year than women.

Stop and think about that for a moment.

Now change the activity in question, and maybe turn the groups being distinguished into ones that have a little less political affect:

Say you have some working stiffs who play golf once a week on the weekends, and some biometrically equivalent trust fund golfers who get to hit the links three times a week. Next, breathlessly report that the weekend duffers “clock 5.7 bogies per 18 holes, in contrast to the trust funded group, who cut their bogie rate to 5.1, even though they play 300 percent more holes than poor folk.

D’oh.

From that you could conclude, I guess, that the evolutionary history that produces poor people also contains genes for lousy golf, and the reverse for the rich folk. Or you could propose that maybe hitting a few more balls might improve your game.

I don’t truly know if practice makes you a better driver. It seems a reasonable hypothesis — but you’d have to do some real research to say so with any confidence, of course.

I don’t even care if this fact has something to do with whatever the Johns Hopkins people observed.

Here’s the point: A couple of times over the last few years I’ve given a talk in which I’ve come out “against science literacy.” Allen’s article is an illustration of what I mean. She’s marginally literate in science-yness at least. She uses words like “genes” and botches that old chestnut about brain sizes and so on. But it doesn’t matter how many Tuesday Science Times section she reads. If she can’t think, it doesn’t matter how many words she knows. And thinking in this context means, at a minimum being able to understand the basics of the statistics she chooses to bandy.

So that’s one point. The other derives from the second half of my job title: I’m a science writer, and I have to say that there is one other quality of Allen’s piece that has not, I think, received all the ridicule it deserves. Boy, is that one badly written stretch of fish-wrap!

And here, I’m not talking about the global issues of logic, accuracy and argument, but the butt-ugly sentences and phrases she unleashed on who knows how many Sunday hangovers.

This post is long enough, so one example will do:

This female taste for first-person romantic nuttiness, spiced with a soupçon of soft-core porn, has made for centuries of bestsellers — including Samuel Richardson’s 1740 novel “Pamela,” in which a handsome young lord tries to seduce a virtuous serving maid for hundreds of pages and then proposes, as well as Erica Jong’s 1973 “Fear of Flying.”

That’s damn near unreadable. Look at that construction:”This female taste for first-person…blah, blah, blah…as well as Erica Jong’s 1973 Fear of Flying.”

Did Allen miss that fourth grade lesson on run-on sentences, in which vital topics of syntax, usage and style were covered, as well as her college expos class? (Sorry. Couldn’t help myself.)

I teach writing at the college and graduate level. My least articulate MIT freshman, fluent in Python, not in English, doesn’t commit crap like this.

Which leads to my last half point. The real damage here is to whatever is left of the Post’s editors’ reputation. Leave aside all the things that can (and have) been said about the vapidity or worse of every paragraph in the piece. How did anyone licensed to wield a red at the paper let uglification like this get through?

Just askin…

Images: James Gilray, engraver, “Physical air,-or-Britannia recover’d from a trance,” 1803. Source: Wikimedia Commons.

Gunnar A. Sjögren, “Saab Formula Junior,” illustration on page 23 of The SAAB Way – the first 35 years of SAAB cars, 1949-1984, 1984. Released for uses clearly not contradictory to Saab’s interests.” Source: Wikimedia Commons.

Horace Hutchinson, British Golf Links, 1897 J. S. Virtue & Co, London, page 9. Source: Wikimedia Commons.

John Leech, ” published in The Comic History of Rome, p. 88 c. 1850. Source: Wikimedia Commons.