Orwell Award Announcement SusanOhanian.Org Home


Can You Read as Well As a Fifth-Grader? Check the Formula

Susan Notes:


I can't resist including (at the bottom of the page) the verbless book review of the verbless book mentioned in this article. But the real point here is how suspect readability formulas are.

by Carl Bialik

If you've checked the grammar of a Microsoft Word document, you may have
encountered a baffling number. The readability formula purports to
represent the text's appropriate grade level. But it has its roots in
research from 60 years ago.

Before computers, reading researchers attempted to quantify the ease of
a work of writing using short excerpts and simple formulas. Despite
computing advances, Word still follows the same model: It multiplies
0.39 by the average number of words per sentence, adds that to 11.8
times the average number of syllables per word, and subtracts 15.59 from
the total. The result is the supposed minimum grade level of readers who
can handle the text in question.

Similar formulas are used by textbook publishers and in dozens of
states' guidelines for insurance policies.

From the beginning, these formulas were known to be problematic. A 1935
paper laid out more than 200 variables that affect readability. Most
formulas incorporate just two, and not because they are the most
important but because they are the easiest to measure. Then they're
mashed together, with weights set according to how the formulas work on
standard texts.

"Everyone is waiting for this magic bullet that's very easy," says Karen
Schriver, who runs an Oakmont, Pa., communication-design research
company. But her experience with clients who have overly relied on these
formulas have suggested that "maybe it's just a stupid idea."

Noting that the same passage's score can differ by three grade levels or
more, depending on the formula, readability consultant Mark Hochhauser
says, "One of the things the field really needs is an updated formula."

Even neurolinguist G. Harry McLaughlin says of his own, widely used SMOG
Readability Formula, "The theoretical basis is c---."

The formulas treat writing as a mere collection of words and spaces.
Word meaning and sentence structure don't figure. George Weir, a
philosopher and computer scientist at the University of Strathclyde in
Glasgow, says Word's readability test thinks grade-schoolers could
handle the nonsense passage, "Acuity of eagles whistle truck kidney.
Head for the treacle sump catch and but. What figgle faddle scratch dog
and whistle?" Similarly, "Had lamb little a Mary" and "Mary had a little
lamb" score identically.

I asked Micro Power & Light Co., which sells readability-testing
software, to evaluate a memorable 2004 Wall Street Journal front-page
article. Four different formulas found it to be comprehensible to
10th-graders, thanks in part to its short sentences. The reason for the
frequent periods: The article was about a new book written without
verbs, and the article mimicked its subject, making for intentionally
tough reading.

Word length is an imperfect measure. "Important" and "elephant" are long
words that are easy for most readers, Dr. Schriver notes. Conversely,
frustrated crossword solvers encounter plenty of uncommon three-letter
words, such as adz, auk and lea. She adds that no formulas account for
document layout -- even short sentences with lean words are challenging
when printed in an eight-point type.

The formulas have their defenders. Readability consultant William DuBay
calls them "good enough," and adds, "They've been extremely beneficial
for millions of readers." Among other uses, they were implemented to
simplify newspaper writing a half-century ago, he says.

Some researchers are trying to make the formulas better, using new
databases and computing power. Prof. Weir aims to create a formula that
incorporates the frequency of words and word combinations in typical
English writing, meaning "the" and "adz" finally can be distinguished.

Several more-advanced readability formulas already have been developed.
None are as convenient, or as criticized, as the Flesch-Kincaid formula
Microsoft uses. Developed by readability researcher Rudolf Flesch in
1948, it was modified by psychologist J. Peter Kincaid in a study for
the U.S. Navy in 1975, using reference passages. "Do not swing, twirl,
or play with the nightstick" is part of a passage deemed appropriate for
seventh-graders. Instructions that included, "All the jet streams of the
Northern Hemisphere have their southern analogues" required a college
degree.

The formula was tweaked once more by Microsoft when the company
incorporated it into Word in 1993. Grade-level scores were capped at 12.
Reed Shaffner, Microsoft's product manager for Word, told me that the
formula was changed in 2003, at least for Windows users. Those users can
see results up to grade level 14, while Mac users won't get results
above level 12.

Why cap the results at all? "It's a user-experience thing," Mr. Shaffner
says. Essentially, Microsoft is concerned about the readability of
readability-formula results.

Prof. Kincaid, who today is the head of a modeling and simulation
program at the University of Central Florida, tried unsuccessfully to
get the formula corrected years before it finally was. Nevertheless,
when he wants to use his own formula, he lets Word do the calculation.

That's rare. "I write long sentences and no computer is going to tell me
how to write," Prof. Kincaid says. "I'm going to write the way I want to
write."

. Email me at numbersguy@wsj.com. Read
daily commentary about numbers and join a discussion with readers at my
free blog, WSJ.com/numbersguy .

NUMBERS GUY BLOG

Is it possible to quantify the readability of a given text? Do you ever
use these formulas? Share your thoughts in the blog comments

All Talk, No Action:
A Funeral for Verbs,
With Few Pallbearers

In Mr. Dansel's Curious Book,
Something ... Not There;
'Strangely Unappetizing'
By CASSELL BRYAN├ół'LOW and ANNE├ół'MICHELE MORICE
Staff Reporters of THE WALL STREET JOURNAL
July 16, 2004; Page A1


PARIS ├ół'├ół' The scene: a hundred or so scholars, writers and others in a packed
amphitheater at the Sorbonne. The event: a funeral.

But not for a person. Mais non. Instead, an au revoir to a part of speech.
The verb.

The mastermind of the mock memorial service: Michel Dansel, freelance
intellectual, author of books such as "The Rat, Our Brother in Darkness," as
well as a dictionary of spelling errors and a guide to complaint letters.

The purpose behind his June 1 ceremony: publicity. For his latest work; for
his bid for literary immortality; for his contribution to the language of
Rimbaud, Flaubert and Stendahl ├ół'├ół' a 233├ół'page novel without verbs.

"Le Train de Nulle Part," or "The Train to Nowhere." A torrent of
adjectives, adverbs, interjections and funky nouns. Also heavy doses of
commas, semicolons, colons ├ół'├ół' and dashes, too. The result, in the words of
the publisher, the first├ół'ever French novel without verbs, as well as
"brilliant, baroque, and original."

As for the plot: descriptions of a train journey and the narrator's
encounters with passengers (not the most appealing people in the world). A
typical passage: "In that carriage, between the grumpy woman oozing
vulgarity and the similarly asinine creature with her, the progenitor and
her eczematous brat, the purple├ół'faced fatso, the half├ół'bald guy like a
vegetarian may├ół'bug, the verbose matinee idol and the crazy witch, no room
for me."

A tough sell, even in a nation with many intellectuals and a well├ół'known book
without the letter 'e.' ("La Disparition," by Georges Perec, 1969.) A
challenge, the unloading of a print run of 3,500 books. So, at the ceremony,
as in the novel, gimmicks galore. Mourners, eulogies, a flowery wreath with
a banner "to the defunct verb." Even a reception at a nearby caf├â┬ę, with
wine, and a buffet of meats and cheeses, just like a traditional French
funeral.

So why this particular literary quirk? A challenge to tired, old ideas about
language, according to 65├ół'year├ół'old Mr. Dansel. The written verb, in his
opinion, such a boor; so pushy; so insistent on action. (Verbs in speech? No
such problem.)

Mr. Dansel's manifesto, according to the book's introduction: "The verb this
invader, this dictator, this usurper of our literature since always!" Then,
a call to arms, "to all the followers of this new movement."

The reaction among French critics to this feat? Mostly chilly. Like the Alps
in January. Brusque. Like a bistro waiter. In the monthly magazine Lire, a
bible for bookworms, just an 18├ół'line snippet, with this sentiment: "a bad
paper by an 11├ół'year├ół'old student."

And from the British press, worse. Sneers! Scoffs! Denunciations! John Walsh
in the Independent: "a jungle of similes and essences, desperate for the
oxygen" of a verb. Echoes from U.S. critics. A verbless review in the
Chronicle of Higher Education: "Enigmatic. Disturbing. Strangely
unappetizing."

The very notion ├ół'├ół' in the words of linguistics professor Geoffrey Pullum, on
a Web site about language ├ół'├ół' "nuts, bonkers, round the bend."

A recent interview with Mr. Dansel, in his crumpled suit, over espresso in
an artsy quarter of Paris. About his book, nearly two years in the making.
His long, lonely struggle against those pesky verbs ├ół'├ół' like "a shadow
boxer." His disappointment at the conservative opinions of the "small world"
of French literary circles.

From whence the idea?

First, Mr. Dansel's childhood in a working├ół'class neighborhood of Paris.
There, the development of "an aversion to orders and commands." Next, to
college at the Sorbonne, two doctorates in literature and a focus on
language, in particular an obscure, nineteenth├ół'century French poet by the
name of Tristan Corbi├â┬Ęre. Meanwhile, extensive travel around the gloomy
reaches of northern Europe and an extended stint in Lapland.

Ingredients for the adult writer's interest in "the hidden side of things."
Among his works, a crime novel with intellectual gangsters and philosopher
policemen.

But also, for a regular income, a slew of more mundane publications. The
total tally, maybe 100 or more, according to Mr. Dansel. But the exact
number? A secret. His reason? The protection of his reputation from French
disdain of overly productive writers. Too pushy, too unseemly, too contrary
to the spirit of the 35├ół'hour workweek.

Hence several pseudonyms, plus one for the verbless book (Michel Thaler ├ół'├ół'
Thaler, his mother's maiden name).

Back at the mock funeral, a chatty mourner: the author's elementary├ół'school
teacher, Georges Rivault. A chuckle. A revelation of a childhood secret, Mr.
Dansel's weakness: spelling and grammar. Above all, "bad at verb
conjugation."


— Carl Bialik
Wall Street Journal
2008-03-14


INDEX OF RESEARCH THAT COUNTS


FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.