Orwell Award Announcement SusanOhanian.Org Home


Common Core and Fuzzy Wuzzy Complex Text Readability Formulas Part 1

Susan Notes: I've been working on this for months, struggling to figure out how to make people care about the outrage it contains. There are a lot of numbers masquerading as science. I guarantee that people who have the patience for it will find things they've seen nowhere else, things that our literacy organizations ignore.


by Susan Ohanian

In 1923, Theophilus Painter published a study of his results from using a microscope to count chromosomes, putting the number of chromosome pairs at 24. These observations were tricky, and when others repeated his experiment, some got the same number; others got 19, others 23. But, as Robert Matthews reports, "as everyone knew the correct answer was 24,those who got anything different could be sure they had done something wrong." Painter's number stood for 30 years. Undisputed. Then,in 1956, Joe Hin Tjio and Albert Levan, tried a new technique for looking at cells. Every time they used it, they came up with 46. Samuel Arbesman reports in The Half-Life of Facts: Why Everything We Know Has an Expiration Date that they talked with other researchers who had discontinued studies because they, too, could find only 46. Instead of discontinuing their own studies Tjio and Levan made a bold statement, asserting there are 46 chromosomes in the human cell. Matthews points out that for years textbook pictures clearly showed 23 chromosomes--and yet captions under the photographs declared the figure to be 24. Today we think that worse than Painter's blunder was the willingness of other scientists to bow to authority rather than to believe their own eyes. These days in public school policy people traveling under the name of scientific research bow to the lure of money from the Bill and Melinda Gates Foundation. This brings us to The Book Thief counting problem. Book reviewers praised The Book Thief, a Common Core State (sic) Standards exemplar text for grades 9-10, as deserving the attention of sophisticated teen and adult readers. Here's Janet Maslin in The New York Times:

Markus Zusak has not really written "Harry Potter and the Holocaust." It just feels that way. The Book Thief is perched on the cusp between grown-up and young-adult fiction, and it is loaded with librarian appeal. It deplores human misery. It celebrates the power of language. It may encourage adolescents to read. It has an element of the fanciful. And it's a book that bestows a self-congratulatory glow upon anyone willing to grapple with it.
And Maslin's just getting started. Likewise, from USA Today to The Guardian, reviewers had a lot of good things to say, everyone agreeing it's a book for sophisticated teens and adults.

But there's a numbers problem. Flesch-Kincaid, a readability formula used to determine the grade level of a text based on word syllables and sentence length, rates the book at grade level 2.3--a book for seven-year-olds. Accelerated Reader, which makes the claim that its Atos "is the first formula to include statistics from actual student book-reading (more than 30,000 students, reading almost 1,000,000 books--and is "a verified measure of quantitative text complexity for the Common Core State Standards," finds The Book Thief to be grade 5.1. Lexile,® which reports 146,055 books rated in their system and announces its pride in being a Endorsing Partner of the Common Core State(sic) Standards Initiative,rates the book at 730, which corresponds to grade 4.3. Note the specificity of this data: Not just second or fourth or fifth grade but third month of second fourth grade. This specificity is used to convince the user that the product is scientific.

But nobody points out that no matter what year and month children are on a purported reading continuum, second, fourth, and fifth grade might be a bit young for a 550-page book in which Death is the narrator.

Text Complexity and the Data Softshoe

The Common Core State (sic) Standards (The sic is a reminder that the states had nothing to do with developing this school curriculum monolith but are now fulfilling their assigned roles as sponsors, boosters, publicists, and enforcers), put text complexity at the very core of their operation and give great credence to computerized readability formulas. First computer formulas measure the books; then, they measure the kids and their teachers.

Supplemental Information for Appendix A of the Common Core State Standards for English Language Arts and Literacy: New Research on Text Complexity appears on the letterhead of the National Governors Association and Council of Chief State School Officers, beneficiaries of $26,524,137 and $79,033,200, respectively, from the Bill and Melinda Gates Foundation to lead the drive to establish Common Core State (sic) Standards (CCSS). This supplement to Appendix A refers readers to the Achieve the Core website (run by Student Achievement Partners) and a study, Measures of Text Difficulty: Testing their Predictive Value for Grade Levels and Student Performance, by Jessica Nelson, Charles Perfetti, David Liben, and Meredith Liben. This study was conducted with a grant from Student Achievement Partners (Gates money: $6,533,350; General Electric Foundation money: $18,000,000), where the Libens are employed. David does webinars on the Common Core for ASCD (Gates money: $3,024,695), complete with a rubric on text complexity (ASCD CCSS Text Complexity Webinar Handout #3)[Gates money to ASCD to support and implement Common Core: $3,269,428].

Some background information seems relevant. Both Libens served on the Common Core K-12 Standards Development English Language Arts Work Team. Reminder: Two founders of Student Achievement Partners, David Coleman and Sue Pimentel, are non-teachers who co-led the writing of the Common Core English Language Arts standards. Coleman now heads the College Board (Gates money: $31,178,497). Nelson, Perfetti, and the Libens report that after examining Atos by Renaissance Learning, Degrees of Reading Power® (DRP®)by Questar Assessment, Inc., Flesch-Kincaid (public domain),The Lexile® Framework For Reading by MetaMetrics, Reading Maturity by Pearson Education, SourceRater by Educational Testing Service, Easability Indicator by Coh-Metrix, "The major comparability finding of the research was that all of the quantitative metrics were reliably and often highly correlated with grade level and student performance based measures of text difficulty across a variety of text sets and reference measures." Maybe they didn't notice what happens with The Book Thief. More about this later.

Zeroing in on three programs--the Lexile® Framework, Atos by Renaissance Learning (popularly known as Accelerated Reader), and Easability Indicator by Coh-Metrix, you have to be startled by how many outliers exist in the grade designations assigned by quantitative metrics that pepper the CCSS exemplar text list. Something is definitely catawampus when Accelerated Reader pegs both The Tale of Peter Rabbit and The Color Purple at a 4.0 reading level. (See Accelerated Reader: The Data Softshoe.)

Focusing on what happens when Accelerated Reader dominates a school gives fair warning against those trying to push Common Core reading formulas in the door. An alarming number of schools put all their faith in parking students in Accelerated Reader(AR), labeling existing library books according to the AR system, and limiting new purchases to books in the AR system. School libraries in thrall to AR abandon the Dewey Decimal or Library of Congress systems and shelve books by AR numbers instead of by the Dewey Decimal or Library of Congress system. This means that in an AR-arranged library, the 2.6 Zone books, for example, hang out together, so Junie B. Jones Smells Something Fishy sits next to High interest/low readability titles also clocking in at 2.6--such as Nuclear Submarines and Keeping Cholesterol Low, not next to Junie B. Jones and Her Big Fat Mouth (2.3) or Junie B. Jones Is Not a Crook (3.0), which would be helpful to kids who, having enjoyed one Junie B. Jones, want another one.

Clearly, there is a certain anal retentive fixation in insisting any quantitative metric can distinguish and differentiate between the reading abilities of a child in the third month of the second grade and one in the sixth month of that same grade, but the AR program also debases reading by coupling rewards with reading level. Readers gain points for each book read, points which can be turned in for gimcrack prizes. A 4.4 reader can rake in 10 points for reading Hemingway's The Sun Also Rises, the quintessential lost-generation novel featuring the narrator's sexual impotence. Surely that's affront enough, but here's the real tragedy. A young reader who likes hands-on projects and figuring out how things work will accrue only ½ point from Everyday Science Experiments in the Backyard (4.0). This is because The Sun Also Rises is 350 pages long and Everyday Science Experiments just 28. This bean-counting system traveling under the protective coloring of science doesn't take into account that a child bubbling over with curiosity may well spend a week on one page from that ½ point book--working out an experiment. Such a child faces AR meltdown.

Think of this 1/2-point child as you listen to the corporate-politicos yelp for schools to be more rigorous, to produce more mathematicians and scientists. Reformists insist this can be done by worshiping at the altar of text complexity and rigor, catastrophically narrowing the curriculum (taking away art, music, and recess for starters) in elementary school and increasing the number of Advanced Placement classes in high school. Not to mention de-humanizing the curriculum: Slashing narrative writing and imposing Common Core architect David Coleman's bizarre perversion of New Criticism that minimizes use of students' prior knowledge or biographical information about the author under study.

With the books managed by the point system, more often than not librarians disappear; replaced by paraprofessionals who make sure the computers delivering the AR tests are plugged in. The teacher's role is reduced to that of data tracker. Here's how AR describes it: AR systematically gathers student-level information on daily practice. The software produces reports, which helps teachers track individual progress and consequently make instructional decisions based on the data they receive. The "information" is student scores on multiple choice test; no enjoyment, insight, or curiosity. The science behind this sounds like kissing kin to Clark Stanley's liniment popular around the turn of the last century. Stanley claimed his cure-all for rheumatism, neuralgia, sciatica, lame back, lumbago, sprains, toothache, frostbites, sore throats, and animal bites contained oil from slaughtered snakes but when the feds seized a shipment in 1917, they found it contained not snakes but beef fat, kerosense, traces of red pepper, turpentine, and camphor. (See Lotions, potions, and Deadly Elixirs: Frontier Medicine in the American West)

In the old days before the data softshoe replaced schools staffed by librarians, professionals chose the books to fill library shelves. Click on "library" on a school web site these days, and frequently the school's AR booklist pops up. There, absent a librarian, one can find a collection of stories Roald Dahl wrote for Playboy as well as his children's fare. In the AR data juggernaut, elementary schools in Alabama, Arizona, California, Delaware, Florida, Georgia, Idaho, Indiana, Maryland, Massachusetts, Michigan, Minnesota, New Hampshire, North Carolina, Ohio, Oregon, Pennsylvania, South Carolina, Virginia, Washington, and Wisconsin offer Dahl's Skin and Other Stories rated at the same grade level: 5.4, as Eleanor Estes' classic The Hundred Dresses. Whereas Estes offers readers 1 point, Dahl's book, written for an adult audience, delivers 9. Alexandria Ripley's Scarlett: The Sequel clocks in at 5.5 and delivers 47 points. According to the AR count, Estes' book has 7,329 words, Dahl's 60,306, and Ripley's 306,004.

The fact that library books now come with a consumer sticker price is a sign of our times. Corporate education reformers want one of those stickers on every teacher too.

You don't have to be a literacy scholar to know that something is seriously wrong with a system that puts The Tale of Peter Rabbit at 4.0, along with The Color Purple, and Curious George Rides a Bike 4.1 and Sound and the Fury and Charlotte's Web at 4.4. Readers gain 0.5, 9,0.5, 14, and 5 points, respectively. Peter Rabbit's tale is told in 975 words; Alice Walker uses 66,556.

Syllable count is part of the formula used to rate a book's difficulty, so Potter's repeated use of Mr. McGregor's name and H. A. Rey repetition of the word "curious" to describe George skews the grade level results. With multi-syllabic characters like Captain Underpants and Professor Poopypants, it's not hard to figure out why Dav Pilkey outranks William Faulkner on the difficulty scale. Perhaps the 83 repetitions of "pagoda" in Jack Gantos' Jack's Black Book are what drive its readability (4.8) higher than Faulkner's Sound and the Fury (4.4), where the most frequent word is "Dilsey."

I predict the coming of teacher evaluations that will provide "value added" to the teacher ratings according to the numbers attached to the books their students read. Go ahead and apply for a grant from the Bill and Melinda Gates Foundation to make this a reality.

How does a picture book aimed at the preschool set gets slotted for 6th grade readability? By using "Bambolona" 30 times, De Paola makes his readability index soar. Counting machines traveling under the name of science don't notice that this is a character's name.

Moving on to The Lexile® Framework, we'll stick with 730, the number assigned to The Book Thief (clocking in at 5.1 at Accelerated Readers). Put in the number and Lexile spits out 1,000 titles with the exact same ranking--everything from some Hardy Boys and Nancy Drew mysteries, Captain Underpants and the Invasion of the Incredibly Naughty Cafeteria Ladies from Outer Space by Dav Pilkey to Farewell to Arms by Ernest Hemingway and Tell Me a Riddle by Tilly Olsen. A Publishers Weekly review of another 4.3, Shipping News by Annie Proulx, raises questions about just what that Lexile computer is counting: "Proulx routinely does without nouns and conjunctions." Note: Besides rating all these books at a 4.3 reading level, Lexile tags all of the above mentioned as suitable for ages 3 and above.

Amazon and Barnes and Noble frequently post the Lexile Measure on books and Tess Garrison, an Edgar Award nominee Stephen King describes as "better than Michael Crichton," was ticked off when she found out what the Lexile 730 on her book Gravity means. In her words:
Gravity is so full of NASA and engineering terminology that it requires a glossary to explain the vocabulary. I then did a search of my other titles and found that my medical thriller Harvest was rated even less difficult to read, at 620. And that is chock full of complicated medical terms.
Lexile 620 corresponds to Grade 3. What Garrison doesn't mention is that Lexile also tags her books as "suitable for age 3 and up," which means she keeps company with these other 730/age 3 and up books: Prehistoric Pinkerton by Steven Kellogg, Fell Back by M. E. Kerr, Journey to an 800 Number by E. L. Konigsburg, Heavy Metal and You by Christopher Krovatin. The latter title appears next to James and the Rain by Karla Kushkin ("James puts on his yellow slicker and goes out into the rain to play rainy day . . .").

One book correctly pegged for 3-year-olds, is Bernstain Bears Meet Santa Bear, but at 740, Lexile® ranks this one tougher than the rest of the books under discussion.

Also pegged at 730 are Black Notice by Patricia Cornwall, in which the decomposed remains of a stowaway lead Dr. Kay Scarpetta on an international search to Interpol's headquarters in Lyon, France); Puss in Boots by Paul Galdone; Millions of Cats by Wanda Gag.

All age 3 and up. I made no effort to "arrange" these titles but took them as they came up on the list from Lexile. I could go on and on with loony examples, but the purpose here is to show what a very limited gauge formulas for determining grade level provide. The important thing to remember here is the research "evidence" presented in Supplemental Information for Appendix A of the Common Core State Standards for English Language Arts and Literacy: New Research on Text Complexity:
"The major comparability finding of the research was that all of the quantitative metrics were reliably and often highly correlated with grade level and student performance based measures of text difficulty across a variety of text sets and reference measures."
One can only wonder what text sets they were examining.

Sadly, a Scholastic page directed at informing parents lends credence to this labeling system:
"Lexile levels are scientifically and mathematically assigned based on the difficulty and readability of a book. Once you know your child's Lexile level, you can search for books that match this level to expand your home library and encourage daily reading practice in your own home."
It may be mathematical but the science seems more akin to snake oil than scholarship. Frontier cowboy Clark Stanley listed the actual ingredients on his cure-all lineament, knowing nobody reads the small print. To find Lexile's disclaimer you have to search their website.

Fuzzword of our Time

Text complexity is the fuzzword derigueur of the Common Core State (sic) Standards (CCSS). Appendix B of the CCSS sacred script contains "text exemplars," described thusly: "The following text samples primarily serve to exemplify the level of complexity and quality that the Standards require all students in a given grade band to engage with." Another CCSS document, Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects Appendix A: Research Supporting Key Elements of the Standards, stresses "the importance of test complexity to student success." Furthermore, it asserts that such dimensions of text complexity as word frequency, sentence length, and text cohesion "are more efficiently measured by computer programs." Coh-metrix, one of the computer systems for analyzing text featured in Appendix A, is more flashy in its claims, declaring: "An automated analysis is unquestionably more reliable and objective than approaches that involve humans annotating and rating texts by hand."

The people developing Coh-metrix received $1,425,000 from the Institute of Education Science (IES), the research arm of the US Department of Education, to see what happens when computers measure texts for complexity.(McNamara, D. S., M. M. Louwerse, and A. C. Graesser. (September 2002â"August 2005). Coh-Metrix: Automated Cohesion and Coherence Scores to Predict Text Readability and Facilitate Comprehension. Memphis, TN: University of Memphis.)

Let's check it out, keeping in mind that where Coh-metrix's public version has over 60 measures of discourse, their report their research version having nearly a thousand measures that are at "various stages of testing." This leaves a lot of wiggle room for excuses when absurdities in their ratings are exposed.

At one level, Coh-metrix is kissing kin to Lexile, using the Flesch-Kincaid system for determining grade level. Flesch Kincaid, a formula first used by the U. S. Army to determine difficulty of technical manuals, is based on number of syllables in a word and the number of words in a sentence. Coh-metrix's aim is to move beyond such a standard readabililty formula by examining five other elements to determine text complexity. It provides percentile scores on each: Narrativity, Syntactic Simplicity, Word Concreteness, Referential Cohesion, and Deep Cohesion.

Narrativity refers to story-like characteristics. Syntactic Simplicity refers to the sentence structure. Word Concreteness refers to the number of words in a passage that are easy to visualize. Referential Cohesion is more complex. Here is the authors' description of how it is measured:
Referential cohesion occurs when a noun, pronoun, or noun phrase refers to another constituent in the text. For example, in the sentence "When the intestines absorb the nutrients, the absorption is facilitated by some forms of bacteria," the word "absorption" in the second clause refers to the event associated with the verb "absorb" in the first clause. A referential cohesion gap occurs when the words in a sentence or clause do not connect to other sentences in the text. Such cohesion gaps at the textbase level increase reading time and may disrupt comprehension.
Here is Coh-Metrix's Deep Cohesion definition from the website, The Coh-Metrix Common Core Text Ease and Readability Assessor (T.E.R.A.):
Deep cohesion measures how well the events, ideas and information of the whole text are tied together. T.E.R.A. does this by measuring the different types of words that connect different parts of a text. These words are called connectives. There are different types of these connectives: time connectives such as after, earlier, before, during, while, later; causal connectives such as because, consequently, thus. Then there are additive connectives such as both, additionally, furthermore, moreover, what is more. There are also logical connectives: actually, as a result, due to. Finally, adversative connectives are words that connect two phrases or notions that on some level conflict with each other, such as "My favorite sport is baseball however I watch more football" or "Whales are not fish yet they spend their life in the water." Some examples of adversative connectives are: but, yet, however, although, nevertheless. All of these connectives help to tie the events, ideas and information in the text together for the reader.
Note the title. No longer is it Coh-Metrix Text Easability Assessor; it has evolved into The Coh-Metrix Common Core Text Easability and Readability Assessor (T.E.R.A.). And guess what? Deep Cohesion pops up in Common Core standards for third graders:
  • Core Standard W.3.2b: Develop the topic with facts, definitions, and details.
  • Core Standard W.3.2c: Use linking words and phrases to connect ideas within categories of information.
  • The American Federation of Teachers Share My Lesson website, which describes itself as "a significant resource bank for Common Core State Standards, covering all aspects of the standards, from advice and guides to help with dedicated resources that support the standards," delivers a Deep Cohesion lesson for third graders. Its lesson on informative/expository writing tells the teacher to provide children with a worksheet on linking/transition words so they can be sure to use them in their own writing and thereby "link words and phrases to connect ideas within categories of information."
    List of Linking/Transition Words

    To Show Order (Temporal Words)

    after
    before
    during
    first
    following
    next
    second
    since
    then
    > third
    while


    To Add Information:
    additionally
    along with
    also
    another
    as well
    besides
    finally
    for example
    for instance
    furthermore
    in addition

    To Indicate a Purpose or Reason
    in hope that
    in order to
    so that
    with this in mind
    because

    To Conclude:
    all in all
    as a result
    finally
    in conclusion
    in summary
    in brief
    therefore
    lastly
    to sum up
    overall
    To sum up, all in all, eight-year-olds are taught to lose their voice and learn to write like bankers, insurance agents, and Common Core curriculum trainers. Workers for the Global Economy.

    Why is Common Core Ramping up Nonfiction?

    Key researchers on computer text analysis acknowledge that computers are better at analyzing informational text than narrative. And so, activated by Common Core impresario David Coleman, Common Core delivery agents, also known as professional development providers for school districts, travel the country calling on teachers to ramp up the reading of nonfiction. James Carville's campaign strategy message for Bill Clinton--It's the economy, stupid!" is revived as an imperative for teachers: Teach nonfiction so students will be successful workers in the global economy, stupid!

    You don't have to be totally paranoid to figure out that if the people leading the Common Core claim that computers are better than humans in figuring out the appropriate grades for specific texts, then you have to go with what computers can do. Maybe this is the place to note that Common Core development and propagation was and is financed in very large part by the Bill and Melinda Gates Foundation, which bases a large part of its education grant disbursement schedule on what computers can do.

    The Common Core requires that by high school, 70 percent of the text students encounter in school must be informational text--nononfiction--never acknowledging that a poem by Emily Dickinson, a play by William Shakespeare, or a picture book by Kevin Henkes all deliver a whole lot of information, just not the kind the computer understands easily . The folks driving this nonfiction imperative present it as a literacy and economic emergency: If a kid is reading fiction, he's dawdling--and putting the nation in danger of sinking into economic catastrophe.

    Blaming teachers for not making their classroom zones of high rigor diverts attention from hedge funders, bankers, and their kissing kin, but that's another story. Some research done at Williams College offers provocative evidence teachers should consider before ramping up of informational text, formerly known as nonfiction. The Williams College evidence shows that the best predictor of students' success in college is not how much nonfiction they read in K-12, not their GPA, but "the amount of personal contact a student has with professors. . . interaction with real, live human beings, whether in the classroom, or in faculty offices, or in the dining halls." Also running contrary to received wisdom from the Common Core industry, Worcester Polytechnic Institute celebrates Poem in Your Pocket Day, distributing two poems across the campus--not via e-mail, but printed on fine paper. They want people to appreciate the fine craftsmanship of the paper as well as the words; they want people to "keep these poems and read them more than once."

    But the Common Core doesn't have time for that; it is all about quantification, parsing a book by its textual characteristics and labeling the results scientific. Let's take a look at how this works out with some texts rated by Coh-Metrix, which is described by its makers as an automated cohesion metric tool. While acknowledging the limitations of the Flesch-Kincaid system, the Coh-Matrix system reports Flesh-Kincaid scores along with their analysis. F. Scott Fitzgerald's oft-quoted line about "the ability to hold to opposed ideas in the mind at the same time and still retain the ability to function" seems appropriate here.
    CCSS Text Exemplar: Frog and Toad Together by Arnold Lobel
    CCSS Grades K-1 Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 0.6
    Narrativity: 86%ile
    Syntactic simplicity: 94%ile
    Word Concreteness: 76%ile
    Referential Cohesion: 30%ile
    Deep Cohesio: 39%

    CCSS Text Exemplar: Sarah, Plain and Tall by Patricia MacLachlan
    CCSS Grades 2-3 Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 2.1
    Narrativity: 84%ile
    Syntactic simplicity: 91%ile
    Word Concreteness: 83%ile
    Referential Cohesion: 19%ile
    Deep Cohesion: 38%ile

    CCSS Text Exemplar: So You Want to be President, Judith St. George
    CCSS Grades 2-3 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 5.9
    Narrativity: 73%ile
    Syntactic simplicity: 49%ile
    Word Concreteness: 22%ile
    Referential Cohesion: 11%ile
    Deep Cohesion: 13%ile

    CCSS Text Exemplar: Tuck Everlasting by Natalie Babbitt
    CCSS Grades 4-5 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 4.2
    Narrativity: 57%ile
    Syntactic simplicity: 53%ile
    Word Concreteness: 86%ile
    Referential Cohesion: 6%ile
    Deep Cohesion: 22%

    CCSS Text Exemplar: The Secret Garden by Frances Hodgson Burnett
    CCSS Grades 4-5 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 10.2
    Narrativity: 91%ile
    Syntactic simplicity: 19%ile
    Word Concreteness: 19%ile Referential Cohesion: 55%ile
    Deep Cohesion: 75%ile

    CCSS Text Exemplar: The Making of a Hurricane by Patricia Lauber
    CCSS Grades 4-5 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 3.5
    Narrativity: 16%ile
    Syntactic simplicity: 66%ile
    Word Concreteness:
    64%ile Referential Cohesion:
    19%ile Deep Cohesion: 6%ile

    CCSS Text Exemplar: Narrative of the Life of Frederick Douglass an American Slave by Frederick Douglass
    CCSS Grades 6-8 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level:10
    Narrativity: 82%ile
    Syntactic Simplicity: 28%ile
    Word Concreteness: 45%ile
    Referential Cohesion: 52%ile
    Deep Cohesion: 60%ile

    CCSS Text Exemplar: The Adventures of Tom Sawyer by Mark Twain
    CCSS Grades 6-8 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 2.7
    Narrativity: 74%ile
    Syntactic Simplicity: 88%ile
    Word Concreteness: 50%ile
    Referential Cohesion: 6%ile
    Deep Cohesion: 50%ile

    CCSS Text Exemplar: "Letter on Thomas Jefferson" by John Adams
    CCSS Grades 6-8 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 4.6
    Narrativity: 89%ile
    Syntactic Simplicity: 69%ile
    Word Concreteness: 2%ile
    Referential Cohesion: 12%ile
    Deep Cohesion: 90%ile

    CCSS Text Exemplar: The Book Thief by Marcus Zusak
    CCSS Grades Text Complexity Band: 9-10 Coh-Metrix
    Flesch-Kincaid: 2.3
    Narrativity: 77%ile
    Syntactic Simplicity: 88%ile
    Word Concreteness: 70%ile
    Referential Cohesion: 4%ile
    Deep Cohesion: 47%ile

    CCSS Text Exemplar: The Grapes of Wrath by John Steinbeck
    CCSS Grades 9-10 Text Complexity Band Coh-Metrix
    Flesch-Kincaid: 3.0
    Narrativity: 83%ile
    Syntactic Simplicity: 87%ile
    Word Concreteness: 81%ile
    Referential Cohesion: 22%ile
    Deep Cohesion: 32%ile
    CCSS Text Exemplar: To Kill a Mockingbird by Harper Lee
    CCSS Grades 9-10 Text Complexity Band Coh-Metrix
    Flesch-Kincaid: 12
    Narrativity: 68%ile
    Syntactic Simplicity: 10%ile
    Word Concreteness: 84%ile Referential Cohesion: 59%ile
    Deep Cohesion: 76%ile

    CCSS Text Exemplar: "1775 Speech to the Second Virginia Convention" by Patrick Henry
    CCSS Grades 9-10 Text Complexity Band Coh-Metrix
    Flesch-Kincaid: 10.3
    Narrativity: 65%ile
    Syntactic Simplicity: 14%ile
    Word Concreteness: 28%ile
    Referential Cohesion: 38%ile
    Deep Cohesion: 52%ile

    CCSS Text Exemplar: "Letter from Birmingham Jail," by Martin Luther King, Jr.
    CCSS Grades 9-10 Text Complexity Band Coh-Metrix Flesch-Kincaid: 11
    Narrativity: 43%ile
    Syntactic Simplicity: 36%ile
    Word Concreteness: 16%ile
    Referential Cohesion: 25%ile
    Deep Cohesion: 65%ile



    CCSS Text Exemplar: Elements by Euclid
    CCSS Grades 9-10 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 13.5 (College)
    Narrativity: 66%ile
    Syntactic Simplicity: 2%ile
    Word Concreteness: 33%ile
    Referential Cohesion: 100%ile
    Deep Cohesion: 94%ile

    CCSS Text Exemplar: "Pardoner's Tale" from Canterbury Tales by Geoffrey Chaucer (version in modern English)
    CCSS Grade 11 Text Complexity Band Coh-Metrix
    Flesch-Kincaid Grade Level: 2.8
    Narrativity: 68%ile
    Syntactic Simplicity: 91%ile
    Word Concreteness: 46%ile
    Referential Cohesion: 2%ile
    Deep Cohesion: 79%ile

    Think about the way the Coh-metrix system often ignores the grade level measure. More about this in Part 2. Part 2 also helps develop a context for this system by putting text ranging from the National Enquirer and The New York Times to Andy Borowitz through the Coh-metrix system.

    — Susan Ohanian

    2014-06-04


    INDEX OF RESEARCH THAT COUNTS


    FAIR USE NOTICE
    This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.