The Scarers in Print: analyzing a poop test brochure for readability

Now, it's too late for me to begin shovelling and sifting at alphabeds and grammar-books. I'm getting to be a old bird, and I want to take it easy. But I want some reading—some fine bold reading, some splendid book in a gorging Lord-Mayor's-Show of wollumes. ~ Charles Dickens, Our Mutual Friend, Chapt. 5

Those of us who take our ability to read for granted, who live and breathe letters as dogs do feces and effluvia, are sometimes forgetful of the difficulties experienced by so many in the face of nothing more formidable than the back of a cereal box, let alone a "chapter book." In Our Mutual Friend the illiterate dustman Mr. Boffin hires that "ligneous sharper," the peg-legged Silas Wegg, to read to him his newly acquired and highly treasured book Decline-And-Fall-Off-The
-Rooshan-Empire ("Eight wollumes. Red and gold. Purple ribbon in every wollume, to keep the place where you leave off.") Because all print is shut to them, he and Mrs. Boffin are willing to endure night after night of Mr. Wegg's garbled delivery of Gibbon's prose, just for the feeling of participating in what they consider proper culture. They want some fine bold reading, in some splendid book. Instead they have to put up with the malapropisms, mispronunciations and petulant malingering of the scheming Silas Wegg.

In 21st century Canada there are still many Mr. and Mrs. Boffins. According to ABC Canada:

• Four out of 10 adults, age 16 to 65 — representing 9 million Canadians — struggle with low literacy. (Adult Literacy and Life Skills (ALL) Survey, Statistics Canada and the Organization for Economic Co-operation and Development, 2005)
• Nearly 15 per cent of Canadians can't understand the writing on simple medicine labels such as on an Aspirin bottle.
• An additional 27 per cent can't figure out simple information like the warnings on a hazardous materials sheet.
• In total, 42 per cent of Canadians are semi-illiterate. The proportion is even worse for those in middle age. And even when new immigrants are excluded, the numbers remain pretty much the same.

The statistics for our neighbours to the south are roughly the same, nor has there has been much improvement over the last generation in either country. This surprises me, for reading is ostensibly such a popular pastime and a prominent feature of our popular culture — certainly much more so than in that now distant, pre-postmodern period when there were few book clubs, no lit blogs, no Chapters-Indigo, no Amazon, no Oprah, no Harry Potter. In the United States, a 2004 NEA report on Reading at Risk lamented the sharp decline in the reading of books of "literature." Some hapless bloke's complaint is recorded there for posterity: "I just get sleepy when I read." To which Ursula Le Guin, in a feature article in the February Harper's, replies resignedly, yes, but there are actually many people who read wide awake. Le Guin thinks books are here to stay. "It’s just that not all that many people ever did read them. Why should we think everybody ought to now?" (Warning: the Harper's website only provides a teaser version of the article to non-subscribers.)

I have to agree. Books are not a threatened species. However, when it comes to health information, the large ranks of the functionally illiterate, possessing a merely nodding acquaintance with the printed word, face a very real threat. In English-speaking North America, as more than 300 studies indicate, health-related materials cannot be understood by most of the intended audience [1]. Now that two thirds to three quarters of our populations are seeking out consumer health information on the Internet, one wonders how this massive group of so-so readers is dealing with the often challenging vocabulary and syntax to be found on sites like MedlinePlus and the Canadian Health Network, not to mention the huckstering puffery of Health 2.0.

'Why, truly, sir,' Mr Wegg admitted, with modesty; 'I believe you couldn't show me the piece of English print, that I wouldn't be equal to collaring and throwing.' 'On the spot?' said Mr Boffin. 'On the spot.' 'I know'd it! Then consider this. Here am I, a man without a wooden leg, and yet all print is shut to me.'

Trying to digest the ColoScreen brochure
My own recent experience with "patient information" has convinced me that the literate need to learn how to write as badly as the illiterate need to learn how to read. Not long ago I had occasion to do some stooping and scooping — for my own, not my Retriever's. A lab test was required, and for the first time in my life I was introduced to the mysteries of ColoScreen. My doctor gave me a package including a special envelope, three sample collection areas with fold-over flaps, and three small wooden "applicator sticks" with which to provide "specimen" smears. The ColoScreen kit came with a 600-word set of instructions: everything I needed to know about the poop test. To this extraordinary work they give the title ColoScreen: a test for fecal occult blood. Now there was some fine bold reading!

Talk about occult. Here is a representative selection of the kind of prose some committee probably laboured over for hours in order to ensure I would make no error in carrying out what any two-year-old can do with the greatest of ease all over the nursery wallpaper:

Two days prior to, and including the test period, a red-meat free, high-residue diet should be followed ... Do not ingest high doses of aspirin or other anti-inflammatory drugs, for 7 days prior to and during testing ... However, consult a medical professional before discontinuing any prescribed medication ... Discontinue the use of toilet tank/bowl cleaners or deodorizers throughout the test period to avoid interference ... Flush tissue with stool, and discard stick in waste container ... On the next two subsequent bowel movements, repeat above steps ...

What possible excuse could there be for this laughable effort? Precision? There certainly is lots of that. Ass-covering? No one can say they weren't told in excessive detail how to scrape their own excrement into an envelope. Just who was this written for? The patients, their health care professionals, the government, or the lawyers? The strenuous, stilted syntax, the jargon, the Latinisms, the pathological avoidance of common idiom — all this adds up in my mind to a truly deplorable effort at communication. What were the authors thinking? Half their potential readership is left to puzzle at expressions like "discontinuing any prescribed medication" and "subsequent bowel movements." And couldn't they have found a simpler way to say "waste container?" Do any of us use such language in our daily lives? "That's alright, Junior. Just make sure you throw any of Fido's subsequent evacuations into the waste container."

Helena Laboratories in Beaumont, Texas, is the manufacturer of the fecal occult blood test package, and, I assume, responsible for the accompanying patient instruction. Their website advertises a number of educational brochures, and I shudder to think of the squinting and squirming and cocking of heads going on right at this moment as millions of helpless Boffins decline and fall under these heavy catapults of English prose.

Different versions of the ColoScreen brochure may be found on the Helena.com website. I checked to see how they compare to the print version I received. Sad to say, they are even more prolix and impenetrable. Here is a not untypical excerpt (perhaps it reads more easily in the Spanish version):

Because of the nonhomogeneity of the stool, it is recommended that the test be performed on three (3) consecutive evacuations, or as close together as possible.

Testing for readability
In the face of such a frontal assault one can do little else but evacuate the wounded and regroup. I wanted to find out just how bad the ColoScreen brochure really was. I needed to see some data that would allow me to compare it against a benchmark. So I resolved to analyze the text using a number of standard readability measures: Coleman Liau, Flesch Kincaid, ARI, SMOG, Gunning Fog, and Lexile. Microsoft Word does a basic readability test as part of its word count feature, and I started there.

According to Word's word count my printed ColoScreen brochure scored 57.2 in the Flesch Reading Ease measure, and 8.8 in the Flesch-Kincaid Grade Level. Not satisfied to rely on a single test, I used a number of freely available online readability analyzers. The sites I used are stored under my readability tag on del.icio.us.

Running my brochure through these online tools produced slightly varying readability results, but they averaged out at roughly Grade 9 or higher. Testing against the Lexile measure returned a Grade 10 (1100L to 1200L). By way of comparison, the Harry Potter series measure between 880L and 950L; Don Quixote (in English translation presumably) rates a fairly high 1410L.

The scores were much too high. For consumer health information, the literature is full of admonitions to employ plain language at the fifth grade level or lower to accommodate differing literacy levels [2,3,4]. The ColoScreen instructions fail to inform patients in an appropriate manner. Not only is this preposterous brochure well-nigh impenetrable to the semi-literate, it is prudish and officious to boot. In fact, it stinks.

Everyone advocates health literacy. Library shelves sag with literature on the subject and the web does the same digitally. Barbara Nail-Chiwetalu reviews the issue of health literacy in a way I found useful [5]:

Health literacy may be defined as the ability to obtain, read, comprehend, and use health information to make appropriate health decisions. The development of appropriate and effective health communication is an initiative recognized in Healthy People 2010. To this end, improving health communication may call for use of a variety of approaches, which may include:

• Improving the accessibility of appropriate health materials in communities [6]
• Emphasizing readability and comprehension of health communication materials (e.g., pamphlets, instruction guides, package inserts, books, Web sites) by

o acquiring materials that are written in conversational style (active voice) [2]
o using short sentences of ten to fifteen words [2]
o translating complex medical terms [3]
o using plain language at the fifth grade level or lower to accommodate differing literacy levels [3]
o using caution with medical textbooks written for physicians or other health professions with consumer due to the high readability level and comprehension of terms [2]
o considering use of nonwritten materials (e.g., charts, diagrams, photographs , picture books, videotapes, audiotapes, multimedia presentations) with persons having limited literacy [3,4]

• Showing sensitivity to language and cultural needs by

o providing materials that are culturally relevant [3]
o translating materials into different languages [3]
o using interpreter services to provide direct translations of what is said [4]

• Adjusting oral communication of health information by

o slowing down the rate of speech when delivering health information [3]
o using a “teach back” or “show me” approach to ensure understanding [3]
o including important family members or close friends in discussions including “surrogate” readers [3]

I don't claim expertise in health literacy, and I realize that simple readability scores are not the last word in assessing the quality of consumer health information [7,8,9]. A recent study by Rosemblat et al. [10] enlightened me as to the importance of the "main point" for readability measurement, while at the same time admitting the difficulty of measuring it:

Only two features, "Vocabulary" and "Main Point," significantly predict whether the annotators rated consumer health texts as readable for general audiences. Traditional readability formulas incorporate syntactic (words per sentence) and semantic (vocabulary) features to predict readability. While the annotators verified familiarity with vocabulary as a predictor, they also found that effective communication of the main point is a significant attribute. These results may contribute to understanding consumer seeking and browsing health information online. For example, eye-tracking studies indicate that users typically scan a Web page for the "take-home" message and move on to another page if not found in a few seconds. However, "ability to communicate the main point" is difficult to define operationally and measure.

So with all this theory and the outstanding efforts of experts and advocates, how did I end up with the ColoScreen atrocity? What applicator stick smeared this rank specimen of English prose onto my slide? Here is the take-home message I extract from my experience. We all must work a little harder at communicating to the system that it must serve real people by promoting reading and assisting the non-readers among us. That is a professional and a political commitment. If the main goals of a system of public health are to increase quality and years of health life and to eliminate health disparities, then we health librarians must re-dedicate ourselves to the dissemination of health information and the promotion of healthy lifestyles, not just for the educated and comfortable, but for everyone who would not be able to read this sentence. More simply put, let's get rid of the Scarers in Print.

'... upon-my-soul to a old bird like myself these are scarers. And even now that Commodious is strangled, I don't see a way to our bettering ourselves. ... I didn't think this morning there was half so many Scarers in Print. But I'm in for it now!' ~ Mr. Boffin, after an evening of "declining and falling" with Mr. Wegg (Our Mutual Friend, Chap. 5)

References:

1. Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health literacy: a prescription to end confusion. Washington: National Academies Press; 2004.

2. Baker LM, Gollop CJ. Medical textbooks: can lay people read and understand them? Library trends. 2004;53(2):336-347.

3. Parker R, Kreps GL. Library outreach: overcoming health literacy challenges. J Med Lib Assoc. 2005;93(Suppl. 4):S81-S85.

4. Allen M, Matthew S, Boland MJ. Working with immigrant and refugee populations: issues and Hmong case study. Library trends. 2004;53(2): 301-328.

5. Nail-Chiwetalu, B. Libraries meeting community health needs: trends and issues (Revised January 18, 2006) [monograph on the Internet]. Washington: NCLIS, 2006 [cited 2008 Jan 24]. Available from: http://www.nclis.gov/award/NCLIS-TrendsinLibrariesReport.pdf

6. Burnham E, Peterson EB. Health information literacy: a library case study. Library trends. 2005;53(3):422-433.

7. Gemoets D, Rosemblat G, Tse T, Logan R. Assessing readability of consumer health information: an exploratory study. Medinfo. 2004;11(Pt 2):869-73.

8. Keselman A, Tse T, Crowell J, Browne A, Ngo L, Zeng, Q. Assessing consumer health vocabulary familiarity: an exploratory study. J Med Internet Res 2007;9(1):e5. doi:10.2196/jmir.9.1.e5

9. Wang, Y. Automatic recognition of text difficulty from consumers health information. 19th IEEE International Symposium on Computer-Based Medical Systems (CBMS '06). 2006:131 - 36. doi:10.1109/CBMS.2006.58

10. Rosemblat G, Logan R, Tse T, Graham L. How do text features affect readability?: expert evaluations on consumer health web site text [article on the Internet]. Mednet Congress 2006 [cited 2008 Jan 24]. Available from: http://www.mednetcongress.org/fullpapers/MEDNET-192_RosemblatGracielaA_e.pdf