"Filter, flavor, flip-top box": getting PubMed filters to work good and draw easy

While attempting to assist a medical resident with a difficult search, I was trying out the Cochrane filter (revised strategy) as discussed in an article by Robinson & Dickersin in the International Journal of Epidemiology [1]. They present a highly sensitive search strategy to retrieve reports of controlled trials using PubMed. In short, you get a lot to like. Excellent for use in systematic review searching, the authors' filter creates a subset much larger than what you would get from clicking on the various clinical trial check boxes in the Limits menu.

This is the search string in all its wonky glory:

(randomized controlled trial[pt] OR controlled clinical trial[pt] OR randomized controlled trials[mh] OR random allocation[mh] OR double-blind method[mh] OR single-blind method[mh] OR clinical trial[pt] OR clinical trials[mh] OR ("clinical trial"[tw]) OR ((singl*[tw] OR doubl*[tw] OR trebl*[tw] OR tripl*[tw]) AND (mask*[tw] OR blind*[tw])) OR ("latin square"[tw]) OR placebos[mh] OR placebo*[tw] OR random*[tw] OR research design[mh:noexp] OR comparative study[mh] OR evaluation studies[mh] OR follow-up studies[mh] OR prospective studies[mh] OR cross-over studies[mh] OR control*[tw] OR prospectiv*[tw] OR volunteer*[tw]) NOT (animal[mh] NOT human[mh])

I have been saving this article for years, but never got round to trying the filter out. Upon entering the strategy directly into the PubMed search box I was alerted by a pink-banded message telling me that the two descriptors highlighted above were not found. Of the two problems, the first is an error in the search strategy itself (I found no correction in the literature). The second is the result of a recent "major change" in PubMed.

This was more filter flavour than I had counted on. Now I was really huffing and puffing as I inhaled a man-size portion of PubMed technical detail in order to discover what had gone wrong with my peer-reviewed, much treasured filter in the flip-top box of my knowledge base. Here is what I found:

1. "Comparative Study" is used only as a publication type. The field delimiter or tag [mh] must be replaced with [pt]. This appears to be an error on the part of the creators of the filter. Comparative Study has only ever been a publication type since 1966. (An aside: this term has never found its way into the PubMed Help list of publication types. However, it does show up on the official NLM Publication Characteristics (Publication Types) - Scope Notes web page.)

2. "Evaluation Studies" was once a MeSH heading but is now a publication type. The tag must be changed to [pt] or PubMed gets tetchy. More on this change below.

Here is a corrected version of the filter (it works in PubMed without error reports):

(randomized controlled trial[pt] OR controlled clinical trial[pt] OR randomized controlled trials[mh] OR random allocation[mh] OR double-blind method[mh] OR single-blind method[mh] OR clinical trial[pt] OR clinical trials[mh] OR ("clinical trial"[tw]) OR ((singl*[tw] OR doubl*[tw] OR trebl*[tw] OR tripl*[tw]) AND (mask*[tw] OR blind*[tw])) OR ("latin square"[tw]) OR placebos[mh] OR placebo*[tw] OR random*[tw] OR research design[mh:noexp] OR comparative study[pt] OR evaluation studies[pt] OR follow-up studies[mh] OR prospective studies[mh] OR cross-over studies[mh] OR control*[tw] OR prospectiv*[tw] OR volunteer*[tw]) NOT (animal[mh] NOT human[mh])

So I had stubbed out the problem filter and made my corrections, but what had happened to require the change to Evaluation Studies?

Not that many of us took much notice in the annual yuletide neuronal storm, but the National Library of Medicine announced a major revision of publication types and corresponding subject descriptors in a Technical bulletin dated 26 Nov 2007 (final update 13 Dec 2007). These were bundled with the usual announcements of new MeSH headings. You know the type: score-settlings amongst the specialists (Coronary Occlusion — not to be confused with Coronary Stenosis); the exotic and somewhat frightening (Leukemia, Myeloid, Chronic, Atypical, BCR-ABL Negative; Shiga-Toxigenic Escherichia coli; Weapons of Mass Destruction); and the when-would-I-ever-use-this? puzzlers: Pollination; Muscle, Striated).

PubMed now distinguishes between articles ABOUT evaluative studies and articles that ARE actually evaluative studies. For the former you must use the brand new MeSH heading Evaluation Studies as Topic. For the latter you use the publication type delimiter: Evaluation Studies [pt]. A useful distinction. Makes sense when you think about it. Direct from NLM's last Technical bulletin, here is a list of the new MeSH Headings that correspond to the Publication Types used for journal article indexing:

Bibliography as Topic
Biography as Topic
Clinical Trials as Topic
Clinical Trials, Phase I as Topic
Clinical Trials, Phase II as Topic
Clinical Trials, Phase III as Topic
Clinical Trials, Phase IV as Topic
Congresses as Topic
Consensus Development Conferences as Topic
Consensus Development Conferences, NIH as Topic
Controlled Clinical Trials as Topic
Correspondence as Topic
Dictionaries as Topic
Directories as Topic
Duplicate Publication as Topic
Evaluation Studies as Topic
Government Publications as Topic
Guidelines as Topic
Interviews as Topic
Legislation as Topic
Meta-Analysis as Topic
Multicenter Studies as Topic
Patient Education as Topic
Practice Guidelines as Topic
Randomized Controlled Trials as Topic
Retraction of Publication as Topic
Review Literature as Topic
Twin Studies as Topic
Validation Studies as Topic (New for 2008)

In what looks like a gaff or an oversight, the indexers at NLM did not see fit to create a MeSH heading Comparative Study as Topic, nor have they come up with rationalizations for the following: Follow-up Studies and Prospective Studies. Nor have they resolved the singular/plural confusion in headings of this type. Perhaps they wish to spare us too much excitement at once. Leave room in your Xmas stocking next December.

If you enter the revised Cochrane filter for controlled trials into PubMed, the database puffs out more than 3,450,000 hits. This creates a nice subset of the PubMed database consisting of controlled trials (or, more precisely, consisting of articles that at least contain terms that would lead one to suspect that they might be controlled trials of some sort). It works good and draws easy. You'd expect it to cost more, but it doesn't.

As Robinson and Dickersin state in concluding their article, "To continue to be an effective and efficient strategy, the revised strategy should be examined periodically to take into account new features available on PubMed, as well as developments in indexing by the National Library of Medicine." I can vouch for that. Pass me that flip-top box.


References:

1. Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. Int J Epidemiol. 2002 Feb;31(1):150-3