Printer Friendly

Rethinking information delivery: using a natural language processing application for point-of-care data discovery.


Clinicians often encounter information needs in their work of caring for patients. In their 2005 study, Ely and colleagues discovered that physicians developed an average of 5.5 questions for each half-day observation, yet could not find answers to 41% of the questions for which they pursued answers [1]. Ely cited time constraints as one of the barriers preventing clinicians from finding answers. In another study, Chambliss and Conley also found that discovering answers is excessively time consuming [2].

Chambliss and Conley determined that references found in MEDLINE could answer or nearly answer 71% of clinicians' answerable questions; however, PubMed is not a tool exclusively designed for point-of-care information delivery. It generally returns excessive, irrelevant data, even when implementing diverse search strategies [3]. Clinicians can spend an average of 30 minutes answering a question using references from MEDLINE [4]. This time span is, by and large, due to the process of literature appraisal, which is naturally lengthened by excessive retrieval [5]. This information discovery process is not practical for a busy clinical setting [4].

Semantic MEDLINE

Natural language processing (NLP) applications such as Semantic MEDLINE can filter PubMed results for a user's specific information need and summarize them to facilitate literature appraisal [6]. Semantic MEDLINE ([double dagger]),a resource developed by the National Library of Medicine (NLM), if enhanced by an adaptive algorithm known as Combo [7], can simplify MEDLINE results for many information needs. The user activates the Semantic MEDLINE application by submitting a search query expressing an information need to PubMed. Semantic MEDLINE then uses the individual processes of SemRep, Summarization, and Visualization to quickly transform the citations' title and abstract text into a compact form and identify data that are salient to a specific information need, which then can be displayed in a visual graph. Currently, NLM hosts an online Semantic MEDLINE application that consists of a publicly accessible demonstration site and a restricted-access portal. The following paper describes these individual processes. This study evaluated a separate, enhanced Semantic MEDLINE system that accommodates additional information needs. This paper also briefly describes how an organization could develop it to serve its own users.


SemRep [8], a rule-based NLP application in Semantic MEDLINE, interprets the meaning of PubMed title and abstract text and rephrases it into compact declarations called semantic predications. For example, consider the following citation abstract text:

"Sandoglobulin significantly reduced the incidence of pneumonia (28 cases in the IGIV group, 43 cases in the placebo group, p=0.0111)" [9]

SemRep rephrases the text with this semantic predication:


SemRep identifies "Sandoglobulin" and "Pneumonia" as the respective subject and object of the text and maps them to the Unified Medical Language System (UMLS) Metathesaurus preferred concepts "Sandolobulin" and "Pneumonia" [10]. It also recognizes the phrase "reduced the incidence of" as the concept that binds the subject and object terms, mapping it to the predicate "PREVENTS," as found in the UMLS Semantic Network. SemRep also identifies the logical UMLS semantic group classifications associated with the arguments, which in this case are "Pharmacologic Substance" (associated with "Sandoglobulin") and "Disease or Syndrome" (associated with "Pneumonia").


Semantic MEDLINE's Summarization phase identifies SemRep semantic predications that are relevant to a user's indicated information need. This process begins by prompting the user to select a topic from a list of UMLS preferred concepts that appear in the SemRep data. A summarization software application in Semantic MEDLINE processes the SemRep output according to the following sequential phases:

* Relevance gathers semantic predications containing the user-selected seed topic. For example, if the chosen topic were "Septicemia," this filter would collect the semantic predication "Blood culture_ DIAGNOSES_Septicemia."

* Connectivity augments Relevance predications with those that share a non-seed argument's semantic type. For example, in the above predication "Blood culture_DIAGNOSES_Septicemia," the semantic type of the non-seed argument "Blood culture" is "Laboratory Procedure." This filter would augment the Relevance semantic predications with others such as "Measurement of serum lipid level_DIAGNOSES_ Sepsis of the newborn," because "Laboratory Procedure" is also the semantic type of the subject argument "Measurement of serum lipid level."

* Novelty eliminates vague predications, such as "pharmaceutical preparation_TREATS_patients," that present information that users already likely know and are of limited use.

* Saliency limits final output to predications that occur with adequate frequency. For example, if "Blood culture_ DIAGNOSES_Septicemia" occurred enough times, all occurrences would be included in the final output.

To operationalize the final Saliency phase, the summarization software in this study used a statistical algorithm known as Combo. Combo [7] analyzes predicate frequencies using an adaptation of the Kullback-Leibler Divergence [11] and measures the strength of predicate/semantic type pairings with Riloff's RlogF metric [12] and PredScal, a scaling metric developed for the Combo algorithm. Prior to this approach, summarization depended on conventional, static applications, called schemas, which are limited to specified "subject_predicate_object" patterns. A different schema was required to summarize for each subheading-type refinement, limiting use to five options: treatment of disease [13], substance interaction [14], diagnosis [15], pharmacogenomics [16], and genetic etiology of disease [17]. Because of its advanced computational methodology, Combo adapts to the properties of each set of SemRep output in determining what is relevant to the user's information need, thus enabling summarization for many subheading concepts.


The semantic predications produced by the Summarization phase can be visually displayed. Figure 1 presents an interface used by NLM to display Summarization output. Due to the nature of the data's compact structure, users can quickly focus on desired data. For example, in Figure 1, the Summarization seed topic is "Septicemia," and the user has limited displayed output to items containing the predicate "DIAGNOSES." In Figure 2, the user has clicked on the arc connecting "Septicemia" and "blood culture" and is presented with the citations addressing blood culture's use as a diagnostic tool for septicemia.


The objective of this study was to evaluate the effectiveness of Semantic MEDLINE, with the Combo statistical algorithm enhancement, in identifying decision support information for disease prevention. The authors wanted to explore its potential use as a point-of-care information delivery application. They wanted to determine if this approach could retrieve recommended preventive interventions found in a commercial, manually annotated product. Prior efforts in applying Semantic MEDLINE, with the Combo algorithm, in a simulated database curation task to identify information relevant to genetic disease etiology were successful [7]. The authors wanted to evaluate the system in a simulated clinical decision support task.


The authors wanted to evaluate this system's performance in retrieving prevention information, because the concept is fluid and especially difficult to capture with such an NLP approach. For example, preventing congestive heart failure includes treating hypertension in vulnerable patients. To prevent lung cancer, clinicians counsel patients on smoking cessation. Therefore, the authors hypothesized that, in addition to finding relevant output in the form of "Intervention X_PREVENTS_Disease Y," they would also find relevant semantic predications containing other predicates, such as "TREATS. Currently, there is no conventional static schema in NLM's Semantic MEDLINE designed to accommodate a disease prevention subheading refinement. The results of this study may offer commentary on the potential enhancement offered by Combo-driven Summarization in expanding Semantic MEDLINE's functionality.

This study also served as a pilot for a larger project to examine Semantic MEDLINE's efficiency, when enhanced with the Combo algorithm, in aiding decision support for disease prevention and drug treatment. ([section])


Disease topics and data

The authors chose the three disease topics: acute pancreatitis, coronary artery disease, and malaria. These three diseases have various etiologies and call for a variety of types of preventive interventions. These differences in disease characteristics and ranges of interventions motivated their selection. The authors executed the following PubMed searches and downloaded the resulting citations.


Acute pancreatitis search session:

#11 Search #8 OR #9

#9 Search (pancreatitis/prevention and control[mesh] NOT Pancreatitis, Chronic[mesh]) AND "systematic review" Limits: Review, Publication Date to 2010/08/31

#8 Search pancreatitis/prevention and control[mesh] NOT Pancreatitis, Chronic[mesh] Limits: Clinical Trial, Meta-Analysis, Randomized Controlled Trial, Publication Date to 2010/08/31

Coronary artery disease search session:

#13 Search #10 OR #11

#11 Search coronary artery disease/prevention and control[mesh] AND "systematic review" Limits: Review, Publication Date to 2010/10/31

#10 Search coronary artery disease/prevention and control[mesh] Limits: Clinical Trial, Meta-Analysis, Randomized Controlled Trial, Publication Date to 2010/10/31

Malaria search session:

#15 Search #12 OR #13

#13 Search Malaria/prevention and control[mesh] AND "systematic review" Limits: Review, Publication Date to 2010/10/31

#12 Search Malaria/prevention and control[mesh] Limits: Clinical Trial, Meta-Analysis, Randomized Controlled Trial, Publication Date to 2010/10/31

The search sessions were conducted February 7, 2011. To garner evidence-based data, retrieval was focused on clinical trials, meta-analyses, randomized controlled trials, and systematic reviews. Retrieval was also limited to match the time period represented by the study's evaluative reference standards, as described below. Two rationales were behind the search queries structure. First, in evaluating Combo-enhanced Semantic MEDLINE for other related projects (addressing genetic disease etiology [7] and drug treatment), information retrieval for text summarization was based on a single disease topic, paired with a subheading-type concept, while drawing on all citations in the database (instead of selected intricate subsets). This provided some standardization across all projects, thus facilitating the eventual comparison of all results. Researchers combined Medical Subject Headings (MeSH) terms with subheadings, keyword phrases (e.g., "systematic reviews"), and publication types when needed. Second, this specific study was designed to simulate an instance in which a clinician would create the search query. Realistically, clinicians searching skills vary; one can expect anything from a very general keyword search to a more sophisticated search profiting from many of the PubMed value-added search tools. The search queries employed in this study represented a type of middle ground in this spectrum.

Semantic MEDLINE processing

The citations were processed with SemRep. SemRep output was processed with the Combo algorithm-enhanced Summarization application. The authors selected the following UMLS preferred concepts as seed topics for the Summarization phase:

* Pancreatitis (for the acute pancreatitis citations)

* Coronary Arteriosclerosis and Coronary heart disease (for the coronary artery disease citations)

* Malaria (for the malaria citations)


To evaluate the results, the authors compiled a reference standard for each disease, consisting of preventive interventions recommended by DynaMed, a commercial decision support product. The authors chose DynaMed because it was one of three top-ranked products in a recent study [18], presented information in a straightforward bulleted list structure, and was readily available. Preventive interventions prefaced with text such as "controversial or not well established with evidence were not included in the study's reference standards. The preventive intervention reference standards for the three disease topics are listed in Tables 1-3.

As previously mentioned, the authors noted the most recently published primary articles that DynaMed used in identifying recommendations and then limited the dates of citation retrieval to avoid including data published after DynaMed's source references. This approach to data acquisition was used in a similar study conducted by other investigators [13]. One of the authors (Workman) captured DynaMed data addressing prevention of the three diseases on February 6, 2011.

The primary analysis examined Semantic MEDLINE output in the general form "Intervention X_PREVENTS_Disease Y" for summarized output for each of the three disease topic groups, along with the associated citation from which each semantic predication originated. If a citation's text confirmed the retrieval of a reference standard intervention, it was counted as a true positive (i.e., a reference standard intervention that theoretically should have been retrieved). For example, if the citation included wording such as "[the intervention] is recommended for prevention of [the disease], the intervention received a true positive status. Because UMLS preferred concepts are sometimes broad in nature, the authors determined that if a general term was associated with citation text containing a reference standard intervention's precise wording, the reference standard would receive a true positive status (this is also demonstrated in the "Results section). All reference standard interventions not represented in the system s output were classified as false negatives (i.e., reference standard interventions that theoretically should have been retrieved by the system but were not). The authors limited the primary analysis to examining output in the form of "Intervention X_PREVENTS_Disease Y, because if a clinician were to use Semantic MEDLINE as a decision support tool for preventive care, that physician would likely begin by reviewing data with the "PREVENTS predicate. Findings were measured according to recall, precision, and F-score (the weighted harmonic mean of precision and recall). Recall consisted of the percentage of reference standard interventions found in the system output for each disease topic. Precision scores were calculated in the primary analysis by grouping the interventions in the summarized data by name and assessing what percentage of these groups led to related citation text containing a reference standard intervention.

The secondary analysis examined semantic predications that included predicates other than "PREVENTS. The authors used the same strategy of using the associated citation data to confirm a given reference standard intervention's true positive status. Because the authors primary interest was whether these additional data supplied additional reference standard interventions, these findings were factored into the final recall calculations, yielding one precision score and two recall scores for each of the three disease topics. Reference standard interventions already identified in the primary analysis received the designation "N/A," or not applicable, in the secondary analysis.


Data acquisition and processing

One of the authors (Workman) performed the information retrieval phase, SemRep processing, Summarization processing using the Combo algorithm-enhanced software, and evaluation of the output. The 3 PubMed search sessions retrieved a total of 3,276 citations; the acute pancreatitis session produced 156 citations, while the coronary artery disease and malaria sessions yielded 2,440 and 680 citations, respectively. SemRep produced 999 semantic predications from the acute pancreatitis citations, 14,781 semantic predications from the coronary artery disease citations, and 3,374 semantic predications from the malaria citations. Using the associated SemRep disease topic outputs, Summarization identified 1,397 unique semantic predications salient to the "Coronary Arteriosclerosis and "Coronary heart disease seed topics, 178 semantic predications salient to the "Pancreatitis seed topic, and 389 semantic predications salient to the "Malaria seed topic.

Evaluation: primary analysis

Semantic MEDLINE with the Combo algorithm enhancement produced an average recall of 70% in the initial examination of output in the form of "Intervention X_PREVENTS_Disease Y." The average precision was 45%, resulting in an F-score of 0.54. The primary analysis recall results for each disease topic are listed in Tables 1-3. Precision results are indicated in Table 4.

Evaluation: secondary analysis

Examination of output semantic predications containing predicates other than "PREVENTS identified additional reference standard interventions and increased average recall to 79%, with an adjusted F-score of 0.57. Reference standard results for each disease topic group are listed in Tables 1-3. Because all reference standard interventions for acute pancreatitis appeared in the primary analysis, no secondary analysis was necessary for this disease topic.


Findings of two analyses

Interesting patterns emerged from both analyses. In the primary analysis (examining output in the form "Intervention X_PREVENTS_disease Y") of the twenty-seven true positive findings for all three disease topics, eighteen were pharmaceutical-type substances or supplements in the associated reference standards. The additional nine true positives consisted of other types of interventions, ranging from behavior issues (e.g., diet) to therapeutic techniques (e.g., guidewire cannulation). In this study, Semantic MEDLINE with the Combo algorithm enhancement was more efficient at expressing preventive drug and supplement interventions with the "PREVENTS predicate than for other kinds of interventions.

The secondary analysis confirmed the hypothesis that some reference standard interventions would be expressed with predicates other than "PREVENTS. The secondary analysis found two of the six interventions not found in the primary analysis for coronary artery disease and four of the thirteen interventions not located for malaria. The relevant semantic predications located in the secondary analysis included the following.

Coronary artery disease:

Diabetic care_USES_Glucose control

Secondary prevention_TREATS_Coronary arteriosclerosis ("Secondary prevention" referencing smoking cessation)


Prophylactic treatment_USES_Amodiaquine

Prophylactic treatment_USES_Artesunate

Prescription of prophylactic anti-malarial_USES_ Pyrimethamine

Malaria Vaccines_TREATS_Child

Malaria Vaccines_TREATS_Infant

As noted earlier, all reference standard interventions for acute pancreatitis were found in the primary analysis.

As predicted, in some cases in both analyses, raw Semantic MEDLINE output did not precisely identify a reference standard item, but the associated citation text named the specific intervention. For example, the semantic predication, "Cannulation_PREVENTS_Pancreatitis," does not specifically name guidewire cannulation for acute pancreatitis; however, the associated citation text, "GW [guidewire] cannulation is associated with a higher cannulation success rate and less PEP [post-ERCP pancreatitis] after pancreatic duct entry [19], identifies the specific cannulation technique corresponding to the reference standard intervention. Nevertheless, for a reference standard intervention to receive true positive status, the specific intervention had to be named in the citation text. For example, in multiple instances, "exercise was mentioned as a preventive intervention in citations associated with the system output for coronary artery disease. Because the precise term "aerobic exercise did not occur, the reference standard intervention aerobic exercise received a false negative status for recall assessment. To fully utilize Semantic MEDLINE with the Combo enhancement as a decision support tool, a clinician should consult the system's output of semantic predications and its associated citation text. An ideal interface would likely combine both, allowing the user to simultaneously review interesting semantic predications and their associated citations.

Precision and variety of output

The performance scores reflect in part the percentage of reference standard interventions included in the output. However, a clinician may find the additional preventive interventions mentioned in Semantic MEDLINE's output useful. For example, the reference standard for acute pancreatitis prevention included five interventions (Table 1). Semantic MEDLINE additionally identified antibiotic prophylaxis [20] and ulinastatin [21] as potential preventive interventions, based on the findings of randomized controlled trials. The associated DynaMed text does not discuss these potential interventions. However, other interventions in Semantic MEDLINE's output may not suit a clinical need. For example, Semantic MEDLINE also identified nafamostat mesilate [22] as a potential preventive intervention. The associated citation text notes that this intervention is "partially effective and highlights independent risk factors associated with the disease. It is again recommended that a Semantic MEDLINE user consult the citation text (and the original article, if desired) associated with a semantic predication, to assess the relevance and strength of evidence pertaining to the original information need. Ideally, an interface (such as the one used by NLM) would present citation text with its associated semantic predication for simultaneous viewing, along with immediate access to the original PubMed record, where links to full text might be present.


Based on these findings, Semantic MEDLINE with the Combo algorithm enhancement potentially serves as a decision support resource. It is a flexible approach to point-of-care information delivery that could be integrated into multiple environments. The authors developed the summarization software with Perl, an interpreted programming language that is compatible with multiple platforms. This Perl application provided adequate computing speeds for this project; however, to increase speed, the software could also be coded with a compiled language like Java. A locally accessible database of SemRep output for several years worth of MEDLINE data is also needed (for a more detailed description of how the system works, please see Workman and Hurdle [7]). Currently, there is not a publicly accessible, Combo-enhanced Semantic MEDLINE web portal; however, this paper provides a brief description of how a library could customize its own application to serve its particular clients.

While a robust Combo-enhanced Semantic MEDLINE is still under development, it offers interesting options for customized search systems. Even though there is no existing application outside of NLM, the framework for a system that accommodates even more information needs exists and could be translated into a product that suits an organization's particular requirements. Libraries could partner with the organizations they serve to customize Combo-enhanced Semantic MEDLINE for their specific user groups. For example, a library serving a health care organization could conduct user studies for various clientele groups to determine their information needs and preferences. The outcomes of these user studies would enable a web designer to tailor a graphic interface for each user group. The designer could create an interface for consumers and patients, using the simplified, summarized output as a means to assist users in navigating and understanding PubMed citation text. Another interface could assist clinicians in executing searches and accessing desired data on a single screen, organized according to their collective preferences and workflow-driven needs. Because Semantic MEDLINE, with the Combo algorithm enhancement, is a dynamic application, users would be free to build and execute their own searches. Resources would be needed (e.g., a trained web designer, hardware, and software) to create a system customized for an institution's needs. A parent organization such as a hospital or health care system should contribute these resources if the sponsoring library cannot.

Combo-enhanced Semantic MEDLINE could either complement existing decision support products or stand alone. Because it automatically produces information relevant to multiple topics and subheading refinements, this application can potentially address the information needs of many individual users. A technician could implement the Summarization software, SemRep semantic predication database, and desired interface to serve clients information needs. No subscription or licensing fees would be required. Each decision support application contributes point-of-care information in its distinctive way. Each product also has requirements that enable its practical use. Commercial products often require payment of very expensive fees and possibly some onsite technical support. At present, Combo-enhanced Semantic MEDLINE would require substantial onsite technical support to establish the customized, user-centered application described in this paper. Organizations should consider their own resources and needs in choosing what value-added products they provide to their clientele.

This study is an example of a technology created in part by librarians and demonstrates a new, dynamic approach to information delivery. It surpasses the functionality of simple information retrieval, freeing users from the difficult, unrealistic task of reviewing many citations, providing instead compact summarizations of text that have been filtered for individual information needs. This approach to information delivery could reinforce the importance of libraries as vital components in the organizations they serve.


This study has limitations that warrant mention. It examined the performance of Combo algorithm-enhanced Semantic MEDLINE in terms of three disease topics, for a single subheading-type refinement. However, in an earlier study [7], the application demonstrated improved performance for a different disease topic (bladder cancer) and subheading-type refinement (genetic disease etiology) over Semantic MEDLINE with conventional, static schema summarization. Other recent research has also examined Combo-enhanced Semantic MEDLINE s performance while processing data for additional disease topics and an additional subheading refinement, with positive results. As previously noted, a manuscript describing this larger project has been submitted to another publication. The authors evaluated output using recommendations found in a single product (DynaMed). Similar comparisons using other commercial decision support products may shed additional light on the application s performance.


* PubMed data can potentially serve clinicians at the point of care.

* Semantic MEDLINE, when enhanced with the Combo algorithm, can summarize PubMed results to provide information needed at the point of care.


* Natural language processing applications that summarize text may facilitate appraisal of PubMed results for automated decision support.

* Natural language processing applications offer an opportunity for libraries to provide value-added information delivery.

* Natural language processing applications can be tailored to serve specific user groups.

* Combo-enhanced Semantic MEDLINE could complement commercial decision support products or independently provide point-of-care information.

DOI: http://dx.doi.Org/10.3163/1536-5050.100.2.009


The authors express gratitude to Dr. Thomas Rindflesch and Dr. Marcelo Fiszman for their essential work in text summarization. They also thank the National Library of Medicine for funding this work through grant number T15LM007123.


[1.] Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians clinical questions: obstacles and potential solutions. J Am Med Inform Assoc. 2005 Mar-Apr;12(2):217-24.

[2.] Chambliss ML, Conley J. Answering clinical questions. J Fam Pract. 1996 Aug;43(2):140^.

[3.] Golder S, McIntosh HM, Duffy S, Glanville J. Developing efficient search strategies to identify reports of adverse effects in MEDLINE and EMBASE. Health Info Lib J. 2006 Mar;23(1):3-12.

[4.] Hersh WR, Hickam DH. How well do physicians use electronic information retrieval systems? a framework for investigation and systematic review. JAMA. 1998 Oct 21;280(15):1347-52.

[5.] Hoogendam A, Stalenhoef AF, Robbe PF, Overbeke AJ. Analysis of queries sent to PubMed at the point of care: observation of search behaviour in a medical teaching hospital. BMC Med Inform Decis Mak. 2008;8:42.

[6.] Fiszman M, Rindflesch TC, Kilicoglu H. Abstraction summarization for managing the biomedical research literature. Proceedings of the HLT-NAACL Workshop on Computational Lexical Semantics; 2004. p. 76-83.

[7.] Workman TE, Hurdle JF. Dynamic summarization of bibliographic-based data. BMC Med Inform Decis Mak. 2011;11:6.

[8.] Rindflesch TC, Fiszman M. The interaction of domain knowledge and linguistic structure in natural language processing: interpreting hypernymic propositions in bio medical text. J Biomed Inform. 2003 Dec;36(6):462-77.

[9.] Glinz W, Grob PJ, Nydegger UE, Ricklin T, Stamm F, Stoffel D, Lasance A. Polyvalent immunoglobulins for prophylaxis of bacterial infections in patients following multiple trauma. a randomized, placebo-controlled study. Intensive Care Med. 1985;11(6):288-94.

[10.] Lindberg DA, Humphreys BL, McCray AT. The Unified Medical Language System. Methods Inf Med. 1993 Aug;32(4):281-91.

[11.] Kullback S, Leibler RA. On information and sufficiency. Ann Math Stat. 1951;22(1):79-86.

[12.] Riloff E. Automatically generating extraction patterns from untagged text. Proceedings of the Thirteenth National Conference on Artificial Intelligence. Menlo Park, CA: The AAAI Press/MIT Press; 1996. p. 1044-9.

[13.] Fiszman M, Demner-Fushman D, Kilicoglu H, Rindflesch TC. Automatic summarization of MEDLINE citations for evidence-based medical treatment: a topic oriented evaluation. J Biomed Inform. 2009 Oct;42(5):801-13.

[14.] Fiszman M, Rindflesch TC, Kilicoglu H. Summarizing drug information in MEDLINE citations. AMIA Annu Symp Proc. 2006:254-8.

[15.] Sneiderman C, Demner-Fushman D, Fiszman M, Rosemblat G, Lang FM, Norwood D, Rindflesch TC. Semantic processing to enhance retrieval of diagnosis citations from MEDLINE. AMIA Annu Symp Proc. 2006. p. 1104.

[16.] Ahlers CB, Fiszman M, Demner-Fushman D, Lang FM, Rindflesch TC. Extracting semantic predications from MEDLINE citations for pharmacogenomics. Pac Symp Biocomput. 2007:209-20.

[17.] Workman TE, Fiszman M, Hurdle JF, Rindflesch TC. Biomedical text summarization to support genetic database curation: using Semantic MEDLINE to create a secondary database of genetic information. J Med Lib Assoc. 2010 Oct;98(4):273-81. DOI: -5050.98.4.003.

[18.] Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res. 2010;12(3):e26.

[19.] Cheung J, Tsoi KK, Quan WL, Lau JY, Sung JJ. Guidewire versus conventional contrast cannulation of the common bile duct for the prevention of post-ERCP pancreatitis: a systematic review and meta-analysis. Gastro intest Endosc. 2009 Dec;70(6):1211-9.

[20.] Raty S, Sand J, Pulkkinen M, Matikainen M, Nordback I. Post-ERCP pancreatitis: reduction by routine antibiotics. J Gastrointest Surg. 2001 Jul-Aug;5(4):339-45; discussion: 45.

[21.] Tsujino T, Komatsu Y, Isayama H, Hirano K, Sasahira N, Yamamoto N, Toda N, Ito Y, Nakai Y, Tada M, Matsumura M, Yoshida H, Kawabe T, Shiratori Y, Omata M. Ulinastatin for pancreatitis after endoscopic retrograde cholangiopancreatography: a randomized, controlled trial. Clin Gastroenterol Hepatol. 2005 Apr;3(4):376-83.

[22.] Choi CW, Kang DH, Kim GH, Eum JS, Lee SM, Song GA, Kim DU, Kim ID, Cho M. Nafamostat mesylate in the prevention of post-ERCP pancreatitis and risk factors for post-ERCP pancreatitis. Gastrointest Endosc. 2009 Apr; 69(4):e11-8.


T. Elizabeth Workman, PhD, MLIS,, Postdoctoral Research Associate, Department of Biomedical Informatics, University of Utah, HSEB 5775, 26 South 2000 East, Salt Lake City, UT 84112; Joan M. Stoddart, MALS, AHIP,, Deputy Director, Spencer S. Eccles Health Sciences Library, University of Utah, 10 North 1900 East, Salt Lake City, UT 84112

Received July 2011; accepted November 2011

* Based on a presentation given at MLA '11, the 111th annual meeting of the Medical Library Association; Minneapolis, MN; 15 May 2011.

([dagger]) Funded by the National Library of Medicine, grant number T15LM007123.

[IRP] This article has been approved for the Medical Library Association's Independent Reading Program <http://www.mlanet .org/education/irp/>.

([double dagger]) A public demonstration of the National Library of Medicine's Semantic MEDLINE, without the Combo enhancement, is available at

([section]) The first author and others describe the results of the "text summarization as a decision support aid project in a manuscript currently under review elsewhere.
Table 1
Semantic MEDLINE recall * of DynaMed preventive intervention
reference standard for acute pancreatitis ([dagger])

                                       Recall in Semantic MEDLINE

                                         Primary       analysis
                                         analysis       (other
DynaMed prevention intervention         (PREVENTS)    predicates)

Guidewire cannulation                       TP            N/A
Nonsteroidal anti-inflammatory
  drugs (NSAIDs)                            TP            N/A
Octreotide                                  TP            N/A
Prophylactic nitroglycerin                  TP            N/A
Interleukin 10 (IL-10)                      TP            N/A
                                       Recall: 100%

* Recall is the percentage of reference standard interventions found
in the system output for each disease topic.

([dagger]) TP=true positive; FN=false negative; N/A=not applicable,
found in primary analysis.

Table 2
Semantic MEDLINE recall * of DynaMed preventive intervention
reference standard for coronary artery disease ([dagger])

                                      Recall in Semantic MEDLINE

                                        Primary      analysis
                                       analysis       (other
DynaMed prevention intervention       (PREVENTS)    predicates)

Proper diet                               TP            N/A
Aerobic exercise                          FN            FN
Smoking cessation                         FN            TP
Modifiable lifestyles                     TP            N/A
Weight loss                               TP            N/A
Treatment of diabetes                     FN            TP
Treatment of hypertension                 TP            N/A
Treatment of hyperlipidemia               TP            N/A
Prophylactic low-dose aspirin             TP            N/A
Use of ACE inhibitors                     TP            N/A
Complete avoidance of tobacco smoke       FN            FN
Angiotensin receptor blockers             TP            N/A
Aldosterone blockade                      FN            FN
Beta blockers                             TP            N/A
Influenza vaccine                         FN            FN
                                      Recall: 60%   Recall: 73%

* Recall is the percentage of reference standard interventions found
in the system output for each disease topic.

([dagger]) TP=true positive; FN=false negative; N/A=not applicable,
found in primary analysis.

Table 3
Semantic MEDLINE recall * of DynaMed preventive intervention
reference standard for malaria ([dagger])

                                       Recall in Semantic MEDLINE

                                         Primary      analysis
                                        analysis       (other
  DynaMed prevention intervention      (PREVENTS)    predicates)

Long sleeves                               FN            FN
Long pants                                 FN            FN
Window screens                             FN            FN
Mosquito nets                              TP            N/A
Insecticide-treated clothes                FN            FN
Insecticide-treated nets                   TP            N/A
Insect repellent                           TP            N/A
Indoor spraying                            FN            FN
Insecticide treatment of livestock         FN            FN
Atovaquone/proguanil                       TP            N/A
Trimethoprim-sulfamethoxazole              FN            FN
"Antimalarial agents"                      TP            N/A
Artesunate plus amodiaquine or
  sulfadoxine-pyrimethamine                FN            TP
Mefloquine                                 TP            N/A
Sulfadoxine-pyrimethamine                  TP            N/A
Amodiaquine                                TP            N/A
Pyrimethamine plus dapsone                 FN            TP
Routine malaria chemoprophylaxis
  (i.e., during pregnancy)                 TP            N/A
Chloroquine                                TP            N/A
Recombinant vaccine based on fusion
  of circumsporozoite protein and
  HBsAg                                    FN            FN
RTS,S/AS02 (vaccine)                       FN            FN
RTS,S/ASO2A (vaccine)                      TP            N/A
RTS,S/AS01E (vaccine)                      FN            TP
RTS,S/AS02D (vaccine)                      FN            TP
MSP/RESA (vaccine)                         TP            N/A
Vitamin A supplementation                  TP            N/A
                                       Recall: 50%   Recall: 65%

* Recall is the percentage of reference standard interventions found
in the system output for each disease topic.

([dagger]) TP=true positive; FN=false negative; N/A=not applicable,
found in primary analysis.

Table 4
Precision results by disease topic, from primary analysis of data
using DynaMed reference standards

Disease topic               Precision

Acute pancreatitis             29%
Coronary artery disease        45%
Malaria                        61%
Average precision              45%
COPYRIGHT 2012 Medical Library Association
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2012 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Workman, T. Elizabeth; Stoddart, Joan M.
Publication:Journal of the Medical Library Association
Article Type:Report
Geographic Code:1USA
Date:May 1, 2012
Previous Article:Provision of pandemic disease information by health sciences librarians: a multisite comparative case series.
Next Article:Comparing patient characteristics, type of intervention, control, and outcome (PICO) queries with unguided searching: a randomized controlled...

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters