Chapter 11Clinical Decision Support Systems

Wasylewicz ATM, Scheepers-Hoeks AMJW.

Publication Details

Clinical decision support (CDS) includes a variety of tools and interventions computerized as well as non- computerized. High-quality clinical decision support systems (CDSS), computerized CDS, are essential to achieve the full benefits of electronic health records and computerized physician order entry. A CDSS can take into account all data available in the EHR making it possible to notice changes outside the scope of the professional and notice changes specific for a certain patient, within normal limits. However, to use of CDSS in practice, it is important to understand the basic requirements of these systems.

This chapter shows in what way CDSS can support the use of clinical data science in daily clinical practice. Moreover, it explains what types of CDSS are available and how such systems can be used. However, to achieve high-quality CDSS which is effective in use requires thoughtful design, implementation and critical evaluation. Therefore, challenges surrounding implementation of a CDSS are discussed, as well as a strategies to develop and validate CDSS.

Keywords:

Clinical decision support systems, CDSS, Clinical data science, Validation

11.1. Introduction on CDSS

11.1.1. What Is CDSS?

Clinical decision support includes a variety of tools and interventions, computerized as well as non- computerized. Non-computerized tools include clinical guidelines or digital clinical decision support resources like ClinicalKey® or UpToDate ® [1, 2]. Such clinical decision support systems (CDSS) are characterized as tools for information management. Another category of CDSS sometimes also called basic or simple clinical decision support systems are tools to help focus attention. Examples of such CDSS include laboratory information systems (LISs) highlighting critical care values or pharmacy information systems (PISs) presenting an alert ordering a new drug and proposing a possible drug-drug interaction [3, 4]. Most focus in the past few decades however has gone to tools to provide patient-specific recommendations called advanced CDSS. Advanced CDSS mayinclude, for example, checking drug disease interactions, individualized dosing support during renal impairment, or recommendations on laboratory testing during drug use.

11.1.2. Why CDSS?

The quantity and quality of clinical data are rapidly expanding, including electronic health records (EHRs), disease registries, patient surveys and information exchanges. Big data and digitalization however, does not automatically mean better patient care. Several studies have shown that only implementing an EHRand computerized physician order entry (CPOE) has rapidly decreased the incidence of certain errors, introducing however many more [5–7]. Therefore, high-quality clinical decision support is essential if healthcare organizations are to achieve the full benefits of electronic health records and CPOE. In the current healthcare setting when facing a decision, healthcare providers often do not know that certain patient data are available in the EHR, do not always know how to access the data, do not have the time to search for the data or are not fully informed on the most current medical insights. It is said the healthcare providers often drown in the midst of plenty [8–10].

Moreover, decisions by healthcare professionals are often made during direct patient contact, ward rounds or multidisciplinary meetings. This means that many decisions are made in a matter of seconds or minutes, and depend on the healthcare provider having all patient parameters and medical knowledge readily available at that time of the decision. Consequently, current decisions are still strongly determined by experience and knowledge of the professional. Also, subtle changes in a patient’s condition taking place before hospital- or ward admission are often overlooked because clinicians regularly perceive a patient in his current state without taking into account changes within normal range. A computer however, takes into account all data available making it also possible to notice changes outside the scope of the professional and notices changes specific for a certain patient, within normal limits.

11.1.3. Types of CDSS

To understand literature on the topic of CDSS and familiarize oneself on the subject it is important to categorize the vast array of CDSS. Categorization of CDSS is often based on the following characteristics: system function, model for giving advice, style of communication, underlying decision making process and human computer interaction which are briefly explained below [11].

The characteristic ‘System function’ distinguishes two types of functions. Systems determining: what is true?: These include purely diagnostic CDSS like many popular differential diagnosis websites like Diagnosaurus® or WebMD® [12, 13]. These CDSS base their advices on a fixed set of data that is user inputted or readily available. The other type of CDSS determine: what to do?, advising which test to order with the purpose of further differential diagnosis or which drug to prescribe for the patient’s current condition. However, this distinction is of limited value as most current integrated CDSS almost always do both: first determine what is true about a patient and then suggest what to do.

Another parameter of CDSS is the approach to give advice, either passive or active. Passive CDSS require the user to do something to receive advice, for example clicking a button or opening a tab. These passive types however, have been abandoned for most part because of their lack of efficacy and dependence of human involvement [14, 15]. A challenge of active systems is to avoid the generation of excessive amount of alerts, causing alert fatigue with the user. This topic is discussed further on in the paragraph on alert fatigue. A closely related characteristic commonly used to categorize CDSS is thestyle of communication, distinguishing a consulting and critiquing model. In a consulting model the system is an advisor, asking questions and proposes subsequent actions. For example, when entering a medication order, the computer asks for the diagnosis and advises the right dose or an alternative treatment. A critiquing system lets the user decide the right dose for itself and only afterwards alerts the user that the dose prescribed for this therapy is too low.

Human computer interaction is another clinical decision support system characteristic. How does a user interact with the computer? Historically CDSS were slow, difficult to access and difficult to use. However, modern day computing power, electronic health record integration and computer mobility have made these problems of the past. However, human computer interaction is still a good way to categorize CDSS describing EHR integration or overlay, keyboard or voice recognition and advice by means of pop-ups, acoustic alarms or messaging systems.

The last commonly used characterization of CDSS, and perhaps the most interesting, is the underlying decision-making process ormodel. The simplest models are problem-specific flowcharts encoded for computerized use, these are discussed further on. With the availability of additional statistical models, mathematical techniques and increasing computing power, much more complex models have been researched and used since, like Bayesian models [16, 17], artificial neural networks [18], support vector machines [19] and artificial intelligence [20]. Many of these systems are used to improve prediction of outcome, prioritize treatment or help choosing the best course of action. Use of such systems in practice however is delayed mainly because of trust issues towards ‘black box’ systems. If a computer tells you to start drug A for a patient based solely on a mathematical model, without a guideline to back it up, are you convinced to do it? Linked to the major trust issue towards ‘black box’ systems is the current model of evidence based medicine and concurrent guidelines based on these studies. Are you willing to ignore an international guideline saying you should start a patient on drug A only because your CDSS says you should start the patient on drug B?

Decision tree models are the oldest but still most used models in clinical practice today. These CDSS use a tree-like model of decisions consisting of multiple steps of ‘if then else’ logic. Figure 11.1 shows an example of such a decision tree model. These models have the advantage of being interpretable by humans and follow logical steps based on conventional medical guidelines. Such decision tree models are also called clinical rules (CRs), computer-interpretable guidelines (CIGs) or decision support algorithms. [15] Instead of predicting outcome or best therapy, a CDSS only automatizes information gathering and provides advice in accordance to a guideline.

Fig. 11.1. Part of the clinical rule gastric protection, represented in GLIF, created in CDSS Gaston (Medecs BV).

Fig. 11.1

Part of the clinical rule gastric protection, represented in GLIF, created in CDSS Gaston (Medecs BV). (Picture adopted from Scheepers et al. 2009 [14])

The next few paragraphs will focus on CDSS that determine both what is true? and what to do?, as well as the use of mainly active critiquing advice and the use of a decision tree model as underlying decision making process.

11.1.4. Medication Related CDSS

From a historical point of view, medication related CDSS seem to go the farthest back and are likely to have the largest potential for benefit [21]. They date back as long as the 1960s [22]. They supported pharmacists with drug allergy checking, dose guidance, drug-drug interaction checking and duplicate therapy checking. Medication related CDSS took further shape when directly linked to computerized physician order entry (CPOE) [23]. CPOE being the system that enabled physicians to prescribe medication using electronic entry. The combination of CPOE and CDSS helped physicians choose the right drug in the right dose and alert the physician during prescribing if for example the patient is allergic. Combining CPOE with basic medication related CDSS meant a giant leap in safer medication prescribing [24, 25]. However, all of the checks mentioned above follow simple ‘if then else’ logic and do not combine multiple patient characteristics when producing alerts. This addition came with the introduction of advanced medication related CDSS.

Such advanced CDSS follow decision tree based models and can assist the physician in dosing medication for patients with renal insufficiency, provide guidance for medication-related laboratory testing and perform drug – disease contraindication checking [23, 26]. Parameters incorporated into medication related CDSS rose steadily in the past few decades including pharmacogenetics and more and more drug disease interactions.

Many current EHRs with integrated CDSS however, still fail to provide guidance relevant to the specific patient receiving care, poorly presenting data and causing alert fatigue to health care providers [27]. One of the main issues with these systems is that they combine only one or two parameters to provide alerts, thereby only increasing the number of alerts. For example, prescribing nortriptyline to a patient with hepatorenal syndrome and being an intermediate metabolizer of CYP2D6 will generate a total of 3 alerts with different advices. An advice on how to dose nortriptyline in a patient with renal insufficiency, another alert with an advice how to dose nortriptyline in patients with liver failure and last but not least an advice how to start treatment in a patient being an CYP2D6 intermediate metabolizer. So which advice should we follow? Therefore, effort should be made into combining multiple parameters and clinical rules to provide one correct advice to the healthcare provider. Designs should incorporate the engagement of all clinicians involved in the delivery of health care and combine multiple patient characteristics and context simultaneously, to ensure that CDSS are actually helpful to clinicians, rather than interrupt health care delivery.

11.2. Challenges for Implementing a CDSS

CDSS are an evolving technology with potential for wide applicability, to individualize and improve patient outcome and health care resource utilization [24, 28]. However, to make CDSS more helpful it requires thoughtful design, implementation and critical evaluation [29].

As mentioned earlier thepromise of CDSS has been around since the 1960s. In 2008, Simon et al. still found that the vast majority of EHRs across the U.S.A. implemented little or any decision support [30]. A recent survey send out to all Dutch hospital pharmacies showed similar disappointing results, only 48% of them using some kind of advanced CDSS [31].

Such alarming results were one of the main reasons the American Medical Informatics Association (AMIA) published the Roadmap for National Action on Clinical Decision Support. The paper acknowledged six strategic objectives, divided into three main pillars, for achieving widespread adoption of effective clinical decision support system capabilities [32]. The three main pillars being: (1) High Adoption and Effective Use. (2) Best Knowledge Available When Needed. (3) Continuous Improvement of Knowledge and CDSS Methods [32]. In the following paragraphs these three pillars will be highlighted to give an overview of tasks and challenges that lay ahead.

11.2.1. High Adoption and Effective Use

To ensure high adoption and effective use, it is important to fine-tune the CDSS in order to suit end-users wishes. Only then alert fatigue can be minimized.

11.2.1.1. Alert Fatigue

Alert fatigue is the concept of poor signal to noise ratio caused by CDSS with an active alerting mechanism. Alert fatigue is defined as the “Mental fatigue experienced by health care providers who encounter numerous alerts and reminders from the use of CDSS” [33]. Alert fatigue causes physicians to override 49–96% of the current medication safety alerts from basic CDSS as well as advanced medication related CDSS. The main reasons for overriding alerts are: low specificity, unnecessary workflow disruption and unclear information [34, 35]. Many of these aspects are caused by lack of user- and patient context. More on the subject of context can be read in the paragraph on context factors, later on.

Because CDSS are offering more and more options characterization of the CDSS itself is not enough. Characterization of the clinical rules used by decision tree CDSS is also key to understand the background of alert fatigue. In the upcoming paragraphs the taxonomy of clinical rules is explained using two fundamental concepts, being triggers and context factors.

11.2.1.2. Triggers

In an effort to characterize clinical rules, Wright et al. used four functional categories: triggers, input data, interventions and offered choices. Triggers were identified as one of the key functional dimensions of CDSS and are the start of each clinical rule. Wright and colleagues reviewed and analyzed their own extensive rule repository, using these four functional dimensions to identify and quantify the use of different taxonomic groups. They identified nine different triggers. However, by far the trigger most often used is the ‘order entered’ trigger, accounting for 94% of all the studied clinical rules and 38% of all clinical rule types. Combined with the knowledge that a patient’s drug list is also the most used ‘input data element’ in all of the studied rules, medication orders (MOs) and drug lists seem to play a key role in CDSS currently used [36, 37].

11.2.1.3. Context Factors

‘Context’ , in computer science, refers to the idea that a system, in our case a clinical decision support system, is both capable of sensing and reacting, based on its environment. An often provided definition of the term ‘context’ is the one provided by Dey, being: “Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves”. Using this definition a system providing ‘context’ also tries to make assumptions about the current situation in relevance, dependent on the user’s task or patient’s status [38].

Riedmann et al. performed a review of literature and subsequently performed an international Delphi study to identify the most important context factors to medication related CDSS [39, 40]. The most important context factors found were ‘severity of the effect’, ‘clinical status of the patient’, ‘complexity of the case’ and ‘risk factors of the patient’. All of these context factors are gained from input data elements such as diagnosis, prior disease history, laboratory results and hospital unit [36].

Another study group of Berlin et al. found that the most targeted clinical tasks of clinicians were associated with drug dosing (46%) and drug treatment (22%) [41, 42]. These findings are in agreement with the study of Wright et al. although using a completely different taxonomy [41].

When combining the results from the studies performed by Wright et al. and Berlin et al., the most CDSS targeted clinical tasks were ‘start of treatment’ and ‘dose adjustment’. As stated earlier, medication ordering was the most frequently used trigger to a clinical rule and a patient’s drug list was the utmost used and most easily available input element. Therefore, providing the right context to medication orders using the drug list should be an important priority. Context factors like ‘severity of the effect’, ‘clinical status of the patient’, ‘complexity of the case’ and ‘risk factors of the patient’ found by Riedmann et al are logical context factors from a physician’s point of view. However, adding such context only adds value when trigger related contexts like ‘start of treatment’ and ‘dose adjustment’ are also included. Moreover, data input like those described by Riedmann et al is not always distinct and readily available in the EHR [36, 39, 41].

In our own experience, gained in the Netherlands, integrated medication related CDSS are still unable to correctly interpret the simple contexts of medication orders. During development and validation of clinical rules, basic contexts like start of new treatment or dose adjustment proved to be elusive and are a frequent cause of suboptimal positive predictive value (PPV) and sometimes suboptimal negative predictive value (NPV). Experts also frequently disagree upon the definitions and clinical relevance of these contexts [43, 44]. Is a medication order a dose adjustment or start of new treatment? An example is a digoxin order. If the clinical task would be starting a patient on digoxin therapy, the CDSS should advice the prescriber on ordering serum potassium levels, perform therapeutic drug monitoring and review new drug-drug interactions. However, entering the same digoxin order to change drug administration time or change drug form, the above monitoring is not applicable. Providing the physician or pharmacist with notifications during this process would cause frustration and alert fatigue [45].

11.2.2. Best Knowledge Available when Needed

The second pillar in the Roadmap provided by the AMIA is best knowledge available when needed. The pillar contains three key challenges:

  • When needed: Integration in clinical workflow
  • Knowledge is available: so it has to be written, stored and transmitted in a format that makes it easy to build and deploy CDSS interventions
  • Best knowledge: Only CDSS which provides current and additional information has potential

11.2.2.1. When Needed: Integration in Clinical Workflow

A key success factor of CDSS is that they are integrated into the clinical workflow. CDSS not integrated into clinical workflow will have no beneficial effect and will not be used [46]. Messages should be presented at the moment of decision-making, though with as less disturbance for the physician as possible. Therefore, different alert mechanisms (pop-up, automatic lab order, prescription order, emails, etc.) should be developed, suitable for different alerting priorities [47]. Understanding how to prompt physicians successfully at the point of care is a complex problem, and requires consideration of technological, clinical, and socio-technical issues. As mentioned earlier, interruptive (active) alerts show significantly higher effectiveness than non-interruptive (passive) reminders [48]. Additionally, a greater positive impact was observed when recommendations prompted an action and could not be ignored [49]. Thoroughly understanding the clinical workflow and users’ wishes strongly increases the probability for success [49]. One of the more recent attempts to incorporate CDSS into clinical workflow was to incorporate CDSS advice into checklists often used in ward rounds [50]. An example of such a particular system is Tracebook. This is a process-oriented and context-aware dynamic checklist, showing great promise and good user acceptability [51].

11.2.2.2. Knowledge Is Available

One of the other major challenges of effective CDSS adaptation is keeping theclinical rules up to date [49]. However, keeping these clinical rules up to date is a massive time and money-consuming task. Therefore, sharing clinical rules seems to be a sensible and financially attractive choice. One of the strategic objectives described in the roadmap was to create a way to easily distribute, share and incorporate clinical knowledge and CDSS interventions into own information systems and processes. With this concept clinical rules could be externally maintained, making a huge leap in efficacy of development and maintenance. A healthcare provider could then just subscribe to certain clinical rules. This should work in “such a way that healthcare organizations and practices can implement new state of the art clinical decision support interventions with little or no extra effort on their part” [32].

Today many clinical rule repositories exist, however none of them are fully functioning. They rely on software vendors to rebuild them into their own CDSS modules. Progress on this objective has been especially problematic when attempting to make or share clinical rules outside an ecosystem of the software vendor [52]. The progress being made using integrated EHR systems, also called second phase CDSS, is commendable however; it strictly limits sharing clinical rules outside of the EHR ecosystem. Newer standards-based systems, third phase and service model systems like the Arden syntax, GLIF, SAGE and SEBASTIAN solve many issues concerning sharing clinical rules [53, 54]. Although all very good initiatives, none of the architectures have really found use in clinical practice.

One of the issues in sharing fully functioning clinical rules are the difference in clinical terms as well as language. Clinicians starting to program clinical rules should keep in mind using standardized terms to make exchange of their CDSS modules possible. Using standardized clinical health terminologies like SNOMED CT would resolve a lot of issues surrounding sharing CDSS [55].

One of the other challenges however is to standardize definitions of context, as these are essential to minimize signal to noise ratio. To study the obstacles left to make sharing a reality, an initiative was started to develop clinical rules which would work across different EHRs, CPOEs, PISs and institutions using the GASTON framework [56]. The framework, derived from GLIF architecture, facilitates sharing guidelines and facilitates integration with institution specific medical knowledge sources and information systems such as EHRs and CPOEs without changing the clinical rules themselves. The most important lesson learned from this project was that despite consensus on the content of a clinical rule, local adaptation was always necessary to achieve sufficient specificity of the alerts.

11.3. Best Knowledge & Continuous Improvement of Knowledge and CDSS Methods

To ensure the best knowledge and retain continuous improvement, validation and verification is indispensable. Much research has been done on the validation of clinical rules itself and focuses on clinical relevance of the recommendations produced by the CDSS. However, to assure correct clinical rules and recommendations we depend on data from the EHR and the correct functioning of the clinical decision support system. The next few paragraphs will give an overview over the levels of validation and verification of CDSS.

11.3.1. CDSS Verification and Validation

Successful adaption and functioning of clinical rules vastly depends on the CDSS used. Tendering, choosing or implementing a new CDSS requires a comprehensiveuser requirement specification (URS) or user requirement documentation (URD). A URS specifies what the users of the software expect the software to do. It is often seen as the contract between the user and the software supplier. Not explicitly or correctly stating user requirements for a software system is the major factor contributing to failed software implementations and massive budget overruns. Maybe not a very appealing job for clinicians, we cannot stress enough the importance of working together with IT personnel to write an all-encompassing URS. Adding or improving functionality afterwards is difficult and costly.

It is important to test all functions of software products such as CDSS. Deepening the topic of software verification and validation requires a book on its own. However, to prevent running into issues during clinical rule development and use of the CDSS in practice it is key to perform software verification and validation using the URS and lower level specifications. Software validation and verification can be performed at many levels using many tools. If your hospital does not have IT personal qualified to plan and perform software verification and validation it is highly recommended to hire external help. Thorough verification and validation of the CDSS software can save expenses and spare frustration later on or even failure of implementation.

When using a CDSS we should keep in mind that a CDSS relies on high quality data to work. Assuring the correct collection of data and their quality is vital before starting to program the clinical rules themselves. A part of the requirements should therefore be a thorough description and testing of items to be used in the clinical rules. If you state: “the system must present the age of a patient” for example; the CDSS probably will present the age of the patient in years. Designing clinical rules using this parameter however for a neonatal care unit could be unwanted and unspecific. Testing if items used in clinical rules result in the expected answer requires clinical knowledge, often scares IT personnel. Clinicians eager to program clinical rules themselves are therefore encouraged to assist in this stage of CDSS validation.

After the successful implementation of the CDSS itself we are ready to start building our own clinical rules.

11.3.2. Development and Validation Strategy

Key to preventing alert fatigue in active CDSS is structured development and validation of clinical rules . Much has been published on the validation of these clinical rules focusing on providing maximal clinical relevance of the recommendations outputted by the CDSS [47, 57–59].

Two key components of a good validation strategy described in most studies are: (1) the use of a multidisciplinary expert panel as well as (2) offline test and revision cycles [58].

A framework was published by McCoy, describing a potentially effective method for assessing clinical appropriateness of medication alerts. A key attribute of this framework is that it determines appropriateness at the time of a triggered alert and by applying expert knowledge [60]. Weingart et al. examined a subset of all displayed alerts to determine alert validity and expert agreement with overrides, although no measures of unintended adverse consequences were reported [58]. Sucher mentions factors that need to be tested, such as verification, validation and worst case testing, but these factors are not explained in detail [59]. A practical validation approach is described by Osherhoff et al., using cases and testing scenarios to validate clinical rules [47]. This method however has limited usefulness due to lack of a detailed description of the method and outcome. To prevent alert fatigue, CDSS implementers must monitor and identify situations that frequently trigger inappropriate alerts and take well-defined steps to improve alert appropriateness [60]. Studies examining CDSS content validation often lack a complete and reproducible method that is demonstrably leading to appropriate alerts.

11.3.2.1. Strategy for Development and Validation of Clinical Rules

Below we describe a four-step strategy to develop and implement clinical rules, which we ourselves use as part of development [57, 61].

Step 1: Technical Validation

The objectiveof this step is to determine whether a clinical rule functions as we expect it to do. Are the parameters in the CDSS linked correctly to the EHR and are we using technically valid definitions. Of course the first step starts by designing a clinical rule. Most often such a clinical rule is based on an evidence-based medicine (EBM) guideline. The EBM guideline is first translated into a computer-interpretable format with measurable and specific parameters. This regularly requires translating clinical terms used in guidelines to standardized clinical terms before use. For example, how to define diarrhea? Is it enough a patient has watery stool or should it also be more than 3 times a day? Such definitions are not solved using only standardized terms. After definitions are clear and build into the clinical rule the clinical rule is tested on a historical EMR database. Subsequently, results are analyzed to determine the amount of true positives (PPV) and true negatives (NPV). These results are discussed in a plenary meeting together with an expert team. Here possible improvements are identified, which could later on be implemented. When the objectives are met (positive predictive value >90% and negative predictive value >95%), the second step of the development strategy is started.

Step 2: Therapeutic Retrospective Validation

The second step is intended to check whether the alerts produced by the CDSS are clinically relevant, useful and actionable. This step of therapeutic validation is of greatest importance for user acceptance further on. Although alerts at this stage are technically valid and based on evidence-based guidelines, health care professionals may not always consider them useful or relevant. This step starts with a meeting between the building team and the expert team to discuss the therapeutic value of the alerts. The expert team should include experts on the subject at hand from different medical disciplines. Moreover, opinion leaders from the clinic should also be included. The expert team reviews all of the alerts generated and classifies them as being relevant or not. Differences between theory and practice are discussed and the expert team formulates modifications to the clinical rule. After modifications are implemented, the clinical rule is tested in the same manner as in step 1 using the same set of patients from historical EHR database. After this test, outcome is once again evaluated by the technical team and expert team together in order to maximize therapeutic PPV and NPV.

Step 3: Pre-implementation Prospective Validation

The third step is used to prepare the CDSS and clinical rule for implementation in practice. The CDSS is linked to a real live EHR, allowing to generate alerts of actually admitted patients. Adaptations are made to assure timely alerting and integration into clinical workflow. The expert team is consulted once again however now focusing on the content of the message (e.g., proposal, command), the recipient of the message (e.g., nurse, physician, pharmacist), the frequency (e.g., once daily, continuously) and the alerting method (e.g., on-demand, automatic). When the rule is refined on these issues, it once again returns to step 1 to proceed through the validation cycle. After completing step one and two again, the rule is implemented into operation and made accessible to a selected group of users to do the final validation. Based on user feedback some final minor technical adjustments are mostly directed to optimize user satisfaction. Frequently, the issues requiring adjustment are the result of only testing the clinical rule in a retrospective setting on a static database instead of prospective on a dynamic real live EHR database. Depending on the frequency of alerting, usually after 2 months, the results from the prospective testing are evaluated by the technical and expert team together to calculate the final positive predictive value. Now the clinical rule is ready for implementation in daily practice.

Step 4: Post-implementation Prospective Validation

The fourth step, after implementation of the clinical rule in daily practice, is continuous maintenance. This step corresponds to the third pillar of effective CDSS implementation suggested by Osherhof and colleagues in their Roadmap. In this step the clinical rule is monitored while operational. Monitoring consists on reviewing performance, follow-up and PPV. The step also encompasses technical and therapeutic maintenance to ensure continuous accuracy of the alerts. We found that every clinical rule needs adjustments after implementation in practice, which were not foreseen during the development phase (step 1–3). First, technical adjustments may be necessary due to updates or new functionalities in the CDSS or EHR. These technical adjustments are developed, validated and implemented by the technical team. When the changes also had therapeutic consequences, the expert team was consulted. Secondly, the content of the clinical rule should be updated regularly, due to changes in the underlying evidence-based medicine or end-users preferences. For example when a new version of the clinical guideline was available, clinical rules were checked and differences reviewed. This step finalizes the strategy, through continuously optimizing suitability of the rule in practice.

11.3.2.2. Adaption in Practice

The adaptation of a CDSS in practice is a key component to success. The validation strategy described above especially benefits from including experts in all of its development cycles. These experts and opinion leaders help support the adaptation of clinical rules in practice and are the main success factor of this strategy.

11.4. Future Perspectives

This chapter shows that clinical decision support systems can definitely support the use of clinical data science in daily clinical practice. However, adoption in practice remains a slow process and many are still reinventing the wheel instead of supporting national initiatives. Decision support systems today mainly use the ‘if then else’ logic. And even using this method, validation is already very time-consuming and complex.

We are very curious to see combinations of systems using tree-based logic using current EBM guidelines and suggestions made using Bayesian models and artificial intelligence. It is a great and promising challenge to make healthcare really benefit more from big data, draw conclusions humans haven’t drawn themselves. However, validation, acceptance and adaptation of ‘black box’ systems will require a paradigm shift, challenging the basic principles of current day EBM practice. Nevertheless, believe in decision support keeps attracting health care professionals to work with these powerful and promising systems.

References

1.
Kronenfeld MR, Bay RC, Coombs W. Survey of user preferences from a comparative trial of UpToDate and ClinicalKey. J Med Libr Assoc. 2013;101(2):151–4. [PMC free article: PMC3634379] [PubMed: 23646031] [CrossRef]
2.
Isaac T, Zheng J, Ashish J. Use of UpToDate and outcomes in US hospitals. J Hosp Med. 2012;7(2):85–90. [PubMed: 22095750] [CrossRef]
3.
Tate KE, Gardner RM, Weaver LK. A computerized laboratory alerting system. MD comput. 1990;7(5):296–301. [PubMed: 2243545]
4.
Kuperman GJ, Teich JM, Tanasijevic MJ, Ma’Luf N, Rittenberg E, Jha A, et al. Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc. 1999;6(6):512–22. [PMC free article: PMC61393] [PubMed: 10579608] [CrossRef]
5.
Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF. High rates of adverse drug events in a highly computerized hospital. Arch Intern Med. 2005;165(10):1111–6. [PubMed: 15911723] [CrossRef]
6.
Magrabi F, Ammenwerth E, Hypponen H, de Keizer N, Nykanen P, Rigby M, et al. Improving evaluation to address the unintended consequences of health information technology: a position paper from the Working Group on Technology Assessment & Quality Development. Yearb Med Inform. 2016;1:61–9. [PMC free article: PMC5171579] [PubMed: 27830232]
7.
Lehmann CU, Seroussi B, Jaulent MC. Troubled waters: navigating unintended consequences of health information technology. Yearb Med Inform. 2016;1:5–6. [PMC free article: PMC5171549] [PubMed: 27830225] [CrossRef]
8.
Mamlin BW, Tierney WM. The promise of information and communication technology in healthcare: extracting value from the chaos. Am J Med Sci. 2016;351(1):59–68. [PubMed: 26802759] [CrossRef]
9.
Frost & Sullivan White Paper, “Drowning in Big Data? Reducing Information Technology Complexities and Costs For Healthcare Organizations”. 2012. Retrieved from http://www​.emc.com/collateral​/analystreports​/frost-sullivan-reducing-information-technology-complexities-ar.pdf.
10.
Bresnick J. The difference between big data and smart data in healthcare. Available from: https:​//healthitanalytics​.com/features/the-difference-between-big-data-and-smart-data-in-healthcare.
11.
Musen MA, Shahar Y, Shortliffe EH. Clinical decision-support systems. In: Shortliffe EH, Cimino JJ, editors. Biomedical informatics: computer applications in health care and biomedicine. 4th ed. London/New York: Springer; 2014.
12.
Zeiger RF. McGraw-Hill’s Diagnosaurus. 4.0 2018. Available from: http:​//accessmedicine​.mhmedical.com/diagnosaurus.aspx.
13.
Smith M, Nazario B, Bhargava H, Cassoobhoy A. WebMD: WebMD LLC. 2018. Available from: https://www​.webmd.com/
14.
Scheepers-Hoeks AM, Grouls RJ, Neef C, Korsten HH. Strategy for implementation and first results of advanced clinical decision support in hospital pharmacy practice. Stud Health Technol Inform. 2009;148:142–8. [PubMed: 19745244]
15.
Latoszek-Berendsen A, Tange H, van den Herik HJ, Hasman A. From clinical practice guidelines to computer-interpretable guidelines. A literature overview. Methods Inf Med. 2010;49(6):550–70. [PubMed: 21085744] [CrossRef]
16.
Stojadinovic A, Bilchik A, Smith D, Eberhardt JS, Ward EB, Nissan A, et al. Clinical decision support and individualized prediction of survival in colon cancer: bayesian belief network model. Ann Surg Oncol. 2013;20(1):161–74. [PubMed: 22899001] [CrossRef]
17.
Neapolitan R, Jiang X, Ladner DP, Kaplan B. A primer on bayesian decision analysis with an application to a kidney transplant decision. Transplantation. 2016;100(3):489–96. [PMC free article: PMC4818954] [PubMed: 26900809] [CrossRef]
18.
Jalali A, Bender D, Rehman M, Nadkanri V, Nataraj C. Advanced analytics for outcome prediction in intensive care units. Conf Proc IEEE Eng Med Biol Soc. 2016;2016:2520–4. [PubMed: 28268836]
19.
Shamir RR, Dolber T, Noecker AM, Walter BL, McIntyre CC. Machine learning approach to optimizing combined stimulation and medication therapies for Parkinson’s disease. Brain Stimul. 2015;8(6):1025–32. [PMC free article: PMC5015434] [PubMed: 26140956] [CrossRef]
20.
Tenorio JM, Hummel AD, Cohrs FM, Sdepanian VL, Pisa IT, de Fatima Marin H. Artificial intelligence techniques applied to the development of a decision-support system for diagnosing celiac disease. Int J Med Inform. 2011;80(11):793–802. [PMC free article: PMC3768292] [PubMed: 21917512] [CrossRef]
21.
Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38. [PubMed: 15755945] [CrossRef]
22.
Yamada RH. An overview of computers in medicine. Can Fam Physician. 1968;14(3):15–7. [PMC free article: PMC2280996] [PubMed: 20468194]
23.
Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc. 2007;14(1):29–40. [PMC free article: PMC2215064] [PubMed: 17068355] [CrossRef]
24.
Nuckols TK, Smith-Spangler C, Morton SC, Asch SM, Patel VM, Anderson LJ, et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev. 2014;3:56. [PMC free article: PMC4096499] [PubMed: 24894078] [CrossRef]
25.
Wolfstadt JI, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med. 2008;23(4):451–8. [PMC free article: PMC2359507] [PubMed: 18373144] [CrossRef]
26.
Eppenga WL, Derijks HJ, Conemans JM, Hermens WA, Wensing M, De Smet PA. Comparison of a basic and an advanced pharmacotherapy-related clinical decision support system in a hospital care setting in the Netherlands. J Am Med Inform Assoc. 2012;19(1):66–71. [PMC free article: PMC3240762] [PubMed: 21890873] [CrossRef]
27.
Nanji KC, Seger DL, Slight SP, Amato MG, Beeler PE, Her QL, et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc. 2018;25(5):476–81. [PMC free article: PMC7646870] [PubMed: 29092059] [CrossRef]
28.
Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health. 2014;104(12):e12–22. [PMC free article: PMC4232126] [PubMed: 25322302] [CrossRef]
29.
Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, et al. Grand challenges in clinical decision support. J Biomed Inform. 2008;41(2):387–92. [PMC free article: PMC2660274] [PubMed: 18029232] [CrossRef]
30.
Simon SR, McCarthy ML, Kaushal R, Jenter CA, Volk LA, Poon EG, et al. Electronic health records: which practices have them, and how are clinicians using them? J Eval Clin Pract. 2008;14(1):43–7. [PubMed: 18211642] [CrossRef]
31.
Workgroup Clinical Rules of the Dutch Association of Hospital Pharmacists (NVZA). Questionare on current state of clinical rule implementation in hospital pharmacy. 2015. http://www​.nvza.nl.
32.
Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc. 2007;14(2):141–5. [PMC free article: PMC2213467] [PubMed: 17213487] [CrossRef]
33.
U.S. National Library of Medicine. Medical Subject Headings (MeSH) [Alert fatigue, health personnel]. 2017. Retrieved from https://www​.ncbi.nlm​.nih.gov/mesh?term=alert%20fatigue. At 01 Oct 2018.
34.
van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138–47. [PMC free article: PMC1447540] [PubMed: 16357358] [CrossRef]
35.
van der Sijs H, Mulder A, van Gelder T, Aarts J, Berg M, Vulto A. Drug safety alert generation and overriding in a large Dutch university medical centre. Pharmacoepidemiol Drug Saf. 2009;18(10):941–7. [PubMed: 19579216] [CrossRef]
36.
Wright A, Goldberg H, Hongsermeier T, Middleton B. A description and functional taxonomy of rule-based decision support content at a large integrated delivery network. J Am Med Inform Assoc. 2007;14(4):489–96. [PMC free article: PMC2244910] [PubMed: 17460131] [CrossRef]
37.
Wasylewicz ATM, Gieling E, Movig K, Grouls RJE, Egberts TCG, Korsten HHM. Clinical rules in Santeon Collaboration Pilot Study (CRISPS): an exploration of joint developement and sharing of CDS content. Unpublished. 2016.
38.
Dey AK. Understanding and using context. Pers Ubiquit Comput. 2001;5(1):4–7. [CrossRef]
39.
Riedmann D, Jung M, Hackl WO, Ammenwerth E. How to improve the delivery of medication alerts within computerized physician order entry systems: an international Delphi study. J Am Med Inform Assoc. 2011;18(6):760–6. [PMC free article: PMC3197982] [PubMed: 21697293] [CrossRef]
40.
Jung M, Riedmann D, Hackl WO, Hoerbst A, Jaspers MW, Ferret L, et al. Physicians’ perceptions on the usefulness of contextual information for prioritizing and presenting alerts in computerized physician order entry systems. BMC Med Inform Decis Mak. 2012;12:111. [PMC free article: PMC3522054] [PubMed: 23031275] [CrossRef]
41.
Berlin A, Sorani M, Sim I. A taxonomic description of computer-based clinical decision support systems. J Biomed Inform. 2006;39(6):656–67. [PubMed: 16442854] [CrossRef]
42.
Berlin A, Sorani M, Sim I. Characteristics of outpatient clinical decision support systems: a taxonomic description. Stud Health Technol Inform. 2004;107(Pt 1):578–81. [PubMed: 15360878]
43.
van Wezel RAC, Scheepers-Hoeks AMJW, Schoemakers R, Wasylewicz ATM, ten Broeke R, Ackerman EW, et al. Application of clinical rules for therapeutic drug monitoring and their impact on medication safety. PW Wetenschappelijk Platform. 2011;5(11):183–6.
44.
Scheepers-Hoeks AMJW, Grouls RJE, Neef C, Ten broeke R, Ackerman EW, Korsten HHM. Compliance to alerts generated by clinical rules, applying three active alert presentation methods in clinical practice. PW Wetenschappelijk Platform. 2014;8:199–202.
45.
Phansalkar S, van der Sijs H, Tucker AD, Desai AA, Bell DS, Teich JM, et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc. 2013;20(3):489–93. [PMC free article: PMC3628052] [PubMed: 23011124] [CrossRef]
46.
Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765. [PMC free article: PMC555881] [PubMed: 15767266] [CrossRef]
47.
Osheroff J, Pifer E, Teich J, Sittig D, Jenders R. Improving outcomes with clinical decision support: an implementer’s guide. Osheroff J, Pifer E, Teich J, Sittig D, Jenders R, editors. Chicago: Health Information Management and Systems Society; 2005.
48.
Scheepers-Hoeks AM, Grouls RJ, Neef C, Ackerman EW, Korsten EH. Physicians’ responses to clinical decision support on an intensive care unit – comparison of four different alerting methods. Artif Intell Med. 2013;59(1):33–8. [PubMed: 23746663] [CrossRef]
49.
Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523–30. [PMC free article: PMC264429] [PubMed: 12925543] [CrossRef]
50.
Nan S, Van Gorp PME, Korsten EHM. Tracebook: a dynamic checklist support system. Comput Base Med Syst. 2014:48–51. https://research​.tue​.nl/en/publications/tracebook-a-dynamic-checklist-support-system-2.
51.
De Bie AJR, Nan S, Vermeulen LRE, Van Gorp PME, Bouwman RA, Bindels A, et al. Intelligent dynamic clinical checklists improved checklist compliance in the intensive care unit. Br J Anaesth. 2017;119(2):231–8. [PubMed: 28854530] [CrossRef]
52.
McCoy AB, Wright A, Sittig DF. Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development. J Am Med Inform Assoc. 2015;22(5):1081–8. [PMC free article: PMC5009930] [PubMed: 26104739] [CrossRef]
53.
Wright A, Sittig DF. A framework and model for evaluating clinical decision support architectures. J Biomed Inform. 2008;41(6):982–90. [PMC free article: PMC2638589] [PubMed: 18462999] [CrossRef]
54.
Wright A, Sittig DF, Ash JS, Sharma S, Pang JE, Middleton B. Clinical decision support capabilities of commercially-available clinical information systems. J Am Med Inform Assoc. 2009;16(5):637–44. [PMC free article: PMC2744714] [PubMed: 19567796] [CrossRef]
55.
International SNOMED. SNOMED CT. 2018. Retrieved from https://www​.snomed.org​/snomed-ct/get-snomed-ct/. At 01 Oct 2018.
56.
de Clercq PA, Hasman A, Blom JA, Korsten HH. Design and implementation of a framework to support the development of clinical guidelines. Int J Med Inform. 2001;64(2–3):285–318. [PubMed: 11734393] [CrossRef]
57.
Scheepers-Hoeks AMJW, Grouls RJE, Neef C, Ackerman EW, Korsten HHM. Strategy for development and pre-implementation validation of effective clinical decision support. Eur J Hosp Pharm. 2013;20:155–60. [CrossRef]
58.
Weingart SN, Seger AC, Feola N, Heffernan J, Schiff G, Isaac T. Electronic drug interaction alerts in ambulatory care: the value and acceptance of high-value alerts in US medical practices as assessed by an expert clinical panel. Drug Saf. 2011;34(7):587–93. [PubMed: 21663334] [CrossRef]
59.
Sucher JF, Moore FA, Todd SR, Sailors RM, McKinley BA. Computerized clinical decision support: a technology to implement and validate evidence based guidelines. J Trauma. 2008;64(2):520–37. [PubMed: 18301226] [CrossRef]
60.
McCoy AB, Waitman LR, Lewis JB, Wright JA, Choma DP, Miller RA, et al. A framework for evaluating the appropriateness of clinical decision support alerts and responses. J Am Med Inform Assoc. 2012;19(3):346–52. [PMC free article: PMC3341775] [PubMed: 21849334] [CrossRef]
61.
Scheepers-Hoeks AM, Grouls RJ, Neef C, Wasylewicz ATM, van’t Geloof W, Korsten EH. Succesfull implementation of clinical rules in daily practice: two years follow-up by pharmacy intervention. Thesis: Alert methods as success factors, influencing effectiveness of a clinical decision support system in clinical practice. Eindhoven. 2014.