Study identifies keys to success of evidence-based decision making

Evidence-based decision making (EBDM)—using the best available evidence from multiple sources to make informed decisions—is critical to the success of any organization. In a new study, researchers have explored what makes EBDM work. They found that conversations and interactions involving three distinct networks are key to its success.

The study, by researchers at Carnegie Mellon University and York University, is published in Behavioral Science & Policy.

“Most studies on EBDM have focused on the decision maker as an individual, but to be most efficient and effective, EBDM requires extensive communication with others,” explains Denise M. Rousseau, professor of organizational behavior and public policy at Carnegie Mellon’s Heinz College, who coauthored the study. “These participants provide the skills and knowledge needed to incorporate diverse types of evidence into the decision-making process.”

EBDM involves using multiple data sources that have been vetted for quality. But it is not enough to search databases online. Effective EBDM requires connecting with other people. This approach can be used in a variety of contexts, from the criminal justice system and construction to hospitals and policy shops.

In this study, researchers reviewed articles on the implementation of EBDM in organizations, searching more than 5,000 articles published since 2000. They narrowed their selection to approximately 100 studies that were empirical and focused on implementation. The studies were conducted in public health departments, hospitals, environmental quality facilities, and nongovernmental organizations.

The researchers then evaluated the quality of the EBDM described in each study by the extent to which a decision maker used four types of evidence: science (e.g., research reports, review articles), organizational context (data relating to patients, clients, or the organization that can improve understanding of the circumstances in which a decision is to be implemented), stakeholder insights, and practitioner expertise (e.g., a doctor or manager’s experience and judgment). They also sought to determine whether the study carried out the six key phases of EBDM—asking, acquiring, appraising, aggregating, applying, and assessing.

Based on their findings, researchers identified several factors that promote EBDM in organizational decisions, including an organizational climate that emphasized science over politics, educated decision makers, and access to reader-friendly summaries and reviews. The most striking attribute of successful EBDM, they found, was its collective nature.

When EBDM is taught in professional settings or discussed in scholarly literature, the focus is typically on the actions of an individual decision maker, the authors note. But when researchers examined the practice of EBDM in real organizations, what stood out most was the involvement of other people in the process of figuring out what evidence is needed, where to find it, and how to use it in making decisions.

They also found that EBDM was enhanced by the interactions and conversations that occurred in three networks, each consisting of the decision maker (alone or as part of a team) and 1) researchers (internal or external to the organization), 2) stakeholders (people in an organization or community who are directly or indirectly affected by a decision), or 3) communities of practice (people who have job responsibilities similar to those of the decision maker). Creating and maintaining ongoing ties with people in these areas is key to an organization’s ability to engage in EBDM effectively, the study concluded.

“Building social connections with people in each of these networks enhances a person’s ability to make good decisions for an organization,” suggests Fairmah HakenZadeh, assistant professor of human resource management at York University, who co-authored the study. “Therefore, organizations engaging in EBDM must have ongoing programs and policies geared toward creating and maintaining the three connections we identified.”