By Carol Haig, CPT, and Roger Addison, CPT, EdD
Our guest this month is Dick Clark, CPT, EdD. Dick, firstname.lastname@example.org, is professor of Educational Psychology and Technology, and clinical research professor of Surgery at the University of California Rossier School of Education. He is also co-director of the Center for Cognitive Psychology, an organization that specializes in unbiased research evaluation, and the CEO of Atlantic Training, Inc.
Dick is a thought leader, long-term ISPI member, and a recipient of the Thomas F. Gilbert Distinguished Professional Achievement Award. He is the keynote speaker for ISPI’s upcoming 50th Anniversary Conference and celebration in Toronto in April where he will discuss evidence-based practice. Among his many talents, Dick is a dedicated researcher. He joins us to share his knowledge of the complexities of evidence-based practice as it applies to the field of performance improvement and to offer guidance for practitioners.
What Is Evidence-Based Practice?
Recent conversations within ISPI reflect a growing interest in providing clients with proven solutions substantiated by research, with evidence that the solutions will be effective. Evidence-based practice is the conscientious use of current results from peer-reviewed empirical research. Peer-review refers to a process that is implemented before research is published in a journal, and means that top experts in the author’s field have critically reviewed the design of the research and the claims made about it to ensure that it is balanced and accurate.
Human performance technology (HPT) is the combination of scientific evidence and practical experience used to provide clients with proof that the recommended solution will work for their problem or opportunity. Thus, evidence-based practice is a keystone of HPT and of ISPI.
Evidence-based practice matters because without it we risk advising clients to use ineffective solutions. By carefully researching a potential solution and combining it with judicious implementation planning and change management actions, we can ensure success for our clients. To learn more, join Dick and other researchers at the Research-to-Practice afternoon at the ISPI conference in Toronto.
Origins of Evidence-Based Practice
It is likely that evidence-based practice originated in ancient China where empirical research was conducted for engineering projects and in 16th and 17th century Europe, where medical research began and gradually transformed health care. Today, both engineering and medicine continue to be heavily immersed in evidence-based practice. It is worth noting that both engineering and medicine have relied on collaborations between researchers, experienced practitioners, and clients to solve problems.
In the past couple of decades, the volume of research on individual, team, and organizational performance has increased dramatically. There is research available on almost any problem or opportunity. In addition, the proliferation of the Internet and its many powerful search engines and research aids make vast quantities of research available to anyone with computer access. Yet, along with accessibility comes the responsibility for doing the research for any course of action we recommend to our clients.
Dick cautions us that just because there is reliable research about a possible solution that does not guarantee that the client will accept it. In medicine, the biggest challenge is not finding an effective treatment but instead convincing a patient to comply with what is prescribed. Likewise, in performance improvement, client compliance with recommendations is key to successful change.
This is where our systematic approach shows its true value. Clients are most apt to comply with performance improvement recommendations when we include:
- Evidence that the proposed solution is likely to work in the client’s environment
- Pilot-tested success with the recommended solution
- An evidence-based change management process to implement the solution
- Focused monitoring, ongoing evaluation, and scheduled updates
In the course of his work in education, Dick became interested in how the most expert surgeons solve complex, life-threatening problems by combining their clinical experience and knowledge of research. His research resulted in an approach for capturing the analytical strategies of highly successful experts so that they can be taught to novices. And no, Dick does not perform surgery, but his years of working with surgeons in the operating room have provided unique insights into the medical profession and the many useful parallels we in the performance improvement business can draw from the medical field. One of these is evidence-based practice.
In business, it is easy to get caught up in trends that espouse stellar results and fail to investigate their efficacy, as Jeanne Farrington discussed in this space. A common belief is that if research says something works it will, and, in actuality, it may not. Dick shares some examples of things we think work that do not:
- Course reaction evaluation sheets provide no useful information about what was learned but may offer insights into trainee motivation.
- Discovery learning, in which students are given a problem to solve alone or in small groups, learn how to solve problems.
- The Myers-Briggs Type Indicator is an unreliable instrument because people tend to get a different score each time they take it.
- Experts as trainers–the best experts are unable to describe 70% of what they know when they attempt to train novices.
- Downsizing an organization does not produce expected results according to the National Research Council.
For more examples of solutions that are widely believed to be effective, see Jeff Pfeiffer and Robert Sutton’s book, Hard Facts, Dangerous Half Truths and Total Nonsense: Profiting from Evidence-Based Management.
Advice About Finding and Using Evidence-Based Research
Dick offers this advice for discovering and using evidence-based research with your clients:
- Focus on the problem to be solved or the opportunity desired
- Avoid starting with a performance solution and looking for a problem to solve
- Look for research that was carefully designed to eliminate all other causes of the outcome except for the solution being tested
- Examine the reliability of the measures used–did the researchers test what they said they tested?
- Determine if the researchers provide a clear operational definition of the solution so that it can be reconstructed in the client’s environment
Consider studies by researchers who disagree with the solution–if possible, find researchers who initially disagreed but changed their minds due to compelling evidence
- Where possible, pilot test the solution in the client organization
- If the solution requires organizational change, develop a change management plan
- Communicate with your client and in your organization about the evidence you discover
Links to the Performance Technology Landscape
Use of evidence-based practice supports these principles of performance technology:
|R||Focus on Results–Research is conducted to produce results.|
|S||Takes a System view–Research requires a systematic approach.|
|V||Adds Value–Research undertaken and presented properly enhances practitioners’ credibility and that of our profession.|
|P||Establishes Partnerships–Evidence-based practice enlists the client in planning and implementing the selected solutions.|
Before you suggest a performance improvement solution to your client, explore what the research tells you using the advice provided above. A helpful guide is Dick’s book, written with Fred Estes, Turning Research Into Results.
ISPI in Five Years?
Dick tells us that the key to evidence-based practice in the field of performance improvement is dialog about evidence sought and found in the service of our work with clients. ISPI is the best place for this dialog, and it is in progress right now. See Chasing Down the Elusive Credits for Facts and Fictions in Learning and Improvement. Dick challenges ISPI to energize its members around the value and ethics of evidence-based practice. ISPI’s future depends on this.
Find all the models and tools featured in TrendSpotters at www.ispi.org/archives/perfXpress.htm#trendToolkit.