by Harold D. Stolovitch, CPT and Erica J. Keeps, CPT

For the last three years, we have repeatedly heard that the economy will turn around “very soon.” Meanwhile, budgets grow tighter and every new training and performance support initiative is scrutinized with a magnifying glass in one hand and an ax in the other. Despite the austerity on the learning and performance support side, the pressure is still on to produce and maintain an increasingly productive workforce and prepare employees for new systems, regulations, and products.

How do we as performance improvement specialists cope with these seemingly impossible challenges…credibly? The usual route seems to be via technology—e-learning, reusable learning objects, knowledge management systems, learning management systems/learning content management systems, webinars, and other forms of distance learning. However, we all know these are fraught with uncertainties, frequent lack of user adoption, and early abandonment. There are also the high initial financial investments, infrastructure costs, and time required to develop and implement new technology-based interventions. Worst of all, what results can we anticipate? There is little research to guarantee high return-on-investment (ROI).

Why not turn to low-cost solutions with strong track records of high yield that are much easier to develop and implement? Here are a dozen of our favorites:

• Cleaning up performance expectations. Research in human performance identifies a lack of clarity of expectations to be the number one cause of inadequate performance. Despite its high toll on productivity, this cause tends to be neglected when searching for improved efficiencies and results. Yet, reviewing, verifying, and redefining expectations so that they align with desired behaviors and outcomes costs little while producing extremely high ROI.

• Developing feedback systems. Inadequate feedback constitutes a close second to unclear expectations in causes of deficient performance. Feedback comes in two useful varieties: corrective feedback that informs performers as to where they are off-target, and confirming feedback that tells them they are doing the job right. Feedback systems are far less costly than virtually any technological intervention and produce powerful impact.

• Creating performance support systems. No matter how good the training, without adequate support mechanisms, acquired skills and knowledge tend to deteriorate. Review of procedures, processes, and tasks, guidance in initial stages of application, and emotional support to build and maintain confidence all provide inexpensive support that results in fluent performance.

• Designing simple and effective job aids. This is still one of the most cost-effective means for achieving performance results quickly. Decision tables, algorithms, step-by-step procedures, directories, cookbooks, troubleshooting guides, and checklists are some of the many job aid options. When well designed, they result in immediate success. If training is required, it is usually brief to help performers use the job aids.

• Ensuring adequate policies, processes, procedures, and resources. When the roadmap is unclear, one is bound to get lost, especially if the vehicle is also faulty. Review of policies, processes, procedures, and adequacy of resources is a first step. Stakeholder consensus, support, and consistent application rules are the next steps. Communication, practice, feedback, and enforcement constitute the final steps. Compared to high-tech interventions, this is a low-cost, high-yield solution to obtain desired performance.

• Providing and aligning incentives and consequences. In a recent year-long, rigorous study, we discovered that incentives, especially tangible ones, can improve performance 13-40% (Stolovitch, Clark, & Condly, 2002). Well-designed incentive systems and well-administered consequences that are perceived as fair and equitable can have dramatic results on performance.

• Increasing motivation to perform. Motivation is critical in both learning and performance along with ability and prior knowledge. By influencing perceived value to perform and confidence in one’s ability to succeed as well as eliminating factors that create negative mood, we can strongly affect workplace motivation. Motivated workers tend to perform at much higher productivity rates.

• Making sure mechanisms for training transfer are in place. Studies have shown that several months after a training intervention, usually less than 30% of what was learned remains. Simple interventions such as having supervisors prepare workers for training, ensuring that training includes considerable practice closely aligned to back-on-the-job work, and support and follow-up post-training immensely increases on-job transfer.

• Reviewing and tightening performer selection systems. Poor selection of performers results in poor performance. Select individuals with characteristics, prerequisite skills, and knowledge that you are not prepared to invest in. Train for specific, job-required competencies. By tightening selection criteria and procedures, you can better fit the right person to the right job. This decreases training investments, supervision time, and turnover—all costly items in a tight economy.

• Eliminating tasks that interfere with job performance. Lower priority or administrative tasks often interfere with getting important jobs done. For example, a salesperson with a specified number of calls to make or amount of revenues to generate may be held back due to mandated report writing and meeting attendance. Eliminating unnecessary, counterproductive tasks can result in considerable performance improvement.

• Adopting structured OJT and PAL. More money and time are spent in informal on-the-job training (OJT) than in formal classes, including e-learning—perhaps four to five times as much. Imagine the efficiencies if OJT were structured. Peer Assisted Learning (PAL), in which more advanced peer-level buddies are trained to assist novice workers, can also achieve remarkable results at a relatively low cost.

• Strengthening existing training. When all else fails, review and clean up existing training? You can retrofit ineffective training by applying sound instructional design to it. Break existing training into logically connected units. Then, build in meaningful rationales, objectives, learning activities, evaluation, and feedback. By strengthening your current training instructionally, you can increase its effectiveness at a fraction of the cost of building new, more sophisticated learning systems.

Tough times call for creative cost-cutting measures. Let’s get back to the fundamentals of performance improvement. There certainly is a place for technology, but let’s not start with expensive solutions until we have clearly defined both the goals and the gaps. Front-end analysis allows us to do this systematically. Then, with a sharp eye on engineering the best performance improvement solutions at the least cost, we can make a significant contribution to achieving remarkable organizational results at low cost in this tight economic environment.

Reference:
Stolovitch, H., Clark, R., & Condly, S. (2002). Incentives, motivation & workplace Performance: research and best practice. Silver Spring, MD: ISPI and SITE Foundation.

Harold D. Stolovitch and Erica J. Keeps share a common passion—developing people. Together they have devoted a combined total of more than 70 years to make workplace learning and performance both enjoyable and effective. Their research and consulting activities have involved them in numerous projects with major corporations worldwide. Harold and Erica are the principals of HSA Learning & Performance Solutions LLC, an international consulting firm that specializes in the application of instructional technology and human performance technology to business, industry, government, and the military. They are co-editors of the Handbook of Human Performance Technology and co-authors of the best-selling, award-winning, Telling Ain’t Training. Harold and Erica may be reached at info@hsa-lps.com.

Advertisement  Would you like to advertise in this space? Contact marketing@ispi.org

by Carol Haig, CPT and Roger Addison, CPT

This month we share predictions that affect all performance improvement practitioners, made by a long-term ISPIer whose experience and interests led him to focus on software technologies and business process change. Paul Harmon is a founder and the executive editor of Business Process Trends, an information portal and web ’zine aimed at business process change managers and information technology specialists and author of the book Business Process Change. He may be reached at pharmon@sbcglobal.net.

In Paul’s view, business process refers to everything from strategic organizational goals and policies to the automation of specific activities, and all of the tasks in between. The people who work with supply chains, value propositions, information technology, Six Sigma, etc., all need to connect through their business processes to identify trends, directions, and best practices for their organizations.

Top Three Predictions
First, Paul predicts that the transition to the process-centric organizational structures we are starting to see will continue and accelerate in the next two to three years. The resulting new organizational designs will be matrix-based to encourage collaboration and information sharing.

Second, Paul predicts that on-going process change programs will emphasize human performance improvement. Information technology is expensive to develop and carries a high risk factor. In the past, the focus was on the latest software technology and making it work. Now, many senior leaders realize that it is easier to change the processes and the people responsible for them than it is to fix an automated system. “People technologies” often give us more for our investment dollar than do new automated technologies, especially in recessionary times like these. As organizations see results from human performance improvement initiatives, they will further emphasize this element in their process change activities.

Paul’s third prediction is combined with his hope that ISPI will reach out to new audiences in different functional areas to show how human performance technology (HPT) applies to them. He cites a number of highly competent, recognized practitioners in ISPI whose work is focused on process improvement, such as Geary Rummler and Don Tosti in the United States, Klaus Wittkuhn in Germany, and Artur Nunes in Portugal. It is time for all ISPIers to step away from talking only to each other and bring their HPT tools and techniques to other parts of the organizations they serve.

Why These Predictions?
Historically, systems weren’t built to talk to each other because the technology wasn’t there and the cost was too high. Then the Internet was born, making it technically possible and economically feasible to tie business processes together.

Today, organizations can think broadly about a specific application and identify all the parts of the business process that the application touches, making the transition to a process-centric organizational structure a logical move. Indeed, systems are being refitted across all industries. And this work is focused on processes—how best to connect the pieces to serve the customer or tie in the supplier.

Process change programs will emphasize human performance improvement. The Gartner Group, the leading information technology research company, reports that even today only 20% of the activities performed in an organization can be automated. This means 80% of decisions and steps in an organization’s processes are still done by people. Senior managers have learned that slick, smooth processes must include people along with the automation, bringing people-processes to the forefront of all design work.

Like many of us, Paul observes that ISPIers talk about processes and teach each other, but don’t reach out to other disciplines. We need to change this if we are to truly make a difference in the workplace and remain a viable professional Society. As Klaus Wittkuhn says in his response to the February 2003 special Performance Improvement journal issue on the HPT value proposition, “ISPI’s value proposition…should target decision makers and line managers.”

At this time, we have an extraordinary opportunity to move our technology out to the business process level in our organizations. We, in performance improvement, are uniquely positioned to do this right now. If we work in the training department, we usually have clients throughout the company, and we know the projects that different functions are working on. We can connect clients working on similar projects and help them use performance improvement tools. Such activities reduce the duplication of effort that silos foster and pave the way for cost reductions. And that’s how HPTers can add real value.

So What Do We Do Now?
Organizations are shifting to a business process structure for all the reasons described above. As they re-align around business processes, opportunities to use HPT tools and models can only grow.

And so we put these critical questions to you, our readers: What should performance improvement practitioners do in response to such changes? How do we make ourselves proactive and teach our clients to use our tools? If our organizations are currently describing the same problems and opportunities from different silos, how do we get them to open a dialog from a business process perspective?

What do you suggest we do, individually and collectively? Email either Carol or Roger at the email addresses provided below, and we will publish your ideas in a future TrendSpotters article.

If you have any predictions about the future of HPT that you feel would be of interest to the PerformanceXpress readership, please contact Carol Haig, CPT, at carolhaig@earthlink.net or Roger Addison, CPT, at roger@ispi.org.

by Ann W. Parkman and Karen VanKampen

One question many training professionals are asking themselves is “How can we directly impact our organization’s profitability?” This question underscores the transition companies are making from traditional training to performance-based training.

What does it mean to be performance based? It means your department’s primary focus is on improving job performance as a means of directly impacting the quality of your organization’s products and services and the efficiency with which they are produced. To achieve this, the following critical elements must exist:

• The training organization’s goals must be aligned with the goals of the business.
• A thorough analysis must be conducted before identifying solutions.
• Identified solutions must focus on job performance.
• When training is identified as appropriate, it is designed using a performance-based methodology such as Criterion-Referenced Instruction (CRI).

Here are some recommendations to consider when assessing ways to transform your department into a true performance-based training organization:

Focus on Results
Your department’s focus should be on improving job performance and sustaining desired performance levels. To maintain a results-oriented focus, ask yourself if your organization:

• Requires that all training requests be analyzed to ensure the right solution (training, non-training, or both) is developed and implemented.
• Views training as appropriate only if the performance problem is due to a lack of skill or knowledge.
• Follows analysis procedures to ensure objectives are derived from job performance needs.
• Gives business leaders training and tools to guide them in providing feedback and coaching to their employees.

Training Methodology
Adopting a performance-based methodology such as CRI will ensure that training provides the job-critical skills learners need to meet management’s expectations. Here are some questions to help you determine how your training organization’s methodology compares to CRI:

• Does training focus on providing learners with the skills needed to improve job performance?
• Do practice situations match on-the-job conditions as closely as possible?
• Do learners receive feedback immediately after practicing so they know what they have done right or wrong?
• Does training include appropriate skill checks to ensure learners have mastered essential skills?

Performance Measurement
Performance-based training organizations are in the best position to influence the development of a plan for standardizing and aligning performance measures. To help you assess how your training organization measures up, ask yourself:

• Are job tasks and performance standards aligned with organizational goals?
• Are pre-employment assessments, training skill checks, and performance appraisals aligned with the job tasks and performance standards?
• Are performance standards stated in quantitative terms?
• Do end-of-course evaluations assess the learners’ perceived ability to apply their newly learned skills back on the job?
• Do procedures exist to evaluate performance at key stages (e.g., after completing training, after learners have returned to their jobs for a set amount of time, etc.)?
• Do procedures exist to determine the ROI of your solutions?

If you answer “no” or “don’t know” to any of these questions, you may need to close some gaps between your current training focus and your desired performance improvement focus.

Ann W. Parkman is executive vice president, managing partner, and co-founder of CEP (The Center for Effective Performance), as well as a past president of ISPI. She will be facilitating a session at ISPI’s Fall Conference entitled, “How to Recognize Performance-Based Training When You See It.” Ann may be reached at aparkman@cepworldwide.com.

Karen VanKampen is director of performance consulting for CEP. She has spoken at a number of ISPI conferences and local ISPI chapter meetings. Karen will be facilitating a session at ISPI’s Fall Conference entitled, “Becoming a Performance-Based Training Organization: Beyond the Basics.” She may be reached at kvankampen@cepworldwide.com.

 

Advertisement  Would you like to advertise in this space? Contact marketing@ispi.org

This strategy, called Open Questions (OQ), presents a question that permits more than one acceptable answer. OQ requires and rewards reflective and creative thinking. It also facilitates mutual learning by providing related webpages for you to explore:

• Scoring Key: A checklist to objectively evaluate your answer—and other people’s answers.
• Expert Answers: SMEs’ responses to compare with your own.
• Peer Answers: A complete list of all answers given by other participants.
• Comments: Space for feedback, questions, and sarcastic remarks about the scoring key, expert answers, peer answers, and earlier comments.

Results
We kicked off the inaugural OQ with a reflective question (“What are the advantages and disadvantages of the OQ approach?”) and a creative question (“If GAME were an acronym, what does it stand for?”) Within 24 hours, we were inundated with enthusiastic responses from several PX readers. Answers are still trickling in. We have cleaned up the clutter and summarized the key points.

You can check out the responses about advantages and disadvantages of OQ by clicking here. To enjoy creative expansions of GAME, visit here. Feel free to contribute your answer for these earlier OQs. As several readers have pointed out, OQ permits continuous improvement of your mastery of the topic. So write a new and improved answer even if you contributed an answer earlier.

A New OQ
Here’s an open-ended question that has been around ISPI circles for several decades: “How would you explain HPT to your mom?” Click here to check out the OQ pages and contribute your answer.

As an extra incentive, I will have a panel of judges select the “best” answer on August 15, 2003. We will publicize this answer in a future issue of PerformanceXpress.

by Clare Elizabeth Carey, CPT, ISPI Director

As an ISPI Director, one of my key responsibilities is to provide updates on the progress of my assigned committee and/or task force. As I’m writing this article by my pool, on another perfect 85 degrees, crystal blue sky, gentle winds day in Hawaii, I can’t help but smile. But it is not because of the weather or my lush tropical setting. It’s because I have the good fortune to be designated as the Board Liaison to the Chapter Partnership Committee (CPC). I say, “good fortune” with all sincerity. The CPC is an engaging and fun group who brings out what is best in ISPI: results, research, replication, and recognition packaged in a professional and collegial network.

Paul Cook leads the CPC leadership team whose members include: William Dudeck (chair elect), Jordan Brun (communications chair), and Dan Topf (former chair). The “Core 4” are supported by the energetic contributions of Roger Chevalier (ISPI), Kim Kidwell, Mia Logan, Robyn Skarbek, Sharon Dwyer, with additional wisdom provided by Diana Vansickle, Jim Wright, Stephany Prodromides, Andreas Kuehn, and James Andrews. This diverse collection of talented HPTers, invests in the continuing growth and development of our chapters while strengthening relationships with International. This is a formidable challenge as our Society increases its global scope amidst austere financial conditions.

This year the CPC will focus its energies on the following goals:

• Conduct the Chapter Leader’s Workshop: The plan is to make the 2004 Chapter Leader’s Workshop bigger, better, and more valuable to chapter representatives. This full-day event designed specifically for the needs and interests of our chapter leaders is offered at no cost to participants. All ISPI members are welcomed and encouraged to take advantage of this conference opportunity, as well.

• Promote and support greater use of the CPC website: Given ISPI’s global community, the Internet highway is the road of choice for our members. The CPC’s dedicated webmaster (Kim Kidwell) works to maintain currency and accuracy of information posted on the CPC website, as well as expanding the Intranet linking of chapters.

• Promote and share the best practices of chapters: The CPC promotes the application of the Award of Excellence criteria as a strategic mechanism to guide local chapters. Chapter of Excellence recipients are useful sources of best practices and serve as mentors and benchmarks for other chapters.

• Improve communication between CPC and local chapter leadership: Communication flows in multiple directions and languages. In addition to publishing its own “Almost Monthly Newsletter,” the CPC has initiated the Chapter Leader Forum. This e-mail discussion group supports the mission of ISPI and the successful operation of chapters. The CPC team encourages the free exchange of ideas and perspectives and the application of HPT to solve chapter problems.

• Improve the CPC as a resource for local chapters: Responding to the feedback from chapter leaders about their greatest challenges, the CPC provides links on its webpage to many resources including the Chapter Management Handbook, newsletter articles, program schedules, available presenters, and successful marketing strategies. Emerging chapters do not need to struggle to create their infrastructure or operating procedures. There is much that has already been done by other chapters—with successful results. The CPC is working to eliminate redundancy and the normal day-to-day frustrations with local chapter operations. They recognize chapters are volunteer organizations with limited time and resources. By harnessing the available CPC resources and references, local chapters can save enormous amounts of time and capitalize on the success of their sibling chapters.

As a conduit for performance improvement, the CPC capitalizes on its collective expertise to share information, tools, and resources among ISPI’s 75 chapters around the world. The CPC has a global view. They want to “internationalize” committee representation, products, and activities. Overcoming geographical boundaries and time zone challenges, the CPC leadership team conducts regular conference calls to plan, organize, and delegate its tasks. They are focused in their efforts and optimistic in their charge. They believe in ISPI’s mission, they value our Society, and they want to serve as lifelines to ISPI’s frontlines.

I can’t help but smile. It’s so refreshing to witness passion in action.

by Todd Packer

Ah, summertime, and even performance technologists deserve a vacation. Through our “I-Spy” column, we hope to offer some useful leisure-time reading for our readership through relevant, interesting, and useful websites for performance technologists. Each month, we take readers to off-the-beaten-path sites that help them find similar thinkers, resources, work, new ideas, and sometimes just plain old fun.

Quick recap: Every month, three sites, one theme. While far from comprehensive, hopefully these sites will spark readers to look further and expand views about human performance technology (HPT). Please keep in mind that any listing is for informational purposes only and does not indicate an endorsement either by the International Society for Performance Improvement or me.

These are the general categories I use for the sites featured:

1. E-Klatch: Links to professional associations, research, and resources that can help refine and expand our views of HPT through connections with other professionals and current trends
2. HPT@work: Links to job listings, career development, volunteer opportunities, and other resources for applying your individual skills
3. I-Candy: Links to sites that are thought provoking, enjoyable, and refreshing to help manage the stresses and identify new ideas for HPT

The theme for this month’s column is Holidays. In many places, August is a time for vacation, travel, and holidays. Allowing yourself some time to reflect and renew can improve your personal and business performance. So, take a deep breath, read this PerformanceXpress issue, then turn off your computer and plan for some time away from work. Don’t forget to pack your toilet paper.

E-Klatch
For some interesting links to information for business training professionals on the go, take a trip to the website of the Association for Development, Advancement & Productivity through Technology Training (ADAPT) “a non-profit organization providing information technology training professionals with a forum to exchange ideas and expand industry knowledge.” Under the resources link, you can find training and learning organizations across the U.S. (yes, ISPI is listed!), connections to some periodicals for travel reading (such as Business Travel News), job postings, and links to help you with everything from airfares to weather.

HPT@work
Are you ready to plan some time away from your work? A visit to several sites designed by Purdue University Professor Alastair M. Morrison, PhD can help. Dr. Morrison’s resources can be quite valuable, whether you’re looking for work (Hospitality Travel Hotel Tourism Jobs and Employment on the Web—contains a very extensive set of links for countries around the world), or you’re seeking ideas for planning a sabbatical/time off from work (Sensational Sabbatical Suggestions). And, if you’re still having difficulty justifying your travels, take along an academic journal (such as Journal of Quality Assurance in Tourism & Hospitality or Visions in Leisure and Business) for some reading at the beach.

I-Candy
Well, if you would like to travel, but can’t seem to tear yourself away from the computer, pay a visit to the Virtual Library of Museums around the world. You’re likely to find a unique museum to meet your interests, including The Baghdad Museum Project (a virtual museum of the Iraq National Museum), the Te Papa Tongarewa Museum of New Zealand, and France’s Paleolithic Cave of Chauvet-Pont-D’Arc. And, now you can really take a break as you review 4,500 years of sanitation at India’s Sulabh International Museum of Toilets.

If in your journey across cyberspace, you find other websites with beautiful travel pictures or that may be of interest to PX readers, email your finds to I-Spy.

Until next time, best wishes for safe travels across the globe or the keyboard. See you in our September PerformanceXpress!

When he is not Internet trawling for ISPI, Todd Packer can be found improving business, non-profit, and individual performance through research, training, and innovation coaching as Principal Consultant of Todd Packer and Associates based in Cleveland, Ohio. He may be reached at tp@toddpacker.com.

by Carl Binder

I’m pleased to announce the “new” GOT RESULTS? Campaign that Timm Esque, Julie Capsambelis, I, and others in ISPI are about to launch. We’re excited about the prospect of it becoming an even more valuable contribution to ISPI and the field of human performance technology (HPT).

A Little History
If you’ve attended ISPI’s Annual Conference in the last two years, you have probably seen the GOT RESULTS? exhibits. They have showcased several dozen examples of performance systems or HPT interventions that produced results in the form of directly measured improvements in behavior, job outputs (accomplishments), or business outcomes. The exhibits have been part of an effort that goes back to the mid-1990s when Ogden Lindsley (1999) observed how few actual performance data make their way to ISPI publications. That observation inspired Timm Esque and Pat Patterson to publish their book, Getting Results: Case Studies in Performance Improvement, and prompted a small group of us to begin formulating strategies and tactics to increase the number of measured results captured and shared among HPT professionals and ISPI members.

From an Event to a Campaign
The GOT RESULTS? exhibits have made a public showing of some great case studies, and we are very proud of the effort so far. Most of those cases have been posted as PDF files on ISPI’s GOT RESULTS? webpage. Our objectives have been to provide practitioners with examples of practical measurement, and to demonstrate with data that HPT methods produce valuable, measurable outcomes.

After the 2003 conference exhibit, we had discussions with a number of ISPI’s officers and thought leaders and began to shift our vision for GOT RESULTS? from a conference-based event to an ongoing campaign in which we will accumulate an expanding variety of results case studies and build an informal network of HPT practitioners skilled in measurement and interested in helping others to measure and share the results of their own work. We emphasize the word practitioner, because this is not about academic research. It is about practical measurement and communication of valuable performance outcomes.

With this new vision, we will still have an exhibit at the conference. But our new emphasis will be on building an archive and network of practitioners to serve the promotion, education, and marketing of HPT and ISPI.

A Performance System
My colleague, Timm Esque, helps his clients establish performance systems in which the most essential element is ongoing measurement of results and databased decision-making aimed at improving outcomes (2002). We envision the GOT RESULTS? campaign as a performance system in which we will measure how many promises to submit cases, how many cases actually submitted, and how many people join our informal network of measurement-based practitioners per month. We’ll use those data to adjust our strategies and tactics over time, with the goal of maximizing involvement of our colleagues and submission of GOT RESULTS? cases.

Get Involved
There are many ways that you can get involved. You can check out the revised webpage. You can download the new submission form and be on the lookout for opportunities in your own work or in the work of your clients and colleagues, for measuring performance in ways that will satisfy the GOT RESULTS? criteria. Also, keep an eye out for publications or conference presentations with results that would fit our criteria, and then help make the connection so we can include those cases in the ISPI archive. And finally, if you’d like to help in the overall campaign, please contact me by email.

Over the coming months, we’ll be enhancing the GOT RESULTS? webpage, linking it to more places on the ISPI website, and doing everything we can to solicit cases. Please join us in helping to build the GOT RESULTS? archive and network of practitioners.

References
Esque, T.J. (2002). Making an impact: Building a top-performing organization from the bottom up. Atlanta, GA: CEP Press.

Esque, T., & Patterson, P.A. (Eds.). (1998). Getting results: Case studies in performance improvement. Amherst, MA: HRD Press, Inc.

Lindsley, O.R. (1999). From training evaluation to performance tracking. In H.D. Stolovitch and E.J. Keeps (eds.). Handbook of Human Performance Technology, second edition. San Francisco: Jossey-Bass/Pfeiffer, 210-236.

by Dick Clark, CPT

Many training studies have been conducted in the past 40 years with conflicting results. A new summary, published in the Journal of Applied Psychology, gives the most accurate view to date of the evidence for the average impact of training found in all adequately designed and published field studies from 1960 to 2000 (Arthur, Bennett, Edens, & Bell, 2003).

The authors of the summary, researchers from Texas A&M University and the Air Force Research Laboratory, reviewed all 636 studies published in the past four decades and selected only those that were adequately designed. They limited their review to studies that had actually been conducted in organizations (they eliminated all “laboratory” studies from their review). They subjected the results of the 162 studies (26% of the total) reporting 397 comparisons that survived their design review to a statistical process called “meta analysis.” This process is thought by many to reveal the most accurate “average” impact of a performance improvement strategy such as training. The authors used the Kirkpatrick four-level system for categorizing the results of the studies they summarized and averaged.

The quick bottom line of this huge review is that, on the average, training has resulted in an approximately 20% performance increase at all of the Kirkpatrick levels.

You will find the article that reports the study in detail in a good university library, but until you have the opportunity to read it yourself, here are some of the findings they report:

• The average learning (level 2) impact of training is a 20% increase in knowledge and skills. So if the average trainee received a score of 50% on a performance test before training, they would have received about 70% on the same test after the training.

• Only 7% of studies report level four impact results—but in those studies level four results were a very impressive 18%. This can be interpreted as indicating that the division or organizational performance that was supposed to be improved by the knowledge and skills that were transferred to the job increased 18% as a result of training.

• Level three (transfer of training to work settings) impact was also about 19-20%. This implies that in these well-designed studies that presumably reflect the best training models, most of what was learned was transferred back to the job and applied.

• The most effective training content was interpersonal skills (such as listening, teamwork, communication) and resulted in 27% learning—but when a cognitive approach to interpersonal skills was used, the result was a whopping 64% increase in learning and a 25% transfer level.

• Only 6% of the studies reported that a needs assessment was conducted before the training was designed and implemented. Since these studies probably reflect the best training programs (people are not inclined to conduct experiments on programs they do not trust), and since people are inclined to report a needs analysis when it has been conducted, it is doubtful that needs analysis is well established in the training community.

• When the researchers asked what training medium was associated with the highest levels of learning, transfer, and impact, the winner was clearly “live instruction” by a trainer (they called it “lecture”)—a 44% increase in learning.

• It is interesting that 6% of the published studies reported a negative result of training on learning, transfer, and/or impact. This means that some training programs significantly decreased learning, transfer, and impact. Even more interesting is the author’s claim that this percentage should be about 10% and that the missing negative impact studies suggest that when experiments find negative impact, they are not often published. This gives the wrong impression that training almost always has a positive result when about 1 study in 10 is negative.

• Finally, there seemed to be no “fall off” or “decay” of training results over time. In the studies where delayed measurements of the four levels were obtained, the results of training seemed to hold up over time.

One might ask whether the studies that were reviewed are representative of the training that is currently being implemented in today’s organizations. I constructed a quick frequency count of the percent of studies that investigated each of the four Kirkpatrick levels and compared the percentages with those reported last year in the ASTD training survey (2002). You can see the results in the table below.

Table 1. Percent of Kirkpatrick Levels studied
in Arthur et al. and reported by ASTD.

You can see that many more training organizations report collecting reaction data than was collected by the authors of the studies that were reviewed. Yet it is possible that the researchers conducting the reviewed studies reasoned that reaction data were less important than learning, transfer and impact data. Notice that the frequency of the “bottom line” data is exactly the same in the two samples and that the learning and transfer data is closer to the averages reported in the ASTD survey. These comparisons lead me to conclude that the studies reported in the review might be representative enough of the training that is currently taking place.

References
Arthur Jr., W., Bennett Jr., W, Edens, P., & Bell, S.T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88(2), 234-245.

Van Buren, M.E., & Erskine, W. (2002). The 2002 ASTD state of the industry report. Alexandria, VA: American Society of Training and Development.

Dick Clark is the principal of Atlantic Training Inc., a company that offers consulting on training design and performance improvement strategies. He is also a professor of Educational Psychology and Technology in the Rossier School of Education at the University of Southern California where he serves as the mentor of USC’s doctoral program emphasis in Human Performance at Work. Dick received the Thomas F. Gilbert Award for Outstanding Professional Achievement from ISPI last year. He and Fred Estes recently authored Turning Research into Results: A Guide to Selecting the Right Performance Solutions (CEP, 2002). Dick may be reached at clark@usc.edu.

by Guy W. Wallace, CPT, ISPI President

The third Board of Directors meeting, of the six in total, was held in San Francisco, July 18-20. It was the halfway point in the year for this Board, and some of us are already feeling a bit of sadness regarding the “end being near.” ISPI Board experience is like that—I know, as this is my second time hitting the mid-point. Ahhh.

But this also reminds me to encourage you to serve our professional home on committees and task forces, at the Board level, as a Director, and then as the President-elect and President. Not for your own ego gratification, which these experiences can indeed offer, but for your own development, both professional and personal.

I have personally learned so much from these experiences over the past 24 years of my active membership that I cannot imagine where my career would be without the insights, knowledge, skills, and friendships that ISPI has afforded me. Talk about a Value Proposition! Click here to check out the nominations section on ISPI’s website for more information including position descriptions and qualifications criteria. The deadline for nominations for the ISPI Board of Directors is August 29, 2003.

But I have digressed.

Executive Director Rick Battaglia, Treasurer Barbara Gough, and I were joined by Past President Jim Hill for a pleasant round of golf on the Thursday prior to the Board meeting. I am sure he has some of those pangs, missing the dialogue, the action, and the learning take-aways from Board meetings. It was great catching up with Jim, who continues his involvement with ISPI and his advocacy of HPT in his post-presidency period.

Thursday evening included a dinner meeting with long-time ISPI/HPT advocate Paul Harmon—check out his interview with Carol and Roger featured above in this month’s TrendSpotters. We discussed his involvement and thoughts regarding both Six Sigma and SCOR and HPT. Paul worked for Tom Gilbert and Geary Rummler in the 1960s and with ISPI’s President-elect Don Tosti in the early 1970s. He has deep roots in HPT.

The Board meeting began on Friday morning with the standard “check-in” updates from the Board regarding their personal and professional lives (we each have both!). As it had only been one month since our last meeting, we moved quickly into updates regarding Society happenings, and then got into the meat of the agenda.

The major topic was the continuation of the development of the ISPI’s Strategic & Operations Plan, which we are deliberately “not rushing through” and taking (what seems like) the whole year to complete. My hope is that we will provide the Society with an overview of the details sometime in September or October.

Other items addressed included the review of the initial action plans and budgets from those committees and task forces that met our window for review and reaction prior to finalizing both for the September Board of Directors meeting, where we will lock down the budgets for the next fiscal year. Until the October PX…Cheers!

Cingular, Capital One, CEP Press, University of Toyota, Walgreens…and members like you!

by Conrad G. Bills, PhD

Conrad G. Bills is training manager for F-16 Training Programs. He serves on multiple training system programs in ISD for Lockheed Martin Naval Electronics & Surveillance Systems, Akron, Ohio, covering all phases of the ISD process. He is responsible for the F-16 MTC pilot training system concept, and his understanding in the Training Systems Integrated Product Team was incorporated during the concept phase of the Joint Strike Fighter (JSF) Program. He joined Lockheed Martin in 1993 upon his retirement from the U.S. Air Force. Conrad holds a PhD from Kent State University. Conrad may be reached at conrad.bills@lmco.com.

 

It is time once again for you, the ISPI membership, to determine the future direction of ISPI by nominating those members that you feel have the qualifications, experiences and vision to lead our Society. Up for nominations this year are the President-elect and three Board members. They will join the President, two continuing Board members and the non-voting Executive Director who make up the eight-member Board.

The duties of the Board are to manage the affairs of the Society and determine the strategic direction and policy of the Society.

Brief Job Descriptions

President-elect
The President-elect assumes the Presidency of ISPI for a one-year term at the conclusion of his/her one-year term as President-elect. The President-elect’s efforts are directed to assuming the Presidency and assignments are designed on preparation for that transition. The President-elect serves to provide continuity of programs, goals, objectives, and strategic direction in keeping with policy established by the Board of Directors.

Director
Each Director on the Board serves a two-year term and is a leader in motivating support for established policy. He or she serves to develop new policy and serves to obtain support for ISPI’s programs. A director should provide an objective point of view in open discussion on issues affecting the membership and profession. He or she should thoroughly analyze each problem considered, vote responsibly and then support those actions adopted by majority vote. Individually, each member of the Board is considered a spokesperson for ISPI and represents the integrity, dedication, and loyalty to established policy.

The deadline for nominations is August 29, 2003. If you would like to nominate someone, please send the your name and contact information along with the nominee’s name and contact information to april@ispi.org. Be sure to indicate the Board position you are submitting for in the Subject line. If you are interested in additional information on the nominations process or the complete job descriptions and qualifications required, click here.

The Certified Performance Technologist (CPT) designation is awarded by ISPI to experienced practitioners in the field of performance improvement and related fields such as instructional design and organizational development whose work meets the 10 Standards of Performance Technology and other application requirements.

Until October 31, 2003, performance improvement professionals with more than six years of experience can apply under the Grandparenting application provision for the CPT designation. After that date, anyone seeking the designation will have to apply under the more rigorous Regular application provision. For more information on the CPT designation and to download the application forms, visit www.certifiedpt.org.

Write and submit a one-page story by August 15, 2003, and you can be published in an upcoming ISPI collection of human performance technology (HPT) success stories. This short, performance improvement book will give readers examples of successful applications of HPT, as well as the inspiration needed to apply HPT in the organizations they serve. The end result will be a book that provides a catalog of success stories from the field of HPT. Click here to learn more!

The ASTD Dissertation Award is given each year to foster and disseminate research in the practice of workplace learning and performance. This year’s award will be presented to the person who has submitted the best doctoral dissertation for which a degree was granted between September 21, 2002-September 20, 2003. The topic must focus on some issue of relevance to the practice of workplace learning and performance. Illustrative areas of concentration include: training and development, performance improvement/analysis, career development, organization development/learning, work design, and human resource planning.

All research methodologies will be considered on an equal basis including, for example, field, laboratory, quantitative, and qualitative investigations. The candidate must be recommended and sponsored by his or her committee chair. All materials submitted must be in English and in Word format by email. Submission requirements correspond to the full manuscript requirements of the Academy of Human Resource Development’s (AHRD) Dissertation of the Year procedures that require applicants to follow the full manuscript conference proposal submission guidelines.

The award winner will receive a $500 cash prize, a commemorative plaque presented at the awards ceremony during the 2004 ASTD International Conference and Exposition, and a designated place on the conference program to present the research (with conference registration fee paid).

Submissions must be sent via email by September 22, 2003 to: Dr. John J. Sherlock, Assistant Professor of Human Resources, Western Carolina University, at sherlock@email.wcu.edu. For further information and submission guidelines, please contact Dr. Sherlock.

• Short “I wish I had thought of that” Articles
• Practical Application Articles
• The Application of HPT
• Success Stories

In addition to the article, please include a short bio (2-3 lines) and a contact email address. All submissions should be sent to april@ispi.org. Each article will be reviewed by one of ISPI’s on-staff HPT experts, and the author will be contacted if it is accepted for publication. If you have any further questions, please contact april@ispi.org.

Go to printer-friendly version of this issue.

Feel free to forward ISPI’s PerformanceXpress newsletter to your colleagues or anyone you think may benefit from the information. If you are reading someone else’s PerformanceXpress, send your complete contact information to april@ispi.org, and you will be added to the PerformanceXpress emailing list.

PerformanceXpress is an ISPI member benefit designed to build community, stimulate discussion, and keep you informed of the Society’s activities and events. This newsletter is published monthly and will be emailed to you at the beginning of each month.

If you have any questions or comments, please contact April Davis, ISPI’s Senior Director of Publications, at april@ispi.org.

ISPI
1400 Spring Street, Suite 260
Silver Spring, MD 20910 USA
Phone: 1.301.587.8570
Fax: 1.301.587.8573
info@ispi.org
http://www.ispi.org