|
Training and performance consulting leaders need fast and credible methods for finding out what is working, and what is not. Traditional evaluation studies that attempt to artificially isolate training “causes” rely on suspicious statistical gyrations, or lead to wild presumptions about causation and outlandish ROI claims. What makes training work has more to do with the performance system than with the training design and delivery itself. The difference is the manner in which the company (or business unit) uses the training, as well as the influence of the prevailing systemic factors such as work habits, reward systems, preparedness of learners, measures, and feedback procedures, to name a few. Enter the Success Case Method: When we discover, as we inevitably do, that some people used their new learning in effective ways, and others do not, the crucial question is why? Why were some trainees able to persevere despite performance system obstacles and make good use of their training? What helped them? Why did some not try at all? What (or who) got in the way? What factors discouraged the large proportion of trainees and eventually forced their performance back to the pre-training levels? The Success Case approach is deceptively simple. It achieves efficiencies by purposive versus random sampling, focusing the bulk of inquiry on only a relative few trainees. The notion is that we can learn best from those trainees who have done the best at applying their learning in their work, and from those trainees who have been the least successful. First, we use a brief survey to identify a small group of exceptionally successful, and unsuccessful, trainees. We then probe trainees from these small “core” samples through telephone interviews. For successes, we (1) document the nature and impact of their application of learning with concrete and indisputable evidence that would “stand up in court” and (2) identify the performance factors (e.g., supervisory support) that enabled these few to achieve the greatest possible results. With the unsuccessful trainees, we identify the performance system obstacles that kept them from using their learning. Opening the Door to Performance Consulting
NOTE: Based on the book The Success Case (Berrett-Koehler 2003).
|
|
||||||||
| |
|||||||||
|
Top Three Predictions Causes of These Predictions In recent years, the big consulting firms have found South Africa to be a welcoming and fertile field for their business. Their presence was sought and valued by senior managers who held consulting firms from the UK or Europe in higher regard than local consultants. Unfortunately, many of the large consultancies worked exclusively at the organizational level, sometimes neglecting to align people and systems throughout the entire organization. Now, local practitioners are challenged to align these interventions to the work and worker levels, solving problems created by these one-level implementations. SAP installation will remain a popular solution-driven practice, and in the process many South African organizations will, no doubt, experience performance problems at the worker and workplace levels, with impact on individual jobs and work processes. Now, belatedly, local practitioners will apply a systematic and systemic approach to an already-implemented project to integrate the pre-determined solutions in their organizations. The alignment of the work approaches used by OD, HR, and training practitioners will potentially provide a huge opportunity for those who work in these functions. Because they offer similar products and services to the same internal customers, practitioners in all three disciplines will use human performance improvement methodologies to align their work processes, educate their customers, and add value for their organizations. ISPIs Standards of Performance Technology could be instrumental in shaping this critical alignment. How Organizations Will Be Different Organizations will be better able to identify and target issues and opportunities. Critical business initiatives will have more staying power when implemented through alignment. The user-friendly, approachable solutions possible through an HPT approach will provide visible increased value, demonstrating that outstanding results are possible from simple interventions. Implications for HPT Work
|
|
||||||||
|
Not all performance is worth improving. Some results should be improved, some maintained, and some reduced or eliminated. In Performance-Based Instruction, Brethower and Smalley (1998) point out that performance improvement interventions always add cost, and only sometimes add value. Many organizations evaluate the worth of an intervention by way of comparing the cost of a solution against the value of results returned. Less frequently considered is the cost of the problem—the cost of doing nothing. When costs go unmeasured, its irresponsible. When worth goes unmeasured, its irrational. If we only measure costs, we act as accountants who simply keep track of debits. If we only measure worth, we are little different than sports commentators whose job it is just to keep score. When both costs and worth are measured, we become informed consumers...and wise investors in process improvements. The interventions we develop in pursuit of improved performance—improved results—always have some costs associated with them. But if the value of the results accomplished does not outweigh both the cost of the problem and the cost of the solution, we subtract value. Value is only added when the benefits of an intervention exceed both the cost of doing nothing and the cost of solving known problems. Consider a $50,000 performance problem with a $20,000 training solution. A typical designer would claim “value added” if the return was greater than $20,000. This, however, ignores the fact that the organization started from a deficit of $50,000. So, to be credible, claims of “value added” and “effectiveness” must be backed by data regarding the cost of the problem, the cost of the solution, and the worth of results. Unless we have evidence that an intervention provides worthwhile results, cost reductions in and of themselves are of little consequence. Indeed, it is only when we have this evidence that the discussion of an interventions efficiency (or cost reductions) become meaningful. Only in this case do we truly improve performance. The term “value” in “value added” and “value subtracted” requires that we know not only about what we “give” (expenses and costs), but also about what we “get back.” This notion is also the foundation of return-on-investment analysis. Performance technologists who seek to provide measures of both the costs and the value of performance improvement initiatives are more likely to make certain that returns are positive ones. Thanks to Tom Gilbert for introducing the idea of “worthy performance” to the field. Thanks to Dale and Karolyn for helping to explain it. References Watkins, R. & Leigh, D. (2001). Performance improvement: More than just bettering the here-and-now. Performance Improvement, 41(8), 10-15.
|
|
||||||||
by Sivasailam “Thiagi” Thiagarajan |
|||||||||
Remember your childhood days when you made up stories with your friends, each person contributing different segments to the story? Have you ever watched (or participated in) an exciting improv game in which players take turns to add a sentence (or a word) to an ever-growing story? An Online Version An Invitation The website has two pages. The story page contains the title for a story and the first two paragraphs. These are written by me to get things started. From now on, I will be an outsider, and let you and the other players take over. To participate, you read the current version of the story and use a simple “form” to add a paragraph to it. You type your name (as a co-author) and type your paragraph to advance the story from where it ended. Then you click a button and immediately see the evolving story with your paragraph appended at the end. You can come back to the story and watch it grow. You may add additional paragraphs, but with this important constraint: You can add more paragraphs only after at least two other players have contributed their paragraphs. The website also contains a welcome-and-debriefing page. This page uses a similar “form” for your comments about the story and the collaborative story-making process. It enables me, as the facilitator, to post additional instructions and suggestions. You may choose to lurk. You may visit the interactive fiction website to see what is happening and to read the story without actively participating in the process. You may also visit and read the debriefing page and add your comments. Suggestions for Play
If you are ready, click here to go to Pattis Interactive Fiction website.
|
![]() |
||||||||
|
The goal of our efforts was to break down organizational “silos” that inhibited effective communication and coordination. More and more organizations are realizing that strong teamwork within groups needs to be balanced by cooperation across groups to avoid internal barriers that interfere with effectively accomplishing organizational results. Partnering is critical to ISPI, not only as a base for establishing relationships with other organizations, but as one of the four keystones of performance technology consulting—defining our relationships with our clients. There are four key elements of successful partnering, that we sometime refer to as the “what” and three “hows.” What: The foundation for partnering. The essential foundation for a partnering relationship is agreement on a mutual goal. A mutual goal is defined as one that both parties can contribute to, and that both parties benefit from. A clear mutual goal creates the foundation for a relationship that goes beyond “win-win”—it is winning together. And that is the key to the power of partnering—an understanding between equals. How: The principles that guide a partnering relationship. The three “hows” define the way people treat each other—the values and practices that guide a successful partnering relationship.
Over the years, we have found this basic partnering model to be effective in several arenas: to break down organizational silos, to establish working relationships with clients, and to ensure the effectiveness of joint ventures. Partnering is but one example of how the power of our technology—our focus on results/goals/outcomes, and analysis of the behaviors that contribute to those results—can be used to deal with some of the most pressing issues in organizations today. To cite an example close to home: ISPI is currently developing partnering relationships with other human resource and performance technology organizations, such as ASTD and IFTDO. The purpose is to combine resources to work toward mutual goals that benefit the members of both organizations. For instance, ASTD and ISPI are cooperating in the development of a performance technology certification process. By working as partners in the development of a single certification process, we have the potential to create far greater benefit to performance technology and its practitioners than by working independently or in competition.
|
|
||||||||
Who Should Attend? Keynote Address Full-Day Workshops Workshop topics include:
Performance Improvement Institute Conference CD-ROM ISPI Membership Registration
|
![]() |
||||||||
|
|
|||||||||
| Last month I promised to show more about how charts can facilitate communication about results. Ill use three charts to illustrate how using Lindsleys (1999) standard charting system can support decisions and analysis without statistics or fancy math.
Figure 1 shows a small businessperson using data to decide about the value of distributing brochures. You can see a decelerating trend in client appointments before she began distributing the brochure, and how the brochure accelerated client volume by about 60% per six months. The chart clearly depicts a change in trend without the usual distortions in “bounce” or variability that occur on more typical “stretch to fill” charts. Lindsleys charting system can do the same for individual count-per-minute behavior, widgets per hour from a process, positive comments per week, dollars per month, or any other count we might collect over time. It allows us to see and understand trends, levels, and bounce independently and quite easily, without statistics. Let me show you how using charts earned me a new client engagement. In the early 1980s I was developing training at a national weight management program for people at serious health risk due to obesity. With about 40 clinics around the country, they were attempting to grow the start-up business. The example here is from a clinic that did not do so well—largely because management failed to understand the data they were collecting, and did not heed my warning based on projected trends. (In fact, they used spreadsheets, not any form of graphic representation to manage their business, with the all-too-common result that missed critical information about trends or variability that might have led to different decisions.)
Figure 2 shows revenues in thousands of dollars per month for a single clinic in Southern California—one of their largest. I had requested copies of clinic spreadsheets to analyze the information graphically to see if I could make any helpful suggestions. The charted trend was clear: I predicted that clinic revenues would continue to decelerate if they did not make key changes in sales, marketing, or client management. (I used other charts as well, e.g., of new and departing clients per month, but this one is representative.) Because the revenues were still among the highest in their national network, no one listened and they continued to run the business without significant change.
Figure 3 shows what happened. With astounding (to them) regularity, revenues continued to decline at about 60% per six months until management was forced to close the clinic. Note that neither the trend nor the bounce changed until close to the end when the clinic more or less imploded. My fulfilled prediction got the attention of senior management who asked me to use my “fancy charts” to predict the divisions annual revenues. I used charts from all 40 clinics, projected the trends and “envelopes of bounce” for each, and made estimates based on purely visual inspection. My estimate was less than 5% off of their eventual $12 million revenue, while the CFO with the assistance of the MIS department made a projection of around $20 million—almost twice actual revenues. After that demonstration, I received new assignments beyond training and documentation because they thought I knew something that they did not. In fact, all I knew was how to use a standard chart to monitor counts, and how to project trends and bounce using purely visual methods to support decisions and predict outcomes. Since that time, I have used this charting system in all kinds of situations to support decisions and to better understand processes, behavior, and results. Using the right chart can make a big difference. If youd like to know more about Lindsleys measurement and charting methods, please let me know. Reference
|
|
||||||||
![]() by Bob Bodine, 2004 Conference Chair |
|||||||||
|
Academic Forum (30-, 45-, 60-, or 90-minute sessions). Headed by Jim Pershing, ISPI Board member, this new track seeks session proposals offering papers, discussion forums, panels, or debates on HPT philosophy, history, theories, research meta-analysis, evaluation methods, research projects, or dissertations/thesis presentations. Innovative Forum (60- or 90-minute sessions). Headed by Andrea Moore, this repeat track seeks session proposals offering experiential learning opportunities that break the concurrent session mold. International Forum (60- or 90-minute sessions). Headed by Jolanda Botke, this new track seeks session proposals about practicing HPT in a specific country/culture, practicing HPT internationally, partnering cross-culturally, or other international perspectives. Concurrent Session (HPT Fundamentals, HPT Applications, and HPT Research) (90-minute sessions). Track Chairs welcome session proposals from the academic community and students on recent research. This provides students with opportunities to speak before their professional community and enriches the knowledge base of our Society. Pre-conference Workshop. This repeat feature headed by Michael Peters will bring practitioners together to offer half-, one-, and two-day pre-conference workshops, at an additional cost to those who participate. Look for more details on these workshops in later issues of PerformanceXpress. Performance Gallery. This format allows you to create a visual display about a successful project. The Performance Gallery provides an opportunity for participants and attendees to view and discuss each presentation. How can you participate? Attend! Present! Volunteer! It is not too early to mark these dates on your calendar:
Here are some suggestions to help you prepare a successful conference proposal submission, especially if you are a novice speaking at ISPI:
We look forward to receiving your ideas for making the 2004 Conference program the best ever!
|
|||||||||
| |
|||||||||
|
Quick recap: Every month, three sites, one theme. While far from comprehensive, hopefully these sites will spark readers to look further and expand views about human performance technology (HPT). Please keep in mind that any listing is for informational purposes only and does not indicate an endorsement either by the International Society for Performance Improvement or me. These are the general categories I use for the sites featured:
The theme for this months column is “Schools out.” Summertime and the learning is easy...? School is out this time of year in many places. However, as performance technologists and lifelong learners, we challenge our knowledge resource environments (including schools) to respond to our needs. Pack your lunch, remember your number 2 pencils, and lets visit some sites where school never ends. Oh, and you cant use the excuse that the 1-D totalistic cellular automata ate your homework. E-Klatch HPT@work I-Candy Until next time, this ever-evolving “I”-life wishes all of our readers a great summer. See you in our July PerformanceXpress!
|
![]() |
||||||||
| |
|||||||||
|
“Why does the class have to be two full days, back-to-back? I cant have all my employees out at the same time for that long. Can you condense it to a half day?” “Sorry about the employees who didnt come back to training after lunch, but we had a lot of fires to put out.” “I want an entire series on teamwork skills, but not during the first and last weeks of the month, and only on a Tuesday, Wednesday, or Thursday. And be sure you avoid conflicts with the technical training schedule. Oh, and we need to get this done in the next two months.” Sound familiar? Many workplaces today function with extreme urgency, large or unpredictable volumes of work, and increasing pressure to demonstrate shareholder value. The slogan in this kind of environment could easily be, “Revenue is king—short-term production is queen!” This means trainers often hear a mixed message from management: “Yes, I want employees trained. I want training myself, but I cant spare a lot of time for it. Getting orders processed is more important—we need to make our objectives because we need the revenue. Maybe next quarter.” How can trainers resolve this problem of not enough time for training? Persuade managers to change their priorities and perceptions of training? Better educate managers on the financial consequences of lack of training to the business? Change careers? Clearly, persuasion and education are long-term efforts, and changing careers may not be desirable. However, there is another option: trainers can ask themselves, “How can I deliver effective training in shorter periods of time?” Albert Einstein once said that problems cannot be solved at the level at which they were created. Is there something in the way that trainers think about training that contributes to the corporate belief that theres not enough time for training? Are firm, but outdated, beliefs held about how training is supposed to be—such as:
These beliefs sound wonderful—and, in part or total, do not fit anymore. For instance, many newspapers report how Americas workplaces are changing in dramatic ways—including reorganizations, layoffs, and budget cuts for “expendable” areas such as training. Also, first-time managers are often technical experts who can “do the job” but do not yet know how to coach people. Trainers need to think of new ways to deliver training within time constraints—regardless of whether they think those constraints are real. The workplace requires trainers to rise to a new level of thinking. As key players in organizational change, trainers have an opportunity to model change within their own profession. Why would trainers want to develop new strategies to deliver training in shorter timeframes? One reason is to increase managements perception that trainers are valuable members of the management team, focused on meeting the same business goals as management. If trainers play an active role in day-to-day operations by using new strategies to deliver training, it may be more difficult to see training as expendable when budgets are cut. If not the trainers demonstrating a higher level of thinking, who? If not now, when?
NOTE: Part two of this series, Not Enough Time: New Strategies for Resolving the Training Dilemma, will provide new and practical approaches for delivering effective training within time constraints.
|
|
||||||||
| |
|||||||||
|
The session explored the breadth of research activities in which ISPI members are engaged, from new syntheses of yesterdays efforts, to todays cutting-edge investigations, to tomorrows issues. Engaging in lively interactions, session participants and contributors jointly explored, questioned, learned, debated, and most of all, showcased the critical role of research in our community of practice. Research is our foundation, our current practices, and our future. Without research, ISPI would not have the tools, models, theories, or practices that we use today. Without research, we will have none for tomorrow. Naturally, we have both qualitative and quantitative data to support the sessions success. The results are presented below, somewhat tongue-in-cheek. Qualitative Results
General
Value & Demand
Format
Quantitative Results
The measures of central tendency and dispersion, shown in the bar chart, illustrate respondents overwhelming approval ratings. (Of note is the ceiling effect.) The range of central tendency measures are reported to accommodate appropriate measurement scales. The data began life as nominal, therefore modes (5 for all items) are reported. To accommodate the evaluation instruments transformation of the native nominal data into ordinal data by assignment of the 5-point scale (1 = strongly disagree; 5 = strongly agree), medians (5 for all items) are reported. Although the data are neither interval nor ratio, and acknowledging the unfortunately ubiquitous violation, means are reported. Dispersion (SD, standard deviation) was notably and consistently narrow (less than a single unit of measure) across all items, indicating (a) the reported values of central tendency represent the data well and (b) between-subject reliability was high. Summary
|
|||||||||
| |
|||||||||
|
Why a conference with such a title? Thinking about this question takes us back nine months when the people involved with ISPI Europe started planning this conference. Building on the success of ISPI Europes first conference focusing on Global Fluency, we wanted to find a new topic that would be worth exploring from a variety of different angles, with people coming from many different cultures, and working for all kinds of organizations. In other words, we wanted to find a conference theme thats rich and interesting enough for such a holistic investigation. When you look at this title, you will find that each word carries a key to the ideas and concerns we debated when planning the conference. Naturally, it all starts with performance. Improving performance is at the center of ISPIs raison dêtre. The models, tools, and practices of our members differ in many aspects, but we all share this common goal. Performance is such a versatile word stretching from the vocabulary of finance to the world of performing arts. Is it not synonymous with doing? Doing to get a result that adds value to customers, employees, clients . However, even after successful interventions to improve performance, organizations tend to revert back to their former status. Practitioners in and outside organizations are puzzled about how to sustain achieved improvements. Here we use the word sustaining in a concrete performance context. Sustaining has another, wider connotation: Sustainability is a balancing act between an organizations need to be profitable and innovative in order to survive, and societys need for a decent living for all, security, and justice, as well as bequeathing to future generations a livable planet. The emphasis of human performance technology (HPT) is rightly on its scientific approach: methodical, proven, measurable, and replicable. That is why we call it a technology. But if it were that simple as the word technology might imply, why are more people not successfully doing it? Maybe it is because HPT is a technology about people, it is about human performance. And people are often irrational, unpredictable, anarchic, and unique. We therefore need an artists approach: create, adapt, improvise, envision with humor, and tolerate. No matter how artistic (and maybe humanistic) we sometimes want to be, the bottom line is results, valuable results, delivered effectively and efficiently to customers. Only valuable results lead to profits and only profits allow organizations to exist in the long run for the sake of their owners, employees, customers, and society at large. This is where the circle closes: only sustainable performance achieved with the individual human being in mind will deliver lasting results for all of us—the employees, the organizations, and the societies. Its a balancing act. Our conference invites you to come together with other people who grapple with this balancing act. We want to discuss, share, and learn theory and practices leading to success on the ideas and concerns the titles keywords imply: Sustaining Performance: The Art of Getting Results. We look forward to welcoming you to a challenging and fun conference. For more information or to register, click here.
|
| ||||||||
ISPI recently became an Associate sponsor of the 2003 Workforce Innovations Conference, July 8-10 at the Marriott Wardman Park Hotel in Washington, DC. Workforce Innovations is the nations premier conference for Americas successful workforce investment leaders, decision-makers, and change agents. The conference will explore the importance of linking employment, education, and economic development to respond to Americas changing economy. For more information, visit: www.workforceinnovations.org.
|
![]() |
||||||||
|
|||||||||
ISPI is looking for Human Performance Technology (HPT) articles (approximately 500 words and not previously published) for PerformanceXpress that bridge the gap from research to practice (please, no product or service promotion is permitted). Below are a few examples of the article formats that can be used:
In addition to the article, please include a short bio (2-3 lines) and a contact email address. All submissions should be sent to april@ispi.org. Each article will be reviewed by one of ISPIs on-staff HPT experts, and the author will be contacted if it is accepted for publication. If you have any further questions, please contact april@ispi.org.
|
|||||||||
|
|
|||||||||
|
PerformanceXpress is an ISPI member benefit designed to build community, stimulate discussion, and keep you informed of the Societys activities and events. This newsletter is published monthly and will be emailed to you at the beginning of each month. If you have any questions or comments, please contact April Davis, ISPIs Senior Director of Publications, at april@ispi.org. ISPI |
|||||||||