by Jack J. Phillips


A previous article by Patricia Phillips
focused on the benefits and barriers to implementing ROI, based on her new book, The Bottomline on ROI. This article focuses on the actual ROI process model, detailed in Measuring Return on Investment in Training and Performance Improvement Programs.

Building a comprehensive measurement and evaluation process is best represented as a puzzle in which the pieces are developed and put in place over time. Figure 1 depicts this puzzle and the pieces necessary to build a comprehensive measurement and evaluation process. The first piece of the puzzle is the selection of an evaluation framework, which is a categorization of data. The framework selected for the process presented here is a modification of Kirkpatrick’s four levels, to include a fifth level: return on investment (Kirkpatrick, 1975).


Figure 1. ROI: The Pieces of the Puzzle

Next, an ROI process model must be developed, showing how data are collected, processed, analyzed, and reported to various target audiences. This process model (see Figure 2) ensures that appropriate techniques and procedures are consistently utilized to address almost any situation.


Figure 2. ROI Process Model

The third piece of the puzzle is the development of operating standards. Labeled guiding principles, these standards help ensure the results of the study are stable and not influenced by the individual conducting the study. Guiding principles allow for replication so that if more than one individual evaluates a specific program, the results will be the same.

Next, appropriate attention must be given to implementation issues, as the ROI process becomes a routine process. Several issues must be addressed involving skills, communication, roles, responsibilities, plan, and strategies.

Finally, there must be successful case applications and practice describing the implementation of the process, the value that a comprehensive measurement and evaluation process brings to the organization, and the impact the specific program evaluated has on the organization.

The remainder of this article focuses on the individual pieces of the evaluation puzzle: developing a comprehensive ROI process.

Evaluation Planning
One of the most important and cost-saving steps in the ROI process is planning the evaluation. By considering several key issues during the planning, time, money, and frustration can be significantly reduced.

Collect Data
Following the planning process, implementation begins. Collecting both hard and soft data is central to the ROI process. Data are usually collected during two time frames. Reaction and learning data (Levels 1 and 2) are collected during the learning or performance improvement program. Application and business impact data (Levels 3 and 4) are collected after the program is implemented. A variety of methods are used to collect the post-program data to be used in the ROI evaluation, ranging from surveys to monitoring business records.

Isolate the Effects of the Program
In this step of the ROI process, specific strategies are explored that determine the amount of output performance directly related to the program. This step is essential because there are many factors that will influence performance data after a program is conducted. The specific techniques utilized at this step will pinpoint the amount of improvement directly related to the program. A few of these techniques include control groups, trend line analysis, or expert estimates.

Convert Data to Monetary Values
This step in the ROI model is absolutely necessary to determine the monetary benefits from a program. This step requires a value to be placed on each unit of data connected with the program. Approaches range from using standard values to expert estimates. The process is challenging, particularly with soft data, but can be methodically accomplished using the correct techniques.

Tabulate Costs of Program
The next step is tabulating the costs of the program. This involves monitoring or developing all of the related costs of the program targeted for the ROI calculation. The conservative approach is to include all of the costs—direct and indirect—so that the total is fully loaded.

Calculate the ROI
The return on investment is calculated using the program benefits and costs. The cost/benefit ratio (BCR) is the program benefits divided by cost. In formula form it is

BCR =

Program Benefits

 

Program Costs


The return on investment uses the net benefits divided by program costs. The net benefits are the program benefits minus the costs. In formula form, the ROI becomes

ROI (%) =

Net Program Benefits

X 100

Program Costs


This is the same basic formula used in evaluating other investments where the ROI is traditionally reported as earnings divided by investment.

Identify Intangibles
In addition to tangible monetary benefits, most programs will have intangible non-monetary benefits. Data items identified that are not converted to monetary values are defined as intangible benefits.

Summary
The ROI process generates six types of data (reaction, learning, application, business impact, ROI, and intangibles), representing a balanced profile of success of a learning or performance improvement solution.

As a world-renowned expert on measurement and evaluation, Dr. Jack J. Phillips provides consulting services for Fortune 500 companies and workshops for major conference providers throughout the world. Phillips is the author or editor of more than 30 books— 10 about measurement and evaluation— and more than 100 articles. He may be reached at roiresearch@mindspring.com.

 

Would you like to advertise in this space? Contact marketing@ispi.org

 


  The ROI process generates six types of data (reaction, learning, application, business impact, ROI, and intangibles), representing a balanced profile of success of a learning or performance improvement solution.



by Todd Packer


I’m honored to introduce I-Spy, a new feature of PerformanceXpress that will highlight relevant, interesting, and useful websites for Performance Technologists. In my work, I often use creative research techniques to uncover resources on the world wide web that can help individuals and organizations make effective decisions and achieve strategic goals. For this column, I hope to locate off-the-beaten path websites that can help you find similar thinkers, resources, work, new ideas, and sometimes just plain old fun.

Each month, I will highlight three sites connected by a general theme. While far from comprehensive, hopefully these sites will spark readers to look further and expand views about human performance technology (HPT). Please keep in mind that any listing is for informational purposes only and does not indicate an endorsement either by myself or the International Society for Performance Improvement (ISPI).

These are the general categories I will use for the sites featured:

  • E-Klatch: Links to professional associations, research, and resources that can help refine and expand our views of HPT through connections with other professionals and current trends.
  • HPT@work: Links to job listings, career development, volunteer opportunities, and other resources for applying your individual skills.
  • I-Candy: Links to sites that are thought provoking, enjoyable, and refreshing to help manage the stresses and identify new ideas for HPT.

The theme for our first column will be Academics. As a bridge between research and practice in HPT, ISPI connects Main Street, Wall Street, and the ivory tower. For a peek into the academic world, here are a few sites of note.

E-Klatch
The combination of research and education forms a basis of HPT and links to the field of educational research. The American Educational Research Association apparently shares ISPI’s interest in connecting process improvement, research, and practical application. Of particular note, given the Lessons in Leadership theme for the 2003 International Performance Improvement Conference & Expo, April 11-15, in Boston, MA, is the Call for Papers (November 12, 2002 deadline) for their journal Educational Evaluation and Policy Analysis for a special issue focusing on Education Leadership and Instructional Improvement. The American Educational Research Association (http://www.aera.net/) is concerned with improving the educational process by encouraging scholarly inquiry related to education and by promoting the dissemination and practical application of research results.

HPT@work
HPT practitioners often find satisfaction in academic work settings. The opportunity to pursue research to validate techniques, to teach and design curricula, to publish, and to apply the techniques within institutions draw many people to this area. Several websites and portals exist that link professionals to positions in academic settings. Keep in mind that HPT skills can also assist in the administration of academic settings, so reviewing positions in this area may be fruitful.

Here’s one:
Academic360.com is a meta-collection of Internet resources that have been gathered for the academic job hunter. It includes links to faculty, staff, and administrative announcements and is not restricted to teaching positions. For faster access the use of graphics and “bells and whistles” is kept to a minimum. Enjoy the site: http://www.academic360.com/!

I-Candy

Who says it’s hard to write like an academic? For an interesting approach to generating quick articles in “academic speak,” visit the Postmodernism Generator, which uses a computer application to randomly generate essays each time you refresh/reload the web page. This may come in handy for your next Performance Improvement article.

The Postmodernism Generator (http://www.elsewhere.org/cgi-bin/postmodern/) was written by Andrew C. Bulhak using the Dada Engine, a system for generating random text from recursive grammars, and modified slightly by Josh Larios.

NOTE: Listings above are for informational purposes only, and do not indicate an endorsement either by Todd Packer or ISPI.

Todd Packer is an independent consultant providing research, coaching, writing, and training in organizational development, creative problem-solving, and stress management. He seeks to improve performance through dynamic new approaches to research and creativity at work. Please contact Todd with your feedback, comments, and ideas at tp@toddpacker.com. For more information, visit www.toddpacker.com.

 

  



by Carol Haig, CPT and Roger Addison, CPT


We spoke recently with Timm Esque, of Esque Consulting, about two emerging trends he thinks will create opportunities for HPT practitioners. Timm is the 2002 recipient of ISPI’s award for Distinguished Service and may be reached at tjesque@esqueconsulting.com.

Two Significant Trends
Organizations are beginning to shift from the short-term focus of the last 10-15 years to one centered on longer-term systemic management. The costs of basing strategic decisions on this quarter’s balance sheet are becoming painfully visible. Stakeholders, and particularly the shareholders among them, are losing patience as they experience the inevitable fallout from too much emphasis on the short term (e.g., unsustainable business practices and in some cases manipulative corporate bookkeeping).

The bookkeeping problems were predictable. The pressure for favorable business information is pushed from the board and executive office to middle management and the workers. As the pressure builds, people respond by presenting the information their superiors want to see or withholding unfavorable results. At one large organization, a department was three months behind in processing critical documents. Employees never reported this to senior management for fear of reprisal. When customers began complaining, the situation was discovered, resulting in a massive reorganization.

All that has changed recently is that some of this inaccurate information is making it all the way out to the stakeholders. This is partly due to the breakdown of checks and balances but also to the over-reliance on automated information systems. The second emerging trend is that managers are working around IT systems if they must, in order to get the information they need to make good business decisions.

Companies make huge capital investments in enterprise-wide systems. Far too often, these behemoths never do what management expected. Timm recently worked with a divisional vice-president who had been asking his finance department for accurate performance data for monthly review. He was repeatedly told that the new system would soon provide the information. Finally, the VP grew so tired of waiting that he told Finance he didn’t care if they had to use paper and pencil to calculate the figures, he wanted his general managers to see the data now and regularly in the future. The Finance people responded by creating some simple Excel-based tools to meet the VP’s requirements. This may seem like a step backwards, but not if that is what it takes to get the right information to decision makers.

Impact of These Trends
As illustrated in the above anecdote, the impact of these trends is that managers are getting serious (and pragmatic) about getting reliable information and using it to make forward looking business decisions. The recent popularity of buzzwords like “balanced scorecard” and “operational excellence” are further evidence that organizations are trying to move to a longer term, more systemic approach to management.

Implications for Performance Improvement
HPT is about systemic improvement. Timm believes the trend toward taking a longer strategic view and the focus on reliable performance data are an opportunity for HPT practitioners to move beyond short-term HPT solutions and help enable a very visible systemic approach. The balanced scorecard and operational excellence are fancy names for an effective performance system. Who knows more about performance systems than HPT practitioners?

One does not have to be a guru to enable systemic change in an organization. See Timm’s latest book, Making an Impact, for more information.

If you have any suggestions about trends driving performance in today’s business environment that you feel would be of interest to the PerformanceXpress readership, please contact Carol Haig at carolhaig@earthlink.net or Roger Addison at roger@ispi.org.

Would you like to advertise in this space? Contact marketing@ispi.org

 


  



ISPI’s “games guy” and QBInternational’s Resident Mad Scientist (aka Director of Research and Development) Sivasailam “Thiagi” Thiagarajan has created another interactive game designed specially for the readers of PerformanceXpress.

As a performance improvement professional, you are probably familiar with the HPT Model. For a quick review of the model, click here.

Play the HPT Model flash game by visiting: http://www.thiagi.com/ispi/sequence/html/TheHPTModel.html. You will find the seven steps of the HPT Model arranged in a random sequence. Rearrange the steps in the correct order by dragging and dropping them (before the timer counts down to zero). Remember, you can play this addictive game repeatedly. Each time you play, you will begin with a new random sequence of steps. You can even select a level to shorten the time limit to match your skills. Before you know it, you will become so fluent with the HPT process that everybody will be impressed.

Future issues of PerformanceXpress will feature new interactive games from Thiagi.

 

  



by Janet Wiscombe


Many people in HR dismiss awards
and incentives programs as “feel good” activities. But evidence suggests there is a strong link between non-cash awards and incentives and improved job performance, says employee-recognition expert and best-selling author Bob Nelson.

Nelson’s study, conducted from September of 1999 to June of 2000, is based on responses from managers and their employees in 34 organizations ranging from Universal Studios to the U.S. Postal Service.

He says that several performance-related variables were found to have broad support from managers in the study, the majority of whom agreed or strongly agreed with the following items (listed with percentage of agreement):

  • Recognizing employees helps me better motivate them. (90.5%)
  • Providing non-monetary recognition to my employees when they do good work helps to increase their performance. (84.4%)
  • Recognizing employees provides them with practical feedback. (84.4%)
  • Recognizing my employees for good work makes it easier to get the work done. (80.3%)
  • Recognizing employees helps them to be more productive. (77.7%)
  • Providing non-monetary recognition helps me to achieve my personal goals. (69.3%)
  • Providing non-monetary recognition helps me to achieve my job goals. (60.3%)

Nelson also found that 72.9% of managers reported that they received the results they expected when they used non-monetary recognition either immediately or soon thereafter, and 98.8% said they thought they eventually would obtain the desired results.

Of the 598 employees who reported to the managers in the study, 77.6% said that it was very or extremely important to be recognized by their manager when they do good work. Employees expected recognition to occur: immediately (20%), soon thereafter (52.9%), or sometime later (18.8%).

“If you look at companies employees love to work for, you’ll find that they recognize their people and tell them they're doing a great job,” says Nelson, whose books include the best-selling 1,001 Ways to Reward Employees (Workman Publishing, 1994) and, most recently, Please Don't Just Do What I Tell You! (Hyperion, 2001). "Non-cash awards and incentives lower stress, absenteeism, and turnover, and raise morale, productivity, competitiveness, revenue, and profit."

Nelson’s mantra: “You get what you reward.”

NOTE: Janet Wiscombe is Associate Editor for Workforce. This article was reprinted with permission from Workforce, April 2002, p. 44.

 


  Sponsored by ISPI and funded by the SITE Foundation, the purpose of the Incentives, Motivation & Workplace Performance: Research and Best Practice study conducted by Harold Stolovitch, Richard Clark, and Steven Condly was to cut through the conflicts and controversies that have existed regarding the use of incentives to improve performance. Order your copy today!


by Michelle Katz and Marilyn Spatz, 2003 Performance Gallery Co-Chairs


In April 2002, a new element
of the regular ISPI Annual Performance Improvement Conference & Expo program lineup was launched. The Performance Gallery is an interactive session, focusing on a visual display or single poster, each telling a story of a successful human performance technology (HPT) project and its results.

This year, the posters were rich and colorful, capturing the imagination and stimulating discussion. Although most participants reportedly stepped inside the room out of curiosity or for a quick look, most ended up engaged in a conversation with the authors of the posters. Among the participants was Eileen Banchoff discussing her model, Francois Lamott d’Incamps from France, and Connie Swartz sharing how to write a Request for Proposal (RFP) for instructional multimedia. The Performance Gallery became an opportunity for conference attendees from across the globe to share their experiences and knowledge in a comfortable, informal manner.

Take the opportunity now to submit a proposal to participate in the 2003 Performance Gallery. The benefits of submitting a proposal include:

  • Provides a forum for sharing your ideas with other conference attendees in a less structured manner than concurrent sessions.
  • For non-native English speakers, allows your poster or visual aid to help you communicate the nature of your project and its outcomes to others.
  • In 2003, the Performance Gallery will be scheduled on the opening night of the conference, providing you with an opportunity to meet new people early and continue to share with them throughout the conference.

Here are the guidelines to follow when creating your poster or visual aid:

  • Size: (119 cm height x 84 cm width)
  • Printed in portrait layout. (Do NOT handwrite your poster)
  • Readable from a distance of at least three meters
  • Self-explanatory, concise, and clearly communicates results
  • Use fewer words and clear illustrations. Your poster should NOT be a reproduction of your academic paper. Rather, it should present a visual summary of your ideas.

You should also be aware that ISPI audiences are sensitive to commercial material. Logos may be used on title and summation graphics, but should not detract from the business message being presented. Advertising and sales promotions in any form are strictly prohibited.

The deadline for submission of your application to participate in the Performance Gallery at the 2003 International Performance Improvement Conference & Expo is October 14, 2002. For a copy of the application form and submission criteria, contact Ellen Bodalski Kaplan, ISPI Senior Director of Meetings, at ellen@ispi.org.

 

 



by Carol M. Panza, Director


The last copy of the International Society for Performance Improvement’s (ISPI) paper-based newsletter, News & Notes (November/December, 2001) contained an article that I contributed about the focus on the first “I” in ISPI which stands for International. Click here for a PDF of the full article. In that issue, I reported on the progress of a multinational group that was in the early forming stages and which was born out of the Chapter Leadership Workshop conducted by the Chapter Partnership Committee at the 39th ISPI Annual Conference in San Francisco. Within the context of that forum, chapter leaders from existing and forming chapters from several European countries had an opportunity to get together and concluded that they could help each other and, potentially, accomplish things beyond any single country-based chapter by thinking more broadly. Well, think more broadly they did!

A forming/organizing meeting was held in early July in Brussels, hosted by MCE (Management Center Europe). Now, just short of one year later, the forming chapter ISPI, Europe, Middle East, Africa (EMEA) has just held a very successful conference in The Netherlands. There were more than 40 participants from 12 countries in attendance. (It was suggested that there were actually 13 countries represented, if we count Scotland as a separate country. But, we’re not going to go there!) The theme of the conference, appropriately enough, was Global Fluency, inspired by an article written by Don Tosti in the Performance Improvement journal (February, 1999). Don generously donated his time as one of the keynote speakers at the conference.

      

In my News & Notes article, I said, “There are many, many valuable professionals who currently belong to ISPI, and I’m not just referring to the Gurus! I’m talking about the folks you can make a call to or contact by email when you need some advice on a difficult issue. Or, it could be someone who can help you to confirm that you’re not crazy. You’re not alone out there.” ISPI EMEA actualized that concept in the design of the conference format. Concurrent sessions were selected based on their relationship to the global fluency theme and their interactive style. That is, the intent was to have facilitators leading each concurrent session, rather than the more traditional presenter format. During each session, one or more participants volunteered to summarize and “report out” on the learning from that session in a synthesis meeting at the end of the conference. The synthesis session was an opportunity for participants to learn a bit about sessions they were unable to attend. For facilitators, the synthesis session was useful feedback on what messages and learning were derived by participants who attended their sessions.

       

ISPI EMEA is already planning for its next conference in September, 2003 in the vicinity of Paris, France. The intent at this point is to keep the conference relatively small so that a highly interactive format can be maintained and, hopefully, expanded. This format clearly emphasizes the belief that we are ALL professionals with value-added contributions to make, not just the person or persons standing at the front of the room! Furthermore, as I said in my keynote address at dinner the evening before the conference sessions were scheduled, “Global fluency depends on global respect.” For anyone interested in viewing a PDF file of the slides used for my talk, click here. There is also a brief article that was distributed as a handout and is available as a PDF file by clicking here.


EMEA Board Members

ISPI EMEA has filed as a forming chapter and is pursuing an important objective of providing enhanced opportunities for active Society participation for professionals living and working outside of North America. One of the decisions made by the original organizers/members, is that all EMEA members will be ISPI members as a prerequisite and no separate chapter dues will be charged. To my knowledge, ISPI EMEA is the only chapter that has chosen to establish itself in this way. I suggest that this structure will support optimal synergy between the Society and the chapter and can potentially serve as a model for the future that other chapters may want to consider. By the way, ISPI membership as a prerequisite for EMEA membership, does not preclude participation in conferences and other activities sponsored by ISPI EMEA. It only means that voting and holding office as EMEA leaders will be limited to its members, all of whom are members of ISPI.

At the conference, I challenged our colleagues who participated to do several things to ensure that HPT (and ISPI) continues to develop its global fluency and international impact.

  • Volunteer for ISPI Committees that require little or no face-to-face interaction (and, thereby, travel).
  • Share your experience and success with the EMEA chapter with other ISPI chapters.
  • Document your performance projects and contribute HPT articles, research, and success stories.

In fact, I would like to challenge everyone, not just the participants at the EMEA conference and the members of that forming chapter. All ISPI professionals need to contribute their ideas and experience if we as a Society are to continue to grow, not only in numbers, but also in the value that we offer to our current and potential membership and their clients.



  


by Carl Binder


This month I want to use my 500 words (or so) to discuss our esteemed colleague and my co-chair for GOT RESULTS?, Timm Esque. I’d like to focus on the elegant model of management that Timm describes in his recent book, Making An Impact, and how it boldly presents the case that measurement is inherent in Human Performance Technology (HPT), not merely an afterthought or optional appendage.

I’ve mentioned the book before in this column, and I suppose some people are beginning to think I’m getting a kickback! But I assure you, there’s nothing in it for me except the belief that Timm’s approach may be the single best method I’ve seen for conveying the message that measurement counts.

Timm has adopted a methodology he learned from Bill Daniels (1995) and has presented it in a remarkably straightforward and lean way that makes it accessible to anyone willing to spend a couple of hours reading. I sent a copy of his book to one of my clients—a Harvard MBA General Manager of a business unit in a $7 billion global corporation. A few weeks later I learned that she’d ordered 3 dozen copies for key people throughout her organization! She told me, “We haven’t seen the results yet, but we’re expecting to very soon.”

The essence of the model is that there are three conditions that must be in place before one can say that something is “managed”—whether an individual, a process, or an entire organization. Those conditions are:

  • clear, measurable expectations
  • self-managed (by the performers) measurement and feedback, and
  • control of resources.

In other words, the performer or group knows exactly what they are supposed to produce and how it will be measured. They monitor their own performance against those expectations. And they have access to resources that can be brought to bear, depending on how it’s going. If any one of these three features is not met, we can say that the individual, process, or organization is not truly being managed.

While this might seem initially to be a rather radical statement, upon further reflection—especially if you put yourself in the shoes of the performer—it’s clear that these are the essentials. The third condition opens the door for all the various performance management and improvement strategies and tactics that we, as HPT professionals, have in our tool bags. But in the absence of the first two conditions, those strategies and tactics are like hammers in search of nails—there is no context for applying them properly.

The important take-away for me is that measurement is an essential component of performance management or performance improvement. It’s not something we or our clients can afford to treat as optional or occasional. Esque and others can show compelling evidence that when these conditions are in place, it is nearly inevitable that performance will begin or continue to achieve expectations—and generally in a relatively short time. It’s getting those conditions in place that should be our primary focus when approaching any request to help manage or improve performance.

References
Daniels, W.R. (1995). Breakthrough performance: Managing for speed and flexibility. Mill Valley, CA: ACT Publishing.

Esque, T.J. (2001). Making an impact: Building a top-performing organization from the bottom up. Atlanta, GA & Silver Spring, MD: CEP Press & International Society for Performance Improvement.

Dr. Carl Binder is a Senior Partner at Binder Riha Associates, a consulting firm that helps clients improve processes, performance, and behavior to deliver valuable results. His easy-to-remember e-mail address is CarlBinder@aol.com and his company’s website is www.Binder-Riha.com.

 

 



by Aimee Boyd


The question “Does technology work?” echoes throughout the education and training sectors of academia and industry. Technology is used in various ways to create products, provide services, educate students, and train employees. Is technology a viable tool for retention of knowledge, skills, and abilities learned during education and training? One process of evaluation is to use technology to provide authentic assessment, thereby assessing learners under conditions similar to on-the-job evaluations. The level of technology in the evaluation process needs to mirror the technology used in training.

An exemplar program, The EnterTech Project, places learners within a computer-simulated environment in which they develop job skills and life skills by completing work tasks within a virtual technology manufacturing company. The web-based instruction engages learners with “virtual” co-workers and supervisors in an ongoing narrative storyline that dynamically changes based on student decisions and input. The curriculum is based on a theoretical framework of constructivism, authentic learning, and social learning theory. The performance objectives were determined by an employer coalition and validated through WorkKeys™ job profiling at nine manufacturing companies, including Dell Computer Corporation, IBM, Motorola, Raytheon, Solectron, and 3M.

Learners spend 70% of their instruction interacting with the computer-based environment. To effectively evaluate this interaction, data are collected from several different sources. Learners, instructors, and employers perform traditional data collection. The computer-embedded tracking system provides assessment “behind the scenes” of the instruction. Performance is monitored and tracked by a learning management system (LMS) through time on task and successful completion of tasks. The computer-embedded tracking allows for assessment without interrupting the learners. As learners complete the interactive computer tasks, their performance is recorded in the LMS. Immediate feedback and easy access to online resources ensure the learner will experience success with little or no frustration. The LMS provides learners with metacognitive opportunities to monitor their progress and presents a skills transcript of learners’ performance at the end of training to be used in job interviews.

Follow-up data collected 90 days post-training includes opinions on EnterTech’s ability to help the learner, suggested improvements, welfare benefits, and employment or educational status. Once learners gain employment, information about their job responsibilities, wages, and benefits is collected. Learners’ employers evaluate their on-the-job performance and the extent to which the EnterTech employees are more likely to stay on the job longer, are more easily trainable, and are more productive then non-EnterTech employees. Instructors and employers help in validating the authentic assessment format of the LMS.

Evaluative results indicate that 67% of the learners gained employment or enrolled in continuing education, 15% received raises or changed jobs for higher salaries, and 33% of the welfare recipients were taken off welfare or had significant re-educations in benefits. Learners report high levels of retention. The use of simulation in an authentic learning curriculum and the evaluation procedures mirroring 21st century technology support the effectiveness of technology.

Aimee Boyd is the evaluation specialist at E-Learning and Training Laboratories, IC2 Institute, The University of Texas at Austin. She is a doctoral student in the department of Educational Psychology at the University of Texas at Austin, specializing in Quantitative Methods. She has an MA in Program Evaluation. Aimee may be reached at aimeeboyd@mail.utexas.edu.

  Aimee Boyd and Melinda Jackson will present a session titled Authentic Assessment through Simulation on Sept. 27, at ISPI's Performance-Based Instructional Systems Design Conference, Chicago, IL, Sept. 26-28, 2002. Register today!



The nominating process for next year’s International Society for Performance Improvement (ISPI) Board of Directors will end on August 30, 2002. Please take a moment to nominate a member (including yourself) for next year’s Board. You can do so by clicking on the link at the end of this message.

Members who have served on the Board of Directors in the past have universally commented on what a wonderful experience and unique opportunity it was to contribute to ISPI in this way. Take a minute to add a deserving member’s name to the process that will provide leadership to the Society for the next two years.

Again, the deadline for nominations is August 30, 2002. If you would like to nominate someone, please send the your name and contact information along with the nominee’s name and contact information to april@ispi.org. Be sure to indicate the Board position you are submitting for in the Subject line. If you are interested in additional information on the nominations process, click here.

 


  Member participation is important—submit your nomination today at april@ispi.org



by Brian Blecke, CADDI


While moderating the HPT-Enterprise-Manager-Client focus group at the 2002 International Performance Improvement Conference & Expo in Dallas this past April, I enjoyed the spirited dialogue between the participants in my meeting. The group consisted of HPT or training directors and managers and a few performance consultants. Taken together, they represented organizations from a number of industries, including the military, information technology, telecom, pharmaceutical, and general services. While most of the members were from the United States, we enjoyed the contributions of a thoughtful, insightful New Zealander. Below are the names of the participants in this session:
  • Myrene McLeod, Fisher & Paykel Appliances, New Zealand
  • James Harbison, DST Output
  • James Harris, SBC Southwestern Bell
  • James Bonfiglio, Healthpoint
  • Connie Perren, Equiva Trading Company
  • R. Erin Brogan, US Coast Guard
  • Indira Nath, Hewlett Packard
  • Dean M. Hebert, Mindset for Performance

As I listened to my group and prodded them to reveal their insights, I learned two critical things about the participants. First, all of the members have strong roots in the training field and are personally and/or organizationally moving toward a more HPT-like model. Additionally, it was clear that training, in their varied roles, was still an important part of their day-to-day activities. Second, most of the participants in this group belonged to both ASTD and ISPI. This should not be a surprise to most of us in the Society (I, too, am a member of both). The richest part of the dialogue centered on, and sprang from, comparisons between the two organizations, and how the organizations interact with each other and their members. The focus group discussion provided a few key themes.

Theme 1
ISPI provides and promotes practical, research-based models, methods, and products. This group saw the research basis as a positive differentiator from other organizations. The members were unanimous in their praise. One member’s comment characterizes the group’s position, “[As a professional,] I need to be constantly refreshed and challenged, and the research is part of that.” Another said, “It is very important to base my choices and recommendations to my organization [on research found at ISPI]. It’s very important to be able to say, ‘It’s not just my idea, but I’ve researched it [at ISPI]’.”

The members roundly praised Performance Improvement as being a must-read on their evidently long list of reading materials.

Theme 2
ISPI should continue to strengthen its relationship with ASTD, in particular, and other societies, in general. From the conversation, I learned that the members perceived that ISPI and ASTD share a complementary, though sometimes overlapped, relationship in the marketplace. They saw a strengthened, differentiated relationship as valuable because it would reduce the amount of time required to “keep up to date.”

Theme 3
ISPI should continue to build through the local and international chapters. Quite a bit of time was spent discussing the process by which new members are acculturated into ISPI at the local and international levels. The members saw this as a critical opportunity for the growth of the Society’s membership ranks.

As the session drew to a close, more than one member expressed their happiness at being invited to participate in the focus group, and that this was a positive, first step in building on ISPI’s strengths and targeting areas for improvement.

The first focus group report for the Enterprise-Manager-Partner session appeared in the July issue of PerformanceXpress. Summaries of the other focus group sessions will continue to appear in future issues of the newsletter. If this summary, or the summaries of the other focus group sessions raise some ideas or comments, please make your opinion known by contacting the members of the Marketplace View 2002 Taskforce, Guy Wallace (guy.wallace@caddi.com) and John Swinney (jswinney@bandag.com).

 


  The richest part of the dialogue centered on, and sprang from, comparisons between the two organizations [ISPI and ASTD], and how the organizations interact with each other and their members.


by Barbara Gough, 2002 Conference Chair


Most of you don’t know that it takes almost two years to plan the ISPI Annual Performance Improvement Conference & Expo. It begins with writing the Call for Proposals and ends with what I hope most people will call a successful event. For those of you who missed this year’s conference in Dallas, we missed having you. The conference was informational and full of opportunities to expand your knowledge and networking circles. Throughout the weeklong event, many people asked me questions that I thought I would address.

Who picks the location of the conference?
The process of selecting a location begins with surveying members and other associations to select dates with the least number of conflicts. ISPI staff looks at many factors when determining when and where to hold a conference. A few of the major considerations include: where the conference was held in past years (West coast, East coast, etc.), accessibility, and price. These factors are weighted to help make the selection process easier. When the conference goes to a more popular city, known as a “first tier” city, it is usually held on less desirable dates to make the rates more affordable. When a “second tier” city, such as Cincinnati is selected, groups normally have more options with dates and less expensive hotel rates.

Why doesn’t ISPI provide lower-cost hotel options?
The ISPI conference is considered a mid-size event. That means that it is too small for most conference centers and too big for most hotels. This makes it challenging to find a suitable hotel to host the conference. When contracts with hotels are negotiated, ISPI must commit to a certain number of sleeping rooms in exchange for a certain number of meeting rooms to hold the concurrent sessions. Alternative hotels are only required when the Headquarter hotel sells all of its sleeping rooms.

Who selects conference educational sessions?
Conference sessions are selected by a group of people serving on the Conference Program Committee. This Committee is comprised of ISPI members who volunteer their time to evaluate all of the proposals received. Each proposal is reviewed and evaluated by three different individuals. Based on the criteria outlined in the Call for Proposals, all proposals receive a score. Proposals receiving the highest scores are accepted and included on the program.

Why do I have to pay to attend the conference if I am a presenter?
ISPI is a non-profit organization that relies on the full participation of its members. Presenting at the conference is considered a professional contribution and a way for our members to give back to their professional Society. In consideration of this contribution, ISPI offers one person designated as the “Lead presenter” a $50.00 discount off their registration fee.

How does the cost of ISPI’s conference compare to the cost of other conferences?
The ISPI conference is about 30% less expensive than other conferences. Other conferences that offer a paid registration to presenters have to recoup those costs by passing those costs on to paying attendees. ISPI’s registration policy, which requires that all individuals register for the conference, allows the Society to offer lower registration fees to all attendees.

Why is the cost of the banquet not included in the registration fee?
The banquet is a perfect opportunity for all members to network with old and new friends, celebrate a successful conference, recognize the Board of Directors, and honor Award of Excellence recipients. Over the years, hotel banquets have grown increasingly expensive. Instead of mandating that all attendees pay for an event they may not be able to attend, ISPI offers the banquet as an option, which helps to keep registration costs down for all attendees.

How do I show a return on the investment to my organization?
When you return home from a conference, there are many activities you can do to pass on the knowledge you gained by attending the conference. For example:

  • Host a lunch and inform your team of the things you learned
  • Call on the network you established at the conference to discuss issues you are having in your workplace
  • Share with your team members the Conference on CD-ROM, including handouts and audio from most of the educational sessions

If we are an International Society, why aren’t flags from other countries displayed at the conference?
Normally, only the flags of the host country and host state are flown at the conference. During the 1999 joint ISPI & IFTDO Conference, flags from a number of different countries were displayed as a way to celebrate the joint effort of both organizations.

How can I get involved with ISPI?
There are many ways you can get involved in ISPI! Volunteer to help out on one of ISPI’s committees, support a local chapter, or even run for the Board of Directors. Visit www.ispi.org for a listing of committees, chapter groups, and Board nomination procedures.

If you have additional questions, suggestions, or comments on future ISPI conferences, contact Ellen Bodalski Kaplan, ISPI Senior Director of Meetings, at ellen@ispi.org.

 

 



Each year since ISPI’s 1998 International Performance Improvement Conference & Expo in Chicago, Illinois, more and more ISPI members have been sharing their RESULTS at the conference. This year in Dallas was no exception. The GOT RESULTS? Poster Session displayed 24 different examples of documented performance results.

With your help, even more quality examples of documented performance results will be displayed at the 2003 GOT RESULTS? Poster Session, which will be in the main exhibit hall.

Why stay on the sidelines? Why not participate in demonstrating what distinguishes ISPI and Human Performance Technology at its best: the ability to produce and directly measure positive changes in individual and organizational results—results that are valuable and that represent significant returns on the investments made to produce them.

Watch PerformanceXpress for further details about participating in the 2003 GOT RESULTS? Poster Session. Be part of the “buzz” in Boston, April 11-15, 2003. For more information on the display or to participate, contact Timm Esque (tjesque@yahoo.com) or Carl Binder (carlbinder@aol.com).

 

 


Rediscover a valuable resource
in your neighborhood, your local International Society for Performance Improvement (ISPI) chapter. A new feature has been added to the ISPI website, Local Chapter Events. Now you can look in one place to see what’s happening at the local level, and chapters are adding new events every day! Chapters with events currently posted include: Florida First Coast; Arizona; Sydney, Australia; and a special Northeast Regional group of chapters. Visit the Local Chapter Events web page today at www.ispi.org, link to Chapters and then to Local Chapter Events, or click here to go directly to the listing.

 


The ASTD Dissertation Award is given each year to foster and disseminate research in the practice of workplace learning and performance. This year’s award will be presented to the person who has submitted the best dissertation for which a degree was granted between July 1, 2001-September 20, 2002. The topic must focus on some issue of relevance to the practice of workplace learning and performance. Illustrative areas of concentration include: training and development, performance analysis, career development, organization development/learning, work design, and human resource planning.

All research methodologies will be considered on an equal basis including, for example, field, laboratory, quantitative, and qualitative investigations. The candidate must be recommended and sponsored by his or her committee chair. All materials submitted must be in English and in Word format by email. Submission requirements correspond to the full manuscript requirements of the Academy of Human Resource Development’s (AHRD) Dissertation of the Year procedures that require applicants to follow the full manuscript conference proposal submission guidelines.

The award winner will receive a $500 cash prize, a commemorative plaque presented at the awards ceremony during the 2003 ASTD International Conference and Exposition, and a designated place on the conference program to present the research (with conference registration fee paid).

Submissions must be sent via email by September 20, 2002 to: Dr. Andrea D. Ellinger, Research Associate, The University of Alabama, Center for Business and Economic Research, at adellinger@cba.ua.edu. For further information and submission guidelines, please contact Dr. Ellinger.

 

 


The International Society for Performance Improvement
(ISPI) would like to congratulate the list of professionals below who have taken advantage of the exemptions available during the grandparenting period and received the designation of Certified Performance Technologist (CPT) last month. Click here for a full list of CPTs. Visit www.certifiedpt.org, and apply today to receive your designation.

  • Ronald J. Ryan, Amgen Inc, California, USA
  • George W. Byars, Saudi Aramco, Dhahran, Saudi Arabia
  • Nancy K. Luca, PRG-Schultz, Georgia, USA
  • Burnes P. Hollyman, Austus Inc, Texas, USA
  • Carol Haig, Carol Haig & Associates, California, USA
  • Susan Coleman, Intelligent Decision Systems, Inc., Tennessee, USA
  • Ellen Sue Menaker, Intelligent Decision Systems, Inc., Maryland, USA
  • Lynn B. Kearny, Independent Consultant, California, USA

 




Performance Marketplace is a convenient way to exchange information of interest to the performance improvement community. Take a few moments each month to scan the listings for important new events, publications, and services. If you would like to post information for our readers, contact ISPI Director of Marketing, Dan Rudt at dan@ispi.org or 301.587.8570.

 

Conferences, Seminars, and Workshops
Internet-Based Performance Improvement Institute, October 7-25, 2002 Principles and Practices of Performance Improvement is coming to your computer! Participants have raved about this class. Find out why. No bags to pack, no airline delays! Just solid performance improvement.

Instructional Design Workshops, In-House and Public — Darryl L. Sink & Associates, Inc. (DSA) workshops include: Designing Instruction for Web-Based Training, The Instructional Developer, The Course Developer, and The Criterion-Referenced Testing. Special pricing for courseWriter Software to DSA graduates.

Performance-Based Instructional Systems Design Conference — September 26-28, 2002, Chicago, Illinois. Take away valuable hands-on solutions to your most critical challenges in Instructional Systems Design. Return to your employer/clients with the tools needed to improve performance and deliver success.

 

Books and Information
CD-ROM Set with 185 Hours of Performance Instruction — Cutting-edge, workplace performance improvement presentations recorded at the recent ISPI Annual Conference. Purchase the Conference CD-ROM at a new, low price and receive substantial savings on ISPI Conferences, Institutes, and Publications.

Report Available: Incentives, Motivation & Workplace Performance: Research and Best Practice — Sponsored by ISPI, funded by the SITE Foundation. The purpose of the study was to cut through the conflicts and controversies regarding the use of incentives to improve performance.

Performance Improvement Quarterly—Now in its 15th year! You can't miss another issue of this scholarly journal that provides cutting-edge research and information necessary for you to keep on top of the business of improving human performance. Subscribe today!

 

 



ISPI is looking for Human Performance Technology (HPT) articles
(approximately 500 words and not previously published) for PerformanceXpress that bridge the gap from research to practice (please, no product or service promotion is permitted). Below are a few examples of the article formats that can be used:

  • Short “I wish I had thought of that” Articles
  • Practical Application Articles
  • The Application of HPT
  • Success Stories

In addition to the article, please include a short bio (2-3 lines) and a contact email address. All submissions should be sent to april@ispi.org. Each article will be reviewed by one of ISPI’s on-staff HPT experts, and the author will be contacted if it is accepted for publication. If you have any further questions, please contact april@ispi.org.

 

 

Go to printer-friendly version of this issue.


Feel free to forward ISPI’s PerformanceXpress newsletter to your colleagues or anyone you think may benefit from the information. If you are reading someone else’s PerformanceXpress, send your complete contact information to april@ispi.org, and you will be added to the PerformanceXpress emailing list.

PerformanceXpress (formerly News & Notes and Quick Read) is an ISPI member benefit designed to build community, stimulate discussion, and keep you informed of the Society’s activities and events. This newsletter is published monthly and will be emailed to you at the beginning of each month.

If you have any questions or comments, please contact April Davis, ISPI’s Senior Director of Publications, at april@ispi.org.

ISPI
1400 Spring Street, Suite 260
Silver Spring, MD 20910 USA
Phone: 1.301.587.8570
Fax: 1.301.587.8573
info@ispi.org

http://www.ispi.org