According to cognitivists, there are several components
of the mind, and each is involved in the learning process in certain
ways. How each component of the mind works has implications for how we
design instruction. The components are:
- perception and sensory stores
- short-term or working memory
- long-term memory
Perception and Sensory Stores
Limits of the sensory stores. Our sensory stores, also
called sensory memories or “buffers,” are capable of storing almost complete
records of what we attend to. The catch is they hold those records very briefly. During
that very brief time before the record decays, we do one of two things:
(1) we note the relationships among the elements in the record and encode
it into a more permanent memory; or (2) we lose the record forever.
Perception is selective. There is more
stimulation in the environment than we are capable of attending to, and
then encoding (internally translating)
for storage in memory. Therefore, we only attend to certain things. We
attend to and see/hear what we expect to see in a given situation. We
attend to those things that interest us because they are either (1) related
to what we already know, or (2) so novel they force us to attend to them.
ID implications. The implications of the selectiveness of perception
and limitations of sensory stores for instructional designers are that
it is crucial to:
Short-term or Working Memory
- get the learner to attend to the parts of the environment
you want him/her to learn (by building expectations, attention-getting,
and establishing motivation and confidence)
- help the learner note relationships among the information
quickly (by organizing the information and relating the new information
to existing knowledge)
Controversy. There is disagreement among cognitive
psychologists about whether there is a short-term memory that is “separate and different” from
long-term memory—whether the two are physically different, or whether
they are just conceptually different constructs.
information is passed from the sensory stores to memory, we mentally
rehearse it. Simply repeating the information over and
over is called passive rehearsal. It does not seem to improve
memory as well as rehearsing the information in a deep and meaningful way,
by doing things like creating associations.
Limited capacity. There seems to be a limit on
the amount of information we can rehearse at one time. The findings
of the study that showed
we can remember 7 +/- 2 bits of information at most, and that to remember
more we have to “chunk” (or group), still apply, with some modifications
of how you define a “bit” (element) or a “chunk.”
Format. At this point in the learning process, the information being
rehearsed is not yet organized and encoded as it will be when it is
finally stored in memory. There is evidence that there are separate
spaces in the brain for storing and rehearsing verbal information and
ID implications. The implications are that instructional designers
- help learners use meaningful ways of rehearsing the information (using
analogies and relating new information to existing knowledge)
- present the information in meaningful “chunks” of
appropriate size for learners
- present the information in multiple formats (verbal, auditory,
- present the information in a way that allows the learner to move
quickly from rehearsal to encoding it in long-term memory.
In general, theorists believe that long-term memory
is organized based on context and experience. That means we encode, store,
information in the way we have used knowledge in the past and expect
to use it again in the future.
strength. Information in memory has a characteristic called strength,
which increases with practice. There is a power law of learning that
governs the relationship between amount of practice and response time
or error rates (Strength = Practice to power x).
Elaboration. Elaboration means adding information to the information
we are trying to learn. The more we elaborate on what we learn through
processing, the better we remember it, because as we tie the new information
to existing information, we create more pathways to get to the new
information as we try to remember it.
Chunking. Memories are stored not as individual
bits or as long strings of information, but in “chunks,” with each
chunk containing about seven elements.
Verbal and visual information. It seems we encode verbal and visual
information differently in memory. We use a linear code for verbal
information, and a spatial code for visual information. We remember
visual information very well, especially if we can place a meaningful
interpretation on the visuals.
Associations and hierarchy. Information is
organized in memory and is grouped in a set of relationships or structures
Using such a structure makes it easier for us to remember because there
more related pieces of information activated when we search for information.
ID implications. The
implications are that instructional designers need to:
- build in a lot of meaningful practice into training
- provide learners with information that elaborates on the information
to be learned
- present the information in meaningful “chunks” of
appropriate size for the learners
- present the information so it uses the abilities to remember both
verbal and visual information
- organize the information hierarchically
- provide many associations to the information being learned
- teach learners to organize/index their memories so
they have many associations, many retrieval paths, and appropriate
structures, which are crucial to effective memory
- use authentic (real-world) contexts for explanations, examples,
Note: Excerpted from Foshay, W.R.,
Silber, K.H., & Stelnicki,
M.B. (2003). Writing Training Materials That
Work: How to Train Anyone to Do Anything. San Francisco: Pfeiffer.
Rob Foshay, PhD, CPT is Vice President of Instructional
Design and Quality Assurance for PLATO
Learning, Inc. He is a former ISPI Board member and recipient
of the Distinguished Service and Honorary Life Member awards.
He may be reached at firstname.lastname@example.org.
Ken Silber, PhD, CPT is Associate Professor, Educational
Technology, Research, and Assessment at Northern Illinois University,
and President of Silber Performance Consulting, where he specializes
in Performance and Needs Analysis, Intervention Design, and Evaluation.
He may be reached at email@example.com.
To learn more from
Foshay and Silber, attend ISPI’s 2004 Performance-based ISD Conference.
See details in this issue.
by Carol Haig, CPT and Roger Addison, CPT
Among the many benefits of attending ISPI’s Annual
Conference is the opportunity to get together with our professional
friends from other
countries and hear their experiences and views in person. This year,
we asked Monique Mueller of Zurich, Switzerland,
to share her perspectives on trends and opportunities in the performance
improvement arena in Western Europe.
heads La Volta Consulting, a firm focused on performance improvement
and culture change. She comes from a Spanish family, has lived and worked
in Europe and the United States, specializes in translating and interpreting
a number of languages, and holds an MBA and a postgraduate degree in
organizational development. Monique credits Klaus Wittkuhn with introducing
her to human performance technology (HPT). She is highly involved in
the ISPI Europe chapter and chaired the popular and successful Europe
conferences of 2002 and 2003.
Monique highlighted three critical
and interrelated opportunities for HPT in Europe at this time. First, performance improvement must be
explained more effectively to suit the Western European business climate
and workplace cultures. It is challenging to explain why and how
performance improvement adds value to organizations in ways that resonate
with the business and societal leaders whose organizations could benefit
from HPT tools and techniques.
Second, small and medium-sized organizations make up the vast majority
of businesses in Europe and the HPT word is not reaching them. Unfortunately,
most examples of HPT successes come from large U.S. companies where
the circumstances do not translate well for this potential client audience.
Third, there is a real need to expand HPT to include the non-North
American voice in the literature and on the European business
scene. Monique observes many opportunities for European public entities
to use HPT to address the issues they face.
Reasons for These Opportunities
While awareness of HPT
has risen in Western Europe, Monique explains, it is a slow process and
lacks the impact needed
to interest decision
makers and launch the widespread use of ISPI’s principles and practices.
There is also the unfortunate stigma attached to HPT of yet “another
thing coming from the U.S.” Europeans require an explanation that will
enable rapid understanding, acceptance, and enthusiasm for performance
To raise awareness among the many untapped small and medium-sized companies,
practitioners should publish and present successes to raise awareness
and facilitate entry into these organizations. We must be creative and
approach trade groups, guilds, and unions to demonstrate the value of
performance improvement and enable organizational leaders to embrace
Finally, Europeans must see and hear non-Americans on the subject of
performance improvement. Of particular value are examples from public
service organizations, government, and education where the introduction
of performance improvement concepts can make such a visible and measurable
difference. It is our responsibility to showcase these results.
How Organizations Will Be Different
Monique points out
that the European Union is growing from 15 to 25 countries. This represents
a huge opportunity to deploy
our tools. For example, the coordination of transportation services and
utilities across Europe raises exciting opportunities for practitioners
to provide support and guidance through complex transitions, and we are
uniquely qualified to do this work.
Western Europe is not untouched by the recent issues
of business ethics in the news, or the practices of “pirate capitalism” we
have observed among CEOs in large U.S.-based companies. Monique suggests
is a unique opportunity, right now, to unite disciplines under the HPT
umbrella to help organizations discover how they can work together using
tools to improve performance and impact business results with all stakeholders
The key to making change, of course, is to begin by changing ourselves
and the way we source and approach opportunities to practice our craft.
Implications for La Volta Consulting
Monique has begun
to alter her firm’s marketing plans
to align with target organizations and their needs as described above.
adapting many of our standard HPT models to serve these new markets and
promises a new focus on demonstrating bottom-line results.
There is a small but growing body of performance improvement-based work
published by European practitioners. Monique challenges ISPIers to add
to it. By publishing success stories from organizations with profiles
similar to those in Western Europe in the European business press, we
can help to raise HPT awareness. For example, the two ISPI Europe conferences
attracted excellent papers that could be expanded and submitted to local
business publications. Monique encourages non-U.S. practitioners to publish
in their own countries and in their native languages in publications
that are read by their target clients.
If you are interested in publishing in Europe and would like help and
guidance, please contact Monique directly. All of us in the performance
improvement business stand to benefit from outreach worldwide.
If you have any predictions about the future of HPT that
you feel would be of interest to the PerformanceXpress readership,
please contact Carol Haig, CPT, at firstname.lastname@example.org or
Roger Addison, CPT, at email@example.com.
In the May
issue of PerformanceXpress Roger
Kaufman rightly asks, “Do we really want to help people to just add
to the bottom line?”
But nothing in business (or anywhere else) is really that simple. There
are more stakeholders than just the owner, or even the customer. Effective
business decisions must balance the requirements of a number of disparate
stakeholders whose requirements may be in conflict with each other.
Here is a model with eight stakeholder categories.
The Government represents the laws and regulations
that guard the interests of the public and business and provide the rules
for conducting business. Government requirements will take precedence,
under the penalty of law, for noncompliance.
Shareholders are the owners of the business. Their
goals are typically financial, long-term growth in equity or short-term
income through dividends, but can be related to other things such as
greater societal enhancement, environmental protection, and so on.
If the executive management group does not achieve shareholder goals,
they will either be replaced or the investors will withdraw their capital
and invest it elsewhere. An entity without a sound capital base will
eventually be crippled.
are those responsible for the operations and results of the entity.
They include an elected board of directors responsible to all the
owners, and the executive management team in charge of overseeing
daily business operations. They must always balance the (conflicting)
interests of various stakeholder groups when determining the course
of action for the organization.
Customers are typically a non-homogenous group.
The enterprise needs to listen to customers to understand their requirements
fully and consider those in light of what the competition is doing
in the marketplace to allow them to determine responsibly what customer
requirements to pursue, that also meet the requirements of other critical
The Employee stakeholder
group includes all ranks of employees below the executive management
level. At the heart of all
employee requirements are a safe workplace and financial security,
but other needs exist among the groups. Global assumptions such as “everyone
wants to be a team player” will only lead to a population of dissatisfied
Suppliers are also a key stakeholder group. As a
business entity themselves, they need to achieve a profit margin that
will allow them to remain in business. The objective should not be
to drive prices for their materials/components so far down that they
The Community stakeholders, although a less formal group,
remain important through the influence they can have on our businesses.
They may ultimately have a voice in public policy and laws, thus completing
a cycle from grassroots demands to the laws of the lands.
seem, given the extent of stakeholders and their requirements, that
there are really three challenges. The first is: identifying specifically
who the primary stakeholders are for any given situation. The second
is: understanding their requirements and the priorities that ought
to be placed on those requirements. The third is the most difficult:
balancing those requirements that are in conflict.
making the tough tradeoffs on whose requirements will take precedence
when conflict arises. What stakeholder requirements should not be met
now, or later, and why? Questions such as these can only be answered
after all requirements are uncovered and judged in relation to one
another to understand the full implications of any decision. It is
inherently complex and takes time and effort.
decisions, you may find that the general default Stakeholder Requirements
Hierarchy model is sufficient. Otherwise, careful analysis is called
for. Good luck, and may the balance of requirements be in your favor!
Guy W. Wallace,
CPT, has been a consultant to government and industry since 1982,
and has served 29 of the current Fortune 500. He is the author of three
and more than 50 articles. He has served on numerous ISPI Committees
and Tasks Forces since joining NSPI in 1979, and also served on the
ISPI Board and as President-Elect and President 2002-2004. His professional
biography was listed in Marquis Who’s Who in America in
2001. Guy may be reached at firstname.lastname@example.org.
At the Opening Session of the recent ISPI
Annual Conference in Tampa, incoming president Don Tosti introduced an
important outcome of the recent
Presidential Initiative. Using an interactive strategy, Don invited hundreds
of participants to react to ISPI’s new Communities of Practice. (Descriptions
of the seven professional communities are at the end of this article.)
If you were not able to interact with other ISPI
members and provide feedback on these communities at the ISPI Conference,
don’t worry! We
have used the Open Questions (OQ) interactive strategy to create a forum
for you to give us your feedback. Even if you had a chance to interact
with others at the conference, this is an opportunity to continue the
Let Us Know What You Think
The task is simple. Click here to get
to the OQ page. Type your inputs, questions, reactions, comments, implementation
ideas, sarcastic remarks, and anything else you want to say. Get it all
off your chest. It will make you feel better. If you wish, include your
name, or pseudonym, or initials in parentheses after your response. Then
click the “send” button. Snoop around the web pages to see what other
participants have said about these professional communities.
WARNING: We truly value your feedback. The planning
and implementation group will analyze and incorporate your comments.
However, we do not
guarantee that all your suggestions will be immediately implemented (because
of logistic, financial, and legal constraints). So, please don’t get
upset if we don’t follow up on everything you say.
HPT Communities of Practice
Management of Organizational Performance: To impact organizational
results by looking at the whole system to determine where the major sources
of variance are, and then addressing them with appropriate organizational
change processes and techniques. (Examples: Change Management, Management & Leadership
Initiatives, Administrative Systems Analysis, Program & Project Management,
Strategic Planning, etc.)
Instructional Systems: Determining when learning should occur
and the best way to achieve learning through manipulation of display,
response demand, and instructional management. (Examples: Instructional
Systems Design, Knowledge Management, Job Aids, Performance Support Systems,
e-Learning, Expert Systems, Fluency Transfer, etc.)
Process Improvement: Increasing the efficiency and/or effectiveness
of the sequence of activities in the value chain that produces outcomes
and results. (Examples: Statistical Process Improvement, Business
Process Re-engineering, Six Sigma, Operations Research, Lean, etc.)
Organizational Design/Alignment: To examine the
allocation of decision-making authority, business processes, values,
and conduct of people in the organization and their performance to ensure
they are aligned to produce the desired results. (Examples: Culture
Change, Group Collaboration, Team Building, Organization Design, Company
Values & Practices, Executive Coaching, Organizational Integration,
Motivation, Incentives, and Feedback: Examining
data about performance and providing the most effective way of delivering
that information to modify the form of behavior or to increase or decrease
the likelihood of the performance. (Examples: Corrective Feedback,
Incentives & Motivation, Coaching, Performance Management, Mentoring,
Performance Appraisal, etc.)
Analysis, Evaluation, Measurement: The process of assessment,
decision, and action relevant to the maintenance and adaptation of the
system. (Examples: Human Factors Analysis, Balanced Scorecard & Dashboard,
Needs Assessment, Statistical Process Controls, Performance Measurement,
Evaluation, ROI, Benchmarking, etc.)
Science of HPT—Foundations: The intellectual pursuit of basic principles and conditions of applications
that impact human performance. (Examples: Behavior Analysis, Educational
Research, Learning Theory, Systems Theory, Motivation, Cognitive Science,
Are you a provider of training? Are you responsible for purchasing
or contracting for training? Are you charged with reporting on the
of training delivered to your organization or individual clients?
It is estimated that $54 billion is spent annually on training activities
in the United States. It is predicted that this year, e-learning alone
will be a $14 billion business.
Clearly, learning is BIG business. With costs of this size, it is no
surprise that organizations want to know what results they are getting
for their considerable investments. Attend the International
Society for Performance Improvement’s 2004 Performance-Based Instructional Systems Design
Conference: Focusing on Results, September 27-October 2, 2004 at
the Hyatt Regency Chicago in Chicago, IL, to discover the latest on reporting
Instructional Systems Design results.
Who Should Attend?
Instructional systems designers and other managers and professionals involved
in design, development, delivery, and evaluation of training and performance
improvement should attend this conference and pre-conference workshops. The
case studies, presentations, and discussions will be equally valuable to
professionals involved in classroom, distance, and electronic learning. All
sectors, including academia, business, consulting, government, military,
and not-for-profit, will benefit.
Tips, Tools, Techniques,
and Other Tantalizing Tidbits
Dr. Harold D. Stolovitch, CPT,
Emeritus Professor, Universite de Montreal, Clinical Professor, University
Principal, HSA Learning & Performance Solutions LLC
Masters Series Presentations
Easiest Thing Is to Build the Wrong Simulation
Dr. Rob Foshay, CPT, Vice President for Instructional
Design and Quality Assurance, PLATO Learning, Inc.
Secret: Ensuring the “Pull” in Performance-Based E-Learning
Deborah Stone, CPT, President and CEO, DLS Group,
in-depth, full-day classes that encourage you to broaden your knowledge
base in a specific topic relating to performance improvement. Workshops
are limited in size ensuring that you will receive individual attention
from expert presenters. Workshops will be conducted on Wednesday, September
29, 2004. Topics include:
- Front-End Analysis & Return
Harold D. Stolovitch, CPT, HSA Learning & Performance Solutions
- The Instructional Design Workshop: Faster/Better/Easier Ways
to Design Instruction
Dr. Darryl Sink, Darryl L. Sink & Associates,
- Leveraging Performance and Business Impact from Training Initiatives
Robert Brinkerhoff, Western Michigan University and Dennis Dressler,
The Learning Alliance
- Training That Works: Developing Training Using The Cognitive
Approach to ID
Dr. Kenneth H. Silber, CPT, Northern Illinois
University - ETRA
Join ISPI today and
register for ISPI’s 2004
Performance-Based Instructional Systems Design Conference: Focusing
on Results at the member rate. Take advantage of all that ISPI membership
has to offer!
This conference is limited to 250 attendees, so make your plans now. Click here to register,
or visit www.ispi.org/isd2004,
for more information.
Safety first. Safe to say, here at I-Spy we seek first of all to explore how the
Internet can be a powerful tool to improve our work together.
Quick recap: Every month, three sites,
one theme. While far from comprehensive, hopefully these sites will
spark readers to look further and expand views about HPT. Please keep
in mind that any listing is for informational purposes only and does
not indicate an endorsement either by the International Society for
Performance Improvement or me.
These are the general categories I use
for the sites featured:
- E-Klatch: Links to professional associations,
research, and resources that can help refine and expand our views of
HPT through connections with other professionals and current trends
- HPT@work: Links to job listings, career
development, volunteer opportunities, and other resources for applying
your individual skills
- I-Candy: Links to sites that are thought
provoking, enjoyable, and refreshing to help manage the stresses
and identify new ideas for HPT
for this month’s
column is Safe
and Sound. When lives are at stake, performance
improvement takes on new importance. Organizations recognize that safety
is a critical
factor for success. We need to protect the human in HPT. This month,
put on your seatbelt, adjust your cybergoggles, and join us as we explore
safe ways to do what we do everyday, whether it’s planning systems
at work, driving home, or just whistling about bovine spongiform encephalitis
in Spanish. Trust me, you may do this everyday soon…
There is safety in
numbers, and numbers in safety! A number of professional associations
focus on safety issues
in workplace performance. One of particular note is the Systems Safety Society. Since
1962, this association has remained “dedicated to the advancement of
the arts, sciences, and technology of system safety in pertinent areas
of endeavor for the benefit of all mankind.” Reminiscent of HPT, we learn: “The
science of system safety is created through systematic and continuous
pursuit of scientific knowledge and understanding of technology and its
application in engineered devices, and of biotechnology (the study of
the human/machine interface) in both expected and unexpected circumstances.” You
can submit articles to and access the Journal of System Safety electronically,
and their Annual Conference is
August 2-6, 2004, in Providence, Rhode Island. While some links are still “under
construction,” this site, and organization, can be a valuable addition
to the PT conceptual toolbox.
June is National Safety
Month! Per the National
Safety Council (NSC), this year’s theme is “Crash-Free June” (no
wisecracks towards the IT department, please!). Seriously, the focus
is on preventing motor vehicle accidents, which in 2002 resulted in
about 120 deaths per day. With information and resources to distribute
to employers and your community, the NSC hopes to make June, and every
month, safe for all drivers. Now that would be some serious performance
improvement. The NSC site
also offers general resources on safety and occupational health, job listings, links to local chapters,
and statistics, including a sobering cost analysis of the negative
economic impact of workplace
injuries and deaths. An extensive library of information can help
you with all types of performance improvement, from a free course on Creating a Safety Culture: Strategies
for Small Business to a fact sheet on setting up your child’s swing set.
How about some toe
tapping tunes about toxicology? Decontaminate your fingertips before
you click over to the Food Safety Music website
of Dr. Carl Winter, food toxicologist, faculty member at the University
of California, and Director of the FoodSafe
Program. With free downloads of clever song spoofs (like “I Sprayed
it on the Grapevine”), you too can sing and dance safely. My personal
favorite? The unsettlingly catchy Beware
La Vaca Loca, a tune about mad cow disease, or bovine spongiform
encephalitis (BSE), sung to “Livin’ la Vida Loca” by Ricky Martin.
Until July, surf safe.
When he is not Internet trawling for ISPI, Todd Packer can
be found improving business, non-profit, and individual performance through
research, training, and innovation coaching as Principal Consultant of
Todd Packer and Associates based in Shaker Heights, Ohio. He may be reached
Many organizations are doomed to implementing the latest
performance improvement fad as an ongoing strategy of “management by
best seller.” It was George Santayana (1863-1952) who said, “Those who
cannot remember the past are condemned to repeat it.” We owe a tremendous
debt to those who came before us. We should all take the time to periodically
return to the original works that form the basis for what we do now in
our field of performance technology.
While there were earlier books that laid the foundations
for our profession, three books stand out as providing the structure
for the way in which we do business today: Joe Harless’s An Ounce
of Analysis (Is Worth a Pound of Objectives) in 1970, Tom Gilbert’s Human
Competence in 1978, and Bob Mager and Peter Pipe’s Analyzing Performance Problems in
1984. I remain amazed at the number of authors who borrow extensively
from these authors (without giving them the credit they deserve) for
their “latest and greatest ideas” on performance improvement.
More recently, three additional books changed the way in
which we view organizations and shaped the role of the performance consultant
in the 1990s: Geary Rummler and Alan Brache’s first and second edition
of Improving Performance:
How to Manage the White Space on the Organization Chart in 1991
and 1995, Dana and Jim Robinson’s Performance Consulting: Moving Beyond
Training in 1996, and Judy Hale’s The
Performance Consultant’s Fieldbook: Tools and Techniques for Improving
Organizations and People in 1998.
But perhaps the greatest impact on formalizing the body
of knowledge for our profession was the publication of the first and
second editions of the Handbook
of Human Performance Technology in 1992 and 1999, masterfully
edited by Harold Stolovitch and Erica Keeps. These two books are still
the best references that we have on our bookshelves, instantly making
available the work of many experts in the field of HPT.
More recently, Darlene Van Tiem, Jim Moseley, and Joan
Dessinger published the first and second editions of Fundamentals of Performance
Technology in 2000 and 2004, which provide a model of the performance
technology process (derived from earlier work by Bill Deterline and Marc
Rosenberg), with text that someone new to our field or a line manager
can read and understand.
And now, three new books have appeared that will further
shape the way in which we understand and apply performance technology. Serious Performance Consulting
According to Rummler by Geary Rummler, and Training
Ain’t Performance by Harold Stolovitch and Erica Keeps, are by
three of our most respected practitioners and authors. The third book, Human
Performance Technology Revisited, collects more than 50 Performance
Improvement articles from many of the best authors in our field.
While it is important that we keep up with the latest books
in our field, it is just as important that we return to the original
works that have shaped our profession. Reading the books that are the
foundation of our profession is similar to enjoying classic movies, such
as The Lion in Winter or Twelve Angry Men, to see the cinematic
roots of the current movies that we see.
your library complete, visit ISPI’s Bookstore at www.ispi.org.
in my measurement-related sessions at ISPI’s Annual Conference
in Tampa made me realize that many of our colleagues and clients have
much about how the frequency with which we measure results can determine
how often we make decisions. This simple, but very important issue
deserves some consideration.
a regular reader of this column, you’ll be aware of several key points
about how and why we measure:
most important reason to measure is to enable
good decisions. Measurement should serve a decision-making or navigational
function, supporting our roles as managers or performance improvement
specialists. Among other things, we decide if our programs, systems,
strategies, processes, or interventions are producing the results we
want or not, whether they should be continued or changed, and if they
are worth the cost.
Timm Esque describes two
conditions that must be in place for us to say that a process or organization
is being managed: 1) that there are measurable expectations for results,
and 2) that results are being measured in such a way that the performers
can make decisions based on the data.
point is that, for us to understand how results are changing, it is
essential that we measure repeatedly over time. For example, to distinguish between a
genuine jump-up in results after a specific intervention and an exceptional
high point (outlier) or a continuing up-trend, either of which might
deceive us into thinking that results have changed dramatically, we must
be able to see each data point in the context of what came before and
using the Standard
Celeration Chart to project straight-line trends into the future
revealed that it takes around 7 to 10 data points to project the same
number of points accurately into the future. Simply comparing one or
two “before points” with one or two “after points” is not a reliable
way to see if results have actually changed. If we want to know whether
a given result—whether performance on a learning exercise or results
on the job—is actually an improvement over what has been occurring
prior to our intervention, we need to measure regularly and frequently.
Frequency Determines Possible Decision Frequency
We all know
that the final test score in a classroom program comes too late to
help the student. If we want to help that student before it’s too late,
we must have daily measures of performance for monitoring reduction
in errors and/or acceleration of correct performance toward a performance
criterion. In fluency-based programs
for call center reps, for example, daily measures repeated over
several weeks allow both coaches and trainees to see if and when they
need help, and whether specific coaching suggestions or changes in
procedure made a difference.
quarterly measures of sales performance don’t help managers or sales
representatives make good decisions about how to improve sales performance.
Indeed, managers in some aggressive sales organizations use the rule
that one bad quarter is a red flag and two bad quarters can lead to
termination—a very expensive approach, given the cost of hiring new
people. Only more frequent measures of results, or intermediate outcomes
(e.g., sales milestones) allow managers to diagnose and improve specific
performance problems before it’s too late.
general set of rules about measurement frequency and decision-making.
In learning or front-line productivity environments, daily measures
(i.e., five data points per week) allow good weekly decisions. For
supervisory or mid-level decision-making, weekly measures of performance
can support good decisions every six weeks or so. Monthly measures
(e.g., samples of customer satisfaction) allow good decisions to occur
every 6 months or so. And for executive level strategic decisions,
quarterly measures can support decisions every 18 months to two years
(e.g., 6 to 8 data points).
choose to make exceptions to these general rules. But if you seriously
consider the significance of your decisions and look at the data you
are using to support them, I think you will recognize these guidelines
as being at least in the general ballpark of what is prudent and reliable.
As usual, I welcome your comments, disagreement, examples, and feedback.
Binder, C. (2001). Measurement: A few important
ideas. Performance Improvement, 40(3), 20-28.
Esque, T.J. (2002). Making an impact: Building a
top-performing organization from the bottom up. Atlanta,
GA: CEP Press.
Pennypacker, H.S., Gutierrez Jr., A., & Lindsley,
O.R. (2003). Handbook
of the standard celeration chart. Cambridge Center for Behavioral
Dr. Carl Binder is a Senior Partner at Binder Riha
Associates, a consulting firm that helps clients improve processes,
performance, behavior and the measurement of results. His easy-to-remember
email address is CarlBinder@aol.com and
you can read other articles by him at www.Binder-Riha.com/publications.htm.
See past issues of this column by clicking on the “Back Issues” link
at the bottom of the blue navigation bar to the left.
The Certified Performance Technologist (CPT) designation
is awarded by ISPI to experienced practitioners in the field of performance
improvement and related fields such as instructional design and organizational
development whose work meets the 10 Standards
of Performance Technology and other application requirements.
Your application to become a CPT must be received at ISPI
headquarters by June 15, 2004, or it will be held
until the next processing deadline of November 15, 2004. For more information
on becoming a CPT, or to download the application, visit www.certifiedpt.org.
The International Society
for Performance Improvement’s 43rd Annual International
Performance Improvement Conference & Exposition in Vancouver,
British Columbia, Canada, April 10-15, 2005 will feature several opportunities
for you to develop your professional skills, learn new HPT tools and
techniques, and hear the latest research findings in our field.
How can you participate? Attend! Present! Volunteer! It is not too early
to mark these dates on your calendar:
- July 30, 2004: Deadline to submit workshop
- August 31, 2004: Deadline to submit session
proposal and early speaker registration for conference
- April 10-12, 2005: Attend an HPT Institute
prior to the conference
- April 11-12, 2005: Attend a pre-conference
- April 12-15, 2005: Attend ISPI’s 43rd Annual
Conference & Exposition
Here are some suggestions to help you prepare a successful conference
proposal submission, especially if you are a novice speaking at ISPI:
- Review the 2005
Call for Proposals, which outlines the review criteria for
session proposals. Then, download the Session Proposal Template.
- Review the Sample Session Proposal. This
is an example of an accepted session proposal, updated to include
all of the required information for 2005.
- Download and review the Sample Handout
and Sample Performance Tool as these will provide guidance as you
are preparing your session proposal.
- Consider a coach! Review the 2004 Conference
Program, and see if you recognize anyone you might contact to provide
feedback on your proposal.
If you have any questions or would like additional
information, please contact ISPI at 301.587.8570 or by email at email@example.com.
Marketplace is a convenient way to exchange information
of interest to the performance improvement community. Take a few
moments each month to scan the listings for important new events,
publications, services, and employment opportunities. To post information
for our readers, contact ISPI Director of Marketing, Keith Pew
at firstname.lastname@example.org or 301.587.8570.
Revisited is an essential
reader for students and practitioners of performance-centered
design (PCD). From job aids and “bolt on” EPSS to ground-up
enterprise performance-centered systems, you will find gems
in terms of methodology, industry trends, and a plethora of
ISD Revisited is
a select collection of 56 articles from ISPI’s Performance
Improvement journal focused ISD as practiced in the 21st Century.
This compendium, with an introduction by Allison Rossett, provides
a fresh perspective on ISD, presenting current thinking and best
Seminars, and Workshops
L. Sink & Associates, Inc. is offering
these workshops in 2004: The Criterion Referenced Testing Workshop,
Atlanta, October 5-6; Designing Instruction for Web-Based
Training, Atlanta, September 14-16; and The Instructional Developer
Workshop, Washington, DC, June 14-16. Visit www.dsink.com for
details, and to register!
Workshop by Thiagi. Learn Thiagis radical
approach to instructional design. Faster, cheaper, better (and
fun at no extra charge). Secrets of training design based on
30 years of fieldwork that challenges the traditional ISD model.
Palo Alto, CA: June 17-18. More
Job and Career Resources
Online CareerSite is your source for performance improvement
employment. Search listings and manage your resume and job applications
Improvement Quarterly, co-published
by ISPI and FSU, is a peer-reviewed journal created to stimulate
professional discussion in the field and to advance the discipline
of Human Performance Technology through literature reviews,
experimental studies with a scholarly base, and case studies. Subscribe
Online Buyers Guide offers resources for your
performance improvement, training, instructional design and organizational
Are you working to improve workplace performance?
Then, ISPI membership is your key to professional development through
education, certification, networking, and professional affinity programs.
If you are already a member, we thank you for your support. If you have been
considering membership or are about to renew, there is no better time to join
ISPI. To apply for membership or renew, visit www.ispi.org, or simply click here.
ISPI is looking for Human Performance Technology (HPT) articles
(approximately 500 words and not previously published) for PerformanceXpress
that bridge the gap from research to practice (please, no product
promotion is permitted). Below are a few examples of the article formats
that can be used:
- Short I wish I had thought of that Articles
- The Application
to the article, please include a short bio (2-3 lines) and a contact email
address. All submissions should be sent to email@example.com.
Each article will be reviewed by one of ISPIs on-staff HPT experts,
and the author will be contacted if it is accepted for publication. If
you have any further questions, please contact firstname.lastname@example.org.
to printer-friendly version of this issue.
free to forward ISPIs PerformanceXpress newsletter to your
colleagues or anyone you think may benefit from the information. If you
are reading someone elses PerformanceXpress, send your complete
contact information to email@example.com,
and you will be added to the PerformanceXpress emailing list.
is an ISPI member benefit designed to build community, stimulate discussion,
and keep you informed of the Societys activities and events. This
newsletter is published monthly and will be emailed to you at the beginning
of each month.
you have any questions or comments, please contact April Davis, ISPIs
Senior Director of Publications, at firstname.lastname@example.org.
1400 Spring Street, Suite 260
Silver Spring, MD 20910 USA