Twenty years ago I was a wide-eyed grad student
immersed in the theory and application of instructional design, performance
factors engineering, and artificial intelligence. The world seemed
impossibly full of opportunities. It seemed I was a sponge, greedily
soaking up the literature. But it wasn’t enough to be an indolent,
inactive sponge. I experimented with fervor! I tried out the theories
I was learning in school- and work-related projects.
The technology at the time was giving us PC-based tools for authoring
courseware, hypertext, and expert system shells. As I worked at the
confluence of these and many other technologies, I began to consider
how hybrid applications might serve as powerful, intelligent, and adaptive
job aids. In 1986, I wrote a couple of articles that predicted that,
at some point in the near future, mechanics would carry in the top
center drawers of their toolboxes a micro-computer that would strap
to their wrists and, operating like Dick Tracy wristwatches, provide
them with just-in-time procedural instructions for making repairs and
decision support for troubleshooting.
I began to call these computer-based job aid applications “intelligent
job aids;” in fact, my doctoral dissertation was entitled Developing
Conventional and Intelligent Job Aids: A Case Study. A better term
was coined in the late 1980s. Gloria Gery was working at the time with
Marc Rosenberg and a project team at AT&T. They
began referring to their applications as Performance Support Systems.
The term was first used in print in 1989 in Gloria’s writing
in CBT Directions magazine. When she wrote a book on the subject,
her publisher, Weingarten Publications, added “electronic” and
titled the book, Electronic Performance Support Systems.
EPSS remains the customary term, though I hear
variations of this, primarily because of the propensity of vendors
to bastardize terms
to attach their own spin and branding. The hallmarks of performance
support, however, have changed little over the past 20 years. An
EPSS is a computer-based job aid that provides just-in-time, just-what’s-needed
assistance to performers on the job. An EPSS typically includes one
or more of the following features:
- Database of job-required information that is organized to facilitate
rapid access and optimize clarity
- Calculators and wizards that simplify and automate procedures
modules that provide intelligent assistance with problem solving
- Embedded tutorials and simulations that provide instruction in work-related
concepts and procedures
Just as a hand-tool leverages physical capabilities,
an EPSS leverages cognitive capabilities. An EPSS can provide adaptive
a full range of cognitive tasks—it makes performers smarter!
From the mid-80s through the entire decade of the 1990s, I worked
on dozens of industrial-flavored EPSS applications, e.g., service for
building climate control, motorcycle troubleshooting, nuclear reactor
coolant pump alignment and balancing, packaging equipment setup and
maintenance, and so on. The prediction about the wrist-mounted EPSS
was fulfilled nearly 10 years ago in several variations ranging from
handheld devices to miniature heads-up displays worn by the performer
over one eye while performing tasks.
The EPSS train had lots of momentum but seemed
to get derailed by the e-learning craze, which took the focus off
of performance and
put it on the Internet delivery channel. Many practitioners get bamboozled,
it seems, every time a new medium comes along, and the Internet craze,
by taking the focus off what’s really important, set our field
back in some ways. The Internet was a new channel through which to
deliver CBT (which became WBT), but bandwidth and other limitations
made it seem like we were regressing.
By now, most practitioners have figured out that the delivery channel,
e.g., the Internet, is generally not the primary driver in the design
of any performance support or instructional application (at least until
the next media fad comes along). More importantly, we must first thoroughly
understand a) the tasks to be supported, b) performer characteristics,
and c) the environment in which the tasks are performed. Then, we can
make good decisions about the delivery channel.
At Siemens Logistics
and Assembly Systems we do use e-learning extensively, but our focus
is on performance,
and EPSS applications are like bread
and butter. Here’s a quick overview of how we are supporting
sales, engineering, and project management staff:
Web-based EPSS applications assist sales
reps throughout our 10-step sales process. These
research and analysis, account
requirements, evaluation, account management, and even best-practice
The online support
job aids, e-learning modules, and some limited instructor-led training.
EPSS applications support
our engineers and project managers in much the same way, addressing
a wide variety of tasks
project planning, engineering
system commissioning. All of these applications
are available directly through intranet
but we’ve also
incorporated many of
them in a large-scale,
A lot has happened in our field the past 20 years.
that EPSS is not a flash-in-the-pan fad. The principles are enduring
and should be continually revisited by performance technologists.
Kim E. Ruyle, PhD, is the Director of Learning and Development for
Siemens Logistics and Assembly Systems. Kim is a long-time ISPI member
and has contributed to publications and many conferences. He may be
reached at firstname.lastname@example.org
||An EPSS can provide adaptive support for a full range
of cognitive tasks—it makes performers smarter!
by Carol Haig, CPT and Roger Addison, EdD, CPT
Bob Horn, Visiting Scholar at Stanford University,
Distinguished Consulting Faculty at Saybrook Graduate School and Research
Center, and Chair of
Information Mapping, Inc., is also the recipient of ISPI’s 2004
Thomas F. Gilbert Distinguished Professional Achievement award. In his
current work, Bob, who may be reached at email@example.com,
is using his Information Mural Visualization Method to help with the
analysis and communication of intricate public policy issues. He talked
with us about three predictions that will affect all of us in the workplace
in the near future.
Top Three Predictions
First, Bob predicts the development of an international auxiliary
language for business and commerce that
will evolve from a rapid increase in the use of visual language in
our communications. Fluency in this new language will become a core
communication skill requirement in the workplace.
Second, today’s available content management
software is robust. The proliferation of knowledge management efforts,
organization, and storage of learning objects, and related needs for
collection and arranging information for reuse will continue to grow
in organizations. This trend will encourage the use of a structured
information architecture to collect and organize information.
Third, there is a growing recognition among leaders
and theorists that organizations are “social messes.” These are “problems about which different people have very different
perceptions and values concerning their natures, causes, boundaries,
and their solutions.”
Reasons for These Predictions
Visual language is widely used today, from advertising to the ubiquitous
PowerPoint® presentation. In some companies, creating an informal
PowerPoint “deck” to describe an idea and emailing it to
colleagues is a standard method of communication. And we have the famous
death-by-PowerPoint presentation that is so much a part of sharing information
People are ready for a better way to present and exchange ideas, and
they are seeing the possibilities in visual language. They are learning what it is that words convey best
and what visuals do best. (For more on this topic, click
here to read the interview with Lynn Kearny, CPT in the April issue
of PX.) There is constantly improving software available,
millions of pieces of clipart to choose from, and best of all, we don’t
have to be able to draw.
As we, in our organizations and in society today,
grapple with ever more complex problems and ideas, we are becoming
more aware of the value
of the tight integration of words and visuals (such as diagrammatic elements
and images) and their power to clarify intricate issues. “You can’t
handle complexity in organizations without visual language,” Bob
says. And, he explains that there are global advantages to visual language:
the use of pictures “disambiguates” words, and vice versa.
If we look around us, we can see that organizations have been struggling
with information management for
some time. Now, senior leaders have identified the need for storing information
in an architectural structure for reference and reuse. The globalization
of commerce means that the architecture must be uniform. It must use
content management software with established standards that let users
identify small chunks of information to assemble into a document, for
example. The Institute for Electronic and Electrical Engineers (IEEE)
has taken the lead and established standards for reusable learning objects.
Soon companies will know exactly how much of their stored information
It is likely that most of us can identify a number of social messes once
we understand what they are. Many government and social programs such
as those for mental health services, long-term care, and health care
in general qualify. Familiar to most of us are the dysfunctional organizations
in which we work, and we could probably have a contest to choose the
one with the biggest social mess. Many long-term predicaments, such
as nuclear waste disposal, are also social messes with stakeholders
worldwide and complicated communications surrounding the issues.
Bob has created mess maps,
large diagrams that represent the many points of view that comprise social
messes. The maps help locate the problems expressed and show where they
intersect. Mess maps are great tools for task forces that must form mental
models of the problems they tackle. Having a common mess map can help
these groups take action quickly.
How Organizations Will Be Different
As organizations experience the failure of various large-scale implementations,
they will develop increased humility. At the same time, the pain of these
defeats will create an openness to learning how to manage broad, complex
issues more effectively. The truth is that there is a lack of expertise
in unraveling the issues of large-scale social change.
With the development of knowledge management architectures, executives
responsible for enormous websites will be better able to manage stored
information. Their organizations will become more adept at solving problems
by locating information quickly and reusing it appropriately.
If organizations fail to manage gaps and the white space (see Improving
Performance: Managing the White Space on the Organization Chart by Rummler & Brache), they will be managing more
Implications for HPT
As performance improvement practitioners, these predictions present
opportunities for us to make significant contributions in the organizations
we serve. We can mitigate messes by ensuring that we stay true to the
foundations of HPT:
a systematic approach to all problems and opportunities
with the end in mind
to business requirements
value for the organization
Using our tools, we can make a difference in our organizations and in
the world. (For more on this topic, click
here to read the interview with Margo Murray, CPT in the March 2002
issue of PX.)
If you have any predictions about the future of HPT that you feel would
be of interest to the PerformanceXpress readership, please contact
Carol Haig, CPT, at firstname.lastname@example.org or http://home.mindspring.com/~carolhaig or
Roger Addison, EdD, CPT, at email@example.com.
As performance consultants, we are increasingly
under the gun to demonstrate the return on investment for our work.
While it’s clear that most
of the value of an organization comes from intangible assets like employee
knowledge and motivation, reputation, and management processes and culture,
there are still no established ways to either measure or maximize them.
We have a great opportunity here since we are
the managers of these important strategic assets.
At the same time, our field has been trying to implement ROI processes,
with marginal success because:
impossible to separate out the effects of various performance improvement
interventions from other factors that impact individual behaviors and
business performance. Therefore, most measures of ROI are not credible.
- Most powerful
interventions are not discrete projects with a specific goal and
end, but rather infrastructure assets that undergird a variety of ongoing
- ROI calculations
are overly simplistic and don’t factor in important variables
such as the impact of taxes and the cost of capital. Therefore, when
present ROI data to executives and financial professionals, they
are not taken seriously.
Over the past six years, I’ve been researching and teaching new
methods for managing performance systems. Reading the literature in finance,
working as a visiting scholar at the Hanken School of Business and Economics
in Helsinki, and applying my new ideas in my consulting work has allowed
me to develop and refine some new approaches that are really exciting.
Let’s just take one short example:
Let’s say that you created a new online system for field repair
technicians; the application allows them to capture and share best practices
and to enroll in short e-learning modules. The system cost $200k to implement,
and within its first year it resulted in about $50k of efficiencies.
However, you expect that as the knowledge base builds, you’ll be
able to gain $100k per year in cost savings so you think the system will
pay for itself in a few years. The ROI isn’t so simple to measure,
because you’ve got to look at important financial factors such
as the cost of capital, depreciation, and ongoing maintenance costs for
the system. But let’s put aside those issues for the moment.
While $100k may look like a lot of money to you,
lunch money to your CEO. However, here’s a more powerful way to
position this: The knowledge system is an infrastructure asset.
If it can be positioned as producing a relatively permanent cost savings,
you can actually calculate its impact on stockholder value.
Your pre-tax savings is $100k, but after taxes, that
comes out to be only about $64k. HOWEVER, let’s say that your company’s Price/Earning
ratio is 24.1 (that’s close to the typical S&P 500 average).
What you’ve really done is to increase shareholder value by more
than $1.5 million (see table below)!
Pre-tax savings after adjustments for overhead and other fixed
After Tax Income
Applicable Price/Earning Ratio
Indicated Increase in Shareholder Value
You’re already a hero, and now you’re talking in REAL dollars.
But there’s more. If you’re thinking like an asset manager,
you may also be able to “productize” your intervention by
copyrighting materials or even patenting certain processes. Not only
does this protect your investment, but it could also provide the framework
for you to market your successful solution or license some of its underlying
algorithms, therefore creating a revenue stream. You should also consider
ways to publicize your intervention by writing articles, making conference
presentations, and getting mentioned in publications that are influential
to your organization’s customers and partners. This helps to strengthen
the brand. Are you seeing new ways to become a more entrepreneurial asset
If we want to truly become business partners, we need to:
- Speak the language
of business (and this means understanding at least some basic financial
measures and concepts)
- Re-position ourselves
from project producers to being managers of strategic infrastructure
- Demonstrate and
document how we impact important measures such as brand strength
and intellectual capital, and get those measures included in published
annual reports, etc.
- Create ways for
training and HPT to become profit centers through protecting and
leveraging an organization’s intellectual property
NOTE: This article is based on Diane Gayeski’s
new book, Managing
Learning and Communication Systems as Business Assets published
Diane Gayeski is CEO of Gayeski Analytics
and maintains academic appointments as Professor in the Park School of
Communications at Ithaca College and Adjunct Professor at Boise State
University. She recently earned the Excellence in Scholarship award from
Ithaca College for her ongoing research contributions to the field of
learning and communications. Diane may be reached at firstname.lastname@example.org.
||Diane will discuss this
topic more in-depth during her workshop at ISPI’s Annual Conference
in Vancouver, British Columbia, Canada, on Tuesday, April 12, 2005.
Did you attend
ISPI’s 2004 Performance-Based Instructional Systems
Design Conference in Chicago? Well, even if you didn’t, here’s
a contest just for you.
When I told a friend that I was making a presentation on performance-based
instructional systems design, she
exclaimed, “What’s that?” She also mumbled something
about the fact that I always talk funny. I wasn’t sure whether
she was referring to my accent or the tendency among ISPIers to create
terms that sound like the title of someone’s doctoral dissertation.
Anyhow, I tried to explain what performance-based instructional
systems design is all about. And
I failed miserably. That gave me an idea for this contest: Can you
define performance-based instructional systems design?
I have an international panel of judges ready to objectively evaluate
the definitions and select the best one. If your entry is selected, you
win! (What exactly do you win? Fifteen seconds of fame. We will publish
your name in a future issue of PX.)
You may be thinking, “But I don’t know
anything about performance-based
instructional systems design!” Don’t
let that stop you. After all, as a PX reader,
you should be able to figure it out.
Your definition should be exactly 16 words
long. No more, no fewer. (Don’t ask “Why?” Please humor
me for the present. It will all become clear at a later date.)
- Email your contest entry to email@example.com.
- Judging criteria include accuracy, creativity, and appeal to PX readers.
- Include your name and email address with your entry.
- You may send more than one entry (but you can win only once).
- The decision of our judges is final.
- Results will be announced in a future issue of PX.
entries will be posted to a website (of course, you will get
for the contest is 11:59 pm (EST), October 15, 2004. All entries
must be received by the deadline.
So you’ve got some questions about certification. Maybe you’re
thinking about taking the plunge. Maybe you’re already a Certified
Performance Technologist (CPT) and are wondering if there is anything
else you need to do. In any case, read on. Hopefully, you’ll find
all the answers to your certification questions below.
10. How many CPTs are there?
As of September 1, 2004, there were approximately 1,000 CPTs. Here is
the yearly breakdown of CPTs joining since the program started in 2002:
2004 (to date)
There were so many CPT applications in 2003 because
of the “grandparenting” clause.
9. What types of companies have CPTs on staff?
The companies that have CPTs are diverse. There are banks, universities,
storage companies, computer companies, printer companies, hospitality
organizations, medical and pharmaceutical companies, and government agencies
that have multiple CPTs on staff. Here is a list of the five companies
that have the most CPTs in their organization:
# of CPTs
State Farm Insurance
Concurrent Technical Group
To view the full list of current CPTs, click here.
8. Are people that are hiring actually looking for CPTs?
you look at the CPT website, you’ll find
a section called, Comments
from CPTs. These are comments from those who have their CPT designation.
You’ll notice that most of them say that the CPT allows them
to open doors with clients and internal customers to speak about performance
improvement using a systematic process. Others mention that the CPT
signifies professionalism, rigorous standards, and sound principles,
which allow the designees greater opportunities within their organizations.
Recently, I did a search on Monster.com for performance
technology. There were 25 hits and most of them had references to preferring
who was certified. One company in particular stated, “Possession
of a Certificate in Human Performance Technology preferred.” Another
company requested in their desired qualifications, “Professional
certification relevant to Learning, Quality, or Performance Technology.” So,
you can see that having the CPT designation is becoming a way for
candidates to distinguish themselves from other applicants.
7. I turned in my CPT application in July. Why
haven’t I heard
No matter when you turn in your CPT application, there are two times
a year that ISPI processes CPT
applications, June and November. It usually takes 2-4 weeks to process
an application so you should receive notice in July or December.
6. Once I’m certified, do I get a certificate?
The CPT designation is based on performance, not class attendance and
a test. If you want a piece of paper, however,
you may feel better
when you receive the 16” x 18” framed CPT recognition plaque.
5. Does certification last a lifetime?
No. To keep your certification current, you must submit re-certification
forms every three years. The purpose of these forms is to show how you
have made strides to stay abreast of the field of human performance technology.
4. Do I fill out the same forms for re-certification
as I did for my CPT?
No. The original certification forms were based
on performance in project work. Now that you’re a CPT, you need
to show how you have stayed current through continuing education, professional
development, and volunteer
service in our field. To review the re-certification requirements, click
As an example, you can earn 12 re-certification points by attending ISPI’s 43rd International
Performance Improvement Conference in 2005. You’ll need to
earn a total of 40 points to re-certify your CPT designation. Keep in
mind that for re-certification, you are once again required to sign
the Code of Ethics.
3. How will I know it’s time to re-certify?
time period for re-certification is every three years. That means that
10% of you will need to re-certify in 2005.
CPTs who received their
designation in 2003 will re-certify in 2006, and the remaining CPTs from
2004 will have until 2007. No matter when your original certification
date was, ISPI will send you an email reminder before your time
is up. That way you’ll have plenty of time to attend a conference
or institute, or volunteer for an organization to get re-certification
2. How can I find out if I’ve already earned
To review your member information, visit
website: www.ispi.org. Once there, log onto My ISPI and
click on your profile. To see which ISPI conferences you’ve attended,
committees you’ve worked on, and presentations you’ve submitted,
click on Activities. Then, you can transfer that information to the re-certification
forms and total up the appropriate number of points.
If you are active in your local chapter, ask them
to provide you with a list of the meetings you’ve attended and the volunteer positions
you’ve held. You’ll want to make sure to get credit for these
types of activities that International ISPI does not track.
1. What is the best thing about being a CPT?
may say it’s the glamour, others may say it’s the fame,
but nothing quite measures up to the pride you feel when you become a
CPT. Pride in collaborating, pride in taking a systems view, pride in
showing value, and pride in focusing on results. It’s these four
principles that you abide by in the work you’ve done in the past,
the work you do today, and work you will do in the future.
and Stay Certified! For more information, visit www.certifiedpt.org.
New in 2005! Explore
Canada’s West Coast
ISPI invites you to journey with us as we explore
Canada’s West Coast
during ISPI’s 43rd Annual International
Performance Improvement Conference, April 10-15, 2005, in Vancouver, British
Columbia, Canada. The experience begins the moment you enter this surreal world
through fresh evergreen trees surrounded by the aroma of a British Columbia
Rainforest. As you make your way into the rainforest, wander into Okanagan
Wine Country or visit Little Italy for delectable pasta. If it’s sushi
you’re craving, plan to spend time in the Pan Asian Square where food
and culture come together. Sample seafood and listen to crashing waves as you
watch a beautiful sunset off the shores of Vancouver Island. Whatever path
you follow on your journey, plan to enjoy food, drink, camaraderie, and an
energetic, electric ambiance that will enrapture you for hours.
One ticket for the Thursday evening networking event is included in all full
conference registration categories. Additional tickets may be purchased for
spouses, guests, or local clients.
With the overwhelmingly positive reaction
to ISPI’s Annual Conference,
it’s no surprise that half of our 2004 attendees in Tampa were
repeat participants. And, one-quarter had attended five or more ISPI
we would like to introduce, and re-introduce, more of your colleagues
by offering the Bring-a-Colleague rate once again. When you register
for the full conference at the member or delegate rate, you may also
register a colleague for only $450—provided your colleague has
not attended an ISPI Annual Conference in the past three years (2002-2004).
When you register, think
of a colleague at your organization, a client organization, your ISPI
or ASTD chapter, or an acquaintance in the field who has not experienced
a recent ISPI conference. Offer that person an opportunity to save
hundreds of dollars while benefiting from the premier educational event
in workplace performance improvement. The deadline is February 4,
2005. Click here to
Travel between the U.S. and the Canadian border now requires
a passport or original birth certificate. ISPI has compiled a list of
to help facilitate the process of obtaining or renewing a passport for
individuals who plan to attend the conference. Find out the requirements
for travel to the Vancouver, instructions on how to obtain a U.S. Passport,
Visas, Customs, and more by clicking here.
It is also important that you confirm the acceptable travel documents
with your air carrier when making your airline reservations.
also want to check out the latest conference updates, including CPT Events and Workshops. In addition, don’t
forget the deadline to submit your Award of Excellence entry is October
15, 2004. For
more information, click
In September 2003, I digressed from my consulting practice to teach
elementary school, making a return to my earliest career as a music teacher.
was an enlightening experience to return to the public school classroom
after many years as both a corporate employee and an independent performance
I found myself asking questions I would ask any corporate client:
results do we expect from our educational system?
competencies should students possess upon graduation?
students fully prepared to enter the workplace and make an immediate
the expectations we have of our educational system realistic?
My assignment was to teach beginning flute,
clarinet, alto sax, trumpet, and trombone classes. I had between five
and ten students
in each class. Spending a year teaching children music taught me that
putting together a concert-quality band is very similar to assembling
an award-winning team.
If you don’t know where you’re
going, anywhere will do. Trite, but very true! Effective leadership is
critical for achieving results in the corporation and in the classroom.
At the outset of each class, the students and I set a year-long strategic
the end of the school year, you will demonstrate your proficiency in
playing the flute, clarinet, alto sax, trumpet, or trombone.
Goals need to be translated into everyday language so people understand
what it is they are expected to do. Translated, this particular goal
be able to play the primary basic notes on your chosen instrument, play
a concert B flat scale and arpeggio accurately and musically.
It was important to state the goal precisely and gain agreement from
Next, we decided how we would measure
our progress toward that goal. Some metrics were already established:
Each music program presented a different repertoire—incremental
in difficulty and sophistication. Our quality metrics included:
the correct notes consistently
the conductor (me) fastidiously
to each other regularly
the tempo markings and dynamic levels accurately
A long-term goal is often easier stated than accomplished. Achieving
results requires focused, consistent effort and a commitment to hang
in there when the going gets tough.
Determining Action Steps
Most goals require several intermediate objectives with specific action
steps. Learning to play an instrument can be a challenge. Getting started
involves basic and incrementally difficult steps:
care of a musical instrument
the instrument; disassembling it
a sound on an instrument without squeaks
the fingering and positions for each note
to read notes
from one note to another to several notes in a musical phrase
the language of music
By mid-October, we were operational; that is, students knew how to care
for their instruments properly, produce reasonably good tones, and play
three to four notes accurately. Preparing to perform was the next step.
That involved teamwork. A music team is like a work team.
group must have a purpose for working together.
musical piece requires several instruments playing different parts
members must be interdependent; that is, they need each other’s
skills and commitment to arrive at mutual goals.
musical selection does not sound whole if some of the instrumental
parts are missing.
group must be accountable as a functioning unit within a larger
family of instruments must play its part of the piece accurately.
group members must coordinate laterally.
all instruments play every note, play the same notes, or play at
the same time.
face-to-face interaction among group members is required.
instrumentalist must be competent at his/her individual part and
each group of instruments must be in balance with the others.
are no individual stars; rather, there is a team of star performers.
- Any team, even a musical group, is only as good as its members.
As we prepared for each concert, all the required skills came into play.
Beginning instruments, now a beginning band, readily achieved their stated
goals for each concert because students were clear at the outset what
they needed to do. They also realized it took teamwork to get there.
For the final concert of the year, the band chose to play Highlights
from Harry Potter, a challenging
piece even at the middle school level. With lots of practice, individually
and collectively as a band, they played the selection extremely well.
It sounded like Harry Potter! Imagine the sense of personal satisfaction
and team exhilaration at being able to stretch and achieve a goal beyond
their expectations. It was heady!
matter what the situation, leaders must provide direction,
set performance expectations, remove obstacles to getting the job
done, and inspire and
motivate their followers.
are more productive when members individually have
a high commitment to deliver performance results.
have more fun, and having fun is integral to successful performance.
Scanlon Wilkins, CPT, a long-time member of ISPI and principal
of The Wilkins Group, works with clients in various industries,
helping them achieve their targeted business goals. She is an Arts
for the City of Walnut Creek, CA. Sheila also works in the
schools teaching instrumental and vocal music. You may reach her
Spending a year teaching
children music taught me that putting together a concert-quality
band is very similar to assembling an award-winning team.
The Certified Performance Technologist (CPT)
designation is awarded by ISPI to experienced practitioners in the field
of performance improvement
and related fields such as instructional design and organizational development
whose work meets the 10 Standards of Performance Technology and other application
requirements. The deadline for submitting your application to become a
CPT must be received at ISPI by November 15, 2004, or it will be held until the next processing deadline
of June 15, 2005. Visit www.certifiedpt.org for
more information on becoming a CPT and to download the application.
past issues, this column has mostly addressed nitty-gritty measurement
about how to decide what to measure, and how to measure in ways that
will optimally support management and instructional decisions. The
focus has been mainly quantitative. With that as background, this month’s
column is a bit of a digression—but an important one, I think.
During Training and Beyond
readers must realize, when my associates and I measure the results
of training or other performance improvement interventions, we look for things to count.
We identify accomplishments (job or process outputs), sub-accomplishments
(sometimes called milestones) or types of behavior to count, and we
graph repeated measures to monitor changes in level,
trend, variability (bounce) or quality over time (per minute, per
hour, per day, per week, per month). We use the “performance
pictures” formed by charted data to decide whether or not we’re
achieving desired results and whether we need to change our approach.
you might also be aware, we use counts of responses or outputs per
practice exercises and tests to give trainers, coaches, and learners
themselves feedback about whether and how rapidly they’re achieving
desired component-level performance goals, and whether they need to
make changes in their learning or practice strategies to optimize progress.
We count and time behavior to determine if and when learners achieve fluency. With these measurement
procedures in place, traditional percent-correct testing or rating
scales became largely irrelevant because they are so far less sensitive
than count/time performance measures.
Simulations to Measure Application
end of training or coaching programs, or after completion of modules
or units intended to develop a particular repertoire or type of performance,
we often use performance tests in the form of high-fidelity simulations to
determine whether and how well individuals can perform. If, as we do,
you distinguish among three stages of learning—1)
initial learning, 2) practice for fluency, and 3) application—performance
tests focus on stage 3 in which learners combine fluent components
to produce important job outputs.
distinguished colleague, Judith Hale (2000), has written an excellent
how to design and
implement performance-based certification programs
in organizations. Covering virtually every aspect and angle related
to certification—including the business drivers and details to
consider when implementing such programs—she touches on the use
of performance tests as part of an overall picture.
such diverse areas as sales, customer service, equipment maintenance,
accounting, we’ve found it possible to combine real or simulated
inputs, tools, and other elements of real-world situations into test
experiences that challenge performers to meet actual on-the-job requirements.
In sales, for example, we’ve designed case study scenarios in
which trainees review telephone transcripts, account notes, financial
records, industry background, and other information and then use sales
reference materials and collateral to prepare for and execute simulated
sales calls with peers or managers. Insurance company representatives
(Enrollers) have used job aids to customize standard presentations
for specified audiences, delivered the presentations to peers, and
responded to tough questions and objections. Maintenance technicians
have been given equipment with specific symptoms to diagnose and repair,
and accounting trainees have been asked to complete portions of audit
procedures with pre-determined errors to detect, etc. In each case,
evaluators use behavioral checklists to monitor critical features of
required performances and evaluate job outputs. In some cases, performers
must meet specific time requirements in order to pass.
In our experience,
most such performance tests are pass/fail rather than numerically evaluated.
When trainees are unable to perform to criterion, trainers or managers
give feedback and direction for practicing missed components in preparation
for another attempt, often with a different scenario or case.
my associates and I hear the term “certification,” we usually
think of this type of high-fidelity simulation or case-study challenge.
We realize that many of our colleagues, including Dr. Hale, use a broader
set of evaluation methods, including percent-correct knowledge tests,
for what they call certification. But when we propose to “certify” that
individuals or groups are definitely able to perform important tasks
or jobs, we use the most realistic and representative scenarios we
can arrange so that performers, their trainers, coaches, and managers
can see without additional interpretation whether and how well they
are actually able to perform.
be most interested in hearing from readers about experiences using
simulations as performance tests, and the results achieved with that
certification: How to design a valid, defensible, cost-effective program. San Francisco: Jossey-Bass/Pfeiffer.
Dr. Carl Binder is a Senior Partner at Binder Riha
Associates, a consulting firm that teaches clients the FluencyBuilding™ performance
improvement methodology, practical measurement strategies and tactics,
and Six Boxes™ performance management. His easy-to-remember
email address is CarlBinder@aol.com, and you can read other articles
by him at www.Binder-Riha.com/publications.htm.
See past issues of this column by clicking on the “Back Issues” link
at the bottom of the blue navigation bar to the left.
This past semester I taught a
graduate course about performance technology for
20 on-campus students and 24 distance students. One assignment, dubbed
asked graduate students to choose a past project
that would benefit from some serious PT “magic.” Sherry
Ryan was asked to improve the technical competence of leaders
in her company by offering training programs. And, that’s exactly
what she did, not surprisingly. But was that what was needed? When
Ryan did a makeover, things looked very different to her.
Read all about it in Moving to a Performance Mindset. If
you would like to contact Dr. Rossett, she may be reached at firstname.lastname@example.org.
Most of the projects I have worked on during my 25-year
career have been training interventions. I saw my job as developing training
and generally did not look at the bigger picture to examine other factors
that can affect performance.
For one major training initiative, I was the project
leader. It involved a perception that many technically savvy manufacturing
were reaching retirement age; and that the new, up-and-coming leaders
didn’t have the same level of technical expertise. This could
weaken our mill system and our ability to remain competitive in the
At the same time, our company was going through a massive
effort to streamline support services, like payroll systems, purchasing,
and many others. All soft skills training was centralized under a corporate
education department, which resulted in downsizing training departments
at the site level. Businesses within the company and individual sites
still “owned” technical training, but there was some confusion
about this and about how training should be handled at the corporate,
business, and site levels.
I was asked to work with the manufacturing leadership for a primary
business to determine key technical competencies needed for front-line
leaders, identify gaps, and seek ways to close the gaps.
What We Did
training or organization design professionals at about half of our
primary manufacturing units (concentrated on facilities that were considered
to be high performers) and key business leaders (corporate level)
- Questionnaires sent out by email (18 open-ended questions)
- Questionnaires were forwarded by respondents to two additional
sites that were not “high performers”
- 80% return rate from site contacts
- Compiled and summarized survey data and sent to all respondents
- Sent executive summary of findings to each site manager, mill training
contacts at each site, and corporate manufacturing leaders
- Asked for confirmation of findings. Did the findings ring
true? How would they expand upon what we’d learned in the
surveys? Received very little additional input, but no disagreement.
- Formed an advisory group to:
- Prioritize gaps
- Identify resources to close gaps
- Assess existing courses, identify vendors
- Write work plan and budget to adapt existing courses, develop
new courses, and deliver
We made quite a bit of progress with the work listed above before
our company went through a series of major management changes that
resulted in our project being curtailed for the most part.
What We Could Have Done Better
- While we did some data gathering, our analysis could have been improved
by including input from:
- Members of target audience (front-line leaders themselves),
including high performers and a random sample drawn from all
- Data from lower performing mills
key assumption was not rigorously validated. We needed more specific
information on the “new” front-line leaders
- Demographics of front-line leaders who are not ready to retire
and those soon to be promoted to leader positions
- Are they, in fact, really less technically knowledgeable?
- What and how much do they need to know? Could the technical
expertise reside elsewhere?
- What are specific ways that front-line leader technical
expertise can improve operating unit performance?
- The role of the first-line leader was changing in many locations—some
were high-performing work teams, some traditional work systems, and
variations in between. We did not have a good way to address this.
- We had a few systemic questions on our survey related to organizational
barriers, succession planning, and knowledge management but did not
use that data to generate a tailored solutions system.
- Our group was committed to training as the primary method to enhance the contributions of front-line leaders
to bring about improved operating reliability and efficiencies
- We had hoped to publish a list of key technical competencies
with resources, self tests, and possibly set up a certification
requirement, but did not vigorously pursue this
- We did not provide any other support for front-line leaders
such as online references, technical contact lists, online community,
- When offering supervisory training (most of which was classroom
courses provided by high-quality vendors), many mills opted to send
operators in addition to, or instead of, first-line supervisors. This
created problems for the instructors and lowered the value for the
target audience in some cases.
- This was partly due to our strategy to offer regional or mill-based
classes to minimize time away from the mills. But in some cases,
the mills couldn’t free up enough supervisors to fill a
- Another issue was that our project looked at front-line leaders’ technical
competency in isolation. We did not look at the technical competency
of the entire mill system to see if there were other segments
with more critical technical training needs.
I am not sure that we could have addressed all of these shortcomings
in a cost-effective, timely way. But, we certainly could have been
more deliberate in deciding if we should address
them before investing in training. In the end, our efforts were well-received
and seen as a positive contribution to improving operating performance.
Could we have had a more significant impact by looking at this as performance
technologists? I certainly think so.
Ryan is an instructional designer at Weyerhaeuser Company in Tacoma,
WA. She is looking forward to completing a Masters degree in Educational
Technology through San Diego State University’s online program
in May 2005. Sherry may be reached at email@example.com
The International Society for Performance Improvement (ISPI) is seeking an ISPI member who has the flexibility to take on the commitment and responsibilities
of Editor for Performance Improvement (PI).
We’re looking for a member who can demonstrate an extensive knowledge
of human performance technology (HPT), has a professional HPT network,
and possesses an editorial review ability. The Editor will be responsible
for acquiring, reviewing, and selecting manuscripts and will contribute
suggestions and ideas toward the editorial direction. The Editor will
work with authors and potential authors to maintain the highest standard
of editorial content and will work directly with ISPI’s Senior
Director of Publications, who is responsible for all production and distribution.
The Editor reports to the Executive Director, who serves as Publisher
of Performance Improvement. The
position requires a two-year commitment, commencing in April 2005. The
Editor will receive $10,000 a year as compensation for the invested time
PI is published 10 times a year and is distributed to more than 5,000 members,
subscribers, and institutions. For an application and instructions,
or for questions regarding the position or the application process,
please contact April Davis, ISPI Senior Director of Publications, by
phone: 301.587.8570 x112; by fax: 301.587.8573; or by email, firstname.lastname@example.org.
Marketplace is a convenient way to exchange information
of interest to the performance improvement community. Take a
few moments each month to scan the listings for important new
events, publications, services, and employment opportunities.
To post information for our readers, contact ISPI Director of
Marketing, Keith Pew at email@example.com or
Performance Consulting According to Thiagi. Thiagi doesn’t
have what it takes to be a serious performance consultant.
But, he has created
techniques and templates to make you a playful performance
technologist. For free stuff (and expensive stuff) on interactive
strategies for improving performance, visit www.thiagi.com.
Performance Consulting According to Rummler uses
an extensive case study to illustrate what a serious performance
consulting engagement looks like, and what a serious performance
consultant does. Do you have what it takes to be a SPC?
Performance is a whimsical, entertaining, and solidly
written book that addresses human performance. From beginning
to end, readers are guided toward an understanding of human
performance improvement and how to use it for real organizational
Darryl L. Sink & Associates,
Inc. is offering these workshops for Fall 2004: Designing
Instruction for Web-Based Training, San Francisco, October 18-20;
The Instructional Developer Workshop, Chicago, October 25-27
and San Francisco, December 13-15; The Course Developer Workshop:
Online Anytime! Visit http://www.dsink.com to register!
Job and Career Resources
Online CareerSite is your source for performance improvement
employment. Search listings and manage your resume and job applications
The International Journal of Coaching
in Organizations (IJCO) is a professional journal, published
quarterly to provide reflection and critical analysis of coaching
in organizations. The journal offers research and experiential
learning from experienced practitioners representing various coaching
schools and methodologies.
Improvement Quarterly, co-published by ISPI and FSU,
is a peer-reviewed journal created to stimulate professional
discussion in the field and to advance the discipline of Human
Performance Technology through literature reviews, experimental
studies with a scholarly base, and case studies. Subscribe
Are you working to improve workplace performance?
Then, ISPI membership is your key to professional development through
education, certification, networking, and professional affinity programs.
If you are already a member, we thank you for your support. If you have
been considering membership or are about to renew, there is no better
time to join ISPI. To apply for membership or renew, visit www.ispi.org, or simply click here.
ISPI is looking for Human Performance Technology
(HPT) articles (approximately 500 words and not previously published)
for PerformanceXpress that bridge the gap from research to practice (please,
no product or service promotion is permitted). Below are a few examples
of the article formats that can be used:
- Short I wish I had thought of that Articles
- Practical Application Articles
- The Application of HPT
- Success Stories
In addition to the article, please include a short bio
(2-3 lines) and a contact email address. All submissions should be sent
to firstname.lastname@example.org. Each article will
be reviewed by one of ISPIs on-staff HPT experts, and the author
will be contacted if it is accepted for publication. If you have any
further questions, please contact email@example.com.
to printer-friendly version of this issue.
free to forward ISPIs PerformanceXpress newsletter to your
colleagues or anyone you think may benefit from the information. If you
are reading someone elses PerformanceXpress, send your complete
contact information to firstname.lastname@example.org,
and you will be added to the PerformanceXpress emailing list.
an ISPI member benefit designed to build community, stimulate discussion,
and keep you informed of the Societys activities and events.
This newsletter is published monthly and will be emailed to you at
the beginning of each month.
you have any questions or comments, please contact April Davis, ISPIs
Senior Director of Publications, at email@example.com.
1400 Spring Street, Suite 260
Silver Spring, MD 20910 USA