By Brett Christensen, MSc, CTDP, Erica Cormack, & Barb Spice

This article explores the twists and turns of a road less traveled when evaluation and human performance technology (HPT) methodologies cross paths. Rather than choosing a fork in the road, the two paths blend together in this evaluation research involving Aboriginal youth of Canada.

Leadership should be a reflection of the community in which it will serve; in Canada, this means incubating and developing leaders representative of the First Nations, Métis, and Inuit, collectively referred to as “Aboriginal” communities. Historically, however, opportunities for Canada’s Aboriginal population to step up and experience leadership and its related activities have been limited, at best. The Aboriginal Leadership Opportunity Year (ALOY) was created by the Canadian Forces to address this need; Aboriginal youth are provided with a unique opportunity to academically prepare for post-secondary study or military service (or both) in a culturally aware setting, which offered multiple opportunities to participate in the full scope of leadership behavior.

The Canadian Forces recognizes that effective leaders are created as a result of the sum total of knowledge, skills, self-awareness, and life experience; in other words, effective leaders manifest their skills across multiple dimensions. It makes sense, then, to evaluate leadership performance using a variety of methods, both in and out of the classroom, and in settings and circumstances that authentically represent the anticipated venue in which the leader will be expected to perform. ALOY achieves this goal through the utilization of a leadership practicum exercise at the end of each program year. This culminating activity requires participants to apply their skills, knowledge, and experience to strategize, plan, and lead a group of peers through an operational exercise, thus demonstrating the full spectrum of leadership skills for the purposes of accomplishing the task and for final evaluation of the learners’ development as a leader.

To evaluate the effectiveness of this dimensionally rich evaluand required an evaluation plan, which was equally able to capture the data of a multidimensional initiative. The introduction of a comprehensive evaluation checklist ensures all stages of evaluation are completed in a logical, iterative fashion; the addition of HPT models and instruments brought added value to the process by ensuring resources, outputs, and outcomes were assessed not only in the context of ALOY, but in relationship to the desired goal of raising up Aboriginal leaders to serve in their communities, in the Canadian Forces, and in the nation as a whole. Did you catch the “twist”?

Without a doubt, Scriven’s Key Evaluation Checklist was the driving instrument behind the evaluation, but the evaluation team’s neat twist on the application was to introduce Rummler and Brache’s Performance Matrix model and Kaufman’s Organizational Elements Model (OEM) as tools to explore the various dimensions of the evaluand as it was described through the Kellogg logic model. Think of the overall model looking something like a Rubik’s Cube; you need to twist and turn the pieces in each layer to arrive at an orderly end result. Only through the churning up of the foundational levels and supports of the ALOY practicum could the team be assured their data was representative of the whole project and the subsequent analysis “complete.”

Brett, Erica, and Barb are three of the 100+ presenters sharing their knowledge and expertise at THE Performance Improvement Conference 2011, April 10-13, in Orlando, Florida. If you would like to learn more, you may attend their 90-minute presentation, “Evaluating Leadership Development in an Academic Program.”

Brett ChristensenAbout the Authors
Brett Christensen, MSc, CTDP, is the wing training development officer at 1 Wing Headquarters in Kingston, Ontario. He is a Certified Training Development Professional and completed his Master of Science in instructional and performance technology (IPT) at Boise State University in September 2010. He may be reached at brett.christensen@forces.gc.ca.



Erica CormackErica Cormack is a curriculum designer and teacher in the justice studies program at a Southern Alberta college. She is pursuing a Master of Science in instructional and performance technology (IPT) at Boise State University with completion set for 2011. She may be contacted at erica.cormack@lethbridgecollege.ca.




Barb SpiceBarb Spice will complete her Master of Science in instructional and performance technology (IPT) at Boise State University in 2011. She is the principal consultant for HRchitecture LLC, helping small to midsize organizations integrate business and HR strategies. Barb may be reached at bspice@hrchitecture.com.