By Ann M.W. Bassolino

Learning to Respect ADDIE
Following 10 years of service as a Coast Guard commissioned officer, I was selected to pursue an MA in Educational Technology at San Diego State University. This degree will direct my next duty assignment in a role responsible for enhancing the performance and productivity of the Coast Guard and its members. Within the first few days of starting this program, I quickly learned the importance of respecting the ADDIE process. Each letter represents a stage in the human performance process and stands for: analysis, design, develop, implement, and evaluate. These stages complement one another in the pursuit of enhanced human performance.

Setting the Stage
During a recent Intro to Performance Technology assignment, I was tasked to find a situation that would benefit from a job aid. I looked to a recent experience of mine as a Coast Guard safety manager. A position that is an extra duty for many personnel as it was for me, I saw a great need for a job aid to help new safety managers assimilate quickly and efficiently into this demanding role.

As a developing junior officer, I was excited for this opportunity as a safety manager to learn about a new field and a chance to lead others. After the first two weeks on the job, worry overcame me as I began to understand the extensive amount of content I needed to learn. Failing to find useful guidance, I fumbled through the position for the next two years. I chose this assignment to concentrate on producing a job aid with the intention to help current safety managers.

Following a brief review of current policies, I administered surveys and conducted interviews with key personnel. During one of the interviews, it was suggested that a field guide would be helpful. Therefore, what better way to help those in need by giving them what they ask for! A six-page field guide was then developed and distributed for feedback. To my excitement, participants were satisfied with the end product! I turned the field guide over to the program managers at the headquarters level for oversight and distribution.

Throughout this assignment I felt confident in my sources, line of questioning, and job aid design. I received positive feedback from the field and the headquarter program managers. I was thrilled that senior personnel were pleased with my work and that the field guide had been added to the Coast Guard’s Office of Safety and Environmental Health resource webpage. Today, a few months after this project and after taking more courses in learning and performance improvement, I now see how much more I could have done while respecting the ADDIE process.

A Second Chance
The Coast Guard’s approach to improving human performance systematically follows the ADDIE process. Upon gaining a better understanding of the Coast Guard’s own respect for ADDIE, I recognize and discuss several aspects in each ADDIE stage that can make this assignment stronger.

Analyze
I found that my data collection results were less than desired due to my limited access to key personnel and to the survey design itself. Conducting an initial focus group with Coast Guard safety leadership would lead to better defined expectations and objectives of safety managers. Some ambiguity exists in the current job descriptions, which leaves individuals questioning their roles and responsibilities. Clarifying these job descriptions would help aid the analysis process in identifying key elements to incorporate into the field guide. Also, developing a larger-scale survey using computer-based programs like Surveymonkey or Qualtrics would generate stronger feedback from a larger audience. I had used a low-tech approach by utilizing a Microsoft Word form template that was emailed directly to a handful of respondents. The response rate was less than desirable.

Another aspect that was not considered was the organizational white space (Rummler & Brache, 1991) [K2] that lives within the Coast Guard safety program. I attempted to address this white space in my solution system with a generic recommendation for an organizational analysis to be conducted; however, no explanation was given to say why this extra analysis is critical to success. An organizational analysis of the safety program would help identify where gaps exist within the program and how they ultimately affect the performance of safety managers in the field.

Design and Development
By involving the information technology (IT) division in the design and development of the field guide, the end product could be more interactive and easier to access by safety managers. I developed the field guide by using Microsoft Publisher and Word with hyperlinks and other helpful information. It was designed to be printed, saved on a computer desktop, or accessed from an online site. If the field guide moves to the Coast Guard cloud, is converted into a Google site, webpage, wiki, or other resource-based platform and supported by the IT division, it will have the technical support necessary for continuous improvements over time.

Implementation
Implementation is one of the five phases of ADDIE that tends to get overlooked during project planning, yet the success or failure of a project rests heavily on how its solution system ultimately gets implemented. Villachica, Stone, and Endicott (2004) highlight two reasons why projects fail: lack of planning during the development phase and implementation is forgotten until the end. Moseley and Hastings (2007) provide an intervention implementation process model to help individuals incorporate a plan into their project’s design. Even though this model is geared toward larger scale projects, certain aspects could help identify a stronger implementation support system to the field guide. For example, identifying and involving the right ownership from the beginning of this project would secure the longevity of this tool.

Evaluation
Evaluation is another element of the ADDIE process that is often forgotten; I only considered it as an afterthought. I did seek out and receive feedback on the field guide by a small pool of users, but this help resulted in only finding spelling and grammatical errors. Incorporating an evaluation plan would provide substance to support continual improvement of the field guide. The size of this assignment does not call for a Level 3 or 4 on Kirkpatrick and Kirkpatrick’s Four Levels of Evaluation (2006), but it would at least benefit from the implementation of feedback surveys to capture Level 1 (reaction) and 2 (learning) data. These levels would identify areas of improvement to the field guide itself and measure safety managers’ perception and feelings about their roles and responsibilities. Evaluating at Levels 1 and 2 will capture beneficial data that can be used in other analyses.

Reflection
As I continue to develop my knowledge about performance technology, I have a better understanding and respect of all five elements of the ADDIE process. Each element requires strong planning skills. With a project management course in my near future, this assignment could benefit from another PT makeover!

References
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating Training Programs: The four levels, 3rd ed. San Francisco, CA: Berrett-Koehler Publications.

Moseley, J., & Hastings, N. (2007). Implementation: The forgotten link on the intervention chain. Performance Improvement, 44(4), 8-14.

Rummler, G. A., & Brache, A. P. (1990). Improving performance: How to manage the white space on the organization chart. San Francisco: Jossey-Bass.

Villachica, S., Stone, D., and Endicott, J. (2004). Project Alignment: Ensuring successful development and implementation from day one. Performance Improvement, 43(10), 9-15.

About the Author
Ann M.W. Bassolino has worked as a Coast Guard instructor at the U.S. Merchant Marine Academy and as both a vessel inspector and marine casualty investigator for the U.S. Coast Guard over the past 10 years. She is currently pursing an MA in Educational Technology at San Diego State University (SDSU) and completed this work in partial fulfillment of her requirements for EDTEC 685 taught by Dr. Allison Rossett. She may be reached at Ann.M.Bassolino@uscg.mil.

 

[1] This field guide can be found at www.uscg.mil/SAFETY/cg1132/safety_manager.asp. Scroll down the page to the “Other Resources & Sources of Information” section and click on “Sector Safety Manager Field Guide V2 (job aid).”