By Jeff Alger, Karen Baerlocher, and Kim Harris

Tales from the Field, a monthly column, consists of reports of evidence-based performance improvement practice and advice, presented by graduate students, alumni, and faculty of Boise State University’s Instructional and Performance Technology department.

A Case Study of Evaluating a High School Associated Student Body Program

Evaluation Request
In the spring of 2013, administrative staff at Wize High School (WHS) (a pseudonym) requested an evaluation of its Associated Student Body (ASB) program be conducted for the 2012-2013 school year to determine if the operations the ASB uses for planning and implementing school-sponsored activities have a positive impact on the ASB student officers, the WHS student body, Wize High School, and the local community. Wize High School is an independent, tuition-free, public charter school located in the northwest region of the United States. The school serves students in kindergarten through 12th grade. The ASB program at WHS is an extracurricular activities program that has been developed as an effort to provide students with an opportunity to learn leadership skills, allow for organized school-sponsored activities, and give participating students a way to explore a variety of interests as they mature and become responsible citizens who will serve in the community. A team of Boise State University’s graduate students conducted this evaluation case study.

Methodology
The evaluation was conducted as a formative goal-based evaluation, based on the fact that Wize High School has clear expectations and desired outcomes for ASB and intends to use the conclusions from the evaluation to improve the program for future ASB operations. A program logic model was created with information provided by the faculty adviser and the ASB student president to outline the process and provide the evaluation team with both the intended and potential performance outcomes expected of the program (see Figure 1).

Tales-HSFigure1

Figure 1. Wize High School’s Associated Student Body Program logic model

A brief analysis of Scriven’s Key Evaluation Checklist (Davidson, 2005) led the evaluation team to ask the questions: “Who asked for this evaluation and why?” and “What is (or should be) the overall design, and why?” To answer these questions in a manner that would align with Wize High School goals and mission, the evaluation team met with the client to discuss the questions and the program’s logic model, to refine it and make any necessary corrections. Working with the administrative staff and the ASB adviser, the team determined the evaluation would be based on four main dimensions of merit (see Table 1).

Dimension Dimensional Evaluation Question
1. ASB Training How well were the students and faculty trained on their respective roles?
2. Communications Plan Was the plan for communications between ASB members and administrators effective, timely, and appropriate?
3. Improvement in Student Skill Set Was the program aiding students in improving the following skills: communication, leadership, delegation, and organization?
4. ASB-Sponsored Activities Was ASB meeting its targeted goals for the number and type of activities it carries out and oversees?

 Table 1. ASB Dimensions of Merit

A second meeting with the client was held to discuss how the evaluation would proceed and to outline the timeline it would follow and who would be responsible for information gathering and reporting. In addition, it was decided that each dimension would be weighted based on the importance of that element. The client agreed that the dimensions of merit should be measured on a four-level evaluation rubric–poor, marginal, good, and excellent.

The evaluation team gathered data in cooperation with the client utilizing a WHS faculty-administered paper survey, along with interviews of select ASB student officers and faculty members, all of whom are directly involved or affected by ASB operations, for triangulation. The team drew conclusions based on this triangulation and identified factors that contributed to ASB success in meeting its goals and outcomes. The team provided recommendations for ASB program improvement based on the triangulation of data and the conclusions derived.

Conclusions
The evaluation team concluded that the overall quality of the ASB program was good when measured on a four-level evaluation rubric (see Table 2). This was determined in collaboration with the client, accounting for the level of importance the client placed on each dimension of merit. Because the ASB Training and Communications Plan were weighted as extremely important, the team focused heavily on these two areas when forming recommendations to be presented to the client.

Dimension Poor Marginal Good Excellent Weighting
1. ASB Training X Extremely Important
2. Communications Plan X Extremely Important
3. Improvement in Student Skill Set X Very Important
4. ASB-Sponsored Activities X Very Important
ASB at Wize High School
Overall Quality: Good

 Table 2. ASB at WHS Evaluation Results

Recommendations
The conclusions and recommendations were presented to WHS administrators and the ASB faculty adviser. The evaluation team found the following recommendations would offer performance improvement opportunities for the Wize High School ASB:

  • Offer ongoing training to ASB members as well as allow for practice of new knowledge and skills
  • Provide documentation to each ASB member that outlines specific duties and responsibilities pertaining to the position held by the individual
  • Train faculty members who work directly with ASB student members as to the specific responsibilities associated with ASB operations and mentoring of students
  • Use ASB weekly meetings to discuss and practice principles of leadership and delegation
  • Identify specific skills and talents of peer students at WHS to realize ASB activities (identification is to be done by ASB student members)
  • Delegate to peer students at WHS who possess a talent or ability related to or supportive of any necessary tasks required to carry out each ASB activity (delegation is to be done by ASB student members)
  • Pursue adoption of an ASB constitution for WHS
  • Hold regular meetings with school administrators and the ASB to discuss ASB event planning

Post-Evaluation Results
Wize High School has implemented the majority of the recommendations for the current school year and plans to eventually implement all of them over the course of the year. Even in these early stages, it is experiencing improvement in all four dimensions of merit. In June 2013, the school appointed a second ASB faculty adviser who is being trained by and works in cooperation with the more experienced co-adviser to mentor students and oversee ASB operations. Students are having success as they work together to plan and carry out ASB-sponsored activities, and the quality of activities has improved, resulting in an increase in school spirit and an increase in the number of students attending ASB-sponsored activities. Due to the recent changes, the ASB as a group has been planning more effectively and has improved its abilities in communicating with one another and the school administrators, in delegating tasks to students both directly and indirectly involved in ASB activities, and in raising interest and involvement on the part of the student body and faculty. The ASB has also planned all of its sponsored activities through the present school year and has clearly outlined who will be responsible for each one and which student groups will assist the ASB and when this is to occur. WHS administrators are pleased with the progress that has come about based on the changes resulting from the implementation of the recommendations and expect that the ASB program will be able to better meet its goals presently and in the future. The long-term outcomes that are currently being affected include personal growth of student leaders, an increased awareness of ASB responsibilities and the need to fulfill them, and a more visible presence of the school in the community as an organization that provides service and produces responsible citizens.

References
Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks, CA: Sage.

About the Authors

JeffAlgerJeff Alger is an active duty Senior Chief Petty Officer in the U.S. Coast Guard pursuing a Master’s of Science in Organizational Performance and Workplace Learning from Boise State University. He plans to graduate in May 2014, and pursue a career in the field once he retires from the military. He may be reached at jeffalger@u.boisestate.edu.

 

 

KarenBaerlocherKaren Baerlocher, SPHR, has over 15 years of experience in the human resources field and is a graduate student in the Organizational Performance and Workplace Learning Program at Boise State University. She will complete her master’s degree in May 2014, and may be reached at karenbaerlocher@u.boisestate.edu.

 

 

KimHarrisKim Harris is a recent graduate from the Department of Organizational Performance and Workplace Learning at Boise State University. She has taught Spanish at the university level for over 13 years and also teaches at a local high school and may be reached at kharris@boisestate.edu.