By Rick Mauney and Shari Stein

Tales from the Field, a monthly column, consists of reports of evidence-based performance improvement practice and advice, presented by graduate students, alumni, and faculty of Boise State University’s Instructional and Performance Technology department.

Problem with Collecting Information
A federal housing compliance contractor, ABC-Contractors (a pseudonym), was seeking a way to improve information gathered from applicants for U. S. Housing and Urban Development (HUD) affordable housing subsidies. A review of recent applications had shown that site managers were frequently failing to obtain all of the information that would lead to an accurate calculation of housing subsidies. These inaccurate subsidy calculations affected ABC- Contractors’ organizational goals which are to ensure HUD does not over pay subsidies to owners, tenants pay the correct amount of rent, and property owners save money on lost rent and assessed fees. One of the measures recommended by HUD (2008, 2009) was to perform strategic interviews for gathering information, but the site managers lacked strategic interview skills.

Learning through Role Play
To address this problem, a team composed of graduate students in Boise State University’s Instructional and Performance Technology program designed an ‘interview skills’ training workshop based on the concept of “problem- based learning.” The team used a learning hierarchy to outline the interpersonal, questioning, and listening skills that site managers need to gather accurate information from applicants. The core content for the program was gathered from HUD documents while subject matter experts from ABC-Contractors provided additional detail based on their own experience in interviewing applicants.

The resulting training program sought to achieve three objectives:

  • Establish a rapport using effective verbal and non-verbal communications skills
  • Apply active and passive listening skills
  • Ask the right questions at the right time

To measure the achievement of these objectives, the team developed an assessment rubric that could be used with role play exercises.

Problem-Based Approach
The team then developed an instructional plan based on Merrill’s “first principles of instruction” (2002), described in Table 1.

Table 1. Merrill’s first principles of instruction.

Principle Learning is promoted when…
Problem Learners are engaged in solving real-world problems.
Activation Existing knowledge is activated as a foundation for new knowledge.
Demonstration New knowledge is demonstrated to the learner.
Application New knowledge is applied by the learner.
Integration New knowledge is integrated into the learner’s world.

 

The team discovered that most site managers prefer to learn by doing. So, the centerpiece of the training program was the “application” principle. To implement this principle, the team created a set of realistic role play exercises that allowed each site manager to practice his/her interviewing skills with a variety of applicants (played by other site managers). Instructors provided coaching and feedback for early role play exercises and reduced the coaching and feedback for later exercises. A final role play exercise, without coaching, was used to test each site manager’s interviewing skills. By allowing them to work through several role play exercises, with reduced coaching, the training program helped the site managers build their confidence and learn how to apply their new skills during interviews with housing subsidy applicants.

In the role plays, the instructor presented scenarios in which the site managers interviewed housing applicants. The role play exercises were completed in groups of three; one participant playing the site manager, another playing the applicant, and a third acting as observer. To set up each role play, the participant’s guide provided a description to the learner playing the applicant (see Exhibit 1).

Exhibit 1. Example of an applicant description used in a role play.


Rose comes in out of the rain, coughing and tells you she’s been sick for the last few weeks.  Rose is 78.  She recently retired from AJAX Corp where she worked for the last 10 years.  Her husband died a year ago and she has just sold her home netting $59,000 after the mortgage and closing costs were paid.

Income information:
Her social security check is for $800.00 per month but her check is being garnished for back taxes so she is only receiving $400.00 per month. She reports the following medical bills:

  • She reports she pays for Medicare of $98.00/mo.
  • She has a lifeline pendant she has been wearing in case of a medical emergency for which she pays $31.00 a month.
  • She had a major surgery last month that wasn’t completely covered by insurance; she put it on her credit card and still owes $2,000.00.

During the role play, the site managers were directed to use their interviewing skills to collect information from the applicant and record it on a sample tenant application. The observer used a criterion checklist (provided in the participant’s guide) to assess the site manager’s rapport building, questioning, and listening.

In the final role play, the instructor observed each participant and scored them using a rubric (see Table 2). The instructor was also given a key that guided their observations. For example, the key described the elements of “positive attitude” as appropriate eye contact, and good posture.

Table 2. Part of the rubric used to rate site managers during role play exercises.

Criterion Beginning 

0-1

Developing 

2-3

Fair 

4-5

Good 

6-7

Score Comments
Positive attitude 

(Obj. 1)

Does not demonstrate a positive attitude Rarely demonstrates a positive attitude Sometimes demonstrates a positive attitude Always demonstrates a positive attitude
Engaging the applicant through small talk 

(Obj. 1)

Does not engage the applicant through small talk Begins small talk but does not allow for response from applicant Begins small talk but is unresponsive to applicant’s statements Appropriately engages the applicant through small talk
Appropriate eye contact 

(Obj. 1, 3)

Does not maintain eye contact at any points Rarely maintains eye contact at appropriate points Sometimes maintains eye contact a appropriate points Always maintains eye contact at appropriate points
Restating applicant’s response when appropriate 

(Obj 2)

Never restates applicant’s response when appropriate Rarely restates applicant’s response when appropriate Sometimes restates applicant’s response when appropriate Always restates applicant’s response when appropriate
Eliciting feedback  

(Obj 2)

Never elicits feedback from applicant Rarely elicits feedback from applicant when appropriate Sometimes elicits feedback from applicant when appropriate Always elicits feedback from applicant when appropriate
Scoring key:6-7 = Good. The site manager is ready to use these skills in her job.
4-5 = Fair. The site manager can begin using these skills effectively, but can also benefit from more practice.

2-3 = Developing. The site manager needs more review and practice. We recommend using the skills outlined in the participant guide and job aids to rehearse these skills on their own.
0-1 = Beginning. The site manager needs more practice and review. We recommend using the participant guide and job aids, as well as more coaching, preferably with an instructor or experienced site manager.


Conclusion

Using training materials developed by the Boise State University team, ABC-Contractors conducted a three-hour pilot test with six site managers. Kirkpatrick (1996) level 2 (learning) data gathered during the pilot test revealed a high degree of mastery by two of the participants, achieving an “Excellent” rating on the final assessment. The remainder of the participants achieved a “Good progress” rating, which indicated areas where continued practice was recommended. Kirkpatrick level 1 (reaction) data gathered during the pilot test revealed that the participants gave the training program high ratings on dimensions such as clarity, thoroughness, and usefulness. In general, data gathered during the pilot test indicated that the use of problem-based learning helped site managers learn the interviewing skills needed to gather complete and accurate information from housing subsidy applicants, which will make it easier for ABC-Contractors to reach its goal of meeting HUD compliance levels while helping property owners save money.

References
HUD Office of Policy Development and Research (2008). Quality Control for Rental Assistance Subsidy Determinations. Available at www.huduser.org.

HUD RHIIP (2009). A Guide to Interviewing for Owners of HUD-Subsidized Multifamily Housing. Available at www.hud.gov/offices/hsg/mfh/rhiip/interviewguide.pdf.

Kirkpatrick, D. L. (1996). Evaluating training programs: The four levels. San Francisco: Berrett-Koehler Publishers.

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59.

Rick MauneyAbout the Authors
Rick Mauney is currently enrolled in Boise State University’s M.S. program in Instructional and Performance Technology. He is an instructional designer at the University of Chicago where he develops information technology training. Rick can be reached at remauney@sbcglobal.net.
Shari SteinShari Stein will complete her Master’s Degree in Instructional and Performance Technology from Boise State University in 2012. She is currently an adjunct faculty in the College of Art and Design at Lawrence Technological University in Southfield, Michigan. Shari may be reached at pidinc@comcast.net.